Audit traceability matrix export crashes when including defect history

ALM Siemens Polarion ALM
, , , , , , , ,
1

Exporting traceability matrices with defect history for compliance reporting causes heap overflow errors and crashes the report generation. We need these exports for audit trails, but the system fails when including more than a few months of defect history.

The crash happens during report generation:


java.lang.OutOfMemoryError: Java heap space
at ReportGenerator.buildMatrix(ReportGenerator.java:342)
at TraceabilityExport.generate(TraceabilityExport.java:156)

Our traceability matrix links requirements to test cases to defects, and auditors need to see the complete defect lifecycle. When we exclude history, the export works fine but doesn’t meet compliance requirements. When we include history (especially for defects with 50+ status changes), the JVM runs out of memory around 60% completion.

We’ve tried increasing heap size to 8GB with -Xmx8g, but it still crashes on large projects. The issue seems to be with how the report generation loads defect history into memory - it appears to load everything at once rather than streaming. Has anyone found workarounds for generating comprehensive traceability matrices without heap overflow in pol-2406?

2

The real issue is that the traceability matrix query doesn’t have history limits configured. By default, it tries to load ALL history for every defect in the matrix. Go to the report configuration and add a history limit parameter - something like ‘maxHistoryEntries=100’ or ‘historyDateRange=90days’. This tells the report generator to only include recent history, which should keep memory usage reasonable while still providing audit trails. You might need to run separate historical exports for older data.

3

Increasing heap size alone won’t solve this - you need to tune other JVM parameters too. Try adding -XX:+UseG1GC -XX:MaxGCPauseMillis=200 to improve garbage collection. Also set -XX:+HeapDumpOnOutOfMemoryError so you can analyze what’s consuming memory. The report generation might be holding references to objects that should be garbage collected. Look at your JVM tuning settings in the Polarion configuration and optimize for report generation workloads.

4

Check your database query performance. The report generation might be timing out on database queries, which causes it to retry and accumulate objects in memory. We found that adding indexes on the history tables significantly improved traceability export performance. Specifically, index the work item ID and timestamp columns in the history tables. Also verify that your database connection pool has enough connections available for report generation - connection starvation can cause memory buildup.

5

I can provide a comprehensive solution that addresses all aspects of this heap overflow issue.

Traceability Matrix Configuration: The core problem is that the default traceability matrix export loads unlimited history. You need to configure history limits in your report template:

  1. Open the traceability matrix report configuration
  2. Add these parameters to the report query:
<parameter name="maxHistoryEntries" value="50"/>
<parameter name="historyDateRange" value="180"/>
<parameter name="batchSize" value="500"/>

This limits each defect to 50 history entries or 180 days, whichever is less, and processes defects in batches of 500.

Report Generation Optimization: Modify your report generation settings to use streaming mode:

  1. Edit the report template XML
  2. Change the generation mode from ‘full’ to ‘streaming’
  3. Enable incremental processing:
<generation-mode>streaming</generation-mode>
<incremental>true</incremental>
<chunk-size>1000</chunk-size>

This processes the matrix in chunks rather than loading everything into memory.

JVM Tuning for Report Generation: Your heap size increase is good, but you need additional tuning. Update your JVM parameters in polarion.properties:

java.awt.headless=true
-Xmx8g
-Xms4g
-XX:+UseG1GC
-XX:MaxGCPauseMillis=200
-XX:+ParallelRefProcEnabled
-XX:MaxMetaspaceSize=512m

The G1 garbage collector handles large heaps better than the default collector. The parallel reference processing helps with the many object references in history data.

History Limits Strategy: For compliance reporting, you don’t need ALL history - you need complete audit trail coverage. Implement a tiered approach:

  1. Recent history (last 90 days): Include all status changes, comments, attachments
  2. Historical data (90-365 days): Include only status changes and key milestones
  3. Archived data (>365 days): Include only initial creation and final resolution

Configure this in your report template with conditional history loading based on defect age.

Database Query Optimization: Add indexes to improve history query performance:

CREATE INDEX idx_history_workitem
ON work_item_history(work_item_id, timestamp);
CREATE INDEX idx_history_type
ON work_item_history(change_type, timestamp);

Faster queries mean less time holding objects in memory.

Alternative Approach - Scheduled Report Generation: For very large matrices, use scheduled background generation:

  1. Configure the report to run during off-peak hours
  2. Increase the report generation timeout to 60 minutes
  3. Enable result caching so subsequent exports use cached data
  4. Set up email notification when generation completes

This allows the report to use more time and resources without impacting interactive users.

Compliance-Specific Configuration: For audit requirements, create a dedicated compliance report template:

  1. Include only fields required for audit (status, assignee, resolution, timestamps)
  2. Exclude verbose fields like full descriptions and comments
  3. Use summary format for history instead of full detail
  4. Link to individual defect pages for detailed history rather than embedding everything

Auditors can drill down into specific defects if needed, but the matrix itself remains manageable.

Monitoring and Validation: After implementing these changes:

  1. Enable JVM monitoring during report generation
  2. Watch heap usage - it should stay below 70% of max heap
  3. If you still see heap pressure, reduce maxHistoryEntries further
  4. Test with progressively larger date ranges to find your optimal settings
  5. Document the configuration for compliance auditors so they understand the history limits

With these optimizations, you should be able to generate comprehensive traceability matrices that meet compliance requirements without heap overflow. The key is balancing completeness with memory efficiency through proper history limits and streaming generation. If you still need more complete history for specific audits, generate targeted reports for smaller subsets of defects rather than trying to include everything in one export.

6

Consider using the REST API to build custom reports instead of the built-in traceability matrix export. The API lets you control exactly how much data you load and when. You can implement pagination and streaming, loading defect history in batches rather than all at once. We built a custom compliance report generator that processes 1000 defects at a time, generates partial matrices, and combines them. Takes longer to run but never crashes, and meets all audit requirements.