Simulation data batch import fails with memory leak and server crashes on TC 12.3

PLM Teamcenter
, , , , , , , ,
1

We’re experiencing critical server crashes during large-scale simulation data imports in TC 12.3. Our workflow processes 500+ simulation results daily from ANSYS and MATLAB, each containing mesh data and analysis outputs.

The import operation starts fine but after processing 150-200 files, we see severe performance degradation. JVM heap usage climbs to 95%+ and eventually triggers OutOfMemoryError. Our current heap is set to 16GB.

ERROR: java.lang.OutOfMemoryError: Java heap space
at SimDataImporter.processLargeDataset(SimDataImporter.java:234)
at BatchProcessor.importSimResults(BatchProcessor.java:156)

We’ve tried increasing heap to 24GB but the problem persists. The memory leak appears related to batch size configuration - currently processing 50 files per batch. GC logs show frequent full GC cycles taking 8-12 seconds each. Server becomes unresponsive and requires restart.

Has anyone dealt with similar memory issues during simulation data imports? Need guidance on JVM heap tuning, optimal batch sizes, and GC log analysis for this scenario.

2

Don’t overlook the batch commit strategy. If you’re keeping transactions open for entire batches, that’s holding references to all imported objects until commit. We implemented micro-commits every 5 files within a batch and saw dramatic memory improvement. Also verify your simulation data cleanup - are temporary analysis files being properly deleted after import?

3

Had the same issue last year with our ANSYS integration. The problem is multi-faceted and requires addressing all three areas you mentioned.

JVM Heap Tuning: Your current approach is counterproductive. Use these optimized settings:

-Xms10G -Xmx10G
-XX:+UseG1GC
-XX:MaxGCPauseMillis=200
-XX:G1HeapRegionSize=32M
-XX:InitiatingHeapOccupancyPercent=45

Fixed heap size (Xms=Xmx) prevents resize overhead. G1GC with 32M regions handles large simulation objects efficiently. The 45% threshold triggers concurrent marking earlier, preventing full GC surprises.

Batch Size Configuration: Reduce to 10-15 files per batch maximum. Simulation data is fundamentally different from CAD data - each file contains dense numerical arrays. Implement this pattern:

// Pseudocode - Optimized batch processing:
1. Initialize batch with maxSize=10, currentSize=0
2. For each simulation file in queue:
3.   Load file metadata only (not full content)
4.   If currentSize + estimatedSize > threshold: commit batch, start new
5.   Process file with streaming API (chunk size 5MB)
6.   Explicitly clear object references after import
7.   If currentSize % 5 == 0: force minor GC suggestion
8. Commit final batch and cleanup temp resources

GC Log Analysis: Enable comprehensive logging:


-Xloggc:/opt/teamcenter/logs/gc.log
-XX:+PrintGCDetails
-XX:+PrintGCDateStamps
-XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=5
-XX:GCLogFileSize=20M

Analyze for these patterns:

  • Old Gen growth rate: Should be < 100MB/minute during imports
  • Full GC frequency: Should be < 1 per hour
  • Pause times: Target < 500ms for young GC, < 2s for mixed GC

If Old Gen grows linearly without dropping, you have a true memory leak. Use -XX:+HeapDumpOnOutOfMemoryError to capture dumps for analysis.

Additional Critical Fix: Modify your SimDataImporter to use WeakReferences for cached data and implement proper resource cleanup:

try {
    processSimulationData(file);
} finally {
    clearCaches();
    closeStreams();
    System.gc(); // Suggest only after batch
}

After implementing these changes, we went from crashes every 6 hours to stable 24/7 operation processing 800+ files daily. Our heap usage stabilized at 55-65% with GC pauses under 300ms. Monitor for 48 hours and adjust batch size if needed - some ANSYS CFD files are exceptionally large and may need batch size of 5.

4

I’ve seen this exact pattern. Your batch size of 50 is way too aggressive for simulation data. We had similar crashes and reducing to 15-20 files per batch made a huge difference. Also check if your GC algorithm is appropriate - for large heap sizes, G1GC performs better than ParallelGC.

5

The GC pause times you’re seeing (8-12 seconds) are definitely problematic. Can you share your current JVM arguments? Specifically the GC settings and heap configuration. Also, are you explicitly clearing object references after each batch completes? Simulation data objects can be quite large and if they’re not properly released, you’ll accumulate memory pressure. Run with -XX:+PrintGCDetails -XX:+PrintGCDateStamps to get detailed GC logging. Look for patterns where old generation fills up rapidly - that’s your smoking gun for the leak.

6

Check your SimDataImporter implementation. Are you using streaming APIs or loading entire datasets into memory? With ANSYS mesh data, you’re probably dealing with multi-GB files. We switched to a streaming approach where we process data in chunks rather than loading complete files. This kept our heap usage stable around 60% even with continuous imports.

7

Your heap increase from 16GB to 24GB might actually be making things worse. Larger heaps mean longer GC pauses. Consider these settings: -Xms8G -Xmx12G -XX:+UseG1GC -XX:MaxGCPauseMillis=200. The G1GC collector is specifically designed to handle large objects and minimize pause times. Also set -XX:G1HeapRegionSize=32M for better handling of simulation data objects.