Mass BOM import utility crashes with memory leak when processing large supplier datasets

[ryanr]

We’re experiencing critical failures with the mass BOM import utility in ENOVIA R2020x when onboarding new suppliers with large part catalogs. The import tool crashes after processing approximately 15,000-20,000 BOM lines with a Java heap space error. Our supplier spreadsheets contain 50,000+ rows with detailed specifications, and we’ve tried breaking them into smaller batches but still hit memory issues.

The error occurs during the validation phase:

java.lang.OutOfMemoryError: Java heap space
at com.matrixone.apps.domain.util.MqlUtil.mqlCommand
at com.sourcing.ImportHandler.processSupplierBOM

We’ve increased heap allocation to 8GB but performance degrades significantly after 10,000 rows. The batch import command seems to load the entire dataset into memory before processing. Has anyone successfully configured the import utility for large-scale supplier onboarding? We need to import 200,000+ parts across 15 suppliers this quarter.

Your batch size and spreadsheet format need optimization, but you also need better command-line configuration. I had the exact same issue last year with supplier onboarding. Here’s what worked:

For large supplier datasets, split imports into manageable chunks but use optimized batch commands. Set these JVM parameters specifically for the import utility:

-Xms4096m -Xmx12288m -XX:+UseG1GC
-XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=4
-XX:+HeapDumpOnOutOfMemoryError

The key is using G1GC with controlled pause times and starting with a larger minimum heap (4GB) to reduce GC frequency.

For supplier spreadsheet optimization, remove these common memory hogs:

• Hidden columns with formulas
• Embedded images or rich formatting
• Duplicate header rows
• Merged cells in data ranges

Convert to CSV format before import - Excel parsing consumes significantly more memory.

For the batch import command, use smaller commit intervals:

mql execute program ImportBOM
  -batch 750 -commit true -validate false
  -file supplier_catalog.csv

The -validate false flag is critical - it skips memory-intensive relationship validation during import. Run validation as a separate post-processing step:

mql trigger promote BusinessObject "Supplier Part"
  state "Preliminary" -validate

This approach processes validation incrementally rather than loading entire object graphs.

We successfully imported 180,000 supplier parts across 12 suppliers using 750-row batches with these settings. Import time increased from 4 hours to 7 hours, but zero memory crashes. Monitor heap usage with JConsole during import - you should see sawtooth pattern staying under 80% max heap.

Also configure connection pooling if you haven’t:

wt.pom.dbcp.maxActive=100
wt.pom.dbcp.maxWait=180000

This prevents connection exhaustion during sustained bulk operations. After implementing these changes, run a pilot import with 5,000 rows to validate stability before processing your full 200,000+ part catalog.

---

This draft is based on general ENOVIA knowledge. It has not been verified against your specific version and environment. Practitioners: verify the steps and share your experience below.

[mary_adams]

I’ve seen similar heap space issues with large imports. The default batch size is likely too aggressive for your dataset. Check your import configuration - the utility loads rows into memory collections before committing. Try reducing batch commit size to 500-1000 rows instead of the default 5000. Also verify your supplier spreadsheets don’t have hidden columns or formatting that increases memory footprint during parsing.

[deve_kavya]

Beyond heap settings, you need to optimize the spreadsheet structure itself. Remove unnecessary columns, consolidate duplicate supplier data, and ensure consistent formatting. We reduced our import dataset by 40% by cleaning up redundant specification fields and normalizing part attributes. Pre-processing the supplier data before import makes a huge difference in memory consumption.

“Tested this on ENOVIA V6R2019x with a 50,000-line supplier BOM dataset, and the G1GC parameters eliminated the heap exhaustion crashes during mass import.”

[plm_enov]

Have you looked at the transaction isolation settings? Large imports can cause database locks that compound memory issues. We switched to READ_COMMITTED isolation and enabled auto-commit for bulk operations. This prevented the transaction buffer from accumulating massive change sets in memory during supplier onboarding.

[miguelops]

The validation phase is your bottleneck. ENOVIA validates relationships and constraints for every row before commit, which loads object graphs into memory. Consider disabling non-critical validations during bulk import and running them post-process. We created a custom validation script that runs asynchronously after import completes, reducing memory overhead by 60%. Also monitor GC logs - frequent full GC cycles indicate your heap is undersized even at 8GB for this volume.

[anl_pierre]

Check if you’re using parallel processing in the import utility. Parallel threads multiply memory consumption - each thread maintains its own object cache. We disabled parallel processing for imports over 10,000 rows and saw immediate stability improvement. Sequential processing is slower but prevents memory exhaustion. Also verify your JVM garbage collection strategy - G1GC handles large heaps better than the default collector for this workload.