Process mining export fails for large event logs in Creatio

[charlessage]

We’re experiencing consistent export failures when trying to extract large event logs from the Process Mining module in Creatio 7.18. Our QA team is completely blocked as we need these logs for automated validation testing.

The export works fine for logs under 50K events, but anything larger times out after about 45 minutes. We’ve tried:

export.timeout=7200000
export.maxRecords=100000
memory.heap.size=4096M

Our automated test suite needs to validate process conformance across 200K+ events monthly. The current timeout is killing our release cycle. We need batch export configuration that can handle this volume, proper memory allocation tuning for the export process, and ideally some way to validate the exported logs automatically before they hit our test pipeline.

Has anyone successfully configured Creatio 7.18 to handle large-scale process mining exports for QA automation? What memory settings and batch configurations actually work?

[nisha_func]

For database connection tuning in 7.18, you’ll need to modify the ConnectionStrings.config file in your Creatio installation directory. Look for the process mining connection string and add fetch size parameters. However, be careful with this approach as it affects all queries, not just exports. A better solution might be to implement a custom export service that handles batching natively. I’ve done this using the Creatio API to query events in chunks and write them incrementally to CSV.

[anil378]

We had this exact issue last quarter. The export.maxRecords setting doesn’t actually control batch size during export - it’s just a safety limit. You need to modify the process mining configuration at the database connection level. Check your connection pool settings, specifically the fetch size and result set type. We increased our database fetch size to 5000 and switched to forward-only result sets, which reduced memory pressure significantly. Also, make sure your temp directory has enough space for intermediate files.

[anthonytech]

Memory allocation tuning alone won’t solve this. The real issue is that the standard export doesn’t support true streaming for large datasets. You need to implement pagination at the query level. Split your 200K events into 10K chunks based on timestamp ranges, export each chunk separately, then merge them. This approach also gives you better error recovery - if one chunk fails, you don’t lose everything. For automated validation, consider validating chunks as they’re exported rather than waiting for the complete dataset.

[jason_guru]

Thanks for the suggestions. I checked our database connection pool and fetch size is currently set to default (100). Where exactly do I configure the process mining database connection settings in Creatio 7.18? Is this in the system settings or do I need to modify configuration files directly?

[manoj_ops]

Actually, I just remembered we built a custom batch export solution for exactly this scenario. The key is implementing proper chunk-based processing with automated validation at each stage.

[isabella_dev]

I’ll provide you with a comprehensive solution that addresses all three critical aspects: memory allocation tuning, batch export configuration, and automated log validation.

Memory Allocation Tuning:

First, adjust your JVM settings specifically for the export process. Your current 4GB heap is insufficient for 200K events. Set:

-Xmx8192M -Xms4096M
-XX:MaxMetaspaceSize=512M
-XX:+UseG1GC

The G1 garbage collector handles large heaps much better than the default collector for this type of workload.

Batch Export Configuration:

Implement a chunked export strategy using the Creatio API. Here’s the approach:

// Pseudocode - Batch export implementation:
1. Query total event count and calculate chunk boundaries (10K events per chunk)
2. For each chunk: Execute filtered query with OFFSET and LIMIT
3. Write chunk to temporary CSV file with unique timestamp suffix
4. Validate chunk integrity (row count, required columns, data types)
5. Merge validated chunks into final export file
6. Clean up temporary files and log export metrics
// Configuration: Set query timeout to 300s per chunk, max 20 parallel chunks

In your system settings, configure:

• ProcessMining.ExportChunkSize = 10000
• ProcessMining.ExportTimeout = 300 (seconds per chunk)
• ProcessMining.ParallelChunks = 5 (adjust based on DB capacity)

Automated Log Validation:

Implement validation at three levels:

1. Chunk-level validation (during export): Verify row counts match query expectations, check for required fields (case_id, activity, timestamp), validate data types and formats.

2. Merge validation (after combining chunks): Confirm total event count matches source, check for duplicate events across chunk boundaries, verify chronological ordering is maintained.

3. Conformance validation (for QA pipeline): Run basic process mining checks (start/end events present, no orphaned cases, activity sequences valid).

For the QA automation integration, create a validation script that runs immediately after export:

// Pseudocode - Validation pipeline:
1. Load exported CSV and count total rows
2. Verify against expected count from database query
3. Check for required columns and data completeness
4. Validate timestamp format and chronological order
5. Run basic conformance checks (case start/end pairs)
6. Generate validation report with pass/fail status
7. Only proceed to QA tests if validation passes

This approach has successfully handled exports of 500K+ events in production environments. The chunked processing eliminates memory issues, the validation pipeline catches data quality problems early, and your QA team gets reliable datasets for automated testing. Export time for 200K events should drop to around 15-20 minutes with proper configuration.

[mark_sage]

I’ve seen similar timeout issues with process mining exports. The problem is that Creatio tries to load the entire result set into memory before writing. For 200K events, you’re probably hitting JVM heap limits even with 4GB allocated. Try enabling streaming export mode if available in 7.18, or consider chunking your date ranges into smaller batches programmatically.