Our production tracking system sends order status updates to CloudSuite via the asynchronous ION API. During daytime (8am-5pm), updates appear in the system within 2-3 minutes. However, during evening shifts (6pm-11pm), the same updates take 15-30 minutes to process. This delay impacts our real-time inventory calculations and creates confusion on the shop floor.
We’re using the async API specifically because we need to handle high volumes - around 200-300 status updates per hour during peak production. The ION event queue seems to be the bottleneck, but we’re not sure about the right tuning parameters. Should we consider switching to synchronous API calls for critical updates, or is there a way to optimize the async processing? We also lack visibility into whether events are being delivered successfully or getting stuck in the queue.
We had similar symptoms last year. The issue was ION’s default thread pool allocation - only 5 threads were assigned to our custom event processing, while 20 threads were allocated to standard Infor workflows. You need to adjust the thread pool settings in ION Configuration. Also verify your connection point isn’t rate-limited. Check the Connection Point settings in ION Desk under Administration.
Thanks both. I found several scheduled jobs running in that time window - master data sync and financial consolidation. Could those really cause 15-30 minute delays though? Our volume isn’t that high compared to what ION should handle.
Absolutely yes. ION uses a priority queue system, and standard Infor processes often get higher priority than custom integrations by default. If your connection point priority isn’t configured correctly, your events will wait while scheduled jobs complete. Navigate to ION Desk > Administration > Connection Points, find your production update connection, and verify the Priority setting is at least Medium or High. Also check the Max Concurrent Messages setting - if it’s set too low (like 1 or 2), that’s your bottleneck right there.
One thing people overlook is ION’s message persistence settings. If your messages are configured for guaranteed delivery with full persistence, each message gets written to disk before processing, which adds significant latency under load. For production status updates that are frequently updated anyway, you might consider using at-most-once delivery semantics instead of exactly-once. This reduces the disk I/O overhead dramatically.
Let me address all three aspects of your situation systematically:
ION Event Queue Tuning: Your evening delays are almost certainly due to queue configuration and resource contention. Here’s what to optimize:
-
Connection Point Settings: Increase Max Concurrent Messages from default (usually 5) to 15-20 for your production update connection. This allows parallel processing of status updates.
-
Thread Pool Allocation: In ION Configuration Manager, navigate to System > Thread Pools. Increase the custom integration thread pool from default 5 to at least 10 threads. This prevents scheduled jobs from starving your real-time updates.
-
Queue Priority: Set your production order connection point priority to HIGH. This ensures your events get processed before batch jobs, even during scheduled task windows.
-
Message Persistence: For frequently-updated status changes, switch from PERSISTENT to NON_PERSISTENT delivery mode in your BOD configuration. Status updates are inherently idempotent - if one is lost, the next update will correct it.
Async vs Sync API Tradeoffs: You’re right to use async for high volume, but consider a hybrid approach:
- Use async API for routine status updates (in-progress, material staged, quality check)
- Switch to sync API for critical state transitions (started, completed, failed) that require immediate confirmation
- This gives you throughput for bulk updates while ensuring critical events are processed immediately
The sync API has a 30-second timeout, so it handles 200-300 updates/hour easily. The key is using it selectively.
Monitoring Event Delivery: Implement these visibility measures:
-
Enable ION Message Tracking in ION Desk > Administration > System Configuration. Set retention to 7 days minimum.
-
Create a custom monitor dashboard querying the ION_MESSAGE_TRACKER table:
- Messages in FAILED status
- Messages with processing time > 5 minutes
- Queue depth by hour
-
Set up alerts in ION for queue depth thresholds. If your queue exceeds 50 pending messages, trigger an email alert.
-
Use ION’s REST API to programmatically check message status:
GET /ion-api/messagetracker/messages?correlationId={yourId}
For your specific evening issue, I’d bet you have financial batch jobs running 6-10pm that are consuming most ION resources. Check the Scheduled Tasks in ION Desk and either reschedule them to off-hours (midnight-6am) or increase overall ION resource allocation if rescheduling isn’t possible. We had the exact same pattern and resolved it by moving consolidation jobs to 2am and increasing thread pools - evening processing time dropped from 20 minutes to under 3 minutes.