Let me provide a comprehensive solution for handling concurrency errors in your sales order updates. You’re dealing with optimistic concurrency control in a high-volume environment, which requires a multi-layered approach:
1. Optimistic Concurrency Control - Proper ETag Usage:
The core issue is that D365’s entity-level ETags are too broad for your use case. Any modification to the sales order invalidates your ETag, even if it’s an unrelated field. Implement field-level change detection:
// Pseudocode - Enhanced update with field-level validation:
1. GET sales order with specific fields: status, shippingMethod, deliveryDate, lastModifiedDateTime
2. Store retrieved values and ETag
3. Build PATCH payload only for fields that need updating
4. Before PATCH, add conditional logic:
If-Match: {etag}
If-Unmodified-Since: {lastModifiedDateTime}
5. On 412 error, re-GET and compare ONLY your target fields
6. If target fields unchanged, retry with new ETag
This approach distinguishes between conflicts that matter (someone changed your target fields) versus noise (background processes modified other fields).
2. ETag Usage - Intelligent Retry Strategy:
Implement a smart retry mechanism that analyzes the conflict:
// Pseudocode - Conflict resolution logic:
1. On 412 error, immediately re-GET the order
2. Compare original values vs. current values for your target fields
3. If NO conflict in target fields:
- Apply your changes to current version
- Retry PATCH with new ETag (max 3 attempts)
4. If CONFLICT in target fields:
- Log conflict details for manual review
- Implement business logic: merge changes or defer to latest
5. Use exponential backoff: wait 500ms, 1.5s, 4s between retries
3. Order Update Retry Logic - Architectural Improvements:
Reduce the window of vulnerability:
Pattern A - Atomic Operations: Use D365 action endpoints for atomic updates when available:
- UpdateOrderStatus action: Handles status transitions with built-in concurrency management
- UpdateShippingDetails action: Updates shipping-related fields as a single unit
Pattern B - Optimistic Batch Updates: For multiple order updates, use batch requests with individual change sets. This ensures partial success - some orders update even if others fail.
Pattern C - Application-Level Locking: Implement distributed locking for your integration:
// Pseudocode - Prevent self-inflicted concurrency:
1. Before processing order, acquire distributed lock: LOCK:ORDER:{orderId}
2. Set lock timeout: 30 seconds
3. Perform GET, validation, PATCH operations
4. Release lock after completion or on error
5. If lock acquisition fails, queue order for retry (another thread processing it)
Pattern D - Queue-Based Processing: During peak hours, implement a processing queue:
- Deduplicate order updates (only process latest update per order)
- Process orders in batches of 10-15 with delay between batches
- This reduces simultaneous update attempts on popular orders
Monitoring and Adaptation:
Implement metrics to track:
- Concurrency error rate by hour of day
- Average retry count per successful update
- Orders requiring manual reconciliation
Use these metrics to dynamically adjust retry behavior - increase retry attempts and backoff delays during known peak hours.
Specific Recommendation for Your 15% Error Rate:
Your high error rate suggests multiple processes are modifying orders simultaneously. Investigate:
- Check if warehouse management system is auto-updating orders during peak hours
- Verify if customer service team is manually modifying orders during this window
- Review scheduled batch jobs that might run at 2-5 PM
Once you identify competing processes, coordinate update timing or implement the queue-based approach to serialize updates. Combined with intelligent retry logic, you should reduce your error rate from 15% to under 3% even during peak hours.