Price list synchronization API fails with duplicate record errors in pricing management

ERP Microsoft Dynamics 365
, , , , , , , ,
1

We’re encountering persistent duplicate record errors when syncing price lists through the pricing management REST API in D365 10.0.41. Our integration pushes product pricing updates from an external system every 4 hours, but about 30% of requests fail with HTTP 400 errors indicating duplicate records.

The payload deduplication logic seems inconsistent - sometimes identical product identifiers pass through, other times they’re rejected. Here’s a sample request that failed:

{
  "PriceListId": "PL-2025-Q1",
  "Products": [
    {"ProductId": "PROD-8821", "Price": 149.99},
    {"ProductId": "PROD-8821", "Price": 149.99}
  ]
}

We need partial success handling - if 1 of 50 products has an issue, the entire batch shouldn’t fail. The outdated pricing data is causing significant issues with customer quotes. Has anyone resolved similar duplicate record problems with the pricing API?

2

Update: we identified the root cause - our external system was generating duplicate ProductIds during certain edge cases. Fixed that first. Then implemented the batch API approach with proper error handling.

3

Thanks for catching that - the example was simplified, but our actual production payloads do have client-side deduplication. The issue is more subtle. We’re getting duplicate errors even when products appear only once in our request. I suspect it’s related to timing - if two integration jobs overlap slightly, the API might see the same product from different requests as duplicates. Is there a way to handle this at the API level?

4

The timing overlap theory is correct. The pricing API has a brief window where concurrent requests for the same product can conflict. We solved this by implementing request-level locking in our middleware - check if a product is currently being processed before submitting. Also, consider using the async batch API endpoints instead of synchronous calls. They handle conflicts better and provide better partial success reporting through the batch status endpoint.

5

Comprehensive solution for price list synchronization with duplicate handling:

1. Payload Deduplication (Client-Side) Implement pre-submission validation to ensure unique product identifiers within each request batch:

var uniqueProducts = products
  .GroupBy(p => p.ProductId)
  .Select(g => g.First())
  .ToList();

2. Use Batch API for Partial Success Handling Switch from synchronous single-item endpoints to the batch pricing API. This provides granular status reporting:

  • Endpoint: `/data/PriceListBatches
  • Submit batch, receive batch ID
  • Poll /data/PriceListBatches('{batchId}')/Status for completion
  • Process response to identify successful vs failed items
  • Retry only failed items in subsequent batch

3. Idempotency Implementation Store request metadata to prevent duplicate submissions:

var requestKey = $"{priceListId}_{productId}_{timestamp}";
if (processedRequests.Contains(requestKey)) return;

4. Unique Product Identifier Validation Before API submission, validate against D365’s product master:

  • Query `/data/Products?$filter=ProductNumber eq ‘{productId}’
  • Verify single match exists
  • Handle cases where external ID maps to multiple D365 products

5. Conflict Resolution Strategy For concurrent update scenarios:

  • Implement distributed locking (Redis/Azure Cache)
  • Use optimistic concurrency with ETag headers
  • Add retry logic with exponential backoff (3 attempts, 2/4/8 second delays)

6. Enhanced Error Handling Parse API error responses to distinguish duplicate types:

  • Same-request duplicates: Fix client-side deduplication
  • Cross-request duplicates: Implement locking/idempotency
  • Data quality duplicates: Cleanse source system

7. Monitoring and Alerting Log all duplicate errors with context:

  • ProductId, PriceListId, timestamp, request payload
  • Track duplicate error rate as KPI
  • Alert when rate exceeds 5% threshold

This approach addresses all three focus areas: proper payload deduplication prevents same-request errors, unique identifier validation catches data quality issues, and batch API enables robust partial success handling. Our implementation reduced duplicate errors by 94% and improved overall sync reliability to 99.2%.

6

You should also look at the API response headers - D365 returns a ‘X-Request-Id’ that you can use for idempotency. Store these request IDs and check before resubmitting. For partial success handling, the batch API is definitely the way to go. It returns granular status for each item in the batch, allowing you to retry only the failed products without reprocessing successful ones. We reduced our failure rate from 28% to under 2% using this approach.

7

I’ve seen this exact behavior. The pricing API doesn’t automatically deduplicate within the same request payload - you need to handle that client-side before submission. Your example shows PROD-8821 twice in the same batch, which triggers the duplicate error. Implement a dictionary-based deduplication in your integration layer to ensure unique product identifiers per request.

8

Another consideration: verify your unique product identifiers are truly unique across your product catalog. We discovered some products had duplicate external IDs due to legacy data migration issues. The API was correctly rejecting them as duplicates because they mapped to the same internal D365 product record. Run a data quality check on your source system to ensure ProductId values are genuinely unique before attempting API synchronization.