Best practices for error handling vs retry logic in RPA API calls

BPM Mendix
, , , , , , , ,
1

I’m building RPA workflows in Mendix that make extensive API calls to external systems (ERP, CRM, document management). The challenge is deciding between robust error handling versus aggressive retry logic. Should we classify errors and only retry transient failures, or implement exponential backoff for all failures? I’m particularly interested in how to balance reliability with idempotency - we can’t have the same order created twice because a retry happened after a timeout that actually succeeded. What are the community’s best practices for error classification, retry strategies, and ensuring idempotency in RPA API integrations?

2

Our retry strategy uses exponential backoff with jitter: first retry after 1 second, then 2, 4, 8, up to max 60 seconds, with random jitter added. We cap at 5 retries for transient errors. For RPA specifically, we also implement circuit breakers - if an external system has 10 consecutive failures, we open the circuit and stop calling it for 5 minutes. This prevents overwhelming failing systems and gives them time to recover. After the cooldown, we try one request (half-open state) to test if the system is healthy again.

3

Don’t forget about monitoring and alerting. We log every retry with the error type, retry count, and outcome. This helps identify flaky external systems or systematic issues. If we see the same endpoint requiring retries frequently, that’s a signal to investigate. Also, set up alerts for when retry exhaustion happens - when all retries fail, someone needs to know immediately so they can intervene manually if needed.

4

Let me synthesize the best practices for error handling, retry strategies, and idempotency in RPA API integrations based on extensive implementation experience.

Error Classification Framework:

Implement a three-tier classification system:

  1. Transient Errors (retry appropriate):

    • Network timeouts, connection resets
    • HTTP 429 (rate limit), 503 (service unavailable), 504 (gateway timeout)
    • Database deadlocks or connection pool exhaustion
    • Action: Retry with exponential backoff

  2. Permanent Errors (don’t retry):

    • HTTP 400 (bad request), 401 (unauthorized), 403 (forbidden), 404 (not found)
    • Validation errors, malformed requests
    • Authentication failures
    • Action: Log, alert, and fail fast

  3. Ambiguous Errors (special handling):

    • Timeouts where response wasn’t received
    • Connection errors after request was sent
    • Action: Check idempotency key/operation log before retry

Retry Strategies for RPA:

Implement layered retry logic:

  1. Immediate Retry (once): For very transient issues like momentary network blips
  2. Exponential Backoff: 1s, 2s, 4s, 8s, 16s (max 5 attempts)
  3. Jitter: Add random 0-1000ms to prevent thundering herd
  4. Circuit Breaker: After 10 consecutive failures, stop calling for 5 minutes
  5. Retry Budget: Limit retries to 10% of requests per minute to prevent cascade failures

For RPA workflows, also implement workflow-level retries - if the entire workflow fails after API retries are exhausted, schedule the workflow to retry in 1 hour with fresh context.

Idempotency in APIs:

Three approaches depending on external API capabilities:

  1. Native Idempotency Keys (preferred):

    • Generate UUID for each operation
    • Include in Idempotency-Key header
    • External system deduplicates automatically

  2. Operation Log Pattern (when API doesn’t support keys):

    • Maintain Mendix entity: OperationLog (uuid, operationType, parameters, status, result)
    • Before API call: Insert ‘pending’ record
    • After success: Update to ‘completed’ with result
    • On retry: Query log first, return cached result if completed

  3. Query-Before-Retry Pattern (for ambiguous errors):

    • After timeout/ambiguous error, query the external system
    • Check if operation succeeded using business key (order number, transaction ID)
    • Only retry if definitively not completed

Mendix Implementation Pattern:

Create a reusable ‘ResilientAPICall’ microflow that encapsulates this logic:

  • Input: endpoint, method, payload, operation type, idempotency key
  • Implements error classification
  • Executes retry logic with backoff
  • Manages operation log for idempotency
  • Returns success/failure with detailed error info

Use this microflow consistently across all RPA API calls for uniform behavior.

Critical Success Factors:

  1. Monitoring: Log every retry with context - you need visibility into retry patterns
  2. Alerting: Alert on retry exhaustion and circuit breaker activation
  3. Testing: Test retry logic with chaos engineering - simulate failures deliberately
  4. Documentation: Document which errors trigger retries for each external system
  5. Tuning: Monitor retry success rates and adjust backoff timing based on real data

For RPA specifically, remember that reliability is more important than speed. It’s better to have a workflow take 2 minutes with proper retries than to have it fail and require manual intervention that takes 2 hours to resolve.