Territory management data quality validation rules fail to trigger during bulk import operations

elena_analyst
Sun, 11 May 2025 11:15:00 +0000

Let me provide a comprehensive solution that addresses all aspects of bulk import validation for territory management.

Bulk Import Validation Configuration: The core issue is that REST API bulk imports bypass data quality validation by default for performance reasons. To enable validation, modify your API requests:

1. Add validation parameters to your POST request:

POST /crmRestApi/resources/11.13.18.05/territories
Header: validateData=true
Header: validationLevel=FULL

2. These headers instruct the API to execute the same validation rules that run in the UI context. Without them, the import service assumes pre-validated data and skips quality checks.

Data Quality Rules Execution Context: Your existing data quality rules need to be configured to execute in bulk import scenarios:

1. Navigate to Setup > Data Quality > Validation Rules

2. For each territory validation rule, click Edit

3. Go to the “Execution Settings” tab

4. Enable these execution contexts:

UI_CONTEXT: Already enabled (working correctly)

API_CONTEXT: Enable for REST API operations

BULK_IMPORT_CONTEXT: Critical for bulk operations

INTEGRATION_CONTEXT: Enable for scheduled integration jobs

5. Set “Validation Timing” to “Pre-Commit” rather than “Post-Commit” - this ensures validation happens before records are saved

6. For territory-specific validations (overlap detection, postal code validation), set “Execution Priority” to “High” to ensure they run even under heavy load

REST API Configuration: Configure your integration to use the proper validation framework:

1. Modify your bulk import payload to include validation directives:

{
  "territories": [...],
  "importOptions": {
    "validationMode": "FULL",
    "onError": "ABORT",
    "validateAsync": false
  }
}

2. The ABORT error handling is critical - it ensures that if any record fails validation, the entire batch is rolled back. This prevents partial imports that leave your territory data in an inconsistent state.

3. Set validateAsync to false for synchronous validation during the import. While this impacts performance, it ensures immediate feedback on validation failures.

Error Handling Strategy: Implement robust error handling to capture and report validation failures:

1. Configure your REST API client to capture validation error responses:

HTTP 400 responses indicate validation failures

Parse the response body for detailed validation error messages

Each failed record will include field-level validation errors

2. Set up error logging in your integration:

if (response.status == 400) {
  log validation errors
  quarantine failed records
  notify data quality team
}

3. Create an error handling workflow:

Failed records go to a quarantine table

Data quality team reviews and corrects errors

Corrected records are resubmitted through the same validation process

Performance Optimization: Full validation on bulk imports can impact performance. Implement these optimizations:

1. Batch Size Optimization:

Reduce batch size from 1000+ to 250-500 records per API call

Smaller batches complete faster and provide better error isolation

If one batch fails validation, you don’t lose the entire import

2. Asynchronous Validation Option: For very large imports where performance is critical, use the async validation framework:

Set validateAsync=true in import options

The import completes immediately, validation runs in background

Invalid records are flagged in a validation report

Use this for initial data loads, not for ongoing operational imports

3. Validation Rule Optimization:

Territory overlap detection is expensive - consider running it as a separate post-import job

Cache postal code validation data to reduce lookup overhead

Use indexed fields in validation rule criteria

Territory-Specific Validation Considerations: Territory management has unique validation requirements:

1. Overlap Detection:

This validation can timeout on large imports

Configure it as a separate validation phase

Run overlap detection as a scheduled job after bulk import completes

2. Postal Code Validation:

Ensure your postal code reference data is up to date

Configure validation to use cached postal code lookups

Set appropriate timeout values (default 30 seconds may be too low)

3. Hierarchy Validation:

Parent territory must exist before child territories are imported

Sort your import data by hierarchy level

Import parent territories first, then children

Verification and Monitoring: After implementing validation:

1. Test with a small batch (50 records) including known invalid data

2. Verify that validation errors are properly returned and logged

3. Confirm that invalid records are rejected and don’t appear in the territory table

4. Monitor API response times to ensure validation doesn’t cause unacceptable delays

5. Set up alerts for validation failure rates exceeding threshold (e.g., >10% failure rate)

Recommended Implementation Approach:

1. Enable validation parameters in your API calls immediately

2. Update execution contexts for all territory validation rules

3. Implement error handling and quarantine workflow

4. Test thoroughly with sample data before processing production imports

5. Monitor performance and adjust batch sizes as needed

The combination of proper API parameters, correct execution context configuration, and robust error handling will ensure your data quality rules are enforced consistently across both UI and bulk import operations.