Auto-numbering fails for documents during bulk import, causing duplicate ID errors

QMS Veeva Vault QMS
, , , , , , , ,
1

I’m running into a serious issue with our document migration project. When importing documents via the bulk loader, the auto-numbering sequence is failing to generate unique document numbers. Instead of using the configured auto-numbering pattern, the import process seems to be attempting to use blank or duplicate values, which triggers validation errors.

Our auto-numbering sequence is configured as DOC-{YYYY}-{0000} and works perfectly when creating documents manually through the UI. However, during bulk import using CSV files with the document number field left empty (expecting auto-generation), we’re getting errors like:


Error: Duplicate document number detected
Value '' already exists for field 'number__v'

I’ve verified the CSV doesn’t contain any document numbers in that column, and the auto-numbering configuration is active. The bulk import process just doesn’t seem to be triggering the auto-number generation. Has anyone successfully imported documents while relying on auto-numbering in 23R3?

2

Auto-numbering doesn’t work the same way in bulk imports as it does in the UI. You need to explicitly tell the loader to use auto-numbering by including a specific flag in your CSV or API call. Check the Vault Loader documentation for the auto-number parameter syntax.

3

Let me provide a comprehensive solution that addresses all three critical aspects of your auto-numbering and bulk import challenge:

Understanding Auto-Numbering Sequence Behavior: The auto-numbering mechanism in Vault QMS is designed primarily for interactive document creation through the UI, not for bulk import operations. When you create a document via the UI with auto-numbering enabled, Vault automatically invokes the sequence generator. However, the bulk loader (CSV-based import) does not trigger this automatic generation when it encounters empty number fields - it treats them as null values, which violates the required field constraint and causes the duplicate error you’re seeing.

Bulk Import Process with Pre-Generated Numbers: The most reliable approach for large-scale document migrations is to pre-generate the document numbers in your CSV file before import. Here’s the step-by-step process:

  1. Access Admin > Configuration > Document Fields and locate your auto-numbering sequence configuration
  2. Note the current sequence value (e.g., if the last document was DOC-2024-0156, the next value is 0157)
  3. Generate sequential numbers for your import batch following the pattern: DOC-2024-0157, DOC-2024-0158, etc.
  4. Populate these numbers in your CSV file’s document number column
  5. Execute the bulk import with the pre-populated numbers
  6. After import completion, update the sequence counter to resume from your last imported number + 1

Duplicate ID Handling and Sequence Management: To prevent conflicts and ensure smooth operation post-migration:

# Example sequence management script
current_sequence = 156  # Current value before import
import_batch_size = 500  # Number of documents to import
next_sequence = current_sequence + import_batch_size + 1
# Update sequence to 657 after import completes

Critical considerations for duplicate prevention:

  • Perform imports during a maintenance window when UI document creation is disabled
  • Lock down document creation permissions temporarily to prevent concurrent creation
  • Validate your CSV for any duplicate numbers before import using a script or Excel formula
  • After import, immediately update the auto-number sequence to the next available value
  • Test with a small batch first (10-20 documents) to verify the number range doesn’t conflict

If you must allow concurrent UI access during migration, use the REST API approach instead of bulk loader. The API supports the ‘useAutoNumber=true’ parameter which properly invokes the sequence generator:

POST /api/v23.3/objects/documents
Content-Type: application/json
{"useAutoNumber": true, "type__v": "quality_document__c"}

However, this is significantly slower for large volumes. For your migration, I recommend the pre-generated number approach with a coordinated maintenance window to avoid any duplicate ID conflicts. Update your sequence immediately after import completion to ensure seamless transition back to normal operations.

4

Another approach is to use the API instead of CSV bulk loader. The REST API supports the auto-number trigger properly. However, for large migrations, this is slower than bulk loading. We typically do a hybrid approach - bulk load with pre-generated numbers, but coordinate the timing so no UI document creation happens during the migration window to avoid conflicts.

5

I encountered this exact issue last month during our migration. The problem is that when you leave the number field empty in the CSV, Vault doesn’t automatically invoke the auto-numbering sequence - it treats it as a null value. You have two options: either pre-generate the numbers before import using the sequence pattern, or use the Vault API with the ‘useAutoNumber’ parameter set to true. The bulk loader CSV approach doesn’t support auto-numbering trigger by default. We ended up pre-generating sequential numbers in our CSV file to avoid the issue entirely.

6

Thanks for clarifying. So if I understand correctly, the bulk import process won’t trigger auto-numbering even if the field is configured for it? That’s frustrating. For pre-generating numbers, how do I ensure they align with the existing sequence and don’t create gaps or conflicts with documents that might be created through the UI during the migration window?