Bulk device registration via API fails with 'Invalid Payload' error on CSV import

IoT PTC ThingWorx
, , , , , , , ,
1

When using the device registry API to bulk import 200+ sensors, I’m getting ‘Invalid Payload’ errors on about 30% of the devices. The CSV file has standard columns: deviceName, deviceType, serialNumber, location. Some devices import successfully while others fail with no specific error details.

I’ve checked the CSV encoding (UTF-8), and the required columns match the API documentation. The strange part is that manually creating the failed devices through the Composer UI works fine with the exact same data.


POST /Thingworx/DeviceRegistry/Services/BulkImportDevices
Content-Type: multipart/form-data
File: device_import.csv (UTF-8)

The API response just says ‘Invalid Payload’ with no indication of which field or row is problematic. This is delaying our device provisioning rollout significantly. Anyone dealt with CSV validation issues in bulk device imports?

2

The debug logs helped! Found that some serialNumber values had leading zeros that were being dropped during CSV parsing (Excel was converting them to numbers). Also discovered that deviceType values are case-sensitive in the API but not in the UI - I had ‘TemperatureSensor’ in CSV but the template is ‘temperatureSensor’. Still seeing a few failures though, and the logs show ‘Missing required field: thingTemplate’ for some rows even though deviceType is present.

3

Check for special characters in your CSV data, especially in the location field. Commas, quotes, or line breaks within field values can break CSV parsing even if the file is UTF-8. Also verify there are no extra spaces in the header row column names - ‘deviceName’ is not the same as 'deviceName ’ with trailing space.

4

Here’s the complete solution for bulk device CSV imports:

1. CSV Encoding and Format:

  • Use UTF-8 encoding (save as UTF-8 in Excel or text editor)
  • Ensure consistent line endings (LF or CRLF, not mixed)
  • No blank rows between data or at the end of file
  • No trailing commas on any line

2. Required Column Headers (case-sensitive, exact spelling):


deviceName,deviceType,thingTemplate,serialNumber,location

Note: The API requires BOTH deviceType and thingTemplate even though they’re related. Set thingTemplate to the exact template name.

3. Data Validation Rules:

  • deviceName: Must be unique, no special characters except underscore and hyphen
  • deviceType: Case-sensitive, must match existing type exactly
  • thingTemplate: Case-sensitive, must match existing template name
  • serialNumber: Wrap in quotes if it has leading zeros: “001234”
  • location: Wrap in double quotes if contains commas: “Building A, Floor 2”

4. Enable API Debug Logging: Add to ApplicationLog before import:


log.setLevel('DeviceRegistry', 'DEBUG');
log.setLevel('CSVParser', 'TRACE');

This shows line-by-line validation errors in application.log.

5. Common Gotchas:

  • Excel auto-formats numbers (leading zeros dropped) - save as CSV UTF-8, not Excel CSV
  • Header column names with spaces or typos cause ‘Invalid Payload’
  • deviceType vs thingTemplate confusion - API needs both columns
  • Special characters in location field must be quoted
  • Case sensitivity differs between API and UI - API is strict

6. Validation Before Import: Test with a small 5-row CSV first to verify schema. Once that succeeds, scale to full dataset. If failures persist, export a working device from the UI using the export API and use that CSV structure as your template.

7. API Headers: Ensure proper content type:


Content-Type: multipart/form-data; boundary=---boundary
appKey: your-key

After fixing these issues, your bulk import should process cleanly. The key is matching the API’s strict validation rules which are different from the more forgiving UI validation.

5

Good catch on the special characters. I found a few location values with commas like ‘Building A, Floor 2’ that weren’t properly quoted. I wrapped those in double quotes in the CSV, but still getting failures on about 15% of devices now. The header row looks clean with no extra spaces. Is there a way to enable detailed API logging to see exactly which field is causing validation to fail?

6

Enable debug logging for the DeviceRegistry subsystem. Add this to your ApplicationLog configuration:


log.setLevel('DeviceRegistry', 'DEBUG');
log.setLevel('CSVParser', 'TRACE');

Then check application.log after running the import - you’ll see line-by-line parsing details and which validation rule is failing. Also verify your CSV doesn’t have any blank rows or trailing commas at the end of lines. The API is stricter than the UI about data cleanliness.