Bulk device registry import fails with 'invalid schema' error code

IOT IBM Watson IoT
, , , , , , , ,
1

Attempting to bulk import 2,400 devices using CSV file through the early access Bulk Import API. Import consistently fails with schema validation error:


POST /api/v0002/bulk/devices/import
Error: SCHEMA_VALIDATION_FAILED
Field: customAttributes.location.coordinates
Reason: Invalid nested object structure

Our CSV includes custom attributes for device location tracking. Need clarification on device registry schema requirements, proper CSV format for nested attributes, and how to define custom attribute structures. Documentation for early access features is limited.

2

Let me provide a complete solution covering device registry schema, CSV format, and custom attribute definition:

Device Registry Schema: Watson IoT Platform requires explicit schema definition before bulk import:

  1. Define Device Type Schema: Navigate to Device Types → [Your Type] → Schema and define the structure:
{
  "customAttributes": {
    "type": "object",
    "properties": {
      "location": {
        "type": "object",
        "properties": {
          "lat": {"type": "number", "minimum": -90, "maximum": 90},
          "lng": {"type": "number", "minimum": -180, "maximum": 180},
          "altitude": {"type": "number"},
          "accuracy": {"type": "number"}
        },
        "required": ["lat", "lng"]
      },
      "deploymentDate": {"type": "string", "format": "date-time"},
      "firmwareVersion": {"type": "string", "pattern": "^\\d+\\.\\d+\\.\\d+$"}
    }
  }
}
  1. Schema Validation Rules:
  • All custom attributes must be pre-defined in device type schema
  • Nested objects require explicit property definitions
  • Use JSON Schema validation syntax for data types and constraints
  • Required fields must be marked in schema
  • Pattern validation available for string types
  1. Schema Update Process: If modifying existing device type:
  • Schema changes are versioned automatically
  • Existing devices aren’t retroactively validated
  • New imports validate against current schema version
  • Test schema changes with small batch before bulk import

Bulk Import CSV Format: Proper CSV structure for nested attributes:

  1. Header Row Format:
deviceId,deviceType,authToken,deviceInfo.description,customAttributes.location.lat,customAttributes.location.lng,customAttributes.location.altitude,customAttributes.deploymentDate,customAttributes.firmwareVersion
  1. Data Row Example:
DEV-2401,AssetTracker,auto-generated,"Warehouse sensor, Zone A",40.7128,-74.0060,15.5,2025-07-01T10:00:00Z,1.2.3
  1. Formatting Rules:
  • Use dot notation for nested objects: parent.child.grandchild
  • Enclose string values with commas/quotes in double quotes
  • Use ISO 8601 format for date-time values
  • Boolean values: true/false (lowercase)
  • Null values: leave cell empty or use explicit null
  • UTF-8 encoding without BOM
  • Unix line endings (LF) preferred
  1. Special Cases:
# Array values (comma-separated in quotes)
customAttributes.tags,"sensor,warehouse,zone-a"

# Escaped quotes within strings
deviceInfo.description,"Device ""Alpha"" - Zone A"

# Optional fields (leave empty)
customAttributes.location.altitude,

Custom Attribute Definition: Best practices for defining custom attributes:

  1. Supported Data Types:
  • string: Text values (use pattern for validation)
  • number: Floating point (use minimum/maximum for ranges)
  • integer: Whole numbers
  • boolean: true/false
  • object: Nested structure
  • array: List of values (define item type)
  • date-time: ISO 8601 timestamps
  1. Validation Constraints:
{
  "firmwareVersion": {
    "type": "string",
    "pattern": "^[0-9]+\\.[0-9]+\\.[0-9]+$",
    "description": "Semantic version format"
  },
  "signalStrength": {
    "type": "integer",
    "minimum": -120,
    "maximum": 0,
    "description": "RSSI in dBm"
  },
  "tags": {
    "type": "array",
    "items": {"type": "string"},
    "maxItems": 10,
    "uniqueItems": true
  }
}
  1. Naming Conventions:
  • Use camelCase for attribute names
  • Avoid special characters except underscore
  • Keep names descriptive but concise
  • Group related attributes under parent objects
  • Don’t exceed 5 nesting levels
  1. Schema Design Tips:
  • Plan for future extensibility
  • Use required fields sparingly (allows partial updates)
  • Include description fields for documentation
  • Define sensible default values where applicable
  • Consider index requirements for searchable fields

For your specific import issue:

  1. Update Device Type Schema:
PUT /api/v0002/device/types/AssetTracker
{
  "customAttributes": {
    "type": "object",
    "properties": {
      "location": {
        "type": "object",
        "properties": {
          "lat": {"type": "number"},
          "lng": {"type": "number"}
        }
      }
    }
  }
}
  1. Format CSV Correctly:
deviceId,deviceType,customAttributes.location.lat,customAttributes.location.lng
DEV-0001,AssetTracker,40.7128,-74.0060
DEV-0002,AssetTracker,34.0522,-118.2437
  1. Validate Before Import: Test with a 10-device subset first:
curl -X POST https://platform.ibm.com/api/v0002/bulk/devices/import \
  -H "Authorization: Bearer $TOKEN" \
  -F "file=@test_devices.csv"
  1. Monitor Import Progress: The API returns a job ID - poll for completion:
GET /api/v0002/bulk/devices/import/{jobId}

Check for detailed error messages if validation fails. The response includes the specific row and field causing issues.

3

One more thing - make sure your CSV encoding is UTF-8 and that you’re not mixing quote styles. The bulk import API is strict about CSV formatting. Use double quotes for all string values, escape any quotes within values, and ensure there are no trailing commas or extra whitespace in headers. We’ve seen schema validation errors that were actually CSV parsing issues in disguise.

4

The issue is likely that you defined location as an object type in the schema but didn’t specify the nested properties. When defining custom attributes, you need to explicitly declare the structure including data types for all nested fields. For coordinates, define lat and lng as float/double types with appropriate validation ranges (-90 to 90 for lat, -180 to 180 for lng).

5

Thanks - I added the custom attributes to the device type schema, but still getting validation errors. My CSV headers look like: deviceId,deviceType,customAttributes.location.lat,customAttributes.location.lng. Is there a specific data type format required for coordinates? Using decimal degrees like 40.7128,-74.0060.

6

CSV format doesn’t support nested objects directly. You need to flatten your custom attributes using dot notation in the CSV headers. Instead of nested location.coordinates, use separate columns like customAttributes.location.lat and customAttributes.location.lng. Each nested level becomes a dot-separated header name.

7

Also check that you’ve pre-defined your custom attributes in the device type schema before importing. The bulk import API validates against the device type’s schema definition. If customAttributes.location isn’t defined in your device type, the import will fail even with correct CSV format. Go to Device Types → Your Type → Schema and add the custom attributes first.