Bulk import jobs using Snowflake 8.0 data preparation API are failing with date format errors. Small test imports work fine, but production datasets with legacy date formats consistently fail.
POST /api/v1/data-prep/import/bulk
{
"sourceFile": "legacy_orders.csv",
"mapping": {"order_date": "date"},
"recordCount": 150000
}
// Error: Invalid date format in row 1247: "31-Dec-2023"
// Expected format: "YYYY-MM-DD"
Our legacy data has dates in DD-Mon-YYYY format. Is there date format validation we can configure? How do we handle legacy data preprocessing for bulk imports? Need better bulk import error handling too - one bad row fails the entire 150K record import.
The partial import mode helps, but I’m still losing data. Is there a way to preprocess the problematic rows and retry them automatically, rather than just skipping them?
Set up a two-pass import strategy. First pass with strict validation captures clean records. Second pass processes the error file with relaxed validation or custom transformation rules. You can automate this using the data prep workflow API to chain preprocessing transformations with bulk imports.
For mixed formats, you need legacy data preprocessing. The API can’t auto-detect multiple formats in one pass. Use the data preparation transformation API to normalize dates before bulk import. Create a preprocessing job that applies format detection and conversion, then feed the cleaned data to the bulk import API.