When exporting large datasets from our ERP system to analytics storage using the OData connector in Power Platform Integration Hub, the export times out after processing about 50,000 records. Smaller exports work fine, but our monthly financial reports need to pull 200,000+ records.
Error from export log:
OData Export Failed
Endpoint: /api/v2/FinancialTransactions
Records processed: 52,341 of 218,467
Error: Request timeout (300s exceeded)
HTTP 504: Gateway Timeout
The OData connector is configured with a 5-minute timeout, and the batch size is set to 1,000 records per request. I’ve tried increasing the timeout to 10 minutes, but that just delays the failure. The ERP system logs show the queries are executing successfully on their end, but the data transfer seems to be the bottleneck. This is creating gaps in our analytics reporting because we can’t get the complete dataset. Has anyone successfully configured OData server-side paging or batch export for large dataset transfers in Integration Hub?
The hang on a specific page suggests there might be a data issue in that particular range - possibly a record with corrupted data or a very large field that’s causing the timeout. Instead of increasing timeouts, try implementing incremental export based on date ranges. Split your 200K records into daily or weekly batches and run separate exports for each batch. You can use Power Automate to orchestrate multiple export jobs in sequence. This also gives you better error handling - if one batch fails, you don’t lose the entire export. The batch export configuration in Integration Hub supports this pattern with date filters.
The timeout is happening because you’re trying to pull too much data in a single request. OData connectors support server-side paging, but you need to configure it explicitly in the connector settings. Look for the “Enable Server-Side Paging” option and set a page size that matches your ERP system’s capabilities. Usually 5,000-10,000 records per page works well. The connector will then automatically handle the paging and combine the results.
I found the server-side paging option and enabled it with a page size of 5,000 records. The export got further (about 85,000 records) but still timed out. The Integration Hub logs show multiple successful page requests, but then it hangs on one particular page and eventually times out. Is there a way to configure retry logic or skip problematic pages and continue?