We’re experiencing timeout issues with our Cloud Functions that process batch API calls from our ERP system for tax calculations. The function handles invoice batches (typically 50-100 invoices per request) and needs to calculate taxes for each line item.
Our current timeout is set to 60 seconds, but we’re seeing failures when batch size exceeds 40 invoices. The tax calculation API call looks like this:
const response = await axios.post(TAX_API_URL, {
invoices: batchData,
calculateDetails: true
});
The ERP workflow expects results within 90 seconds to avoid blocking the invoice approval queue. We’ve tried increasing the timeout to 120s, but that doesn’t solve the underlying batch processing inefficiency. Has anyone dealt with similar Cloud Functions timeout challenges when handling batch API operations? We need to optimize both the timeout configuration and the batch processing approach to keep our invoice workflow running smoothly.
Thanks for the insights. We profiled the API calls and each invoice averages 800ms for tax calculation. So 50 invoices × 800ms = 40 seconds minimum, plus overhead. The parallel processing approach sounds promising. Should we implement this within a single Cloud Function call, or split it into multiple function invocations? Our concern is maintaining transaction consistency for the batch.
I’ve seen this pattern before. Your 60-second timeout is hitting the wall because you’re processing sequentially. Have you considered breaking down the batch into smaller chunks and processing them in parallel? Cloud Functions can handle concurrent execution well. You might want to split your 50-100 invoice batch into groups of 10-15 and process them simultaneously. This would significantly reduce your overall processing time while staying within reasonable timeout limits.
For transaction consistency with batch processing, I’d recommend a hybrid approach. Keep the single Cloud Function entry point, but use Promise.all() to process chunks in parallel within that function. This maintains your transaction boundary while gaining concurrency benefits. You could also consider Cloud Tasks for more complex orchestration if batches grow larger. The key is balancing parallelism with your invoice workflow’s consistency requirements.