We have an automated demand planning workflow that’s supposed to trigger whenever sales forecast files are uploaded to our SFTP directory. The workflow worked fine for the first month after implementation, but now it’s failing to start automatically, causing us to miss several demand planning cycles.
Our sales team uploads CSV files with forecast data to /incoming/demand-forecasts/ every Monday morning. The SFTP integration event mapping should detect these files and trigger the demand planning workflow to import and process them. However, the workflow isn’t starting - we have to manually trigger it through the UI, which defeats the purpose of automation.
When I check the event bus logs, I don’t see any events being published for the file uploads. The files are definitely landing in the SFTP directory (I can see them there), but it’s like the integration isn’t detecting them. The workflow trigger configuration looks correct to me, but clearly something is misconfigured.
Has anyone experienced issues with SFTP file upload events not triggering workflows properly?
Good suggestions. I checked and the SFTP file watcher service is running. Permissions look okay too. However, I noticed something in the event mapping config - there’s a “file age threshold” setting of 300 seconds. Could this be causing the issue? The files are being processed manually within minutes of upload, so maybe the automation is checking for files before they’re considered “ready”?
Look at your event mapping configuration carefully. The file pattern matching might be too restrictive or incorrect. If your sales team is uploading files with names like “forecast_2024_12_03.csv” but your event mapping is looking for “demand_forecast_*.csv”, the events won’t match. Also check if there’s a file size threshold configured - sometimes small test files are ignored by the integration to prevent spurious triggers. Review the exact file naming convention and make sure it matches your event filter pattern.
Based on our discussion, here’s the complete solution addressing all three focus areas:
SFTP Integration Event Mapping:
The core issue is likely a mismatch between the event published by the SFTP integration and the event your workflow is subscribed to. Verify your event mapping configuration:
- Check what event type the SFTP file watcher publishes - it’s probably a generic “file.uploaded” event
- Check what event type your demand planning workflow trigger is subscribed to - it might be looking for “demand.forecast.received”
- If there’s a mismatch, you need to configure event transformation
In your SFTP integration configuration, add an event mapping rule:
- Source Event: file.uploaded
- Filter: file.path matches “/incoming/demand-forecasts/*” AND file.extension equals “csv”
- Target Event: demand.forecast.received
- Event Payload: Include file metadata (path, size, upload timestamp)
This transforms the generic file upload event into the specific event your workflow expects, but only for files in the demand forecasts directory.
Workflow Trigger Misconfiguration:
Review your workflow trigger configuration in the demand planning workflow definition:
- Verify the workflow trigger type is set to “Event-Based” not “Manual” or “Scheduled”
- Confirm the event subscription includes “demand.forecast.received” (or whatever specific event type you’re using)
- Check if there are additional trigger conditions that might be preventing activation - for example, time-of-day restrictions or conditional logic that’s evaluating to false
- Ensure the trigger is enabled (not disabled) - sometimes triggers get disabled during testing and aren’t re-enabled
Test the trigger by publishing a test event manually through the event bus console to verify the workflow responds correctly when the event is properly formatted.
Missed Demand Planning Cycles:
To recover from missed cycles and prevent future occurrences:
-
Implement a fallback mechanism: Configure a scheduled job that runs every Monday at noon to check for unprocessed files in the SFTP directory. If files exist that haven’t been processed (check by comparing file timestamps against workflow execution history), trigger the workflow manually via API
-
Add monitoring and alerting: Set up alerts when files are uploaded to the SFTP directory but no corresponding workflow execution occurs within 15 minutes. This provides early warning before you miss a planning cycle
-
For the immediate backlog: Manually trigger the workflow for each missed file, processing them in chronological order. Check the workflow execution history to identify which Monday uploads were missed
-
Consider implementing idempotency in your workflow so reprocessing the same file doesn’t create duplicate demand forecasts
The file age threshold of 300 seconds you mentioned is fine - it prevents processing incomplete uploads. However, verify that the file watcher is actually checking the directory every 5 minutes. There should be a polling interval configuration. If it’s set to check every hour, that would explain delays in detection even after the age threshold is met.
Another possibility: check if the event bus itself is healthy and processing messages. Use the event bus monitoring dashboard to see if events are being published and consumed. If the event bus is experiencing backlogs or has crashed consumers, events might be published but never delivered to the workflow engine. Look for dead letter queues with unprocessed events.
The file age threshold is important for ensuring files are completely uploaded before processing starts, but 300 seconds (5 minutes) shouldn’t cause the workflow to never trigger. The issue is more likely in the event subscription configuration. Check if your demand planning workflow is actually subscribed to the SFTP file upload event type. In MASC 2023, workflow triggers need to explicitly subscribe to event types. It’s possible the subscription was removed or never properly configured. Look in the workflow trigger configuration for the event subscription list.
I’ve seen this exact issue before. The problem is often that the SFTP integration publishes a generic file upload event, but the workflow is subscribed to a more specific event type like “demand forecast file uploaded”. There’s a mismatch between what the integration publishes and what the workflow expects. You need event transformation or routing logic to map the generic SFTP event to the specific event type your workflow is listening for.