After upgrading to sapiot-25, our billing engine stopped processing device telemetry from MQTT topics. The issue appears related to missing correlation IDs in the message payload. We’re seeing billing calculation errors for approximately 2,300 connected devices.
The MQTT topic structure hasn’t changed (devices/+/telemetry/usage), but messages now fail with “correlation_id_missing” errors in the billing engine logs. The billing configuration was working perfectly in sapiot-23 before the upgrade.
Error: Billing calculation failed
correlation_id: null
topic: devices/sensor_001/telemetry/usage
payload_size: 256 bytes
Has anyone encountered similar MQTT topic mapping issues after upgrading? Our billing cycles are blocked and we need to resolve this urgently.
Found the migration guide section. Our devices send a transaction_id field but not correlation_id. Can I map transaction_id to correlation_id in the adapter configuration? Also concerned about the 2,300 devices already in the system - will this mapping apply retroactively or only to new messages?
The correlation ID configuration is now in the billing engine’s MQTT adapter settings. You need to specify the JSON path where the correlation ID should be extracted from your message payload. If your devices don’t send correlation IDs, you can configure the adapter to generate them based on device ID and timestamp. This is documented in the sapiot-25 migration guide section 4.7.
Thanks for the quick response. I checked the topic mapping but I’m not seeing where to configure the correlation ID explicitly. Is this a new parameter in the billing engine configuration file, or is it handled through the UI? We have hundreds of device types so I need to understand if this requires individual mapping per device type or can be set globally.
I went through this exact scenario last month. Here’s the complete solution covering all three critical aspects:
MQTT Topic Mapping Configuration:
Navigate to IoT Cockpit → Device Management → MQTT Adapter Settings. Add a new field mapping rule:
source_field: "transaction_id"
target_field: "correlation_id"
mapping_type: "direct"
Billing Engine Configuration:
Update your billing engine configuration file (billing-engine-config.xml) to recognize the correlation ID from MQTT metadata:
<mqtt-billing-adapter>
<correlation-id-source>message.metadata.correlation_id</correlation-id-source>
<fallback-generation>device_id + timestamp</fallback-generation>
</mqtt-billing-adapter>
Handling Existing Failed Transactions:
For the blocked billing cycles, use the Billing Reconciliation API to reprocess failed transactions. The API will apply the new mapping rules retroactively:
POST /billing/reconciliation/v1/reprocess
{
"date_range": "2025-03-15 to 2025-03-20",
"apply_current_mapping": true,
"device_filter": "all"
}
The key change in sapiot-25 is that correlation IDs became mandatory for billing calculations to support the new distributed billing architecture. The fallback-generation setting ensures that even if a device doesn’t send transaction_id, the system will auto-generate a correlation ID using device_id + timestamp.
After applying these changes, restart the billing engine service and monitor the logs. You should see successful correlation ID extraction and billing calculations resuming. The reconciliation API typically processes about 500 transactions per minute, so your 2,300 devices should be cleared within 5-10 minutes.
One additional recommendation: update your device firmware to explicitly include correlation_id in future messages to avoid relying on the fallback mechanism, which can impact billing performance at scale.