We deployed a firmware update to our fleet of edge devices last week, and since then the health metrics in our monitoring dashboard have stopped updating. The devices appear online in IoT Core, and we can see MQTT connection events, but the dashboard shows stale data from before the update.
The firmware update changed some of our MQTT topic structures to optimize payload size, and I’m wondering if this is affecting the IoT Rules Engine routing. We’re also seeing inconsistencies in device shadow synchronization - some devices report correctly while others don’t.
Has anyone experienced similar issues after firmware updates? Could the topic structure changes be causing the rules engine to drop messages, or is this more likely a device shadow sync problem?
First thing to check is whether your IoT Rules Engine rules are still matching the new MQTT topic patterns. If you changed the topic structure, the SQL queries in your rules might not be capturing the messages anymore. Go to IoT Core console and check the rule metrics to see if message processing counts have dropped to zero.
This is a common issue after firmware updates. Beyond the rules engine, you should also verify that your device shadows are configured to accept the new reported state structure. If the firmware changed the payload schema, the shadow service might be rejecting updates. Check CloudWatch Logs for shadow update errors - look for rejected or invalid state messages.
Good call - I checked the rules and you’re right, the message processing count dropped significantly after the firmware deployment. Our old topic pattern was device/{deviceId}/health but the new firmware uses device/{deviceId}/metrics/health. The SQL query was using a wildcard but apparently not matching correctly.