Rules engine fails to trigger actions on batch MQTT data ingestion from edge devices

IOT IBM Watson IoT
, , , , , , , ,
1

We’re experiencing issues with our rules engine in Watson IoT Platform wiot-25. Our edge devices send batch MQTT messages containing multiple sensor readings (temperature, humidity, pressure) in a single payload, but the rules engine isn’t triggering the configured actions consistently.

The batch payload structure looks like this:

{
  "readings": [
    {"sensor_id": "T001", "value": 85.2, "timestamp": 1710492180},
    {"sensor_id": "T002", "value": 91.7, "timestamp": 1710492185}
  ]
}

When we send single-message payloads, rules trigger fine. The rule condition script checks if temperature exceeds 90°F, but it seems to only evaluate the first reading in batch messages. We’ve verified MQTT ingestion is working - data appears in the device registry. Has anyone dealt with batch message processing in rules engine configuration?

2

Another approach is using the IoT Platform’s device event preprocessing feature. You can configure a transformation that extracts array elements before rule evaluation. This keeps everything within the platform without external Node-RED dependencies. Check the device type settings under Advanced Options - there’s a JSON transformation section specifically for handling complex payloads.

3

We’re using the visual condition editor currently. I didn’t realize it wouldn’t automatically iterate through array elements. That explains why only sporadic triggers occur - probably when the first element meets the condition. Do we need to switch to custom JavaScript for batch processing?

4

We had the exact same issue last quarter. The problem is the rules engine expects a flat message structure. For batch payloads, you need preprocessing. We solved it by adding a Node-RED flow between MQTT ingestion and the rules engine that splits batch messages into individual events. Each reading becomes a separate message that the rules engine can evaluate properly. Performance impact was minimal - added maybe 200ms latency.

5

I’ve worked extensively with batch MQTT ingestion in Watson IoT. Here’s the complete solution addressing all three critical areas:

1. Batch vs Single-Message Payload Handling: The rules engine processes messages at the document level, not array element level. Your batch structure requires either message splitting or custom evaluation logic. For your use case with multiple sensor readings, I recommend the preprocessing approach.

2. Rule Condition Scripting for Arrays: If you want to keep batch messages, switch to custom JavaScript conditions:

var triggered = false;
msg.readings.forEach(function(reading) {
  if (reading.value > 90) triggered = true;
});
return triggered;

This iterates through all readings and triggers if ANY exceed the threshold. Adjust logic if you need ALL readings to exceed the threshold (use && operator instead).

3. MQTT Ingestion Node Configuration: Verify your device type configuration:

  • Navigate to Device Types → [Your Type] → Advanced Settings
  • Enable “Message Transformation” under preprocessing
  • Add transformation rule: Split array field “readings” into individual events
  • Set the output path to preserve sensor_id and timestamp fields

Alternatively, configure your MQTT client to publish individual messages:

readings.forEach(r => {
  client.publish('iot-2/evt/status/fmt/json',
    JSON.stringify(r));
});

The preprocessing approach is cleaner for production - it maintains batch efficiency on the network while giving the rules engine individual events to evaluate. We’ve processed 50K+ batch messages daily this way with sub-second trigger latency. The key is ensuring the MQTT ingestion node correctly parses and transforms before rule evaluation begins.

6

The visual editor has limitations with array structures. You have two options: restructure your MQTT payload to send individual messages (less efficient but simpler), or implement custom JavaScript conditions. For batch processing, JavaScript gives you full control. You can iterate the readings array and evaluate each element. I’d also recommend checking your MQTT ingestion node configuration - make sure message parsing is set to preserve the JSON structure intact.

7

I’ve seen similar behavior. The rules engine in Watson IoT evaluates against the root message object by default. When you batch readings into an array, the condition script needs explicit iteration logic. Are you using the standard condition editor or custom JavaScript? The batch structure might require array parsing in your rule definition.