Running Factorytalk MES 11.0 with performance-analysis module pulling data from multiple IoT streams (machine counters, downtime sensors, quality inspection systems). When we generate OEE reports that aggregate across these different data sources, we’re getting incorrect performance totals that don’t match reality.
For example, calculated availability shows 94% but manual verification indicates it should be 87%. The issue appears when aggregating IoT metric data with different sampling rates-some sensors report every second, others every 30 seconds, and quality checks are event-driven.
Here’s a sample aggregation query that’s producing incorrect results:
SELECT SUM(good_count)/SUM(total_count) as yield
FROM iot_metrics
WHERE timestamp BETWEEN @start AND @end
I suspect timestamp normalization or weighted averaging issues, but need guidance on proper data quality validation and aggregation logic verification for multi-rate IoT data sources.
The aggregation logic verification needs to account for the semantic differences between your data sources too. Machine counters are typically cumulative (total parts produced), while downtime sensors report state changes (running/stopped events), and quality data is discrete inspections. You can’t aggregate these the same way. Counters need delta calculations between samples, state changes need duration calculations, and quality needs count-based ratios.
Don’t interpolate-that introduces artificial data points that can skew your accuracy metrics. Instead, use time-weighted averaging where each measurement represents its valid time interval. For a 30-second sensor reading, that value is valid for 30 seconds. When calculating period totals, multiply each measurement by its interval duration before summing, then divide by total time period.
Good point about timestamp synchronization. I checked and we do have some clock drift-up to 2 minutes on older sensors. But even if I filter to data where timestamps should be aligned, the totals are still off. How should the weighted averaging actually work? Do I need to interpolate the lower-frequency data to match the high-frequency sampling rate?
Another issue to consider is data quality validation. Not all IoT data points are valid-sensors can report error codes, null values, or obviously incorrect readings (like 300% efficiency). Your aggregation logic should filter these out before calculation. In FT MES 11.0 performance-analysis module, there’s a data quality rules engine that can flag suspect readings based on configurable thresholds. Make sure that’s enabled and properly configured for each sensor type.