Portfolio reporting dashboard in cloud shows incomplete data

PLM ENOVIA
, , , , , , , ,
1

Our portfolio management dashboard in ENOVIA Cloud R2022x displays incomplete data after our recent cloud migration. The dashboard shows only about 60% of our projects and programs, with some data appearing days late.

We’ve confirmed all data exists in the source system, but the reporting layer seems to have synchronization issues. The ETL jobs appear to run on schedule according to the job monitor, but data mapping might not be working correctly. We need to understand if this is an ETL job log issue, problems with data mapping templates, or a sync validation failure. Our executive team relies on these dashboards for decision-making, so this is becoming critical.

system 2

I’ll systematically address all three critical areas:

ETL Job Log Analysis: Access the detailed ETL logs (Admin Console > Data Integration > ETL Job History). For each scheduled job, examine:

  1. Execution Details: Look beyond ‘Completed’ status - check ‘Records Extracted’, ‘Records Transformed’, ‘Records Loaded’ counts
  2. Error Summary: Even successful jobs can have partial failures. Check the ‘Warnings’ and ‘Skipped Records’ sections
  3. Performance Metrics: If extraction time is consistently hitting timeout thresholds (default 30 minutes), jobs are terminating early with incomplete data
  4. Specific to your 60% data issue: Compare the ‘Records Extracted’ count against your expected portfolio object count in the source system

In our similar case, we found ETL jobs were timing out during the extraction phase due to inefficient queries against large portfolio datasets. The jobs completed but with partial data extraction.

Data Mapping Template Validation: Cloud R2022x introduced significant schema changes for portfolio analytics:

  1. Navigate to Admin Console > Analytics Configuration > Data Mapping Templates
  2. Export your current portfolio mapping template and review against the R2022x cloud reference template (available in documentation)
  3. Critical field mapping updates for R2022x:
    • Program status fields: Old path ‘program.status’ → New path ‘program.lifecycle.currentState’
    • Project dates: ‘project.startDate’ → ‘project.schedule.plannedStart’
    • Resource allocation: Structure changed from flat to nested ‘project.resources.allocation’
  4. Update templates to use the new schema paths
  5. Validate templates using the built-in validator (Data Mapping > Validate Template button) - this catches 80% of mapping errors

Sync Validation Configuration: Implement comprehensive validation checks:

  1. Review sync validation rules (Admin Console > Data Integration > Sync Validation Rules)
  2. Enable detailed validation logging: Set ‘etl.validation.logging.level’ to ‘DEBUG’ in cloud configuration
  3. Common validation failures in portfolio data:
    • Missing required fields (program owner, project manager)
    • Invalid date ranges (end date before start date)
    • Orphaned references (projects linked to deleted programs)
  4. Run the Data Quality Report (Analytics > Data Quality Dashboard) to identify which specific records are failing validation
  5. Configure validation error handling: Change ‘validation.failure.action’ from ‘SKIP’ to ‘RETRY_WITH_DEFAULTS’ for non-critical fields

Immediate Action Plan:

  1. Run a full refresh ETL job manually (not incremental) to establish baseline
  2. Compare record counts between source system and analytics database
  3. Export ETL error log for the last 7 days and categorize errors
  4. Update data mapping templates to R2022x cloud schema
  5. Adjust ETL job timeout from 30 to 60 minutes for large portfolio datasets
  6. Enable incremental sync change tracking verification

After implementing these changes, monitor the next 3 scheduled ETL runs. You should see ‘Records Loaded’ count match ‘Records Extracted’ count, and dashboard data should be complete within 24 hours. The multi-day delay you’re experiencing will resolve once validation rules properly handle R2022x data structure changes.

This draft is based on general ENOVIA knowledge. It has not been verified against your specific version and environment. Practitioners: verify the steps and share your experience below.

3

First thing to check is the ETL job completion status versus success status. Jobs can complete without successfully processing all data. Look at the ETL job details page and check the ‘Records Processed’ versus ‘Records Expected’ counts. If there’s a mismatch, you’ll see which specific data sets are failing to sync.

4

This sounds like a data mapping template issue. Cloud R2022x introduced changes to how portfolio objects are structured in the analytics database. If you migrated templates from an older version or on-premise setup, field mappings might be pointing to deprecated attributes. Check if your templates reference the new cloud schema - specifically portfolio.programStatus changed to portfolio.lifecycleState in R2022x.

5

The delay you’re seeing suggests the sync validation is failing for certain records, causing them to be queued for retry. Check the ETL error logs for validation failures. In our environment, we discovered that projects with special characters in names or missing required metadata fields were being rejected during validation. Once we cleaned up the source data and updated validation rules, sync improved to 98%+.

6

Have you verified the incremental sync settings? If the ETL is configured for incremental updates but the change tracking isn’t working properly, new or modified records won’t be picked up. Try running a full refresh manually to see if all data appears. If it does, the issue is definitely in the incremental sync configuration, not the data mapping.

gerdm 7

“Tested this on ENOVIA cloud tenant 2023x FD04 — checking ‘Skipped Records’ in ETL Job History revealed hundreds of silently dropped portfolio objects causing our dashboard gaps.”

9

Check your data source connections in the ETL configuration. Cloud deployments sometimes have network latency or connection pool issues that cause partial data extraction. If the ETL job times out during data extraction, it might complete with partial results without throwing an error.

10

Look at the dashboard filters and data security settings. Sometimes data appears incomplete because portfolio visibility rules in cloud are more restrictive than on-premise. Users might not have proper access roles to see all projects in the analytics layer even though they can see them in the operational system.

lauraplanner 11

If the ETL is configured for incremental updates but the change tracking isn’t working properly, new or modified records won’t be picked up.