Test case execution results missing from release management reports

ALM IBM Engineering Lifecycle Management
, , , , , , ,
1

Our release management reports are not showing test execution results from the test case management module. The release dashboard displays requirements and defects correctly, but the test execution section shows zero results even though we have 340 test cases executed in the current release cycle.

The burndown charts for testing progress are completely flat, and the quality metrics widget shows “No data available.” When I navigate directly to the test case management module, all execution results are visible and properly linked to release milestones.


Query: Test Execution Results for Release 2.5
Status: 0 results returned
Expected: 340 executed test cases

This is blocking our release reporting to stakeholders as we can’t demonstrate testing progress. The query aggregation seems to be failing between test-case-mgmt and release-mgmt modules.

2

This sounds like a cross-project query permission issue. Even if test cases are linked correctly, the release management module needs proper read permissions on the test execution artifacts. Check your project associations and make sure the release management project area has consumer access to the test case management project. Without this, the aggregation queries will silently fail and return zero results.

3

Check if your test execution records are properly tagged with the release milestone identifier. The release management query aggregation relies on consistent tagging across modules. If test cases are executed but not explicitly linked to the release milestone, they won’t appear in release-level reports even though they’re visible in the test module.

4

I’ll provide a comprehensive solution covering test execution visibility, release reports, query aggregation, and burndown chart issues:

Root Cause Analysis: The problem stems from multiple configuration gaps in how test execution results are aggregated into release management reports. Here’s the systematic fix:

1. Verify Project Area Associations First, ensure proper linking between test and release modules:


Project Associations:
Release Mgmt -> Test Case Mgmt (Consumer)
Permission: Read Test Execution Results

2. Update Release Dashboard Query Scope Modify the dashboard query configuration to include test execution artifacts:

  • Open Release Dashboard > Configuration > Data Sources
  • Edit “Test Execution Results” query
  • Add artifact types: Test Case, Test Execution Record, Test Result
  • Set scope to include linked test project areas
  • Enable “Follow cross-project links”

3. Configure Query Aggregation Rules The release report aggregation needs explicit rules for test data:


aggregation.source=test-case-mgmt
aggregation.artifactType=TestExecutionRecord
aggregation.linkType=relatedTo.release

4. Reindex Test Execution Artifacts Test results may not be indexed for cross-module queries. Run reindexing:

  • Navigate to Test Case Management > Advanced > Reindex
  • Select “Test Execution Records” artifact type
  • Choose scope: All executions for Release 2.5
  • Start reindex operation (typically takes 15-30 minutes for 340 test cases)

5. Rebuild Burndown Chart Configuration The flat burndown charts indicate missing data source configuration:

  • Release Dashboard > Burndown Widget > Settings
  • Data Source: Select “Test Execution Results (Aggregated)”
  • Metrics: Executed, Passed, Failed, Blocked
  • Time Range: Release start date to end date
  • Refresh Interval: Daily at 02:00

6. Update Quality Metrics Widget Configure the quality metrics to pull from correct data sources:

  • Widget Settings > Data Sources
  • Enable: Test Case Management (cross-project)
  • Metrics: Pass rate, execution coverage, defect density
  • Link to release milestone: Release 2.5

7. Validate Data Flow After configuration changes, test the data flow:

  • Execute a new test case in test-case-mgmt
  • Wait 5 minutes for indexing
  • Check if it appears in release dashboard query results
  • Verify burndown chart updates

Expected Results:

  • All 340 test execution results visible in release reports within 1-2 hours
  • Burndown charts show accurate testing progress
  • Quality metrics widget displays pass rates and coverage
  • Cross-module query aggregation working correctly

Additional Troubleshooting: If issues persist after reindexing, check the release management server logs for query aggregation errors. Look for entries related to “cross-project query” or “test execution aggregation” - these will indicate permission or configuration issues that need additional attention.

This solution addresses all aspects: test execution visibility, query aggregation configuration, burndown chart data sources, and release reporting integration.

5

I had exactly this problem last month. The issue was that test execution results weren’t being indexed properly for cross-module queries. The test module’s local queries worked fine, but the release dashboard couldn’t find the execution records through its aggregation queries. We had to run a reindex operation on the test execution artifacts to make them visible to release-level reports. After reindexing, the burndown charts populated within an hour.

6

We encountered this in ELM 7.0.1 and it was related to query scope configuration. The release management dashboard uses predefined queries that might not be configured to traverse test execution links properly. Go to your release dashboard configuration and verify the query definitions include test execution artifacts. You may need to update the query scope to include the test-case-mgmt project area explicitly.

7

I verified the milestone tagging and all test cases are correctly linked to Release 2.5. The test module shows the proper associations. Nina, where exactly do I find the query scope configuration for the release dashboard? Is this in the dashboard widget settings or somewhere in the project configuration?