Bamboo build quality metrics not importing to Rally quality-mgmt module

ALM Rally (Broadcom Agile Central)
, , , , , , , ,
1

Our Bamboo plans complete successfully with test results, code coverage, and quality metrics, but Rally’s quality-mgmt module shows no data. We’ve configured the Bamboo Rally Tasks plugin to push metrics after each build.

The Bamboo logs indicate successful API calls to Rally with 200 responses. However, when we check Rally’s quality gates dashboard and metrics reports, all fields are empty. We’ve verified the metrics field mapping configuration in Bamboo matches Rally’s expected field names.

The plan key correlation seems correct, and we’ve tried manual cache refresh in Rally with no improvement. Has anyone resolved quality metrics import issues between Bamboo and Rally’s quality management system?

2

Plan key correlation in Rally SA is case-sensitive and must exactly match your Bamboo plan key format. If your Bamboo plan is “PROJ-BUILD” but Rally has “proj-build”, the metrics won’t correlate. Also verify the subscription config includes quality metric events. We had a similar issue where our subscription was only configured for build status updates, not metric data.

3

Rally’s quality-mgmt module requires metrics to be associated with specific test sets or build definitions, not just sent as standalone data. Check if your Bamboo plugin is creating the proper Rally test set reference before pushing metrics. The 200 response might just mean the API call succeeded, but Rally could be silently discarding metrics without a valid test set association.

4

This is a combination of field mapping, plan correlation, and subscription configuration issues. Here’s the comprehensive solution:

1. Metrics Field Mapping Correction: Rally SA’s quality-mgmt module uses specific field names that differ from standard Rally versions. Your current mapping uses generic names that Rally doesn’t recognize. Update your Bamboo Rally Tasks plugin configuration to use Rally SA’s exact field names:

<qualityMetrics>
  <metric source="coverage" target="CoveragePercentage" type="decimal"/>
  <metric source="tests_passed" target="PassedTestCount" type="integer"/>
  <metric source="tests_failed" target="FailedTestCount" type="integer"/>
  <metric source="defects" target="OpenDefectCount" type="integer"/>
  <metric source="build_duration" target="BuildDuration" type="integer"/>
</qualityMetrics>

Key corrections:

  • CodeCoverage → CoveragePercentage (Rally SA requires percentage suffix)
  • TestsPassed → PassedTestCount (Rally SA uses Count suffix for tallies)
  • DefectCount → OpenDefectCount (Rally SA distinguishes open vs. total defects)

Also specify the data type for each metric. Rally SA rejects metrics without explicit type declarations. Valid types are: integer, decimal, boolean, string.

The field names are case-sensitive and must match exactly. Even small variations like “TestCount” vs. “TestsCount” will cause silent failures.

2. Plan Key Correlation: Rally SA requires the plan key to be registered in the build definition before metrics can be imported. Verify your setup:

  1. In Rally, navigate to Setup > Build Definitions
  2. Find or create a build definition for your Bamboo plan
  3. Set the ExternalBuildID field to exactly match your Bamboo plan key: `PROJECT-QA
  4. Ensure the build definition is associated with the correct project and workspace

The correlation works both ways: Bamboo must send the plan key in the metrics payload, AND Rally must have a matching build definition. If the build definition is missing or the ExternalBuildID doesn’t match exactly (including case and hyphens), Rally will accept the API call but discard the metrics.

In your Bamboo Rally Tasks configuration, ensure the plan key is being sent in the payload:

<buildInfo>
  <planKey>${bamboo.planKey}</planKey>
  <buildNumber>${bamboo.buildNumber}</buildNumber>
  <projectKey>${bamboo.planKey.projectKey}</projectKey>
</buildInfo>

The ${bamboo.planKey} variable should resolve to PROJECT-QA.

3. Cache Refresh Timing: Rally SA’s quality-mgmt module uses a tiered caching system:

  • Metrics data cache: Refreshes every 15 minutes
  • Dashboard aggregation cache: Refreshes every 30 minutes
  • Historical trends cache: Refreshes every 2 hours

When you push new metrics, they may not appear immediately even with manual refresh. However, if metrics don’t appear after 2-3 hours, caching isn’t the issue.

To force immediate cache refresh after fixing configuration:

  1. Navigate to Rally Setup > System > Cache Management
  2. Select “Quality Metrics” from the cache type dropdown
  3. Click “Clear Cache” and then “Rebuild”
  4. Wait 5-10 minutes for rebuild to complete
  5. Refresh your quality gates dashboard

Note: Cache management requires Rally Administrator role.

4. Subscription Configuration: Your subscription must be configured to accept quality metric events from Bamboo. Verify these settings in Rally Setup > Integrations > Bamboo:

  • Event Types: Ensure “Quality Metrics” is checked (not just “Build Status”)
  • Metric Types: Enable specific metrics you’re sending (Coverage, Test Results, Defect Counts)
  • Workspace Scope: Set to include all workspaces where build definitions exist
  • Frequency: Set to “Real-time” rather than “Batch” for immediate processing
  • Validation: Enable “Strict” mode to get error messages for rejected metrics

The subscription config also controls metric retention. Check the “Data Retention” setting - if it’s set too low, older metrics might be auto-purged, making it seem like imports aren’t working.

Complete Bamboo Task Configuration: Update your Bamboo Rally Tasks configuration with this complete setup:

<task>
  <rally>
    <connection>
      <url>https://rally1.rallydev.com</url>
      <apiKey>${bamboo.rally.apiKey}</apiKey>
      <workspace>${bamboo.rally.workspace}</workspace>
    </connection>
    <buildInfo>
      <planKey>${bamboo.planKey}</planKey>
      <buildNumber>${bamboo.buildNumber}</buildNumber>
      <buildResult>${bamboo.buildResult}</buildResult>
      <startTime>${bamboo.buildTimeStamp}</startTime>
      <duration>${bamboo.buildDuration}</duration>
    </buildInfo>
    <qualityMetrics>
      <metric source="clover.coverage" target="CoveragePercentage" type="decimal"/>
      <metric source="junit.passed" target="PassedTestCount" type="integer"/>
      <metric source="junit.failed" target="FailedTestCount" type="integer"/>
      <metric source="sonar.violations" target="OpenDefectCount" type="integer"/>
    </qualityMetrics>
    <testResults>
      <includeTestCases>true</includeTestCases>
      <testSetName>${bamboo.planKey}-${bamboo.buildNumber}</testSetName>
    </testResults>
  </rally>
</task>

Key additions:

  • <testResults> section creates the test set association that Rally requires
  • testSetName uses a unique identifier combining plan key and build number
  • includeTestCases ensures individual test results are linked to metrics

Verification Steps:

  1. Update Bamboo Rally Tasks configuration with corrected field mappings
  2. Verify Rally build definition exists with matching ExternalBuildID
  3. Run a Bamboo build and check the task logs for successful metric submission
  4. Wait 15-30 minutes for cache refresh (or force clear/rebuild)
  5. Navigate to Rally quality-mgmt module and verify metrics appear
  6. Check quality gates dashboard shows coverage, test counts, and defect metrics
  7. Verify historical trends populate with multiple builds over time

If metrics still don’t appear after these changes:

  1. Enable debug logging in Bamboo Rally Tasks plugin
  2. Check Rally’s webhook delivery logs for detailed error messages
  3. Verify the API key has permissions to create quality metrics
  4. Ensure the workspace and project in Rally match your Bamboo configuration

The combination of correct field mapping, proper plan key correlation, understanding cache refresh timing, and complete subscription configuration should resolve your quality metrics import issues.