We’re planning a migration from Jira Data Center to Cloud and need to reassess our multi-environment test execution tracking strategy. Currently using OSLC Connect to integrate with external test tools, but Xray Cloud has different limitations around environment management.
Our DC setup tracks test executions across five environments (dev, qa, staging, pre-prod, prod) with detailed execution history per environment. We’re concerned about losing this granularity in Cloud, especially for hybrid reporting during the migration period.
What approaches have others taken for environment-specific test tracking in Cloud? Are there workarounds for Xray Cloud’s environment limitations, or should we rethink our entire test execution model?
We went through this migration last year. The biggest difference is that Xray Cloud treats environments as labels rather than first-class entities like DC does. You can still track multi-environment executions, but you lose some of the built-in environment filtering and reporting that DC provides.
Our workaround was to use custom fields for environment tracking and build our own dashboards using Jira’s native reporting. It’s more manual but gives us the flexibility we need. The OSLC migration part was straightforward - most OSLC adapters support both DC and Cloud with minor config changes.
Traceability actually improved with separate plans because each environment’s test coverage became explicitly visible. Requirements link to all environment-specific plans, so you can see at a glance which environments have executed tests for each requirement.
For hybrid sync, we used Xray’s REST API to push execution results from DC to Cloud. Wrote a simple script that ran post-execution to duplicate results to Cloud with environment tags. This gave us Cloud reporting without re-running tests. The sync added about 30 seconds per execution but eliminated duplicate effort.
Having guided multiple organizations through DC-to-Cloud migrations with complex environment tracking, here’s a comprehensive strategy:
DC vs Cloud Environment Management:
The fundamental difference is architectural. DC’s environment management is deeply integrated into Xray’s data model with native support for environment-based execution filtering, historical trending per environment, and environment-specific test cycle management. Cloud simplifies this to environment labels and custom fields, which provides flexibility but requires more manual configuration.
OSLC Migration Considerations:
OSLC Connect adapters work in both environments, but Cloud has additional authentication requirements (OAuth vs basic auth). Plan for adapter reconfiguration during migration. The good news is that OSLC’s standardized interface means your external tool integrations (HP ALM, IBM RQM, etc.) don’t need changes - only the Jira-side adapter config.
Xray Cloud Limitations and Workarounds:
Cloud’s main environment limitations:
No native environment-based test cycle organization (workaround: use test plan naming convention with environment suffix)
Limited environment filtering in gadgets (workaround: custom JQL-based dashboards with environment field filters)
Execution history doesn’t auto-group by environment (workaround: use labels + Structure for hierarchical environment views)
These aren’t blockers, but they require rethinking your reporting and organization strategy.
Hybrid Reporting Strategy:
During migration, run a parallel reporting architecture:
Maintain DC as source of truth for historical data
Start writing new executions to Cloud
Use Jira’s native CSV export from DC and import to Cloud for historical migration (environment data preserved in custom fields)
Build unified dashboards that query both instances during transition (use Jira’s REST API to aggregate data)
Typical hybrid period: 3-6 months depending on historical data volume and reporting complexity.
Recommended Approach:
Based on your five-environment setup:
Create environment-specific test plans in Cloud (DEV-TP, QA-TP, STAGING-TP, PREPROD-TP, PROD-TP)
Use a custom “Environment” field (single select) on Test Execution issues
Implement Structure for hierarchical environment views and cross-environment coverage reports
Migrate OSLC adapters to Cloud OAuth and test connectivity before cutover
Build Cloud dashboards that replicate your critical DC reports (most can be recreated with custom JQL)
Script bidirectional sync during hybrid period to validate Cloud reporting accuracy
Migration Phases:
Phase 1 (Month 1-2): Cloud setup, adapter migration, parallel execution in both environments
Phase 2 (Month 3-4): Validate Cloud reporting, train team on new workflows, fine-tune dashboards
Phase 3 (Month 5-6): Gradual cutover starting with dev environment, progressive promotion to prod
Phase 4 (Month 6+): DC decommission after 30-day validation period
The key insight is that Cloud’s flexibility requires more upfront design but ultimately provides better scalability. Your test execution model doesn’t need complete rethinking - just adaptation to Cloud’s label-based rather than entity-based environment handling. Most organizations find Cloud’s approach cleaner once they adjust, though the transition requires careful planning.
The separate test plans per environment approach is interesting. How did that impact your traceability? We have requirements that trace through test cases to executions, and I’m worried about losing that connection if we split into multiple plans.
Also, during the hybrid period, did you sync environment data between DC and Cloud, or keep them independent? We’re trying to avoid duplicate data entry for the same test executions.
I’d recommend keeping DC and Cloud in parallel during migration for hybrid reporting. We ran both for six months, syncing test results bidirectionally. This let us validate that Cloud reporting met our needs before fully cutting over.
For Xray Cloud limits, the main constraint is that environment-based test cycles aren’t as flexible. We ended up creating separate test plans per environment instead of one plan with multiple environment executions. It’s more overhead but provides clearer separation.