Test case execution status stuck on 'In Progress' after test run completes

[ricardosql]

Our automated test execution framework updates Rally test case execution status via REST API, but approximately 30% of executions remain stuck at ‘In Progress’ even after the test run completes successfully. The webhook event handlers are configured to trigger on test completion, but the status transition isn’t happening consistently.

Here’s our status update call:

result_data = {
    "TestCaseResult": {
        "Verdict": "Pass",
        "Date": datetime.now().isoformat()
    }
}
response = rally.put(result_ref, result_data)

The API returns 200 OK, but the UI still shows ‘In Progress’. This is blocking our reporting pipeline because management dashboards filter out incomplete executions. I suspect either permission validation issues or test result mapping problems, but I’m not sure how to debug further.

[apicoder]

Check your workspace permissions carefully. Even with Editor rights, the API user needs specific ‘Modify All TestCaseResults’ permission to update execution status. Also, if your test cases are associated with test sets that have locked status, the individual result updates will fail silently. Review your test set configuration and ensure they’re not in a read-only state during execution windows.

[sqlarchitect]

Your issue involves multiple interconnected problems. Let me break down the complete solution:

State Machine Configuration: Rally’s TestCaseResult state machine requires three mandatory fields for completion: Verdict, Build, and Date. Your original code was missing Build. Additionally, the state machine has validation rules that prevent transitions if the TestCase parent is in certain states (Archived, Deprecated). Verify your test cases aren’t in these states:

test_case = rally.get('TestCase', fetch=['ScheduleState'])
if test_case.ScheduleState in ['Archived', 'Deprecated']:
    # Cannot update execution status

Webhook Event Handlers: The async nature of webhooks means you can’t rely on immediate state changes. Implement a polling mechanism after your API call:

// Pseudocode - Verification loop:
1. Submit TestCaseResult update via PUT request
2. Wait 2-3 seconds for webhook processing
3. GET the TestCaseResult object and check Verdict field
4. If still 'In Progress', retry update with exponential backoff
5. Log failures after 3 retry attempts

Permission Validation: Your API user needs workspace-level permissions, not just project-level. Navigate to Setup > Workspace > Permissions and verify the API key has ‘Workspace Admin’ or ‘Editor’ role with ‘Modify All’ checked for TestCaseResult objects. Project-level permissions aren’t sufficient for state transitions.

Test Result Mapping: The 15% failure rate suggests orphaned relationships. Implement a health check before updates:

result_data = {
    "TestCaseResult": {
        "TestCase": test_case_ref,  # Re-establish link
        "Verdict": "Pass",
        "Build": build_number,
        "Date": datetime.now().isoformat(),
        "Tester": user_ref  # Required for audit trail
    }
}

The Tester field is often overlooked but required for proper state transitions in Rally 2023+. Without it, Rally’s validation engine rejects the update silently.

For your reporting pipeline, implement a cleanup job that runs hourly to detect stuck executions and force-complete them with a ‘Blocked’ verdict. This prevents dashboard pollution while you fix the root cause. The combination of adding all required fields, verifying permissions, and implementing retry logic should resolve 99%+ of your stuck status issues.

[amandacreator]

The remaining 15% failures are likely due to test result mapping inconsistencies. When Rally creates a TestCaseResult, it establishes a link to the parent TestCase object. If that link is broken or the TestCase has been moved to a different project, the status update will appear to succeed but won’t actually persist. Run a query to find orphaned TestCaseResults:

(TestCase = null) AND (Verdict = null)

These are results that lost their parent reference and can’t complete the state machine transition.

[tarunbuilder]

The 200 OK response doesn’t guarantee the state transition completed. Rally’s state machine configuration requires the TestCaseResult object to have both Verdict AND Build fields populated for the status to move from ‘In Progress’ to completed. Check if your API call includes the Build field - it’s mandatory for state transitions even though it’s not marked as required in the schema.

[sarah_rall]

I added the Build field to our API payload and the success rate improved to about 85%, but we’re still seeing 15% stuck. The webhook logs show successful deliveries for all events. Could this be a permission validation issue where the API user doesn’t have rights to transition certain test case states?

[ace_creator]

I’ve seen this exact issue. The problem is that Rally’s webhook event handlers fire asynchronously, and if your API call completes before the webhook processes the state change, you get a race condition. The TestCaseResult is created with ‘In Progress’ status, but the webhook that should update it to ‘Pass’ or ‘Fail’ never fires properly. Check your webhook configuration logs for failed deliveries.