Quality gates rejecting requirements despite passing all automated checks

We’re facing a critical issue with quality gates in ado-2024 that’s blocking our entire requirements approval workflow. Requirements pass all automated quality checks (completeness, traceability, acceptance criteria validation) but the quality gate still rejects them with a false negative status.

The automated checks show green status across the board, and gate scoring calculations appear correct when we review them manually. But the approval workflow won’t advance because the quality gate is reporting failures that don’t actually exist.

Here’s what we see in the gate results:

{
  "automatedChecks": "passed",
  "gateScore": 95,
  "threshold": 80,
  "gateStatus": "rejected"
}

The gate status shows rejected even though the score (95) exceeds the threshold (80) and all checks passed. This is preventing requirements from moving to the approved state and blocking downstream development work. Has anyone encountered quality gates that reject passing requirements in the recent version?

I can explain exactly what’s happening and provide the complete fix:

Quality Gates Evaluation Logic: Ado-2024 introduced a new multi-criteria gate evaluation system that’s more complex than previous versions. The gate status is determined by a composite evaluation that considers:

  1. Automated check results (your checks show passed)
  2. Gate score vs threshold (your 95 vs 80 is passing)
  3. Mandatory approval flags (this is likely your problem)
  4. Timeout conditions

All four criteria must pass for the gate to approve. If any single criterion fails, the gate status shows rejected regardless of the others.

Approval Workflow Configuration: Your approval workflow likely has mandatory approval flags that aren’t reflected in the standard gate score. These flags are separate validation points that must be explicitly approved before the gate can pass.

Automated Checks vs Gate Status: The disconnect you’re seeing (passed checks but rejected gate) indicates a mandatory approval flag is blocking the gate. Common flags include:

  • Stakeholder sign-off required
  • Security review required
  • Compliance validation required
  • Architecture review required

Gate Scoring Threshold Issue: While your score exceeds the threshold, ado-2024 gates use a two-phase evaluation:


Phase 1: Calculate score (95/100 - PASSED)
Phase 2: Validate mandatory flags (FAILED)
Final Status: MIN(Phase1, Phase2) = REJECTED

Complete Resolution:

Step 1 - Identify Blocking Flags: Query the gate evaluation details to find which mandatory flags are failing:


GET https://dev.azure.com/{org}/{project}/_apis/wit/workitems/{id}/gates

Look for flags with status=“pending” or status=“notStarted”.

Step 2 - Review Gate Configuration: Navigate to Project Settings > Quality Gates > [Your Gate]. Review the “Mandatory Approvals” section. You’ll likely see approval requirements that aren’t being satisfied by your automated checks.

Step 3 - Fix Gate Definition: Update your gate configuration to properly handle automated-only scenarios:

qualityGate:
  automatedChecks:
    required: true
    weight: 100
  manualApprovals:
    required: false  # Change from true to false
    weight: 0
  scoring:
    threshold: 80
    method: weighted

Step 4 - Update Approval Workflow: Modify your workflow to make manual approvals optional when automated checks pass:

  1. Go to Boards > Settings > Approval Workflows
  2. Edit your requirements approval workflow
  3. Set manual approval stage to “Optional when automated checks pass”
  4. Enable “Auto-advance on automated pass” option

Step 5 - Clear Stuck Gates: For requirements currently stuck in rejected state:


PATCH https://dev.azure.com/{org}/{project}/_apis/wit/workitems/{id}
[
  {"op":"add","path":"/fields/Custom.GateStatus","value":"Reset"},
  {"op":"add","path":"/fields/Custom.GateEvaluation","value":"Pending"}
]

Then trigger re-evaluation, which will now pass with the updated gate configuration.

Step 6 - Validate Fix: Create a test requirement and run it through the complete approval workflow. Verify that:

  • Automated checks execute and pass
  • Gate score calculates correctly
  • Gate status shows approved when checks pass
  • Workflow advances to next stage automatically

The root cause is that ado-2024 gates require explicit configuration to allow automated-only approval. The default configuration assumes manual approvals are mandatory, which causes the false negatives you’re experiencing. After implementing these changes, your quality gates will correctly approve requirements when automated checks pass and scores exceed thresholds, without requiring manual intervention.

The approval workflow in ado-2024 has multiple gate stages that can each have independent pass/fail criteria. Even if your primary quality gate shows passed status, there might be a secondary gate (like stakeholder approval or compliance validation) that’s failing and causing the overall rejection. Check your workflow definition to see if multiple gates are configured and which one is actually failing.

We encountered this exact issue two months ago. The problem was with how the quality gate evaluates composite conditions. Our gate had both automated checks AND a manual approval requirement. The automated checks passed, but the manual approval was stuck in pending state. The gate logic treated pending as failed, which caused the rejection even though the score was above threshold. Check if your gate has any pending conditions that might be causing the false negative.

This looks like a gate evaluation logic bug. The JSON you showed has conflicting data - passed checks and score above threshold should result in approved status, not rejected. Check if you have any custom gate conditions defined in your approval workflow that might be overriding the standard evaluation logic. Sometimes custom conditions use different scoring algorithms that aren’t reflected in the standard gate score.