AI-powered requirement extraction in Polarion misses critica

ALM Siemens Polarion ALM
, , , , , , , ,
1

Our team is using Polarion’s AI-driven requirement extraction feature to process safety specification documents for an automotive project. The AI does a decent job extracting functional requirements, but consistently misses safety-critical constraints that use domain-specific terminology like “ASIL-D compliance,” “fail-safe mode,” and “redundant sensor validation.”

We’ve imported 15 safety specifications, and manual review shows the AI extracted about 60% of functional requirements correctly but only 15-20% of safety constraints. This creates major gaps in our safety requirements baseline. The missed requirements are clearly stated in the documents with SHALL/MUST keywords, but the AI seems to skip over technical safety terms.

Is there a way to configure the AI extraction model to recognize automotive safety patterns, or do we need a hybrid approach combining AI extraction with manual safety requirement capture?

2

Another approach - use Polarion’s document comparison feature to track what the AI extracted vs what’s in the original document. Import the document with AI extraction, then import again with manual tagging, and run a diff. This highlights exactly what the AI missed so you can build a pattern library of problematic terminology. Over time, you’ll identify which safety terms need manual capture vs which the AI handles well.

3

Here’s a comprehensive solution addressing all three aspects of AI extraction for safety requirements:

1. AI Extraction Model Configuration for Domain-Specific Terminology

Polarion’s AI extraction can be tuned using custom extraction rules. Create a configuration file that defines safety-critical term patterns:

<extractionRules domain="automotive-safety">
  <termPattern priority="high">ASIL-[A-D]</termPattern>
  <termPattern priority="high">fail-safe|fail-operational</termPattern>
  <termPattern priority="high">redundant.*validation</termPattern>
</extractionRules>

Upload this to Polarion → Administration → Document Processing → AI Extraction Rules. This forces the AI to treat these patterns as high-confidence requirement indicators even if surrounding context is ambiguous.

2. Safety Constraint Pattern Recognition and Validation

Implement a two-stage validation workflow:

Stage 1 - Post-Extraction Validation: Create a Polarion workflow function that runs after AI extraction:

// Pseudocode - Safety requirement validation:
1. Query all requirements extracted from document
2. For each requirement, check for safety indicators:
   - Contains ASIL level designation
   - Has failure mode description
   - Includes verification method
3. Flag requirements missing safety metadata
4. Auto-assign to safety engineer for review

Stage 2 - Completeness Check: Use Polarion’s custom field validation to enforce safety constraint completeness. Create mandatory custom fields for safety requirements:

  • ASIL Level (enum: A/B/C/D)
  • Safety Goal Reference (link to parent safety goal)
  • Failure Mode (text)
  • Verification Method (enum: Test/Inspection/Analysis)

Requirements missing these fields are automatically flagged in the traceability matrix.

3. Hybrid Manual-Automated Requirement Capture Workflow

Implement this three-phase approach:

Phase 1 - AI First Pass (Automated):

  • Run AI extraction on full document to capture obvious functional requirements
  • AI typically handles 70-80% of standard requirements well

Phase 2 - Safety Section Manual Review (Semi-Automated):

  • Configure document import to auto-tag sections with safety-related headings
  • Safety engineer reviews only these tagged sections (10-15% of document)
  • Use Polarion’s inline editing to manually create requirements the AI missed
  • Apply the safety requirement template with mandatory custom fields

Phase 3 - Cross-Reference Validation (Automated):

  • Run a validation script that compares extracted requirements against safety standards checklist
  • Generate gap report showing which safety aspects lack requirements
  • Example validation rules:
    • Every ASIL-C/D function must have redundancy requirement
    • Every sensor input must have validation requirement
    • Every failure mode must have detection and mitigation requirements

Implementation Priority:

  1. Start with Phase 2 (manual review of safety sections) - this immediately improves your 15-20% capture rate to 90%+
  2. Add Phase 3 validation to catch systematic gaps
  3. Fine-tune Phase 1 AI rules over time as you identify common missed patterns

This hybrid approach balances automation efficiency with safety-critical thoroughness. The AI handles routine functional requirements while safety engineers focus their expertise on critical constraints.

4

I looked into the extraction template settings but couldn’t find options for custom term dictionaries in pol-2310. The section marker approach sounds promising - can you share what your configuration looked like? Did you use XML-based templates or the web UI configuration? Also wondering if there’s a way to validate extracted requirements against a safety constraint checklist automatically.

5

The AI extraction in Polarion uses generic NLP models that aren’t trained on domain-specific terminology out of the box. For automotive safety, you’ll need to configure custom extraction rules. Check if your Polarion version supports custom term dictionaries where you can define safety-critical keywords that should trigger requirement extraction.