We’re evaluating options for enhanced audit logging of test execution activities in ELM 7.0.2. Our compliance team needs detailed tracking of who executed tests, when results were modified, and what data changed. The built-in audit logging captures basic events, but we need more granular information like parameter changes and result attachments.
I’m curious about others’ experiences: have you relied on the native audit event handlers and JRS parameterized queries, or have you built custom server extensions? Our concern is that custom extensions might complicate upgrades and maintenance, but the out-of-box reporting seems insufficient for our audit requirements. What’s worked well in your organization for maintaining compliance reporting while keeping the system maintainable?
The native audit logging in 7.0.2 is actually quite comprehensive if you configure it correctly. You can capture test result modifications, attachment additions, and execution parameter changes through the built-in event handlers. The challenge is querying this data effectively. We built parameterized JRS queries that pull audit data filtered by date range, user, and test case. It’s not perfect, but it covers 90% of our compliance needs without custom code.
Having implemented both approaches across multiple organizations, here’s my recommendation based on your compliance requirements:
Native Audit Event Handlers Approach:
The built-in audit logging in ELM 7.0.2 handles most test execution tracking needs effectively. Configure audit event handlers to capture:
- Test execution start/completion events
- Result status changes (Pass/Fail/Blocked transitions)
- Parameter modifications during execution
- Comment additions with timestamps
- Attachment uploads with metadata
The key is proper global configuration setup. Ensure your project configuration includes all test execution artifact types in the audit scope, and enable detailed change tracking for test results. This captures sufficient data for most compliance frameworks (SOC2, ISO 27001, FDA 21 CFR Part 11).
JRS Parameterized Queries:
Build custom reports using JRS to query the audit database. Create parameterized queries that filter by:
- Date ranges for monthly/quarterly compliance reviews
- Specific users or teams for access audits
- Test case categories for focused compliance checks
- Change types (modifications vs. deletions)
These queries can join audit events with test execution records and user profiles to generate comprehensive audit trails. The advantage is that JRS reports are version-compatible and don’t break during ELM upgrades.
When to Consider Server Extensions:
Only build custom extensions if you have specific requirements that native logging absolutely cannot meet:
- Real-time audit event notifications to external systems
- Attachment deletion tracking with full metadata preservation
- Integration with third-party compliance platforms
- Custom encryption or data masking for sensitive test data
If you go the custom route, keep extensions minimal and well-documented. Use the official ELM extension APIs and avoid modifying core audit handlers. Design extensions to fail gracefully so audit logging continues even if your custom code breaks.
Maintenance Perspective:
Native audit handlers with JRS reporting require minimal ongoing maintenance - mainly query updates when compliance requirements change. Custom extensions need testing with every ELM patch and major version upgrade. Based on our experience, the native approach reduces maintenance burden by 60-70% while still meeting most audit requirements.
For your specific needs around test result modifications and attachment tracking, I’d recommend starting with the native capabilities. Configure comprehensive audit event handlers, build robust JRS queries, and only add a targeted server extension if attachment deletion tracking proves insufficient for your auditors.
Thanks Chen, that’s helpful. What about attachment tracking? Our auditors specifically want to know when evidence files are added or removed from test results. Does the native logging cover file-level operations, or would that require a custom extension?
That’s encouraging about the native capabilities. Can you share more about how you structure your JRS queries? We’re particularly interested in tracking when test results change from Pass to Fail or vice versa, along with the justification comments. Does the audit event handler capture enough detail for that level of granularity?
Attachment operations are logged in the audit trail, including filename, size, upload timestamp, and the user who added it. However, the native system doesn’t track attachment deletions as robustly - you get a deletion event but not always the details of what was deleted. If deletion tracking is critical, you might need a lightweight server extension that specifically monitors attachment removal events and writes them to a custom audit table.