Quality-mgmt pol-2310: JMeter vs LoadRunner for Polarion load testing

ALM Siemens Polarion ALM
, , , , , , , ,
1

We’re evaluating load testing tools for our pol-2310 quality management workflows and need community input on JMeter versus LoadRunner. Our testing scenarios involve 50-100 concurrent users performing LiveDocs rendering, test execution updates, and defect submissions.

JMeter Challenges: Correlation rules for OAuth2 tokens and CSRF extraction require custom scripting. We’ve built extractors for session management but LiveDocs rendering load tests show inconsistent response times - possibly due to JavaScript rendering not being captured properly.

// JMeter OAuth2 token extraction
String token = vars.get("oauth_token");
if (token == null || token.isEmpty()) {
    log.error("OAuth2 token extraction failed");
}

LoadRunner Considerations: LoadRunner’s protocol modeling supposedly handles Polarion’s REST API interactions more elegantly, but we’re concerned about the licensing costs and complexity of setting up TruClient for LiveDocs scenarios.

Anyone with experience load testing Polarion quality management modules with either tool? Particularly interested in how you handled OAuth2 token handling and CSRF token extraction for authenticated sessions.

2

For OAuth2 token handling in JMeter, use the JSR223 PreProcessor with Groovy scripting. This gives you full control over token refresh logic and header injection. We built a reusable script that handles token expiration automatically during long-running tests. The CSRF token extraction is simpler - just add a JSON Extractor after your authentication request and reference it in subsequent HTTP headers.

3

I’ve implemented comprehensive load testing for pol-2310 quality management with both tools, so I can provide detailed comparison based on your specific requirements.

JMeter Correlation Rules: JMeter requires manual correlation setup but offers complete flexibility. For OAuth2 token handling, implement a token manager using JSR223 elements:

// Token refresh logic
if (tokenExpiry < System.currentTimeMillis()) {
    // Trigger OAuth2 refresh flow
    vars.put("refresh_required", "true");
}

For CSRF token extraction, use JSON Path Extractor with pattern $.csrfToken after authentication. The key is setting up a proper test hierarchy where authentication happens once per thread and tokens are shared across requests using JMeter variables.

LoadRunner Protocol Modeling: LoadRunner’s Web-HTTP/HTML protocol handles Polarion’s REST API well, but the real advantage is TruClient for LiveDocs scenarios. TruClient executes actual JavaScript and captures client-side rendering time, giving you true end-user performance metrics. However, this comes at 3x the cost and requires dedicated performance engineers familiar with LoadRunner scripting.

LiveDocs Rendering Load: This is where tool choice matters significantly. JMeter’s HTTP samplers only measure server response time - you’re not capturing the 2-4 seconds of client-side rendering that users actually experience. If LiveDocs performance is critical, consider hybrid approach: JMeter for API load testing and Selenium Grid for realistic browser-based LiveDocs testing. We run JMeter for backend load (500+ concurrent API calls) and parallel Selenium tests (20-30 browsers) for UI validation.

OAuth2 Token Handling: Both tools handle this adequately once configured. JMeter’s approach is more transparent - you see exactly what’s happening in your correlation rules. LoadRunner abstracts this away, which is convenient but makes debugging harder. For pol-2310, OAuth2 tokens typically expire after 3600 seconds, so implement refresh logic that triggers at 3000 seconds to avoid mid-test authentication failures.

CSRF Token Extraction: JMeter requires explicit extractors but gives you fine-grained control. LoadRunner’s automatic correlation usually catches CSRF tokens but occasionally misses them in AJAX responses. For quality management workflows with heavy AJAX interactions, JMeter’s explicit extraction is actually more reliable.

Recommendation: For your 50-100 concurrent user scenario with budget constraints, JMeter is the better choice. Invest time in building reusable test components for OAuth2 handling and CSRF extraction - once built, they work reliably across test scenarios. Supplement with limited browser-based testing using open-source tools for LiveDocs validation. LoadRunner only makes sense if you need enterprise-level reporting, have existing LoadRunner infrastructure, or require advanced protocol modeling for complex scenarios beyond standard REST API testing.

We successfully load test pol-2310 quality management with JMeter handling 200+ concurrent users performing test execution updates and defect submissions, with response times under 2 seconds at 95th percentile. The OAuth2 and CSRF handling complexity is a one-time setup cost that pays off in flexibility and maintainability.

4

Thanks for the feedback. We’re leaning toward JMeter now due to budget constraints. The LiveDocs JavaScript rendering limitation is acceptable since we’re primarily testing backend performance. Has anyone successfully modeled LoadRunner protocol scenarios for Polarion’s REST API? Curious if the protocol modeling actually provides significant advantages over JMeter’s HTTP request samplers.