Document control JSON export contains corrupted special characters

QMS ETQ Reliance
, , , , , , , ,
1

We’re exporting document metadata from ETQ Reliance 2022 via the REST API to feed our downstream reporting system, but special characters are getting corrupted in the JSON output. Characters like ™, ®, é, ñ, and Chinese characters are showing as � or garbled text.

The document titles and descriptions contain international characters since we have global operations. When I view the documents in the ETQ web interface, everything displays correctly. But the JSON export from the API is breaking our downstream integration.

I suspect this is related to UTF-8 character encoding configuration or how the JSON export API headers are set. We’re also concerned about character normalization in document metadata and whether there’s proper schema validation for international content.

Here’s a sample of what we’re getting:

{"title":"Safety Protocol™ - Espa�ol",
 "description":"Procedimiento de seguridad para operaci�n",
 "author":"Mar�a Gonz�lez"}

The actual characters should be ™, ñ, and í. Has anyone resolved similar encoding issues with document control exports?

2

Classic UTF-8 encoding mismatch. The API is probably returning UTF-8 encoded data, but your HTTP client is interpreting it as ISO-8859-1 or Windows-1252. Check the Content-Type header in the API response - it should explicitly specify charset=utf-8.

3

Before making database changes, try setting the Accept-Charset header in your API requests. Some versions of ETQ’s REST API respond to this header and will transcode the output accordingly. Set it to ‘Accept-Charset: utf-8’ in your HTTP client configuration.

Also check if there’s a query parameter option for encoding. Some ETQ API endpoints support ?encoding=utf-8 as a URL parameter to force UTF-8 output regardless of database collation.

4

Here’s the complete solution addressing all four focus areas:

1. UTF-8 Character Encoding Configuration: The root issue is your database collation, but you can fix this without changing it. Configure your API client to explicitly handle UTF-8:

HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));
headers.setAcceptCharset(Arrays.asList(StandardCharsets.UTF_8));

Also add this to your ETQ system configuration (Admin > System Settings > API Configuration):


api.export.defaultCharset=UTF-8
api.export.forceCharsetConversion=true

2. JSON Export API Headers: Modify your API calls to include explicit charset handling. When making requests:


GET /api/documents/export
Accept: application/json; charset=utf-8
Accept-Charset: utf-8

And configure your JSON parser to handle UTF-8:

ObjectMapper mapper = new ObjectMapper();
mapper.configure(JsonParser.Feature.ALLOW_UTF8_BOM, true);

3. Character Normalization in Document Metadata: ETQ 2022 supports Unicode normalization in export transformations. Enable this in Admin > Document Control > Export Settings:

  • Set ‘Unicode Normalization Form’ to ‘NFC’ (Canonical Composition)
  • Enable ‘Character Entity Encoding’ for special symbols
  • Check ‘Validate UTF-8 Sequences’ to detect corruption early

You can also apply normalization rules via the API by adding a transform parameter:


GET /api/documents/export?transform=unicode_nfc&validateEncoding=true

4. Schema Validation for International Content: Implement validation rules for document metadata to ensure consistent encoding from the start. Create a custom validation rule in ETQ:

  • Navigate to Admin > Document Control > Validation Rules
  • Add rule: ‘Validate Character Encoding’
  • Set validation script to check for valid UTF-8 sequences
  • Apply to all document fields containing international characters

For existing documents with corrupted characters, you’ll need to run a cleanup script. ETQ provides a data normalization utility:

-- Backup first!
UPDATE DOCUMENT_METADATA
SET title = CONVERT(NVARCHAR(MAX), title COLLATE Latin1_General_100_CI_AS_SC_UTF8),
    description = CONVERT(NVARCHAR(MAX), description COLLATE Latin1_General_100_CI_AS_SC_UTF8)
WHERE title LIKE '%�%' OR description LIKE '%�%'

Long-term Solution: While the above fixes your immediate export issues, plan a database migration to a UTF-8 compatible collation (Latin1_General_100_CI_AS_SC_UTF8 for SQL Server 2019+). This is the proper fix but requires:

  • Full database backup
  • Maintenance window (2-4 hours for typical document repositories)
  • Testing of all integrations post-migration
  • ETQ support involvement to ensure compatibility

For now, the API-level fixes and normalization rules will resolve your export corruption. Test thoroughly with documents containing ™, ®, accented characters, and CJK characters to verify all edge cases are handled correctly.

5

I checked the API response headers and they do include Content-Type: application/json but there’s no charset parameter. Our database is SQL Server with Latin1_General_CI_AS collation, which might not be UTF-8 compatible. Could that be the root cause? Would changing the database collation be safe, or is there a way to fix this at the API level?