Integration module fails to sync with external MQTT broker after certificate renewal

IoT Cisco IoT Cloud Connect
, , , , , , , ,
1

After renewing TLS certificates on our external MQTT broker (Mosquitto 2.0), the Integration Services module in cciot-25 can no longer establish connections. The broker logs show TLS handshake failures with “certificate verification failed” errors. We’ve confirmed the new certificates are valid and properly configured on the broker side.

Error from integration logs:


SSLHandshakeException: sun.security.validator.ValidatorException
PKIX path building failed
Unable to find valid certification path

The old certificates expired yesterday, so we urgently need to complete the TLS certificate renewal process and update the trust store in IoT Cloud Connect. We’re also wondering if there are specific requirements for MQTT secure connection configuration when using external brokers. The integration was working perfectly before the certificate renewal.

2

When working with external MQTT brokers, I always recommend using a certificate monitoring script that alerts you 30 days before expiration. This gives you time to plan the renewal without service interruption. Also, during renewal, keep both old and new certificates valid for at least 24 hours to allow gradual rollover.

3

One more thing - if you’re using client certificate authentication (mutual TLS), you’ll need to update both the trust store and the client keystore. The broker needs to trust your client certificates, and your client needs to trust the broker certificates. Both sides must be updated during renewal.

4

The PKIX path building error means the Integration module doesn’t trust the new certificate chain. You need to import the new CA certificate into IoT Cloud Connect’s trust store. The renewal process requires updating both the broker certificates and the client-side trust store.

5

The Integration Services module uses its own trust store located at /opt/cisco-iot/integration/config/truststore.jks. You’ll need to import the new CA certificate there using keytool. Make sure to restart the integration service after updating the trust store, as it only loads certificates at startup. Also verify that the certificate chain is complete - intermediate certificates must be included.

6

Here’s the complete solution addressing all three focus areas:

TLS Certificate Renewal Process: First, obtain the complete certificate chain from your MQTT broker, including the root CA and any intermediate certificates. Export these to PEM format if they’re not already.

Verify the certificate chain is valid:

openssl verify -CAfile ca-chain.pem broker-cert.pem

If verification succeeds, you have a valid chain. If it fails, you’re missing intermediate certificates.

Trust Store Update: Import the new CA certificate chain into the Integration Services trust store:

keytool -import -trustcacerts -alias mqtt-broker-ca \
  -file ca-chain.pem \
  -keystore /opt/cisco-iot/integration/config/truststore.jks \
  -storepass changeit

If you had previously imported the old certificate, remove it first:

keytool -delete -alias mqtt-broker-ca-old \
  -keystore /opt/cisco-iot/integration/config/truststore.jks

Verify the import:

keytool -list -keystore /opt/cisco-iot/integration/config/truststore.jks

Restart the Integration Services module:

sudo systemctl restart cisco-iot-integration

MQTT Secure Connection Configuration: Update your MQTT integration configuration to ensure proper TLS settings:

  1. In the Integration Services configuration file (/opt/cisco-iot/integration/config/mqtt-broker.properties), verify:
mqtt.broker.url=ssl://mqtt-broker.yourdomain.com:8883
mqtt.tls.version=TLSv1.2
mqtt.tls.verify.hostname=true
mqtt.connection.timeout=30
mqtt.keepalive.interval=60

  1. Ensure hostname verification is enabled. The broker certificate’s CN or SAN must match the hostname in your connection URL. If you’re using an IP address, add it to the certificate’s SAN field.

  2. For mutual TLS (if applicable), also update the client keystore:

keytool -importkeystore \
  -srckeystore client-cert.p12 -srcstoretype PKCS12 \
  -destkeystore /opt/cisco-iot/integration/config/keystore.jks \
  -deststoretype JKS
  1. Test the connection using mosquitto_pub/sub to verify broker accessibility:
mosquitto_pub -h mqtt-broker.yourdomain.com -p 8883 \
  --cafile ca-chain.pem -t test/topic -m "test" \
  --tls-version tlsv1.2

Ongoing Certificate Management: Implement automated certificate monitoring:

  • Set up alerts for certificates expiring within 30 days
  • Use Let’s Encrypt or similar for automatic renewal where possible
  • Document the renewal process including all trust store locations
  • Test certificate renewal in a staging environment first
  • Keep certificate renewal history for audit purposes

After following these steps, your Integration Services module should successfully reconnect to the MQTT broker with the new certificates. Monitor the integration logs for successful TLS handshake messages confirming the connection is established.