MQTT broker integration fails with custom endpoint in AWS IoT Core, devices not connecting reliably

IoT AWS IoT
, , , , , , , ,
1

We’re migrating our industrial device fleet to AWS IoT Core and hitting authentication failures with our custom MQTT endpoint. Devices successfully connect to the default endpoint but fail when using our custom domain endpoint (iot-custom.mydomain.com).

The custom endpoint registration appears successful in the AWS IoT Console, but device certificate mapping seems problematic. Getting connection refused errors:


MQTT CONNECT failed: TLS handshake timeout
Endpoint: iot-custom.mydomain.com:8883
Client cert: device-cert-001.pem

We’ve verified the certificates are valid and IoT policy attachment looks correct in the console. The same certificates work fine with the default AWS endpoint. Is there a specific configuration step for custom endpoints that differs from the default? Our devices need the custom endpoint for regulatory compliance requirements.

2

I’ve seen this before with custom endpoints. The device certificate mapping to the custom endpoint requires explicit configuration in the IoT policy. Your policy needs to include the custom endpoint ARN in the Resource field, not just the default endpoint. The TLS handshake timeout suggests the endpoint isn’t recognizing the certificate authority chain. Did you upload your custom CA certificate to AWS IoT when you created the custom endpoint? That’s a mandatory step that’s easy to miss.

3

Here’s the complete solution addressing all three focus areas:

Custom Endpoint Registration: Verify your endpoint configuration includes the correct domain ownership validation. The custom endpoint must have valid DNS records and certificate chain:


aws iot describe-domain-configuration \
  --domain-configuration-name my-custom-config

Device Certificate Mapping: The critical issue is ensuring certificate-to-thing mapping works with custom endpoints. Update your device registration to explicitly reference the custom endpoint:


aws iot attach-thing-principal \
  --thing-name device-001 \
  --principal arn:aws:iot:region:account:cert/certId

Then verify the certificate is active and mapped:


aws iot describe-certificate --certificate-id certId

IoT Policy Attachment: Your IoT policy MUST include the custom endpoint in the Resource ARN. Update the policy document:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": "iot:Connect",
    "Resource": "arn:aws:iot:region:account:client/${iot:Connection.Thing.ThingName}",
    "Condition": {
      "StringEquals": {
        "iot:Connection.Thing.IsAttached": "true"
      }
    }
  }, {
    "Effect": "Allow",
    "Action": ["iot:Publish", "iot:Subscribe", "iot:Receive"],
    "Resource": [
      "arn:aws:iot:region:account:topic/custom-endpoint/*",
      "arn:aws:iot:region:account:topicfilter/custom-endpoint/*"
    ]
  }]
}

Key Configuration Steps:

  1. Ensure the custom domain configuration status is ENABLED
  2. Verify the server certificate chain includes all intermediate CAs
  3. Update device client code to use the full custom endpoint URL
  4. Attach the updated policy to all device certificates
  5. Test connection using AWS IoT Device SDK with verbose logging

The TLS handshake timeout typically indicates the policy evaluation failed before the TLS connection completed. With the policy properly scoped to your custom endpoint and certificate mapping verified, your devices should connect successfully. Monitor CloudWatch Logs for iot:Connect events to confirm successful authentication.

4

Thanks for the quick responses. We’re using iot:Data-ATS endpoint type and DNS resolution checks out fine using nslookup. The custom CA certificate was uploaded during endpoint creation. However, reviewing our IoT policy, I see the Resource field still references the default endpoint ARN. That’s likely the issue - the policy attachment isn’t properly scoped to the custom endpoint.