I want to share our implementation of Databricks connector authentication using Azure Entra ID service principals. Our organization required automated Tableau extract refreshes while maintaining strict credential policies - no personal access tokens stored in Tableau Server and full audit trail of data access.
The challenge was configuring service principal authentication that works with Tableau’s scheduled refresh jobs. We needed token management that handles expiration automatically and complies with our security team’s requirements for credential rotation and monitoring.
After testing several approaches, we found a solution that balances automation, security, and compliance. This might help others facing similar requirements with cloud data platforms and enterprise authentication standards.
Here’s our complete implementation approach for service principal authentication with Databricks:
Service Principal Token Management:
We created a dedicated service principal in Azure Entra ID specifically for Tableau-Databricks integration. The configuration includes:
-
Entra ID Setup: Register the application with appropriate API permissions for Azure Databricks. Generate client secret with 90-day expiration to align with our security policies.
-
Databricks Configuration: Add the service principal to the Databricks workspace using its Application ID. We use Unity Catalog for fine-grained access control - the principal has SELECT permissions on specific catalogs and schemas, plus CAN USE on designated SQL warehouses.
-
Tableau Connector Setup: When creating the Databricks connection in Tableau, select OAuth authentication method. Input the service principal’s client ID, client secret, and tenant ID. Tableau handles token acquisition and refresh automatically through the standard OAuth flow.
Automated Tableau Refreshes:
The OAuth tokens expire after 1 hour, but Tableau’s credential store manages refresh tokens that remain valid for up to 90 days (matching our secret rotation policy). When a scheduled extract refresh runs:
- Tableau checks if the access token is valid
- If expired, it uses the refresh token to request a new access token from Entra ID
- The new token is used to authenticate with Databricks
- No manual intervention needed unless the client secret itself expires
For scheduled refreshes, we store the service principal credentials at the project level in Tableau Server. This allows multiple data sources to share the same authentication without duplicating secrets.
Compliance with Credential Policies:
Our implementation meets enterprise security requirements through several mechanisms:
-
Secret Rotation: We maintain two active secrets in Entra ID during rotation periods. When the primary secret approaches expiration:
- Generate second secret in Entra ID
- Update Tableau data source connections with new secret
- Test scheduled refreshes
- Deactivate old secret after validation period
- This approach provides zero-downtime rotation
-
Audit Trail: Full logging across three layers:
- Entra ID sign-in logs show every token acquisition with timestamp and originating IP
- Databricks audit logs record all queries executed by the service principal
- Tableau Server logs track refresh job success/failure with correlation IDs
- We export these logs to our SIEM for correlation and alerting
-
Least Privilege Access: The service principal has no administrative rights. Unity Catalog grants are scoped to specific data assets. If a workbook needs access to new tables, we update the grants explicitly rather than giving broad permissions.
-
Monitoring and Alerting: We built monitoring around three failure scenarios:
- Service principal lockout (alert from Entra ID)
- Token refresh failures (alert from Tableau Server logs)
- Databricks permission errors (alert from query logs)
Failover Strategy:
We don’t maintain a backup authentication method because that would duplicate secret management burden. Instead, our incident response includes:
- Documented runbook for rapid service principal recovery
- Separate break-glass admin account for emergency Databricks access
- Automated alerts that trigger within 5 minutes of refresh failures
- SLA of 30 minutes to restore service principal functionality
The service principal approach eliminated PAT sprawl, provided centralized credential management, and gave us the audit trail our compliance team required. Initial setup took about a week including testing and documentation, but ongoing maintenance is minimal - just quarterly secret rotation which we’ve automated with Azure DevOps pipelines.
The Databricks connector in Tableau 2023.2+ supports OAuth tokens directly. We created a dedicated service principal in Entra ID with specific permissions scoped to our Databricks workspace. The key was configuring the connector to use OAuth with the service principal’s credentials instead of PATs. Token refresh happens automatically through the OAuth flow - Tableau requests new tokens when needed using the service principal’s client ID and secret stored in Tableau Server’s credential store.
This is helpful context. What about failover scenarios? If the service principal gets locked or disabled for any reason, do you have a backup authentication method, or do all refreshes just fail until it’s resolved?