Let me provide a comprehensive solution addressing managed identity RBAC, storage firewall rules, and Synapse pipeline permissions systematically.
Managed Identity RBAC Configuration:
The core issue is that Synapse pipelines authenticate differently than Studio. Here’s the complete RBAC setup:
- Verify Managed Identity Assignment:
# Get Synapse workspace managed identity
az synapse workspace show --name your-synapse-ws \
--resource-group your-rg \
--query identity.principalId -o tsv
- Assign Correct RBAC Roles:
You need Storage Blob Data Contributor at BOTH the storage account AND container level:
# Storage account level
az role assignment create \
--assignee <managed-identity-id> \
--role "Storage Blob Data Contributor" \
--scope "/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<storage-account>"
# Container level (critical for pipeline access)
az role assignment create \
--assignee <managed-identity-id> \
--role "Storage Blob Data Contributor" \
--scope "/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<storage-account>/blobServices/default/containers/erp-raw"
- Wait for Propagation:
RBAC changes can take 5-10 minutes to propagate. This is often overlooked and causes intermittent failures.
Storage Firewall Rules Resolution:
The firewall configuration is likely your main blocker:
- Enable Trusted Microsoft Services:
In Azure Portal → Storage Account → Networking:
- Select “Enabled from selected virtual networks and IP addresses”
- Check “Allow Azure services on the trusted services list to access this storage account”
- This allows Synapse’s Azure Integration Runtime through the firewall
- Add Synapse Managed VNet (if using):
If your Synapse workspace uses a managed VNet:
az storage account network-rule add \
--account-name <storage-account> \
--subnet "/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Synapse/workspaces/<workspace-name>/subnets/default"
- Resource Instance Rules:
For enhanced security, add a resource instance rule specifically for your Synapse workspace:
- In Storage Networking → Resource instances
- Add: Resource type = Microsoft.Synapse/workspaces
- Select your Synapse workspace
- This is more secure than broad trusted services access
Synapse Pipeline Permissions Fix:
Pipeline-specific configurations that commonly cause 403 errors:
- Linked Service Authentication:
Verify your Data Lake Storage linked service uses managed identity:
{
"type": "AzureBlobFS",
"typeProperties": {
"url": "https://<account>.dfs.core.windows.net"
},
"connectVia": {
"referenceName": "AutoResolveIntegrationRuntime"
},
"annotations": [],
"authentication": "ManagedServiceIdentity"
}
- Dataset Path Permissions:
Ensure the managed identity has execute permissions on parent folders:
# If using hierarchical namespace (Data Lake Gen2)
az storage fs access set \
--account-name <storage-account> \
--file-system erp-raw \
--permissions "rwx" \
--acl-user <managed-identity-id>
- Pipeline Activity Configuration:
In your Copy activity, ensure source dataset uses managed identity:
- Source dataset → Connection → Authentication = System-assigned managed identity
- Verify “Test connection” succeeds with “To linked service” option
- Integration Runtime Considerations:
If using Azure IR (default):
- Ensure storage firewall has trusted services enabled
- Consider switching to Managed VNet IR for better security isolation
If using Self-Hosted IR:
- Add the IR machine’s IP to storage firewall allow list
- Ensure IR machine can reach storage account (test with Storage Explorer)
Troubleshooting Steps:
- Test Managed Identity Access Directly:
# Get access token for managed identity
az account get-access-token --resource https://storage.azure.com/
# Use token to list blobs
curl -H "Authorization: Bearer <token>" \
"https://<account>.blob.core.windows.net/erp-raw?restype=container&comp=list"
- Check Activity Logs:
In Azure Portal → Storage Account → Monitoring → Diagnostic logs:
- Enable logging for read/write operations
- Look for 403 errors with caller identity
- Verify the caller is your Synapse managed identity
- Validate Firewall Rules:
az storage account show \
--name <storage-account> \
--query networkRuleSet
Complete Resolution Checklist:
✓ Synapse workspace has system-assigned managed identity enabled
✓ Managed identity has Storage Blob Data Contributor at storage account level
✓ Managed identity has Storage Blob Data Contributor at container level (erp-raw)
✓ Storage firewall has “Allow trusted Microsoft services” enabled
✓ Storage firewall includes Synapse managed VNet (if applicable)
✓ Linked service authentication is set to Managed Identity
✓ RBAC propagation wait time (10 minutes) has passed
✓ No deny assignments blocking the managed identity
✓ Integration Runtime can reach storage account (network connectivity)
After applying all these configurations, wait 15 minutes for full propagation, then test the pipeline. The combination of proper RBAC at multiple levels, correct firewall exceptions for Azure Integration Runtime, and managed identity authentication in the linked service will resolve your 403 errors.