Our Network Watcher flow logs are experiencing significant delays before appearing in our Log Analytics workspace. We’re seeing 15-30 minute gaps between when traffic occurs and when the logs become queryable, which is making real-time security monitoring nearly impossible.
We have NSG flow logs configured for multiple network security groups across a hub-and-spoke VNet topology with peering. The flow logs are set to write to a storage account and then get ingested into Log Analytics. I’m wondering if the VNet peering setup is causing permission issues that slow down the log collection process, or if there’s something wrong with our Log Analytics integration configuration.
The delay is consistent across all NSGs, so it doesn’t seem to be a problem with individual configurations. Has anyone dealt with flow log ingestion delays like this?
Check the permissions on your storage account and Log Analytics workspace. For VNet peering scenarios, you need to ensure that Network Watcher has the appropriate role assignments across all peered VNets. If Network Watcher doesn’t have ‘Network Contributor’ role on the spoke VNets, it can cause intermittent delays in log collection. Also verify that the storage account isn’t in a different region than your Log Analytics workspace-cross-region transfers add latency.
The storage account and Log Analytics workspace are in the same region. I checked the workspace metrics and we’re only using about 40% of our daily cap, so capacity shouldn’t be an issue. I’ll verify the Network Watcher permissions on the spoke VNets-that could be the problem.
We are using version 2 flow logs with traffic analytics enabled at 10-minute intervals. The storage account shows the flow log files being created regularly, but there’s a significant delay before they show up in Log Analytics queries. Could this be a Log Analytics workspace ingestion bottleneck?
I suspect the issue is with how the flow logs are being ingested from storage into Log Analytics. Are you using the built-in integration or a custom ingestion pipeline? The built-in Traffic Analytics feature processes flow logs asynchronously, and if your workspace is in a different pricing tier or has high ingestion volume from other sources, it can deprioritize flow log ingestion. Check your workspace’s daily cap and ingestion rate metrics.