VPC firewall rules block database connections over Private Service Access after subnet expansion

Cloud Platforms Google Cloud Platform (GCP)
, , , , , , , ,
1

We’re experiencing intermittent connection failures to our Cloud SQL PostgreSQL instance after expanding our VPC subnet range. The database uses Private Service Access for connectivity from our GKE cluster.

The connections were stable before the VPC subnet expansion. Now we see:


ERROR: connection timeout to 10.128.5.42:5432
Failed to establish connection after 30s

We’ve verified the Private Service Access peering is active and the allocated IP range hasn’t changed. The firewall rules appear correct, but I suspect there might be an implicit deny rule interfering with the new subnet range. Has anyone dealt with firewall rule management issues after VPC changes that affected Private Service Access configuration?

2

Thanks Sara. I checked the firewall rules and found our main database access rule only covered the original /24 subnet. The expansion added a /20 range which wasn’t included. However, I’m not sure if I should modify the existing rule or create a new one specifically for Private Service Access traffic. Also concerned about the priority ordering - we have several rules and I don’t want to accidentally break other services.

3

I’ve seen this exact issue. The problem is that VPC subnet expansion creates a new subnet object, and your firewall rules might be targeting the old subnet by name rather than by CIDR. If you’re using subnet-based targeting in your firewall rules instead of IP ranges, the new subnet won’t be covered. Switch to CIDR-based rules for more flexibility. Also, check your Cloud SQL instance’s authorized networks - though with Private Service Access this shouldn’t matter, but worth verifying the configuration is consistent.

4

Update: I found the issue. Our firewall rule was using a tag-based target rather than IP ranges, and the new subnet wasn’t properly tagged. I’ve fixed it now and connections are stable. Appreciate all the help!

5

Check if your firewall rules explicitly allow the new subnet CIDR range. When you expand VPC subnets, existing firewall rules don’t automatically update to include the new ranges. Run gcloud compute firewall-rules list --filter="network=YOUR_VPC" and verify that your ingress rules cover the expanded subnet. The Private Service Access peering itself might be fine, but the firewall layer could be blocking the new IPs.

6

Perfect! Let me provide a comprehensive solution for anyone facing similar issues with VPC subnet expansion and Private Service Access.

VPC Subnet Expansion Impact: When you expand VPC subnets, existing firewall rules don’t automatically adapt. You need to explicitly update or create new rules covering the expanded ranges.

Firewall Rule Management Steps:

  1. List current firewall rules and identify gaps:

gcloud compute firewall-rules list --filter="network=YOUR_VPC" --format="table(name,sourceRanges,allowed)"
  1. Check Private Service Access allocated ranges:

gcloud services vpc-peerings list --service=servicenetworking.googleapis.com --network=YOUR_VPC
  1. Create dedicated firewall rule for new subnet:

gcloud compute firewall-rules create allow-db-new-subnet \
  --network=YOUR_VPC \
  --allow=tcp:5432 \
  --source-ranges=10.128.0.0/20 \
  --priority=1000

Private Service Access Configuration Best Practices:

• Use CIDR-based firewall rules instead of tag-based targeting for subnet expansions - more flexible and automatic coverage

• Maintain separate firewall rules for Private Service Access traffic (priority 900-1000) vs application traffic (priority 1000-2000)

• Always verify the allocated IP range matches your firewall source ranges

• Document your firewall rule hierarchy and priorities to avoid conflicts

• Test connectivity from all subnets after VPC changes before deploying to production

Priority Guidelines:

  • Critical infrastructure (databases): 900-1000
  • Application services: 1000-2000
  • General access: 2000-3000
  • Explicit denies: 100-500 (use sparingly)

Verification: After updating firewall rules, test from multiple pods/instances in different subnets:


psql -h 10.128.5.42 -U dbuser -d production

Monitor Cloud SQL logs and VPC Flow Logs to confirm traffic is allowed. The key is maintaining explicit allow rules that cover all your subnet ranges, especially after infrastructure changes like VPC expansion.

7

For Private Service Access, you need to ensure your firewall rules allow traffic from both your application subnet AND the allocated range used for the private connection. The allocated range (typically something like 10.128.0.0/20 for the service) needs explicit ingress rules. Check your allocated ranges with gcloud services vpc-peerings list --service=servicenetworking.googleapis.com --network=YOUR_VPC. Create a dedicated firewall rule for this range with priority around 1000 to avoid conflicts. Don’t modify existing application rules - keep them separate for easier management.

8

One thing to watch out for: implicit deny rules. GCP has an implicit deny all ingress rule at the lowest priority. When you add new subnets, make sure your allow rules have proper priority (lower number = higher priority). I typically use priority 1000 for application-level rules and 900 for critical infrastructure like database access. Also verify that you don’t have any explicit deny rules that might be catching the new subnet range before your allow rules.