Infrastructure as Code config drift causing supplier management module failures

QMS Sparta Systems TrackWise
, , , , , , , ,
1

We’re experiencing recurring failures in our Trackwise 9.0 supplier management module due to infrastructure drift in our AWS deployment. Our Terraform configurations manage the supplier data infrastructure, but we’re seeing state inconsistencies that break supplier record synchronization.

The main issue is our Terraform state locking isn’t properly configured, allowing concurrent modifications from different team members. We’ve also noticed our CI/CD pipeline lacks automated drift detection, so changes go unnoticed until supplier workflows fail.

terraform {
  backend "s3" {
    bucket = "trackwise-tfstate"
    key    = "supplier-mgmt/terraform.tfstate"
  }
}

Our dev and ops teams work in silos, and there’s no formal approval workflow for infrastructure changes. When drift occurs, supplier onboarding processes halt, blocking critical vendor qualifications. Has anyone implemented proper state locking and drift detection for Trackwise infrastructure?

2

Check if your RDS instances for supplier data are being modified outside Terraform. We found that automated AWS maintenance windows were changing instance parameters, causing drift. Tag all your Trackwise resources with ‘ManagedBy=Terraform’ and set up AWS Config rules to alert on untagged modifications. Also verify your IAM policies restrict manual changes to Terraform-managed resources.

3

For supplier management specifically, you need an approval workflow before any infrastructure changes. We use Atlantis with custom workflows that require approval from both a DevOps engineer and a QMS admin before terraform apply runs. This is especially important for GxP environments where infrastructure changes need audit trails.

4

The drift detection piece is critical here. We implemented Terraform Cloud’s drift detection feature that runs scheduled checks every 6 hours against our Trackwise infrastructure. It catches manual AWS console changes or CloudFormation modifications that bypass Terraform entirely. You can also use terraform plan in your CI pipeline as a pre-deployment check. The key is making drift visible before it impacts production supplier workflows. Have you considered implementing policy-as-code with Sentinel or OPA to enforce infrastructure standards?

5

You need a comprehensive approach addressing all your pain points. Here’s what worked for our Trackwise 9.0 supplier management infrastructure:

State Locking Configuration: First, enable proper state locking with DynamoDB:

terraform {
  backend "s3" {
    bucket         = "trackwise-tfstate"
    key            = "supplier-mgmt/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-lock"
    encrypt        = true
  }
}

Create the DynamoDB table with LockID as the primary key. Enable S3 versioning and MFA delete protection for your state bucket.

Automated Drift Detection: Implement drift detection in your CI/CD pipeline using GitLab CI or GitHub Actions. We run terraform plan -detailed-exitcode every 4 hours as a scheduled job. Exit code 2 indicates drift, which triggers alerts to our monitoring channels. Integrate with Datadog or PagerDuty for immediate notifications.

For continuous monitoring, deploy Driftctl or use Terraform Cloud’s drift detection feature. These tools compare your actual AWS infrastructure against your Terraform state and report discrepancies in real-time.

Cross-Functional Collaboration: Establish a unified infrastructure team with representatives from dev, ops, and QMS compliance. Create a shared responsibility model where:

  • DevOps owns the Terraform modules and CI/CD pipeline
  • QMS team reviews changes for compliance impact
  • Operations approves changes affecting production supplier workflows

Use Slack or Teams with automated drift notifications. We created an #infrastructure-alerts channel that receives Terraform drift reports, AWS Config rule violations, and deployment status updates.

Infrastructure Change Approval Workflow: Implement Atlantis or Terraform Cloud with required approvals. Our workflow requires:

  1. Developer creates Terraform changes in a feature branch
  2. Pull request triggers automated `terraform plan
  3. Plan output posts to PR for review
  4. Two approvals required: one DevOps, one QMS compliance
  5. Only after approvals can terraform apply execute
  6. All changes logged to audit trail with user attribution

For supplier management specifically, add custom validation rules that check:

  • Database backup configurations remain compliant
  • Encryption settings meet GxP requirements
  • Network security groups don’t expose supplier data

This systematic approach eliminated our drift issues and reduced supplier module failures from weekly occurrences to zero over six months. The approval workflow also satisfies FDA audit requirements for infrastructure change control.