Terraform state, backends and drift

What state is and why Terraform needs it

Your code says "there is a bucket called aws_s3_bucket.logs". The cloud has a bucket with the real name college-logs-prod. Terraform needs a record that links the two. That record is the state, stored by default in a file called terraform.tfstate (JSON). It holds the resource address, the real ID and the last known attributes of everything Terraform manages.

At every plan, Terraform reads the state, refreshes it by asking the cloud for the current values, and compares the result with your code. Without state, Terraform would not know which real bucket is "its" bucket, and it would try to create everything again.

Why state is sensitive

State stores attributes in plain text. That includes database passwords, private keys created by Terraform, and connection strings, even for values marked sensitive. Anyone who can read the state file can read those secrets. Treat state like a password vault.

# .gitignore for a Terraform project
.terraform/
*.tfstate
*.tfstate.*
crash.log
*.tfvars          # only if your tfvars files hold secrets; commit safe example files instead
override.tf
override.tf.json

Local versus remote backends

A backend is where state lives. The default local backend writes terraform.tfstate on your laptop. That breaks in a team: your colleague does not have your file, and if your laptop dies the state is gone. A remote backend stores state in shared, durable storage with access control and encryption.

terraform {
  backend "s3" {
    bucket       = "acme-terraform-state"
    key          = "network/prod/terraform.tfstate"
    region       = "ap-south-1"
    encrypt      = true
    use_lockfile = true   # native S3 locking in newer Terraform versions; check the docs for yours
  }
}

State locking

If two engineers run apply at the same moment, both read the same old state and both write a new one. One change is lost and the state may be corrupted. Locking prevents this: the first apply takes a lock, and the second waits or fails with "Error acquiring the state lock". Newer Terraform versions can lock with a lock file in the S3 bucket itself (use_lockfile = true). The older way used a DynamoDB table, which is now deprecated. azurerm and gcs backends lock natively, and HCP Terraform handles it for you.

Working with state safely

terraform state list                                  # every address in state
terraform state show aws_s3_bucket.logs               # stored attributes of one resource
terraform state mv aws_s3_bucket.logs aws_s3_bucket.app_logs   # you renamed it in code
terraform state rm aws_s3_bucket.legacy               # forget it; the real bucket is NOT deleted

Newer Terraform also lets you do these moves in code: a moved block records a rename, and a removed block forgets a resource. They are reviewed in a pull request like any other change, which is safer than running commands by hand.

Importing existing resources

Often a resource was created by hand before IaC. An import block (Terraform 1.5 and later) adopts it into state, so Terraform manages it from now on. terraform plan -generate-config-out=generated.tf can even write a first draft of the resource block for you to clean up.

import {
  to = aws_s3_bucket.logs
  id = "college-logs-prod"
}

Drift

Drift means the real infrastructure no longer matches the state, usually because someone changed it in the console. A normal plan detects drift and proposes to change it back to match the code. terraform plan -refresh-only shows only what changed outside Terraform, without proposing to fix it, and terraform apply -refresh-only updates state to accept reality. Then decide: either update the code to match, or let the next apply restore the code's version.

Common mistakes