- TechOps Examples
- Posts
- Steps to Prevent and Recover Lost Terraform State Files
Steps to Prevent and Recover Lost Terraform State Files
Govardhana M K
January 16, 2025
In partnership with
TechOps Examples
Hey — It's Govardhana MK 👋
Along with a use case deep dive, we identify the remote job opportunities, top news, tools, and articles in the TechOps industry.
👋 Before we begin... a big thank you to today's sponsor MAVEN AGI
Drowning In Support Tickets? Maven AGI is here to help.
Maven AGI platform simplifies customer service by unifying systems, improving with every interaction, and automating up to 93% of responses. Seamlessly integrated with 50+ tools like Salesforce, Freshdesk, and Zendesk, Maven can deploy AI agents across multiple channels—text, email, web, voice, and apps—within days. Companies like Tripadvisor, ClickUp, and Rho slash response times by 60%, ensuring quicker support and exceptional customer satisfaction. Don’t let support tickets slow you down
IN TODAY'S EDITION
🧠 Use Case
Steps to Prevent and Recover Lost Terraform State Files
🚀 Top News
👀 Remote Jobs
GoGuardian is hiring a Site Reliability Engineer
Remote Location: India
Alpaca is hiring a Senior Software Engineer - Platform
Remote Location: Worldwide
📚️ Resources
📢 Reddit Threads
🛠️ TOOL OF THE DAY
gitdiagram - Turn any GitHub repository into an interactive diagram for visualization.
🧠 USE CASE
Steps to Prevent and Recover Lost Terraform State Files
In my recent LinkedIn post, I wrote:
Among the responses, Rob Larsen shared this experience
This pain is all too familiar for many of us in the DevOps world, and I couldn't walk away with just a comment.
Here's my take on it...
What Can You Do to Prevent This?
1. Enable State File Versioning in Remote Backends
Use an S3 bucket with versioning enabled.
Example S3 backend configuration:
terraform {
backend "s3" {
bucket = "tf-state-bucket"
key = "path/to/terraform.tfstate"
region = "us-east-1"
encrypt = true
dynamodb_table = "terraform-lock-table"
}
}
Versioning ensures that you can recover from accidental deletions or corruptions by restoring a previous state.
2. Lock the State File
Always use locking mechanisms (e.g., DynamoDB for S3 backend). This prevents simultaneous writes and ensures data integrity.
Add dynamodb_table to your backend config for distributed locks.
3. Take Regular Backups
Automate state file backups using lifecycle policies or scripts.
aws s3 cp s3://tf-state-bucket/path/to/terraform.tfstate /
s3://backup-bucket/path/to/backup-$(date +%Y-%m-%d).tfstate
4. Use Workspaces Wisely
Avoid overloading a single workspace with multiple environments (e.g., dev, prod).
Keep state files separated for better manageability.
terraform workspace new dev
terraform workspace new prod
5. Access Control and Least Privilege
Limit who can delete state files. Use IAM policies to enforce.
{
"Effect": "Deny",
"Action": "s3:DeleteObject",
"Resource": "arn:aws:s3:::tf-state-bucket/*",
"Condition": {
"BoolIfExists": {"aws:MultiFactorAuthPresent": false}
}
}
Despite the best intentions and efforts, things can still go wrong, as highlighted in Rob Larsen's comment.
How to Approach If This Happens
1. Restore a Previous Version from Backend
aws s3api copy-object
--copy-source tf-state-bucket/path/to/terraform.tfstate?versionId=123456789
--bucket tf-state-bucket
--key path/to/terraform.tfstate
2. Recreate the State File with Terraformer (If Completely Lost)
terraformer import aws --resources=ec2,s3 --regions=us-east-1 --profile=default
terraform plan -refresh-only
terraform apply
aws: Specifies the provider.
--resources=ec2,s3: Defines the resource types to import (e.g., EC2 instances and S3 buckets).
--regions=us-east-1: Targets the AWS region.
--profile=default: (Optional) Specifies the AWS CLI profile to use.
Terraformer automates resource discovery and imports them into Terraform state while generating the corresponding .tf configuration files.
Validate and apply carefully.