Version remote state with the HCP Terraform API

The Terraform state file is the source of truth for your infrastructure. Protecting and backing up this file is critical for practitioners who use Terraform in production. Remote state storage with HCP Terraform offers fail-safes for your infrastructure in the event of disaster-recovery situations and local file corruption. Although Terraform takes steps to prevent state errors, your state file can get corrupted due to partial apply operations or incorrectly running terraform import or terraform taint. Using the HCP Terraform API, you can safely download, modify, and upload your state file to an HCP Terraform workspace.

In this tutorial, you will generate a state file by deploying an AWS instance with web access using the Terraform CLI. Then, you will download your remote state file and use the Terraform API to create a new state version.

Prerequisites

For this tutorial you will need:

• The Terraform CLI 0.15.0+ installed locally.
• Access to HCP Terraform or Enterprise.
• An AWS account with credentials configured for Terraform.
• The awscli configured.
• jq installed.

Clone the example configuration

Clone the example GitHub repository.

$ git clone https://github.com/hashicorp-education/learn-tfc-state-api

Change into the repository directory.

$ cd learn-tfc-state-api

Open the main.tf file to review the configuration. The main resources in this configuration are an AWS EC2 instance and a security group with port 8080 access.

First, update your configuration with your HCP Terraform information. Update <YOUR-ORGANIZATION-NAME> with your HCP Terraform organization name.

terraform {  required_version = ">= 1.1.0"  required_providers {    aws = {      source  = "hashicorp/aws"      version = ">= 3.24.1"    }  }  cloud {    organization = "<ORGANIZATION_NAME>"    workspaces {      name = "Example-Workspace"    }  }}##...

Save your changes.

Run the terraform login subcommand, and follow the prompts to authenticate to HCP Terraform.

$ terraform loginTerraform will request an API token for app.terraform.io using your browser.If login is successful, Terraform will store the token in plain text inthe following file for use by subsequent commands:    /Users/username/.terraform.d/credentials.tfrc.jsonDo you want to proceed?  Only 'yes' will be accepted to confirm.  Enter a value:Generate a token using your browser, and copy-paste it into this prompt.Terraform will store the token in plain text in the following filefor use by subsequent commands:    /Users/rachel/.terraform.d/credentials.tfrc.jsonToken for app.terraform.io:  Enter a value:

For more detailed instructions on logging in, reference the Authenticate the CLI with HCP Terraform tutorial.

After authenticating, initialize your Terraform configuration.

$ terraform initInitializing HCP Terraform...Initializing provider plugins...- Reusing previous version of hashicorp/aws from the dependency lock file- Using previously-installed hashicorp/aws v3.37.0Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to seeany changes that are required for your infrastructure. All Terraform commandsshould now work.If you ever set or change modules or Terraform Settings, run "terraform init"again to reinitialize your working directory.

Add AWS credentials to workspace variables

Navigate to your HCP Terraform state-versioning workspace. Click on "Variables" and add your region variable as a Terraform variable. Add your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables as environment variables. Be sure to mark the AWS credentials environment variables as sensitive

Apply your configuration

In your terminal, apply your configuration. Enter yes when prompted to confirm your changes.

$ terraform applyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely.Preparing the remote apply...To view this run in a browser, visit:https://app.terraform.io/app/hashicorp-learn/state-versioning/runs/run-Lm96BJVNXkRv7dNQWaiting for the plan to start...##...Apply complete! Resources: 3 added, 0 changed, 0 destroyed.Outputs:instance_id = "i-0bcc0850e611cfd6b"public_ip = "3.143.170.167"

Verify that your state file contains your resources using terraform state list command.

$ terraform state listdata.aws_ami.ubuntuaws_instance.exampleaws_security_group.sg_webaws_security_group_rule.sg_web

In the next section, you will configure your HCP Terraform workspace.

Configure the HCP Terraform workspace

You will need your workspace ID and a new HCP Terraform API token to access your workspace's state file.

Retrieve workspace ID

In HCP Terraform, navigate to your new workspace.

In your state-versioning workspace, navigate to "Settings" then "General" and copy your workspace ID.

Create a workspace ID environment variable in your terminal, replacing <YOUR-WORKSPACE-ID> with the workspace ID you just copied.

$ export WORKSPACE_ID=<YOUR-WORKSPACE-ID>

Retrieve HCP Terraform API token

Create a new user token by clicking the icon for your user in the top right corner, then "User settings", then "Tokens". Select "Create an API token".

Name your token "state-versioning" then select "Create API token".

Copy the API token.

Create an environment variable with your token, replacing <YOUR-TFC-TOKEN> with the token you just copied.

$ export TFC_TOKEN=<YOUR-TFC-TOKEN>

Lock your workspace

Click on the lock icon to lock your workspace. Locking your workspace prevents other operations from running and potentially corrupting the state file you are going to download. Your workspace needs to be locked before you can push a new state file via API. You must lock the workspace as the same user you generated the HCP Terraform token for in the previous step.

Edit configuration tags

Run the AWS CLI to add the Org tag to your EC2 resource. Your state file does not have a record of this value.

$ aws ec2 create-tags --resources $(terraform output -raw instance_id) --tags Key=Org,Value=HashiCorp

It may take a few minutes to update your instance.

Your new Org tag is HashiCorp in AWS while your Terraform state file is not aware of this change.

Later in this tutorial, you will reconcile this difference with the Terraform state.

Download the state file

In your terminal, navigate to the helper_scripts folder.

$ cd helper_scripts

The shell scripts in this directory construct your API queries, download your remote state file for editing, and create a payload for uploading your changes.

Open the getstate.sh file to review the API query.

#!/bin/bashHTTP_RESPONSE=$(curl \     --header "Authorization: Bearer "$TFC_TOKEN"" \     --header "Content-Type: application/vnd.api+json" \     "https://app.terraform.io/api/v2/workspaces/"$WORKSPACE_ID"/current-state-version" | jq -r '.data | .attributes | ."hosted-state-download-url"')curl -o state.tfstate $HTTP_RESPONSE

If you are using Terraform Enterprise, change the URL from app.terraform.io to your personalized Terraform Enterprise domain.

This snippet uses your environment variables and authenticates to your HCP Terraform workspace to download the current remote state file. The hosted-state-download-url contains the URL that hosts your remote state file.

In your terminal, run the getstate.sh script.

$ ./getstate.sh  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100  1995  100  1995    0     0   8711      0 --:--:-- --:--:-- --:--:--  8711  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100  8941    0  8941    0     0  33486      0 --:--:-- --:--:-- --:--:-- 33486

Verify your query downloaded the state file and open state.tfstate.

$ cat state.tfstate{  "version": 4,  "terraform_version": "0.15.1",  "serial": 0,  "lineage": "acd79b18-a1a9-3e66-6325-8686bbc77566",##...

Modify and create the state payload

Now that you have the remote state downloaded, open the state.tfstate file in your file editor. Increment your serial number by one then save the file.

 {   "version": 4,   "terraform_version": "0.15.1",-  "serial": 0,+  "serial": 1,   "lineage": "acd79b18-a1a9-3e66-6325-8686bbc77566",   "outputs": {     "instance_id": {       "value": "i-01b237003b468184d",       "type": "string"     },     "public_ip": {       "value": "3.19.246.180",       "type": "string"     }   },## ... }

This is your new state version number. Terraform uses the serial to keep track of the changes made in each new state file and uses it to make sure your operations run against the correct known state file in the HCP Terraform workspace. In standard operations, Terraform updates the serial for you automatically. However, since you're pushing a new state version, you need to manually increment this value.

Edit your instance tags to include the new Org tag. Search for terraform-learn-state-versioning in the state.tfstate file. Add a comma at the end of the Name tag. Terraform parses the JSON-formatting state file and adds elements to your resource records.

##...            "tags": {-              "Name": "terraform-learn-state-versioning"+              "Name": "terraform-learn-state-versioning",+              "Org": "HashiCorp"            },            "tenancy": "default",##...

Save your file.

Now, you will construct your current state payload. Select the tab for your operating system for specific instructions.

#!/bin/bashserial=$(cat state.tfstate | jq '.serial')md5_compute=$(md5 -q state.tfstate)md5=\"$md5_compute\"lineage=$(cat state.tfstate | jq '.lineage')base64_encode=$(base64 state.tfstate)state=\"$base64_encode\"echo "{   \"data\": {   \"type\": \"state-versions\",     \"attributes\": {       \"serial\": $serial,       \"md5\": "$md5",       \"lineage\": "$lineage",       \"state\": "$state"     }   } }" > payload.json

$ ./createpayload.sh

$ cat payload.json{   "data": {   "type": "state-versions",     "attributes": {       "serial": 1,       "md5": "f51e44f5672b40725e283c1bd5556752",       "lineage": "939c75bf-0872-6277-d273-3df86f7ac679",       "state": "ewogICJ2ZXJzaW9uIjogNCwKICAidGVyc## …}

#!/bin/bashserial=$(cat state.tfstate | jq '.serial')md5_compute=$(md5sum state.tfstate | awk '{print $1}')md5=\"$md5_compute\"lineage=$(cat state.tfstate | jq '.lineage')base64_encode=$(base64 state.tfstate)state=\"$base64_encode\"echo "{   \"data\": {   \"type\": \"state-versions\",     \"attributes\": {       \"serial\": $serial,       \"md5\": "$md5",       \"lineage\": "$lineage",       \"state\": "$state"     }   } }" > payload.json

$ ./linux-createpayload.sh

$ cat payload.json{   "data": {   "type": "state-versions",     "attributes": {       "serial": 1,       "md5": "f51e44f5672b40725e283c1bd5556752",       "lineage": "939c75bf-0872-6277-d273-3df86f7ac679",       "state": "ewogICJ2ZXJzaW9uIjogNCwKICAidGVyc## ...}

Upload the new state file to HCP Terraform

Now that you have a JSON payload with your encrypted state file, upload the new state file to HCP Terraform.

In your helper_scripts directory, open the uploadstate.sh file.

#!/bin/bash HTTP_POST=$(curl \     --header "Authorization: Bearer "$TFC_TOKEN"" \     --header "Content-Type: application/vnd.api+json" \     --request POST \     --data @payload.json \     "https://app.terraform.io/api/v2/workspaces/"$WORKSPACE_ID"/state-versions") echo $HTTP_POST

This API query uses the --data flag to upload the payload.json file to your workspace.

Run the uploadstate.sh script.

$ ./uploadstate.sh##...  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100 13247  100  1107  100 12140   1392  15270 --:--:-- --:--:-- --:--:-- 16641{"data":{"id":"sv-VBU3yeG5XMLgK5K6","type":"state-versions","attributes":{"created-at":"2021-04-08T21:59:29.954Z","size":null,"hosted-state-download-url":"https://archivist.terraform.io/v1/object/dmF1bHQ6djE6ZXFONmlaYlVhcHVNOE9WWENZZkljdmJz..."##...

In your HCP Terraform workspace, navigate to your "States" tab and select the most recent state.

In the "Changes in this version" section, confirm your new state file contains a new serial number and tag reference.

Unlock your workspace by clicking on the lock icon and confirming the unlock.

Update your configuration

Because you updated your resource outside of the Terraform workflow with an additional tag, you must update the configuration with the updated resource and run a terraform apply to maintain parity with your state file. Terraform will perform the apply, but will not make any resource changes. This is a "no-operation" or "no-op" apply.

Change into the root directory.

$ cd ..

Open the main.tf file and update your instance tag.

 resource "aws_instance" "example" {   ami                    = data.aws_ami.ubuntu.id   instance_type          = "t2.micro"   vpc_security_group_ids = [aws_security_group.sg_web.id]   user_data              = <<-EOF              #!/bin/bash              apt-get update              apt-get install -y apache2              sed -i -e 's/80/8080/' /etc/apache2/ports.conf              echo "Hello World" > /var/www/html/index.html              systemctl restart apache2              EOF   tags = {     Name = "terraform-learn-state-versioning"+    Org  = "HashiCorp"   } }

Run terraform apply to consolidate your configuration with your remote state. This is a "no-op" apply.

$ terraform applyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely.Preparing the remote apply...To view this run in a browser, visit:https://app.terraform.io/app/hashicorp-learn/state-versioning/runs/run-gGFN9Tdd6cGuaqyNTerraform v0.15.1on linux_amd64Configuring remote state backend...Initializing Terraform configuration...aws_security_group.sg_web: Refreshing state... [id=sg-08df7f3f965e47a6a]aws_security_group_rule.sg_web: Refreshing state... [id=sgrule-4136193275]aws_instance.example: Refreshing state... [id=i-0a8f43386c25bc073]No changes. Infrastructure is up-to-date.This means that Terraform did not detect any differences between yourconfiguration and the remote system(s). As a result, there are no actions totake.

Now that your modified state file matches your infrastructure and configuration, delete the local version of your state file.

$ rm helper_scripts/state.tfstate

Your payload.json file also contains an encrypted version of your state. Delete your payload.json file.

$ rm helper_scripts/payload.json

Destroy your infrastructure

In HCP Terraform, destroy your remote workspace.

Navigate to "Settings" > "Destruction and Deletion".

At the bottom of the page, select "Queue Destroy Plan" and confirm.

Next, delete your workspace from HCP Terraform. At the bottom of the page, select "Delete workspace" and confirm.

Next steps

In this tutorial, you learned how to use the HCP Terraform API to interact with and update your HCP Terraform workspace's state. First, you created infrastructure in HCP Terraform. Then, you downloaded your current HCP Terraform state file and safely modified and versioned your state file. Finally, you uploaded your versioned and updated state file to HCP Terraform.

For more information about the HCP Terraform API or Terraform state, review the following resources:

• HCP Terraform API documentation.
• HCP Terraform Fundamentals tutorials.
• Terraform State tutorials.