Authenticate providers with Vault-backed dynamic credentials

You can use HashiCorp Vault to authenticate infrastructure provisioning in HCP Terraform using short-lived, strictly-scoped credentials. These Vault-backed dynamic credentials prevent you from having to store long-lived cloud provider credentials in HCP Terraform. Once you establish a trust relationship between HCP Terraform and Vault, you can use Vault's secrets engine to manage credentials for your cloud provider. HCP Terraform will then request per-operation credentials from Vault to provision your infrastructure.

Vault-backed dynamic credentials are unique and short-lived for each Terraform run, which limits the blast radius in the event of a security breach. Dynamic credentials also give you fine-grained control over the resources that each of your HCP Terraform projects and workspaces can manage by using your cloud providers' access management controls. You can use dynamic credentials to provision infrastructure on cloud providers including AWS, Google Cloud Platform, Azure, and Vault. Vault-backed dynamic credentials let you consolidate your secrets management with Vault and take advantage of its security controls and features such as integration with identity providers.

After you configure Vault-backed dynamic credentials for a workspace or project, HCP Terraform begins each run by requesting credentials from Vault, passing details about the workload, including your organization and workspace name. Vault then authenticates with your cloud provider, and sends the temporary credentials to HCP Terraform. HCP Terraform uses those credentials to manage your resources for the run. This workflow is based on the OpenID Connect protocol (OIDC), an open source standard for verifying identity across different systems.

In this tutorial, you will set up a Vault secrets engine for AWS, establish the trust relationship between Vault and HCP Terraform, and configure a workspace to use Vault to provision dynamic credentials. Finally, you will use dynamic credentials to provision infrastructure in that workspace.

If you do not currently use Vault to manage your credentials, you can configure HCP Terraform to use dynamic provider credentials directly with your cloud provider instead. Refer to the Authenticate Providers with Dynamic Credentials tutorial for details.

Prerequisites

This tutorial assumes that you are familiar with the Terraform, HCP Terraform, and Vault workflows. If you are new to Terraform, complete the Get Started collection first. If you are new to HCP Terraform, complete the HCP Terraform Get Started collection first. If you are new to Vault, complete the Vault foundations collection.

For this tutorial, you will need:

• Terraform v1.2+ installed locally.
• An HCP Terraform account and organization.
• A Vault cluster. We recommend you use a non-production HCP Vault cluster to complete this tutorial.
• An AWS account with local credentials configured for use with Terraform.

Clone example repository

Clone the example repository, which contains Terraform configuration to create the trust relationship between AWS and HCP Terraform.

$ git clone https://github.com/hashicorp-education/learn-terraform-vault-backed-dynamic-credentials

Change into the example repository directory.

$ cd learn-terraform-vault-backed-dynamic-credentials

Change into the trust subdirectory for AWS.

$ cd aws/trust

Configure trust with your cloud provider

To use Vault-backed dynamic credentials, you must first configure Vault's secrets engine, then establish a trust relationship between Vault and HCP Terraform. You must also create the roles and policies that define which resources and services the dynamic credentials can manage, and the HCP Terraform workspace that will provision your resources.

In this tutorial, you will create the trust relationship and TFC workspace by using the CLI-driven workflow for HCP Terraform.

Review cloud provider configuration

Open aws.tf and review the configuration for your trust relationship.

The configuration first configures the AWS provider, and defines an IAM user that Vault's secrets engine will use to provision your dynamic credentials.

provider "aws" {  region = var.aws_region}resource "aws_iam_user" "secrets_engine" {  name = "hcp-vault-secrets-engine"}resource "aws_iam_access_key" "secrets_engine_credentials" {  user = aws_iam_user.secrets_engine.name}

Next, the configuration defines an IAM user policy and role for the hcp-vault-secrets-engine user to assume.

resource "aws_iam_user_policy" "vault_secrets_engine_generate_credentials" {  name = "hcp-vault-secrets-engine-policy"  user = aws_iam_user.secrets_engine.name  policy = jsonencode({    Version = "2012-10-17"    Statement = [      {        Action = [          "sts:AssumeRole",        ]        Effect   = "Allow"        Resource = "${aws_iam_role.tfc_role.arn}"      },    ]  })}resource "aws_iam_role" "tfc_role" {  name = "tfc-role"  assume_role_policy = jsonencode(    {      Statement = [        {          Action    = "sts:AssumeRole"          Condition = {}          Effect    = "Allow"          Principal = {            AWS = aws_iam_user.secrets_engine.arn          }        },      ]      Version = "2012-10-17"    }  )}

Finally, the configuration defines an IAM policy and attaches it to the role. The dynamic credentials that Vault provides will assume the role to provision the resources allowed by the policy.

resource "aws_iam_policy" "tfc_policy" {  name        = "tfc-policy"  description = "TFC run policy"  policy = jsonencode({    Statement = [      {        Action = [          "ec2:*",        ]        Effect   = "Allow"        Resource = "*"      },    ]    Version = "2012-10-17"  })}resource "aws_iam_role_policy_attachment" "tfc_policy_attachment" {  role       = aws_iam_role.tfc_role.name  policy_arn = aws_iam_policy.tfc_policy.arn}

This policy allows all operations on EC2 instances. IAM policies give you fine-grained control over the permissions granted to your dynamic credentials. You can configure the policies associated with your trust relationships to be as restrictive or permissive as you need.

Review Vault configuration

Open vault.tf and review the configuration for your trust relationship.

First, the configuration enables Vault's AWS secrets engine and configures it to authenticate with the IAM user and credentials defined in aws.tf.

provider "vault" {  address = var.vault_url}resource "vault_aws_secret_backend" "aws_secret_backend" {  namespace = var.vault_namespace  path      = "aws"  access_key = aws_iam_access_key.secrets_engine_credentials.id  secret_key = aws_iam_access_key.secrets_engine_credentials.secret}resource "vault_aws_secret_backend_role" "aws_secret_backend_role" {  backend         = vault_aws_secret_backend.aws_secret_backend.path  name            = var.aws_secret_backend_role_name  credential_type = "assumed_role"  role_arns = [aws_iam_role.tfc_role.arn]}

The Vault secrets engine will use these credentials to access the role defined in the previous section.

Next, the configuration configures a JWT authentication backend, which implements the OpenID standard.

resource "vault_jwt_auth_backend" "tfc_jwt" {  path               = var.jwt_backend_path  type               = "jwt"  oidc_discovery_url = "https://${var.tfc_hostname}"  bound_issuer       = "https://${var.tfc_hostname}"}resource "vault_jwt_auth_backend_role" "tfc_role" {  namespace      = var.vault_namespace  backend        = vault_jwt_auth_backend.tfc_jwt.path  role_name      = "tfc-role"  token_policies = [vault_policy.tfc_policy.name]  bound_audiences   = [var.tfc_vault_audience]  bound_claims_type = "glob"  bound_claims = {    sub = "organization:${var.tfc_organization_name}:project:${var.tfc_project_name}:workspace:${var.tfc_workspace_name}:run_phase:*"  }  user_claim = "terraform_full_workspace"  role_type  = "jwt"  token_ttl  = 1200}

This configuration ensures that Vault will check that requests for dynamic credentials satisfy the following OIDC claims:

• aud: The intended audience of the token, configured by the bound_audiences attribute, matches the unique case-sensitive string for your cloud provider. This ensures that the token is only valid for your configured provider. You may want to customize the audience for advanced use cases, such as using multiple sets of dynamic credentials with a single cloud provider. For this tutorial, use the default value for the audience.

• sub: The subject of the token, defined in the bound_claims attribute, matches the configured HCP Terraform organization name, project, and workspace. This ensures that your cloud provider only authenticates requests for the specified projects and workspaces in your organization, and lets you scope permissions on a per-project or per-workspace basis.

The sub claim includes four parts separated by colons (:):

The example configuration uses the bound claims type glob and the glob character (*) to allow any workflow phase. When you define your trust relationships, you can also use * to allow requests from any workspace within a named project, or any project in your organization. Depending on your organization's needs, you can configure a single trust relationship for each cloud provider, or separate trust relationships for each project or workspace.

For each run, after Vault verifies that the request is signed by HCP Terraform with the provided TLS certificate, HCP Terraform will provide Vault with a Terraform Workload Identity (TWI) token that includes information about the run, such as the organization, project, and workspace. Vault will then use HCP Terraform's OIDC metadata and public signing key to verify that the token was generated by HCP Terraform and the information it contains has not been tampered with. Once verified, if the token matches the conditions for your trust relationship, Vault will dynamically generate credentials for the configured role. HCP Terraform will then use those dynamic credentials to provision your resources.

Finally, the configuration defines the policy that allows Vault-backed dynamic credentials access to the AWS secrets engine.

resource "vault_policy" "tfc_policy" {  name = "tfc-policy"  policy = <<EOT# Allow tokens to query themselvespath "auth/token/lookup-self" {  capabilities = ["read"]}# Allow tokens to renew themselvespath "auth/token/renew-self" {    capabilities = ["update"]}# Allow tokens to revoke themselvespath "auth/token/revoke-self" {    capabilities = ["update"]}# Allow Access to AWS Secrets Enginepath "aws/sts/${var.aws_secret_backend_role_name}" {  capabilities = [ "read" ]}EOT}

Configure HCP Terraform workspace

Open tfe.tf and review the configuration for your workspace.

First, the configuration defines the TFE provider and an HCP Terraform workspace. The organization and workspace names match those you defined in the sub claim for your trust relationship.

provider "tfe" {  hostname = var.tfc_hostname}resource "tfe_workspace" "trust_workspace" {  name         = var.tfc_workspace_name  organization = var.tfc_organization_name}

Next, the configuration enables Vault-backed dynamic credentials for the workspace by setting the workspace environment variables required to enable and configure the trust relationship.

resource "tfe_variable" "enable_vault_provider_auth" {  workspace_id = tfe_workspace.trust_workspace.id  key      = "TFC_VAULT_PROVIDER_AUTH"  value    = "true"  category = "env"  description = "Enable the Workload Identity integration for Vault."}resource "tfe_variable" "tfc_vault_addr" {  workspace_id = tfe_workspace.trust_workspace.id  key       = "TFC_VAULT_ADDR"  value     = var.vault_url  category  = "env"  sensitive = true  description = "The address of the Vault instance runs will access."}resource "tfe_variable" "tfc_vault_role" {  workspace_id = tfe_workspace.trust_workspace.id  key      = "TFC_VAULT_RUN_ROLE"  value    = vault_jwt_auth_backend_role.tfc_role.role_name  category = "env"  description = "The Vault role runs will use to authenticate."}resource "tfe_variable" "tfc_vault_namespace" {  workspace_id = tfe_workspace.trust_workspace.id  key      = "TFC_VAULT_NAMESPACE"  value    = var.vault_namespace  category = "env"  description = "Namespace that contains the AWS Secrets Engine."}resource "tfe_variable" "enable_aws_provider_auth" {  workspace_id = tfe_workspace.trust_workspace.id  key      = "TFC_VAULT_BACKED_AWS_AUTH"  value    = "true"  category = "env"  description = "Enable the Vault Secrets Engine integration for AWS."}resource "tfe_variable" "tfc_aws_mount_path" {  workspace_id = tfe_workspace.trust_workspace.id  key      = "TFC_VAULT_BACKED_AWS_MOUNT_PATH"  value    = "aws"  category = "env"  description = "Path to where the AWS Secrets Engine is mounted in Vault."}resource "tfe_variable" "tfc_aws_auth_type" {  workspace_id = tfe_workspace.trust_workspace.id  key      = "TFC_VAULT_BACKED_AWS_AUTH_TYPE"  value    = vault_aws_secret_backend_role.aws_secret_backend_role.credential_type  category = "env"  description = "Role to assume via the AWS Secrets Engine in Vault."}resource "tfe_variable" "tfc_aws_run_role_arn" {  workspace_id = tfe_workspace.trust_workspace.id  key      = "TFC_VAULT_BACKED_AWS_RUN_ROLE_ARN"  value    = aws_iam_role.tfc_role.arn  category = "env"  description = "ARN of the AWS IAM Role the run will assume."}resource "tfe_variable" "tfc_aws_run_vault_role" {  workspace_id = tfe_workspace.trust_workspace.id  key      = "TFC_VAULT_BACKED_AWS_RUN_VAULT_ROLE"  value    = vault_aws_secret_backend_role.aws_secret_backend_role.name  category = "env"  description = "Name of the Role in Vault to assume via the AWS Secrets Engine."}

The TFC_VAULT_PROVIDER_AUTH variable configures the workspace to use dynamic credentials for the Vault provider, and the next three variables identify your Vault instance and the role that Vault will generate dynamic credentials for.

The next five variables configure the Vault secrets engine for AWS to manage your dynamic credentials and provide them to HCP Terraform.

Configure organization and workspace name

In the trust directory for your cloud provider, copy the example .tfvars file to create a new variables file that configures your organization and workspace name.

$ cp terraform.tfvars.example terraform.tfvars

Open terraform.tfvars and configure the variables for the trust relationship. The configuration will use the organization and workspace name to define the OIDC subject for your trust relationship's policy.

aws_secret_backend_role_name = "edu-app"tfc_hostname        = "app.terraform.io"tfc_organization_name = "<YOUR_ORG>"tfc_workspace_name  = "vault-backed-aws-auth"vault_url = "<YOUR_VAULT_CLUSTER>"

Replace <YOUR_ORG> with your HCP Terraform organization name, and <YOUR_VAULT_CLUSTER> with your Vault cluster's public URL. If you are using HCP Vault to complete this tutorial, navigate to your cluster, find the Cluster URLs tool, and click Public to copy it to your clipboard.

Initialize configuration

Set the VAULT_TOKEN environment variable to authenticate with your Vault cluster. If you are using HCP Vault, navigate to your cluster, find the New admin token tool and click Generate token to copy the token to your clipboard.

$ export VAULT_TOKEN=

Initialize your Terraform configuration.

$ terraform initInitializing the backend...Initializing provider plugins...- Reusing previous version of hashicorp/aws from the dependency lock file- Reusing previous version of hashicorp/vault from the dependency lock file- Reusing previous version of hashicorp/tfe from the dependency lock file- Installing hashicorp/aws v4.62.0...- Installed hashicorp/aws v4.62.0 (signed by HashiCorp)- Installing hashicorp/vault v3.14.0...- Installed hashicorp/vault v3.14.0 (signed by HashiCorp)- Installing hashicorp/tfe v0.43.0...- Installed hashicorp/tfe v0.43.0 (signed by HashiCorp)Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to seeany changes that are required for your infrastructure. All Terraform commandsshould now work.If you ever set or change modules or backend configuration for Terraform,rerun this command to reinitialize your working directory. If you forget, othercommands will detect it and remind you to do so if necessary.

Apply configuration

Apply this configuration to create your trust relationship. Respond to the confirmation prompt with a yes.

$ terraform applyTerraform used the selected providers to generate the following execution plan.Resource actions are indicated with the following symbols:  + createTerraform will perform the following actions:  # aws_iam_access_key.secrets_engine_credentials will be created  + resource "aws_iam_access_key" "secrets_engine_credentials" {##...Plan: 21 to add, 0 to change, 0 to destroy.Changes to Outputs:  + openid_claims   = {    + sub = "organization:hashicorp-learn:project:Default Project:workspace:vault-backed-aws-auth:run_phase:*"    }  + tfc_workspace_url = (known after apply)Do you want to perform these actions?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.  Enter a value: yesvault_jwt_auth_backend.tfc_jwt: Creating...vault_policy.tfc_policy: Creating...vault_policy.tfc_policy: Creation complete after 1s [id=tfc-policy]aws_iam_user.secrets_engine: Creating...##...Apply complete! Resources: 21 added, 0 changed, 0 destroyed.Outputs:openid_claims = tomap({  "sub" = "organization:<YOUR_ORG>:project:Default Project:workspace:vault-backed-aws-auth:run_phase:*"})tfc_workspace_name = "vault-backed-aws-auth"tfc_workspace_url = "https://app.terraform.io/app/<YOUR_ORG>/workspaces/vault-backed-aws-auth"

HCP Terraform can now use this trust relationship to request dynamic credentials from Vault when it provisions infrastructure for the learn-terraform-dynamic-credentials workspace in your default project.

Provision infrastructure with dynamic credentials

Now you will use HCP Terraform to provision infrastructure with Vault-backed dynamic credentials using the trust relationship and workspace you provisioned in the last section.

First, navigate to the infra directory and review the example configuration.

cd ../infra

This configuration defines a single EC2 instance in the us-east-2 region.

provider "aws" {  region = var.aws_region}data "aws_ami" "amazon_linux" {  most_recent = true  owners      = ["amazon"]  filter {    name   = "name"    values = ["amzn2-ami-hvm-*-x86_64-gp2"]  }}resource "aws_instance" "web" {  ami           = data.aws_ami.amazon_linux.id  instance_type = var.instance_type  user_data = <<-EOF    #!/bin/bash    sudo yum update -y    sudo yum install httpd -y    sudo systemctl enable httpd    sudo systemctl start httpd    echo "<html><body><div>Hello, world!</div></body></html>" > /var/www/html/index.html    EOF  tags = var.tags}

Initialize configuration

Set the TF_CLOUD_ORGANIZATION environment variable to your HCP Terraform organization name. This will configure your HCP Terraform integration.

$ export TF_CLOUD_ORGANIZATION=

Now, initialize the configuration.

$ terraform initInitializing HCP Terraform...Initializing provider plugins...- Reusing previous version of hashicorp/aws from the dependency lock file- Installing hashicorp/aws v4.62.0...- Installed hashicorp/aws v4.62.0 (signed by HashiCorp)HCP Terraform has been successfully initialized!You may now begin working with HCP Terraform. Try running "terraform plan" tosee any changes that are required for your infrastructure.If you ever set or change modules or Terraform Settings, run "terraform init"again to reinitialize your working directory.

Apply configuration

In the last section, you configured the trust relationship and the vault-backed-aws-auth workspace with the required variables to use it. Your Vault cluster will trust requests from app.terraform.io that use your organization and the vault-backed-aws-auth workspace.

Apply the configuration to provision your infrastructure using the dynamic credentials.

HCP Terraform will request dynamic credentials from Vault, and use them to perform a speculative plan. Once the plan is complete, respond to the confirmation prompt with a yes to apply your configuration. HCP Terraform will request another set of dynamic credentials and use them for the apply step.

$ terraform applyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely.Preparing the remote apply...To view this run in a browser, visit:https://app.terraform.io/app/<YOUR_ORG>/vault-backed-aws-auth/runs/run-r8bMqNCbjGvhiN1nWaiting for the plan to start...Terraform v1.4.5on linux_amd64Initializing plugins and modules…###...data.aws_ami.amazon_linux: Refreshing...data.aws_ami.amazon_linux: Refresh complete after 0s [id=ami-08c50cb06459e56a4]Terraform used the selected providers to generate the following execution plan.Resource actions are indicated with the following symbols:  + createTerraform will perform the following actions:  # aws_instance.web will be created  + resource "aws_instance" "web" {##...Plan: 1 to add, 0 to change, 0 to destroy.Changes to Outputs:  + domain_name = (known after apply)Do you want to perform these actions in workspace "vault-backed-aws-auth"?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.  Enter a value: yesaws_instance.web: Creating...aws_instance.web: Still creating... [10s elapsed]##...Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Outputs:domain_name = "ec2-52-15-48-71.us-east-2.compute.amazonaws.com"

You successfully provisioned infrastructure with HCP Terraform using dynamic credentials instead of long-lived credentials. Since you do not need to store the credentials themselves in HCP Terraform, this reduces the potential for accidental exposure. Defining the role and permissions for the dynamic credentials lets you specify exactly which operations can use these keys. By using Vault-backed dynamic credentials, you can consolidate secrets management in Vault, and take advantage of its features and integrations.

In your organization, different teams may be responsible for managing trust relationships and deploying infrastructure. For example, a security team could configure and manage the workspaces that define trust relationships, while infrastructure teams configure and manage downstream workspaces that use the trust relationships from the security team. When different teams manage trust and infrastructure, consider how you will coordinate changes to maintain scoped trust relationships while still enabling the downstream teams that rely on them.

Remember that trust relationships reference your organization, project, and workspace names, so changing any of those values within HCP Terraform can potentially break your trust relationships.

Clean up your infrastructure

Remove the infrastructure and trust relationship that you created in this tutorial.

Destroy infrastructure

First, remove the infrastructure you created in this tutorial. In the aws/infra directory run terraform destroy, which will also request dynamic credentials from Vault to remove your instance.

$ terraform destroyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely.Preparing the remote apply...To view this run in a browser, visit:https://app.terraform.io/app/<YOUR_ORG>/vault-backed-aws-auth/runs/run-xSVFzQJofYUjXQVq##...Plan: 0 to add, 0 to change, 1 to destroy.Changes to Outputs:  - domain_name = "ec2-52-15-48-71.us-east-2.compute.amazonaws.com" -> nullDo you really want to destroy all resources in workspace "vault-backed-aws-auth"?  Terraform will destroy all your managed infrastructure, as shown above.  There is no undo. Only 'yes' will be accepted to confirm.  Enter a value: yesaws_instance.web: Destroying... [id=i-08a4cdaab3a518085]aws_instance.web: Still destroying... [10s elapsed]aws_instance.web: Still destroying... [20s elapsed]aws_instance.web: Destruction complete after 30sApply complete! Resources: 0 added, 0 changed, 1 destroyed.

Destroy trust relationship

Destroy the trust relationship you created in this tutorial.

Navigate to the trust directory.

cd ../trust

Use Terraform to destroy the trust relationship, Vault secret engine configuration, and workspace. Respond to the confirmation prompt with a yes.

$ terraform destroyaws_iam_policy.tfc_policy: Refreshing state... [id=arn:aws:iam::841397984957:policy/tfc-policy]aws_iam_user.secrets_engine: Refreshing state... [id=hcp-vault-secrets-engine]vault_policy.tfc_policy: Refreshing state... [id=tfc-policy]vault_jwt_auth_backend.tfc_jwt: Refreshing state... [id=jwt]##...Plan: 0 to add, 0 to change, 21 to destroy.Changes to Outputs:  - openid_claims   = {    - sub = "organization:<YOUR_ORG>:project:Default Project:workspace:vault-backed-aws-auth:run_phase:*"    } -> null  - tfc_workspace_name = "vault-backed-aws-auth" -> null  - tfc_workspace_url  = "https://app.terraform.io/app/<YOUR_ORG>/workspaces/vault-backed-aws-auth" -> nullDo you really want to destroy all resources?  Terraform will destroy all your managed infrastructure, as shown above.  There is no undo. Only 'yes' will be accepted to confirm.  Enter a value: yesaws_iam_role_policy_attachment.tfc_policy_attachment: Destroying... [id=tfc-role-20230418140318135500000001]aws_iam_user_policy.vault_secrets_engine_generate_credentials: Destroying... [id=hcp-vault-secrets-engine:hcp-vault-secrets-engine-policy]##...tfe_variable.enable_aws_provider_auth: Destruction complete after 2stfe_workspace.trust_workspace: Destroying... [id=ws-zK5aBLW85vECGwEs]tfe_workspace.trust_workspace: Destruction complete after 0sDestroy complete! Resources: 21 destroyed.

Next steps

In this tutorial, you created a trust relationship between HCP Terraform and Vault. You also configured Vault's secrets engine to use the trust relationship to provide credentials for your cloud provider. Then you used those credentials to provision your infrastructure.

Refer to the following resources to learn more about dynamic credentials.

• Refer to the terraform-dynamic-credentials-setup-examples GitHub repository for other example configurations.

• Read the dynamic credentials documentation for more details about how to configure your trust relationships.

• You can also manually generate custom auth tokens for custom authentication workflows or providers that do not natively support dynamic credentials.