Vault Provider and Ephemeral Values

In my last post about ephemeral values and write-only arguments, I showed how you could use the updated AzureRM provider to leverage the new azurerm_key_vault_secret ephemeral resource with write-only arguments that have been added to other resources. But the AzureRM provider is not the only provider to introduce ephemeral resources and write-only arguments! Version 5 of the Vault provider also has these new capabilities, as does the Random provider. In this post I thought we could explore bother providers to see how they have implemented ephemerality.

Vault Provider Support

Version 5.1.0 of the Vault provider has two ephemeral resources in it: vault_kv_secret_v2 and vault_database_secret. The vault_kv_secret_v2 is meant as a replacement for the current data source of the same name, and in fact if you are using the data source in your configuration, you’ll now get deprecation message:

│ Warning: Deprecated Resource
│
│   with data.vault_kv_secret_v2.burrito_recipe,
│   on main.tf line 61, in data "vault_kv_secret_v2" "burrito_recipe":
│   61: data "vault_kv_secret_v2" "burrito_recipe" {
│
│ Deprecated. Please use new Ephemeral KVV2 Secret resource `vault_kv_secret_v2` instead

Don’t worry if you’re currently using the vault_kv_secret_v2 data source, it’s still available for now. I’d expect it to be retired in the next major version of the provider.

You can use the new vault_kv_secret_v2 ephemeral resource to retrieve a secret value from a KV V2 secrets engine on a Vault server. The value will not be written to state or to plan files. Ephemeral value rules apply, so you can only use the value with a write-only argument or another ephemeral value type.

Let’s check out an example of using the ephemeral resource to copy a secret value from Vault to Azure Key Vault:

ephemeral "vault_kv_secret_v2" "burrito_recipe" {
  mount = "burrito_secrets"
  name  = "burrito-recipe"
  version = 1
}

# Write the secret to Azure Key Vault
resource "azurerm_key_vault_secret" "write_only" {
  name             = "burrito-recipe"
  value_wo         = jsonencode(ephemeral.vault_kv_secret_v2.burrito_recipe.data)
  value_wo_version = 1
  key_vault_id     = azurerm_key_vault.example.id
}

During plan, Terraform will retrieve the secret value from Vault, but will not include it in the execution plan file. When apply is run, Terraform will retrieve the secret value again, and this time write it to Azure Key Vault. Critically, the value_wo argument is not written to state, so the actual value is not recorded in state. Yay!

Version Confusion

There are two version related arguments that I want to tease apart here, as I think they could be a point of confusion. The version argument in the vault_kv_secret_v2 ephemeral resource refers to the version of the secret you want to retrieve from Vault. I could potentially have several versions of the burrito-recipe secret stored in Vault, so the version argument tells Vault which one to get. If I omit the version argument, the Vault provider will retrieve the latest version.

The value_wo_version argument in azurerm_key_vault_secret has nothing to do with the version of the secret pulled from Vault. The purpose of the vault_wo_version argument is to let Terraform know when the value of the secret has changed. Since Terraform doesn’t have the current value of the secret stored in state, it has no way of knowing if the secret value retrieved from Vault is different from what is stored in Azure Key Vault. The value_wo_version argument value IS recorded in state, and when it changes, Terraform will overwrite the current Azure Key Vault value with whatever is retrieved from Vault.

The number set for version and value_wo_version do not have to be the same. And in fact, there is a version property of the Key Vault secret that can also be different. You could have the following:

That would be totally valid. You are retrieving version 7 of the Vault secret and writing it as version 3 of the Azure Key Vault secret, using the value_wo_version argument set to 2 to make Terraform overwrite the current value.

Confusing? Yes. What can you do? Here’s my humble recommendation.

Set the version argument for vault_kv_secret_v2 and the value_wo_version to the same value. If the latest version of your Vault secret is 7, then use that value for both resources. That way you can know exactly what version you’re pulling from Vault, and you can use a single input variable (var.secret_version) to set both arguments, like so:

ephemeral "vault_kv_secret_v2" "burrito_recipe" {
  mount = var.secret_mount_path
  name  = var.secret_name
  version = var.secret_version
}

# Write the secret to Azure Key Vault
resource "azurerm_key_vault_secret" "write_only" {
  name             = "burrito-recipe"
  value_wo         = jsonencode(ephemeral.vault_kv_secret_v2.burrito_recipe.data)
  value_wo_version = var.secret_version
  key_vault_id     = azurerm_key_vault.example.id
}

When it’s time to update to version 8 of the secret, you only have to change one input value to retrieve the correct secret version and tell Terraform to update the Key Vault secret.

What about the version of the Key Vault secret? That doesn’t really factor into this deployment. Someone else is consuming the secret, and it’s up to them how they want to manage the version being retrieved.

If you want a full example to take for a spin, check out this folder in my Terraform Tuesdays repository. The hashi_vault configuration will set up Vault with a secret, and the hashi_vault_access will retrieve the secret and write it to an Azure Key Vault.

I fully expect more ephemeral resources to be added to the Vault provider over the next few release cycles. In fact, there are already several logged issues and pull requests to add them. If the ephemeral resource type you want is not yet supported, keep your eye on future releases. Maybe even watch the repository?

Random Provider

Version 3.7.0 of the Random provider added the random_password ephemeral resource. This is kind of a weird one. The syntax is straightforward enough:

ephemeral "random_password" "password" {
  length           = 16
  special          = true
  override_special = "!#$%&*()-_=+[]{}<>:?"
}

The use case is less clear. Why do I say that? Bear in mind that Terraform doesn’t store the resulting value in state or plan. But unlike the vault_kv_secret_v2 or azurerm_key_vault_secret ephemeral resources, the value of the random_password isn’t being retrieved from a persistent source.

Every time the random_password ephemeral resource is invoked the value returned will be different. If I use this resource to set the database administrator password for an azurerm_mssql_server, I will have no idea what the value was actually set to. Maybe that’s good if I never plan to log on as the administrator and just need the password set to something sufficiently complex. In fact, no one would know the value, so I guess that’s a bonus.

I suppose you could use the random_password to set a value of a Key Vault secret, and then use that Key Vault secret to set the password for the Azure MSSQL server. That would solve the secret zero problem of how a password is generated before it’s stored. When you need to roll the password, just change the value_wo_version argument for the Key Vaule secret. The random_password ephemeral resource is creating a different value every time it’s invoked.

Okay, so maybe I have convinced myself there’s some utility here. I suppose we’ll have to wait and see.

Conclusion

As more providers implement ephemeral resources and write-only arguments, the ability to keep sensitive data our of state will steadily rise. The write-only version argument is going to become increasingly important. You should start thinking now about how you want to manage that value for different use cases.