Terraform fot d distinct()

This is part of an ongoing series of posts documenting the built-in interpolation functions in Terraform. For more information, check out the beginning post. In this post I am going to cover the distinct() function. The example file is on GitHub here.

What is it?

Function name: distinct(list)

Returns: Takes a list and removes any duplicate values, keeping the first of each duplicate value. Returns the updated list.

Example:

variable "distinct" {
  default = ["one","two","three","one"]
}

# Returns ["one","two","three"]
output "distinct_output" {
  value = "${distinct(var.distinct)}"
}

Example file:

##############################################
# Function: distinct
##############################################
##############################################
# Variables
##############################################
variable "string_list" {
  default = ["So", "so", "long", "long", "so"]
}

variable "int_duplicates" {
  default = [42, 42, 42]
}

variable "string_and_int_list" {
  default = [42, "42", 42]
}

variable "empty_string_list" {
  default = ["", "", ""]
}

variable "empty_list" {
  default = []
}

variable "bool_list" {
  default = [false, true, false, true]
}

variable "nested_list" {
  default = [[1, 2, 3], [1, 2, 3]]
}

##############################################
# Resources
##############################################

##############################################
# Outputs
##############################################

#Test a standard list
output "1_distinct_list" {
  value = "${distinct(var.string_list)}"
}

#Test a standard int list
output "2_int_duplicates" {
  value = "${distinct(var.int_duplicates)}"
}

#Test int and strings with same value
output "3_string_and_int_list" {
  value = "${distinct(var.string_and_int_list)}"
}

#Test list of empty strings
output "4_empty_string_list" {
  value = "${distinct(var.empty_string_list)}"
}

#Test what it will do with an empty list
output "5_empty_list" {
  value = "${distinct(var.empty_list)}"
}

#Test what will happen with a list boolean values
output "6_bool_list" {
  value = "${distinct(var.bool_list)}"
}

#Flatten a nested list and then distinct
output "7_nested_list" {
  value = "${distinct(flatten(var.nested_list))}"
}

Run the following from the distinct folder to get example output for a number of different cases:

#All examples are in variables
terraform apply

Why use it?

This would be immensely useful when combining a bunch of lists from data sources and trying to extract only the unique records for other processes. I would use this in concert with other list functions, such as compact, concat, and flatten to clean up nested lists and empty values. In fact, I would probably run compact prior to distinct, so I can remove all empty strings, and then remove the duplicates. That seems more efficient from a computation perspective. The compact function only has to check if the value is empty, whereas the distinct has to check if that value exists in the list already.

Lessons Learned

The function explicitly doesn’t deal with nested lists, so you’ll need to use flatten first. Boolean values are converted to 0 and 1. Strings and ints are treated as the same value, so “42” and 42 are considered the same. The function is case sensitive, so “Fish” and “fish” are not the same value.

Coming up next is the element() function. I’m partial to the element(var.movies,5) myself.