How to create and manage Secrets in GCP Secret Manager using Terraform

Secret Manager is a secure and convenient storage system for API keys, passwords, certificates, and other sensitive data. Secret Manager provides a central place and single source of truth to manage, access, and audit secrets across Google Cloud. GCP secret manager allows you to securely store and access API keys, passwords, certificates, and other sensitive data.

Terraform is an open-source infrastructure as code software that allow users to define and provide data center infrastructure using a declarative configuration language known as HashiCorp Configuration Language(HCL), or optionally JSON. We will use terraform to manage secrets in this guide.

Please note that terraform will store secrets in clear text in the state file and it is up to you to take protective measures to secure them.

Prerequisites

You need the following to follow along with this guide:

A GCP account with a subscription – If you don’t have a GCP subscription, we can create a free account at https://cloud.google.com before we start.
A GCP Service account for authentication to GCP. Checkout How to Create a Service Account for Terraform in GCP.
An existing GCP project to store our secrets

Before proceeding you have to be logged in to google cloud. Use Cloud SDK:

$ gcloud auth login
$ gcloud auth application-default login

Enabling Secrets manager api

You also need to ensure that the secret manager api is enabled for our project before proceeding. This step is mandatory. Open the GCP Portal and type secret manager in the search box located at the top of the page then click Enable if it is not enabled.

This only needs to be done once per project, but it is an idempotent operation. You can also use gcloud cli:

gcloud services enable secretmanager.googleapis.com

Or terraform

resource "google_project_service" "secretmanager" {
  service  = "secretmanager.googleapis.com"
}

Adding service Account Permissions

We need to update our Service Account with extra permissions. Open the GCP Portal and go to IAM & Admin and select IAM.

Select our Service Account and click on the Edit Permissions button (the pencil icon) and add the Secret Manager Admin role.

Using gcloud:

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/secretmanager.admin" \
  --condition="None";

Adding the terraform code

Providers are used in Terraform to interact with Providers using API calls.

Define provider configuration like shown below:

locals {
  gcp_project      = "citizix-one"
  credentials_path = "../terraform-user-xxxx.json"
  region           = "europe-west6"
}

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 4.20"
    }
  }
}

provider "google" {
  credentials = file(local.credentials_path)

  project = local.gcp_project
  region  = local.region
}

Then we can create a secret named live-app-password with automatic replication policy and a new version of the secret:

resource "google_secret_manager_secret" "secret" {
  secret_id = "live-app-password"

  replication {
    automatic = true
  }

  depends_on = [google_project_service.secretmanager]
}

resource "google_secret_manager_secret_version" "secret-version-basic" {
  secret = google_secret_manager_secret.secret.id

  secret_data = "Secr37P4s5word"
}

Grant a user or service account IAM permissions to access the secret:

resource "google_secret_manager_secret_iam_member" "live-app" {
  secret_id = google_secret_manager_secret.secret.id
  role      = "roles/secretmanager.secretAccessor"
  member    = "user:foo@bar.com" # or serviceAccount:my-app@...
}

Secret with user-managed replication

We also have an option of dictating how the secret replication happens. This is defined in the user_managed replicas in google_secret_manager_secret replication.

In our case, we are opting to have our secrets replicated in europe region west1 and west 2.

resource "google_secret_manager_secret" "secret" {
  secret_id = "live-app-password"

  labels = {
    label = "citizix"
  }

  replication {
    user_managed {
      replicas {
        location = "europe-west1"
      }
      replicas {
        location = "europe-west2"
      }
    }
  }
}

resource "google_secret_manager_secret_version" "secret-version" {
  secret = google_secret_manager_secret.secret.id

  secret_data = "Secr37P4s5word"
}

Reading Secret Data

To read the secret, we will have to query for the stored version. Terraform provides a way to query data. Let us query version 1 of our app password secret:

data "google_secret_manager_secret_version" "secret" {
  secret   = "live-app-password"
  version  = "1"
}

If you specify the version, terraform will return the specified version of the secret. To get the latest version, omit the version parameter:

data "google_secret_manager_secret_version" "secret" {
  secret   = "live-app-password"
}

Consuming the Secret Data

To access the data retrieved from the secret, we can output it as shown below:

output "live-app-password" {
  value = data.google_secret_manager_secret_version.secret.secret_data
}