Security

How to install and set up Hashicorp Vault in Linux

For most organizations, a major concern has been how to best secure data, preventing it from unauthorized access or exfiltration.

HashiCorp Vault allows you to secure, store and tightly control access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API. Secrets are any form of sensitive credentials that need to be tightly controlled and monitored and can be used to unlock sensitive information. Secrets could be in the form of passwords, API keys, SSH keys, RSA tokens, or OTP.

HashiCorp Vault makes it very easy to control and manage access by providing you with a unilateral interface to manage every secret in your infrastructure. Not only that, you can also create detailed audit logs and keep track of who accessed what.

HashiCorp Vault is a free and open source product with an enterprise offering. The enterprise platform includes disaster recovery, namespaces, and monitoring, as well as features for scale and governance.

Using HashiCorp Vault for secrets management is certainly more secure than placing plaintext secrets in your configurations. In accordance with industry best practices for data encryption, HashiCorp Vault utilizes both TLS for data in transit and AES 256-bit encryption for data at rest.

Checkout

# Installation

For Ubuntu/Debian, use the following commands to install vault

Install dependencies

sudo apt install software-properties-common

Then add the repo and install vault.

curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install vault

For Centos/RHEL, install using the following commands:

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
sudo yum -y install vault

After installing Vault, verify the installation worked by opening a new terminal session and checking that the vault binary is available. By executing vault, you should see help output similar to the following:

$ vault
Usage: vault <command> [args]

Common commands:
    read        Read data and retrieves secrets
    write       Write data, configuration, and secrets
    delete      Delete secrets and configuration
    list        List data or secrets
    login       Authenticate locally
    agent       Start a Vault agent
    server      Start a Vault server
    status      Print seal and HA status
    unwrap      Unwrap a wrapped secret

Other commands:
...

If you get an error that the binary could not be found, then your PATH environment variable was not setup properly. Please go back and ensure that your PATH variable contains the directory where Vault was installed.

Otherwise, Vault is installed and ready to go!

# Starting the Vault Dev Server

Once installed, we need to start the vault service before using it. Vault operates as a client/server application. The Vault server is the only piece of the Vault architecture that interacts with the data storage and backends. All operations done via the Vault CLI interact with the server over a TLS connection.

Let us start and interact with the Vault server running in development mode. The dev server is a built-in, pre-configured server that is not very secure but useful for playing with Vault locally. 

To start the Vault dev server, run:

$ vault server -dev
==> Vault server configuration:

             Api Address: http://127.0.0.1:8200
                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
              Go Version: go1.17.7
              Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: false
           Recovery Mode: false
                 Storage: inmem
                 Version: Vault v1.10.0
             Version Sha: 7738ec5d0d6f5bf94a809ee0f6ff0142cfa525a6

==> Vault server started! Log data will stream in below:

...

WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.

You may need to set the following environment variable:

    $ export VAULT_ADDR='http://127.0.0.1:8200'

The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.

Unseal Key: ae1V/woQecalxNIAstoB4kPhqeuBl0Dmt0bxGVoMIKs=
Root Token: hvs.DvT6ReMjPzTUMiJpzUnZlpjJ

Development mode should NOT be used in production installations!

You should see output similar to that above. Notice that Unseal Key and Root Token values are displayed.

The dev server stores all its data in-memory (but still encrypted), listens on localhost without TLS, and automatically unseals and shows you the unseal key and root access key.

Insecure operation: Do not run a Vault dev server in production. This approach is only used here to simplify the unsealing process for this demonstration.

With the dev server started, perform the following:

  • Launch a new terminal session.
  • Copy and run the export VAULT_ADDR ... command from the terminal output. This will configure the Vault client to talk to the dev server.
$ export VAULT_ADDR='http://127.0.0.1:8200'
  • Vault CLI determines which Vault servers to send requests using the VAULT_ADDR environment variable.
  • Save the unseal key somewhere. Don’t worry about how to save this securely. For now, just save it anywhere.
  • Set the VAULT_TOKEN environment variable value to the generated Root Token value displayed in the terminal output. Example:
$ export VAULT_TOKEN="hvs.DvT6ReMjPzTUMiJpzUnZlpjJ"

To interact with Vault, you must provide a valid token. Setting this environment variable is a way to provide the token to Vault via CLI.

# Verify the Server is Running

Verify the server is running by running the vault status command. If it ran successfully, the output should look like the following:

$ vault status
Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    1
Threshold       1
Version         1.10.0
Storage Type    inmem
Cluster Name    vault-cluster-742a46e2
Cluster ID      2dbdadc3-8eeb-c787-8084-f6e7578e6091
HA Enabled      false

# Creating your first Secret

# Key/Value secrets engine

When running Vault in dev mode, Key/Value v2 secrets engine is enabled at secret/ path. Key/Value secrets engine is a generic key-value store used to store arbitrary secrets within the configured physical storage for Vault. Secrets written to Vault are encrypted and then written to backend storage. Therefore, the backend storage mechanism never sees the unencrypted value and doesn’t have the means necessary to decrypt it without Vault.

Key/Value secrets engine has version 1 and 2. The difference is that v2 provides versioning of secrets and v1 does not.

Use the vault kv <subcommand> [options] [args] command to interact with K/V secrets engine.

You can interact with key/value secrets engine using the vault kv command. Get the command help.

$ vault kv -help

# Write a secret

Now, write a secret name with value of citizix to the path secret/blog using the vault kv put command. This command creates a new version of the secrets and replaces any pre-existing data at the path if any.

$ vault kv put secret/blog name=citizix
== Secret Path ==
secret/data/blog

======= Metadata =======
Key                Value
---                -----
created_time       2022-04-15T06:47:19.237861969Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

Please note that it is important that the path is prefixed with secret/, otherwise this example won’t work. The secret/ prefix is where arbitrary secrets can be read and written.

You can even write multiple pieces of data.

$ vault kv put secret/blog name=citizix awesome=yes
== Secret Path ==
secret/data/blog

======= Metadata =======
Key                Value
---                -----
created_time       2022-04-15T06:48:32.982156587Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            2

# Read a secret

Secrets can be retrieved with vault kv get <path>.

$ vault kv get secret/blog
== Secret Path ==
secret/data/blog

======= Metadata =======
Key                Value
---                -----
created_time       2022-04-15T06:48:32.982156587Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            2

===== Data =====
Key        Value
---        -----
awesome    yes
name       citizix

Vault returns the latest version (in this case version 2) of the secrets at secret/blog.

To print only the value of a given field, use the -field=<key_name> flag.

$ vault kv get -field=name secret/blog
citizix

Optional JSON output is very useful for scripts. For example, you can use the jq tool to extract the value of the excited secret.

$ vault kv get -format=json secret/blog | jq -r .data.data.name
citizix

# Delete a secret

Now that you’ve learned how to read and write a secret, let’s go ahead and delete it. You can do so using the vault kv delete command.

$ vault kv delete secret/blog
Success! Data deleted (if it existed) at: secret/blog

Try to read the secret you just deleted.

$ vault kv get secret/blog
== Secret Path ==
secret/data/blog

======= Metadata =======
Key                Value
---                -----
created_time       2022-04-15T06:48:32.982156587Z
custom_metadata    <nil>
deletion_time      2022-04-15T06:51:52.12566776Z
destroyed          false
version            2

The output only displays the metadata with deletion_time. It does not display the data itself once it is deleted. Notice that the destroyed parameter is false which means that you can recover the deleted data if the deletion was unintentional.

$ vault kv undelete -versions=2 secret/blog
Success! Data written to: secret/undelete/blog

Now, the data is recovered.

$ vault kv get secret/blog

== Secret Path ==
secret/data/blog

======= Metadata =======
Key                Value
---                -----
created_time       2022-04-15T06:48:32.982156587Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            2

===== Data =====
Key        Value
---        -----
awesome    yes
name       citizix

# Starting and enabling Vault using Systemd

Most modern Linux distributions uses systemd to manage its services. For production workloads you will need to start the vault server using a service manager.

Start vault using this command:

sudo systemctl start vault

Confirm the status

$ sudo systemctl status vault
● vault.service - "HashiCorp Vault - A tool for managing secrets"
   Loaded: loaded (/usr/lib/systemd/system/vault.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2022-04-15 06:02:29 UTC; 2min 14s ago
     Docs: https://www.vaultproject.io/docs/
 Main PID: 69690 (vault)
    Tasks: 7 (limit: 23167)
   Memory: 58.0M
   CGroup: /system.slice/vault.service
           └─69690 /usr/bin/vault server -config=/etc/vault.d/vault.hcl

Apr 15 06:02:29 rockysrv.citizix.com vault[69690]:                    Mlock: supported: true, enabled: true
Apr 15 06:02:29 rockysrv.citizix.com vault[69690]:            Recovery Mode: false
Apr 15 06:02:29 rockysrv.citizix.com vault[69690]:                  Storage: file
Apr 15 06:02:29 rockysrv.citizix.com vault[69690]:                  Version: Vault v1.10.0
Apr 15 06:02:29 rockysrv.citizix.com vault[69690]:              Version Sha: 7738ec5d0d6f5bf94a809ee0f6ff0142cfa525a6
Apr 15 06:02:29 rockysrv.citizix.com vault[69690]: ==> Vault server started! Log data will stream in below:
Apr 15 06:02:29 rockysrv.citizix.com vault[69690]: 2022-04-15T06:02:29.815Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
Apr 15 06:02:29 rockysrv.citizix.com vault[69690]: 2022-04-15T06:02:29.815Z [WARN]  no `api_addr` value specified in config or in VAULT_API_ADDR; falling back to detection if>
Apr 15 06:02:29 rockysrv.citizix.com vault[69690]: 2022-04-15T06:02:29.845Z [INFO]  core: Initializing versionTimestamps for core
Apr 15 06:02:29 rockysrv.citizix.com systemd[1]: Started "HashiCorp Vault - A tool for managing secrets".

# Accessing vault UI

To activate the UI, set the ui configuration option in the Vault server configuration.

ui = true

listener "tcp" {
  # ...
}

Then go to your browser and access http://server_ip:8200/ui/ to access the UI.

The Vault server is uninitialized and sealed. Before continuing, the server’s storage backend requires starting a cluster or joining a cluster.

Select Create a new Raft cluster and click Next.

Enter 5 in the Key shares and 3 in the Key threshold text fields then Click Initialize. When the unseal keys are presented, scroll down to the bottom and select Download key. Save the generated unseal keys file to your computer.

The unseal process requires these keys and the access requires the root token. Click Continue to Unseal to proceed.

Open the downloaded file. Copy one of the keys (not keys_base64) and enter it in the Master Key Portion field. Click Unseal to proceed.

The Unseal status shows 1/3 keys provided. Enter all the 3 keys until the Vault is unsealed and is ready to operate.

Finally, Copy the root_token and enter its value in the Token field. Click Sign in.

# Conclusion

In this tutorial, we learned how to install and use the powerful CRUD features of Vault to store arbitrary secrets. On its own, this is already a useful but basic feature. Key/Value secrets engine is one of the secrets engines that Vault offers.

comments powered by Disqus
© 2020 - 2024 Citizix, Inc.
Citizix Ltd
Built with Hugo
Theme Stack designed by Jimmy