Password and Secret Management with the Hashicorp Vault command line client.


What is Vault

Vault secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets.  It tightly controls access to secrets and encryption keys by authenticating against trusted sources of identity such as Active Directory, LDAP, Kubernetes, CloudFoundry, and cloud platforms. Vault enables fine grained authorization of which users and applications are permitted access to secrets and keys

Obtaining the Vault Client

Vault can be downloaded at the Vault Project download page.  Versions are available for MacOS, Linux, Windows, FreeBSD, NetBSD, OpenBSD, and Solaris

After downloading Vault, unzip the package. Vault runs as a single binary named vault . Any other files in the package can be safely removed and Vault will still function.  Place the location of the vault executable in your default PATH

Configuring the Vault Client

Vault is configured by setting the location of the vault server in the VAULT_ADDR environment variable.  

MacOS & Linux:  export VAULT_ADDR=https://vault.es.cloud.vt.edu:8200

Windows:  setx VAULT_ADDR "https://vault.es.cloud.vt.edu:8200"  

Logging into Vault from the Command Line

vault login -method=ldap username=<pid> password=<password>

Note:  Vault rechecks ldap group membership often so it may be better to use a JSON Web Token (JWT) instead of your pid password.  The JWT acts as a temporary pid password that doesn't require two-factor auth.  You can get a JWT at ED JWT Generator

Example:


$ vault login -method=ldap username=cmcnabb
Password (will be hidden):			<== Beware the Stealth Two Factor
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key                    Value
---                    -----
token                  s.7NL9RJyde8GSNlIiC7lrdUur
token_accessor         FMj8qTJBUKkI7rSbxr5hezJs
token_duration         8h
token_renewable        true
token_policies         ["default"]
identity_policies      ["es_dbaa" "es_dbaa_database" "transit"]
policies               ["default" "es_dbaa" "es_dbaa_database" "transit"]
token_meta_username    cmcnabb

The token will be saved in .vault-token in your home directory and will be automatically used every time you run vault after logging in.  You can use the token to log into the Vault User Interface at  https://vault.es.cloud.vt.edu:8200

An improperly entered username or password will result in the following:


Incorrect username or password result

Error authenticating: Error making API request.

URL: PUT https://vault.nonprod.es.cloud.vt.edu:8200/v1/auth/ldap/login/cmcnabb
Code: 400. Errors:

* ldap operation failed

Secrets Engines

Secrets engines define how data is stored and accessed.  They appear as mountpoints within vault.  


Listing Secrets Engines

$ vault secrets list
Path                   Type         Accessor              Description
----                   ----         --------              -----------
cubbyhole/             cubbyhole    cubbyhole_dd21087f    per-token private secret storage
es.dbaa/                  kv           kv_23f36bc5           key store for dbaa team
es.dbaa.banner/           kv           kv_a764c67d           key store for dbaa banner team
es.dbaa.data-delivery/    kv           kv_72622700           key store for dbaa data-delivery team
es.dbaa.database/         kv           kv_32539ea8           key store for dbaa database team
es.dbaa.lead/             kv           kv_af975499           key store for dbaa lead team
es.dbaa.web/              kv           kv_e9020c21           key store for dbaa web team
identity/              identity     identity_3dc9bbef     identity store
secret/                kv           kv_4ab6ced2           key/value secret storage
sys/                   system       system_654a5852       system endpoints used for control, policy and debugging
transit/               transit      transit_1b363511      n/a

The Cubbyhole

The cubbyhole secrets engine is used for short term storage of secrets.  They are tied to the token, not the user.  Once the token expires or is revoked its related cubbyhole data is no longer accessible.


Using the Cubbyhole Secrets Engine

$ vault write cubbyhole/mysecret foo=bar
Success! Data written to: cubbyhole/mysecret

$ vault list cubbyhole
Keys
----
mysecret
$ vault read cubbyhole/mysecret
Key    Value
---    -----
foo    bar

Key/Value Secrets Engines

Key/Value Secrets Engines are persistent.  They exist until a vault command is used to create or destroy them.  There are two versions of the Key/Value secrets engines.  Version 1 is the original version while Version 2 adds versioning and a 'check-and-set' option.  Versioning allows older versions of a secret to be retrieved in case of accidental deletion or modification.  Check-and-set is used to keep a secret from being updated if it already exists.  To check the the secrets engine version use the 'vault secrets list -detailed' command and check the 'options' column.

In our configuration, the 'secret' engine is version 1.  It is intended for storing per-user secrets.  es.* secret engines are version 2 with versioning and are intended for team secrets


Using a KV Secrets Engine

$ vault kv list secret/cmcnabb
Keys
----
mykeys
testkey

$ vault kv put secret/cmcnabb/newkey first=who second=what third="I don't know"
Success! Data written to: secret/cmcnabb/newkey

$ vault kv get secret/cmcnabb/newkey
===== Data =====
Key       Value
---       -----
first     who
second    what
third     I don't know

# Store some binary data
$ vault kv put secret/cmcnabb/binary data=$(base64 < binary_file)
 
$ vault kv delete secret/cmcnabb/newkey
Success! Data deleted (if it existed) at: secret/cmcnabb/newkey

$ vault kv list secret/cmcnabb
Keys
----
mykeys
testkey

 

rvault

The 'vault kv list' command currently has no recursive mode.  rvault is a command line utility that can be used to list or search all secrets the user may access.  Versions for Windows, Linux, and MacOS can be downloaded at https://github.com/kir4h/rvault/releases.  Once downloaded the rvault binary should be placed in a directory in the user's PATH


rvault Usage

$ rvault -h
Tool to perform some recursive operations on Vault KV

Usage:
  rvault [command]

Available Commands:
  help        Help about any command
  list        Recursively list secrets for a given path
  read        Recursively read secrets for a given path

Flags:
  -a, --address string          Vault address
      --alsologtostderr         log to standard error as well as files
  -c, --concurrency uint32      Maximum number of concurrent queries to Vault (default 20)
      --config string           config file (default is $HOME/.config/rvault/config.yaml)
  -e, --exclude-paths strings   KV paths to be excluded (applied on 'include-paths' output)
  -h, --help                    help for rvault
  -i, --include-paths strings   KV paths to be included (default [*])
      --insecure                Disables SSL verification
      --log_dir string          If non-empty, write log files in this directory
      --log_file string         If non-empty, use this log file
      --logtostderr             log to standard error instead of files (default true)
  -t, --token string            Vault token
  -v, --v Level                 number for the log level verbosity
      --version                 version for rvault

Use "rvault [command] --help" for more information about a command.

Transit Secrets Engine

The Transit Secrets engine provides cryptography as a service.  It can also sign and verify data, generate hashes of data, and act as a source of random bytes.

Encryption Keys

The Transit Secrets engine supports the following encryption key types:  aes256-gcm96, chacha20-poly1305, ed25519, ecdsa-p256, rsa-2048, and rsa-4096.


Creating an Encryption Key

$ vault write -f transit/keys/mykey
Success!  Data written to: transit/keys/mykey


Encrypting Data

Data should be converted to a base64 representation prior to encryption. 

Encrypting Data Using the Transit Secrets Engine

$ vault write transit/encrypt/mykey plaintext=$(base64 <<< "my secret data")
Key           Value
---           -----
ciphertext    vault:v1:bUF+fx28W/ZRYWkLvEau/CSSvdZ8WoWkoZ3PTm+mGnKUa/8K9awxfdPYmA==

Decrypting Data

Decrypting using the Transit Secrets engine

$ vault write -field=plaintext transit/decrypt/mykey ciphertext=vault:v1:bUF+fx28W/ZRYWkLvEau/CSSvdZ8WoWkoZ3PTm+mGnKUa/8K9awxfdPYmA== | base64 -D
my secret data

Response Wrapping

Response Wrapping is a process where Vault will return a token attached to a cubbyhole containing the desired data.  The token is assigned a time to live, after which the cubbyhole (and its data) will no longer exist.  The token is single use: after it is used its cubbyhole and data no longer exist.  This response wrapping does three things:

  1.  Provides cover for the secret - Only the token is sent to the recipient.  The secret itself is not sent over email or instant message or other means.
  2. Malfeasance detection - The token can only be used once.  If the intended recipient can not use it then it has probably been compromised.
  3. Limited lifetime - Tokens can't sit around unused and forgotten for long periods of time

Specifying the -wrap-ttl parameter with a time-to-live value will generate a wrapping token for the response that would have been returned.  Time-to-live values are a number and a unit designator (s for seconds, m for minutes, h for hours, d for days)

Wrapping a Secret

$ vault kv get -wrap-ttl=30m  secret/cmcnabb/testkey
Key                              Value
---                              -----
wrapping_token:                  s.6SFHFecAbWx4ljjJlMetc51R
wrapping_accessor:               5g80dl3LL59y6q8JGPv0VYnd
wrapping_token_ttl:              30m
wrapping_token_creation_time:    2019-03-06 18:03:11.73953576 +0000 UTC
wrapping_token_creation_path:    secret/cmcnabb/testkey

Looking up a wrapping token's metadata

$ vault token lookup s.6SFHFecAbWx4ljjJlMetc51R
Key                 Value
---                 -----
accessor            5g80dl3LL59y6q8JGPv0VYnd
creation_time       1551895391
creation_ttl        30m
display_name        n/a
entity_id           n/a
expire_time         2019-03-06T18:33:11.760632734Z
explicit_max_ttl    30m
id                  s.6SFHFecAbWx4ljjJlMetc51R
issue_time          2019-03-06T18:03:11.760632512Z
meta                
num_uses            1
orphan              true
path                secret/cmcnabb/testkey
policies            [response-wrapping]
renewable           false
ttl                 29m43s
type                service

Unwrapping a Secret

$ vault unwrap s.6SFHFecAbWx4ljjJlMetc51R
Key                 Value
---                 -----
refresh_interval    768h
foo                 bar

Listing Permissions

the 'vault token capabilities' command will show what permissions a user has on a particular secrets path


Checking Permissions on a Secrets Engine

$ vault token capabilities es.dbaa.database/
create, delete, list, read, update

$ vault token capabilities es.dbaa.web/
deny