Secrets

Working with secrets in chef-cuisine

We want to use chef-cuisine for all environments, including development. This means all information stored there will be available for Ops, Devs, managers and everybody able to steal tarball from repo server. That's why we must encrypt sensible data!

In this example all secrets are stored in keepass database. However you may have another tool for this. From Chef side we use chef-vault to be able to restrict access to particular secret to list of nodes or admins.

Each node that needs access to vaulted data must have pre-generated key pair. Public part is stored in cuisine in clients/nodename.json file and private part should be transferred to a node via secure channel by a Responsible Operations Person. In case you want to use Chef Server you will have to register node first, encrypt secrets with it's public key, now available to chef-vault, and on next chef run secrets can be used.

There are few types of secret we need to protect:

• client.pem for each nodes
• SSL keys
• passwords set in attributes
• secret files (licenses, shared vpn keys, etc)

All secret data is stored in chef-vault, encrypted with public key of each node requiring access to that data. We never have secrets in plain text in git, but adding new 'reader' requires manual actions. Ops person needs to take secret from keepass, encrypt it for specific node and commit changes. As we are not using chef-server, we can't use chef-vault's admin feature. At least until chef-zero will support users.

Adding new client with client.pem

• Create key pair.

$ knife client -z -d create ryoko.tanabata.dev -f clients/ryoko.tanabata.dev.pem
WARNING: No knife configuration file found
Created client[ryoko.tanabata.dev]

• Put private key in keepass database. In group chef_client_keys
• Create item named after node hostname.
• Attach generated .pem file.
• Commit you changes!

Adding SSL secret

• Get ssl certificate tarball from KeePass database ("SSL keys" section)
• Unpack to trusted dir (/tmp/cr)
• Add SSL data to chef vault with access for particular node (ryoko.tanabata.dev)

$ ./misc/ssl2json.rb cert /tmp/cr/STAR_realloc_spb_ru.crt key /tmp/cr/realloc.spb.ru.key  chain /tmp/cr/SSLCA.crt > /tmp/cr/realloc.spb.ru.json
$ knife vault -z create ssl-vault realloc_spb_ru -S "name:ryoko.tanabata.dev" --admins '' --json /tmp/cr/realloc.spb.ru.json

On target node add SSL extraction. This may be done by using ssl_certificate resource directly in cookbooks or by using attributes and ssl_certificate::attr_apply recipe.

run_list(
  "recipe[ssl_certificate::attr_apply]"
)
override_attributes(
  "ssl_certificate" => {
    "items" => [
      { "name" => "realloc.spb.ru",
        "dir" => '/etc/nginx/ssl',
        "item" => 'realloc_spb_ru',
        "source" => 'chef-vault',
        "bag" => 'ssl-vault',
        "key_item_key" => 'key',
        "cert_item_key" => 'cert',
        "chain_item_key" => 'chain',
        "chain_source" => 'chef-vault',
        "chain_bag" => 'ssl-vault',
        "chain_item" => 'realloc_spb_ru',
        "chain_name" => 'realloc.spb.ru.chain.pem' # Need to set chain_name if we need a chain to be generated.
      }
    ]
  }
)

Adding secret file

• Get secret file from KeePass or other safe place
• Add to chef-vault data-bag (file-vault here)

$ knife vault -z create file-vault ms_license_key -S "name:ryoko.tanabata.dev" --admins '' --file /tmp/license.key

On target node add file to node['file-vault']['files'] attributes

run_list(
  'recipe[file-vault]'
)
override_attributes(
  'file-vault' => {
    'files' => [
      { 'name' => '/root/ms_license.key',
        'item' => 'ms_license_key'
      }
    ]
  }
)

Use file

[root@ryoko ~]# cat /root/ms_license.key
j3qq4-h7h2v-2hch4-m3hk8-6m8vw

Adding secret attribute

There are many cases when you want to store something secret in attributes. Proposed solution is to have Hash with secret attributes stored in chef-vault that will be merged and put to force_override to replace non-secret content of attributes. In such way we can have simple default passwords for use in dev environment and in prod roles say to override them with encrypted content from chef vault item.

Let's see an example.

run_list(
  "recipe[attr-vault]",
  "recipe[kludges-tanabata::attr-file]"
)
override_attributes(
  "flurb" => {
    "foop" => {
      "buzz" => "NotASecret"
    }
  },
  "attr-vault" => {
    "configs" => {
      "flurb" => {
        'precedence_level' => 'force_override',
        'bag_cascade' => [
          'chef_vault_item[attr-vault::flurb]'
        ]
      }
    }
  }
)

All magic is done by attr-vault cookbook. On a start it reads node['attr-vault']['configs'] hash and iterates through keys. For each key found it gets hash from bsg_cascade items and does attribute DeepMerge with a section after key's name.

In this example it will take chef-vault item flurb from attr-vault vault and merge all it's contents with node['flurb'] attribute.

In decrypted form this item contains buzz value.

{
  "id": "flurb",
  "foop": {
    "buzz" : "Secret"
  }
}

So at the end we will have node['flurb']['foop']['buzz'] value as Secret.

Adding new attr-vault item is done the same way as other chef-vaults

$ knife vault -z create attr-vault flurb -S "name:ryoko.tanabata.dev" --admins '' --json /tmp/cb/flurb.json