In most systems I come across, private keys are all over the place, made available as secrets to the workloads that need them. The trouble is not just that sometimes, secrets are not really secret, but more fundamentally, that private keys are something that we don't need to handle directly at all, they're basically too hot to handle 🔥.

Instead, we can use a remote service to manage our private keys such as Azure Key Vault, obviating the need to handle them ourselves. In this way, private keys become infrastruture.

Ideally, the keys managed by the service are in fact generated there, too, such that at no point in time is the key ever available to you. We can ask the service to sign or decrypt some data, but the actual key is never revealed.

But before we talk about how we can switch to such managed keys, let's look at a few example use-cases:

• Authentication

  Lots of systems are integrated using public/private key pairs. For example, when we commit to GitHub, many developers use SSH to connect, using key pair-based authentication; or when we connect to a database, the authentication protocol might use key pair-based authentication instead of a password, for an extra layer of transport security.

• Session signing

  Most web applications require the signing of session cookies, a token that proves to the web application that you're logged in, or simply that you have an "open session". Typically, this is configured using a locally available signing key, but again, we can improve security by using a managed key (and it's possible to be clever here and amortize the cost of using the managed key over a period of time or number of sessions created).

The operation we need in both of these cases is the ability to sign a payload. Managed keys are perfect for this!

Case Study: Using managed keys to authenticate to Snowflake

The Snowflake client library for Python has an authentication system that supports multiple methods. Working on a data platform solution for Grundfos, a Danish company and also the world's largest pump manufacturer, I contributed a change to extend this authentication system to accomodate managed keys (the existing system supported only concrete private keys such as a file on disk).

For Python, the cryptography package has become the defacto standard for encryption. The RSAPrivateKey interface represents an abstract private key that uses traditional RSA asymmetric encryption.

With the change linked above, it's now possible to implement a custom private key object which defers operations to a managed key service since the Snowflake library now takes bytes | RSAPrivateKey as input.

I'm collaborating with Microsoft to provide this functionality out of the box with the azure-keyvault-keys package, a feature that's likely to go into the next release. In code, this would look like the following:

from azure.keyvault.keys import KeyClient
from azure.keyvault.keys.crypto import ManagedRsaKey
from snowflake.connector import connect

key_client = KeyClient(
    "https://<keyvault-name>.vault.azure.net/",
    credential=AZURE_CREDENTIAL
)

ctx = connect(
    # Here goes the usual connection parameters
    ...
    private_key=ManagedRsaKey(KEY_NAME, key_client)
)

The prerequisite to have this work is to use ALTER USER ... SET RSA_PUBLIC_KEY = '<key-data>'. You can download the public key in PEM-format (suitable as key data here) using the portal or Azure CLI:

$ az keyvault key download     \
    --vault-name <keyvault-name> \
    -n <key-name>                \
    --encoding PEM               \
    -f /dev/stdout

Private keys disguised as secrets

In many systems, private keys are often being referred to as secrets. For example, in Airflow, there's a secret_key which "cryptographic components can use this to sign cookies and other things".

In order to switch to managed encryption keys, we must recognize these situations and create the necessary interfaces.

Reducing cost

Using managed keys is not free. There is a cost for each operation, but for Azure Key Vault for example, using standard RSA 2048-bit keys, these operations are as cheap as reading a secret. As mentioned previously, in some cases, we can be clever and use a surrogate signing key for a period of time. This also reduces the operational dependence on the remote service.

These costs (and infrastructure complexity) should be easily offset by the much lessened burden of compliance with security protocol. If you never saw a private key, there is no risk having compromised it. Access to using a managed key can be granted on a higher level and revoked just as easily.