Key Management & Encryption (KMS)

The hard part isn't encryption

Encrypting a blob of data is a solved problem. Pick AES-256, call one library function, and you have ciphertext. The first time most teams meet encryption, it feels almost too easy. Then the real question arrives: where does the key live? Because ciphertext plus the key sitting right next to it is just a slightly slower way to store plaintext.

Every hard problem in cryptography in production is a key management problem. Who can use the key? How do you rotate it without re-encrypting petabytes? How do you prove, six months later during an audit, exactly who decrypted that customer record and when? This article is about the lifecycle of the key, not the math of the cipher. The cipher you can trust; the key you have to govern.

One sentence, then a picture

A KMS doesn't store your secrets, it stores the keys that protect your secrets, and never lets those keys leave the building.

That last clause is the whole point. A well-designed KMS performs cryptographic operations on your behalf so the master key material never appears in your application's memory, your logs, or a developer's laptop. You send it data to wrap; it sends back wrapped data. The key stays inside the vault.

Envelope encryption, drawn out

You almost never encrypt large data directly with the master key. The master key is slow to reach (a network call), rate-limited, and you want to minimize how often it touches anything. Instead you use envelope encryption: the master key encrypts a small *data key*, and the fast, local data key encrypts the actual payload. The diagram below shows both the encrypt and the decrypt path.

1. 1

   Ask KMS for a data key

   Call GenerateDataKey against your master key. KMS returns the data key **twice**: once in plaintext (for immediate use) and once wrapped (encrypted by the master key).

2. 2

   Encrypt locally with the plaintext data key

   Use the plaintext data key with a fast local AES cipher to encrypt your payload. This happens in your process, no per-byte network round-trip to KMS.

3. 3

   Throw away the plaintext data key

   Wipe it from memory the moment you are done. The only copy that survives is the wrapped one.

4. 4

   Store ciphertext + wrapped key together

   Persist the encrypted payload alongside the wrapped data key. You never store the plaintext key, so leaking the storage layer alone reveals nothing.

5. 5

   To decrypt, unwrap first

   Send the wrapped data key to KMS Decrypt. KMS uses the master key to return the plaintext data key, which you then use locally to decrypt the payload, then discard again.

Where keys live (and why it matters)

Not all key storage is equal. The single biggest jump in security comes from moving the key out of anything a developer or an attacker can read directly. Here is the spectrum, worst to best.

A Hardware Security Module (HSM) is a physical (or cloud-hosted dedicated) device built so that key material physically cannot be extracted, attempts to tamper with it zero out the keys. A managed KMS is usually *backed* by HSMs you share with other tenants; a dedicated HSM (CloudHSM, Azure Dedicated HSM) gives you single-tenant hardware and full control of the key store, at real operational cost. Most teams should reach for managed KMS first and only graduate to a dedicated HSM when a compliance regime demands it.

Envelope-encrypt a payload

Here is the full envelope flow in Python against AWS KMS. The same shape applies to Azure Key Vault and Google Cloud KMS, only the SDK names change. Notice we never see the master key, and we wipe the plaintext data key after use.

import os
import boto3
from cryptography.hazmat.primitives.ciphers.aead import AESGCM

kms = boto3.client("kms")
MASTER_KEY_ID = os.environ["KMS_KEY_ID"]  # an alias/ARN, NOT key material


def encrypt(plaintext: bytes) -> dict:
    # 1. Ask KMS for a fresh data key (256-bit) wrapped by the master key.
    resp = kms.generate_data_key(KeyId=MASTER_KEY_ID, KeySpec="AES_256")
    plaintext_key = resp["Plaintext"]        # use locally, then destroy
    wrapped_key = resp["CiphertextBlob"]     # safe to store alongside data

    # 2. Encrypt locally with the plaintext data key (fast, no network).
    nonce = os.urandom(12)
    ciphertext = AESGCM(plaintext_key).encrypt(nonce, plaintext, None)

    # 3. Drop the plaintext data key from memory.
    del plaintext_key

    return {"wrapped_key": wrapped_key, "nonce": nonce, "ciphertext": ciphertext}


def decrypt(envelope: dict) -> bytes:
    # 4. Unwrap the data key, KMS uses the master key, then we discard it.
    plaintext_key = kms.decrypt(CiphertextBlob=envelope["wrapped_key"])["Plaintext"]
    plaintext = AESGCM(plaintext_key).decrypt(
        envelope["nonce"], envelope["ciphertext"], None
    )
    del plaintext_key
    return plaintext

At rest vs in transit, rotation, and audit

At rest vs in transit

Encryption at rest protects data sitting on disk, your database files, object storage, snapshots, backups. This is what KMS and envelope encryption are for: if someone walks off with the drive (or dumps the S3 bucket), they get ciphertext. Encryption in transit protects data moving across a network using TLS, so an eavesdropper on the wire sees nothing useful. You need both, they defend against different attackers. At-rest defends the stolen disk; in-transit defends the tapped wire. Covering one and skipping the other leaves a wide-open door. (For the in-transit half, see HTTPS, TLS & Encryption Basics.)

Key rotation

Keys should not live forever. Rotation means periodically retiring a key and creating a new one, so a key that leaks has a limited useful lifetime and you bound how much data any single key ever protected. The beauty of envelope encryption is that rotating the *master* key is cheap: old wrapped data keys can still be unwrapped by KMS (it keeps prior versions), so you do not have to re-encrypt your data to rotate the master. Managed KMS can rotate the master automatically on a schedule (e.g. yearly). Data keys, being per-object, rotate naturally as you write new data.

Access control and audit

A key is only as safe as the policy on it. Use IAM-style key policies to grant least privilege: this service may Encrypt, that service may Decrypt, and almost nobody may administer the key or schedule its deletion. Crucially, every operation against the key is logged. In AWS that means kms:Decrypt calls land in CloudTrail; Azure and GCP have equivalents. That audit trail is how you answer "who decrypted this record, from where, and when", and it is what an auditor will ask for. A key with no access policy and no audit log is a liability dressed up as a control.

Common mistakes that cost hours

1. Keys in code or config. Once a key hits Git, it is in the history forever, rotating it means rewriting history *and* re-keying. Treat any key that touches a repo as already compromised.
2. No rotation. A key that has protected three years of data is a single point of catastrophic failure. If it leaks, every byte it ever wrapped is exposed. Schedule rotation and let envelope encryption keep it cheap.
3. One key for everything. A single master key shared across services, tenants, and environments means one compromised caller decrypts the whole company. Scope keys per environment and per data domain to contain the blast radius.
4. No audit on key use. If you cannot say who decrypted what and when, you cannot detect abuse and you cannot pass an audit. Enable KMS call logging from day one, it is the cheapest control you will ever turn on.

Takeaways

Where to go next

Key management is one pillar of cloud security. Pair it with the credential side and the in-transit side to cover the whole story.

• Secrets Management, the companion piece: managing application credentials (API keys, DB passwords) rather than the cryptographic keys themselves.
• HTTPS, TLS & Encryption Basics, how the *in transit* half works, end to end.
• Cloud Engineer path, where key management, IAM, and network security fit into the full role.