Support for rotation of SSE-C keys.

[akaokunc]

How to categorize this issue?

/area backup
/kind api-change

What would you like to be added:
I'm proposing to add support for reading backups from S3 using multiple keys while keeping one key as the active one for writing. This would allow rotation of these keys which is currently not possible.

Why is this needed:
Currently etcb support using SSE-C key while uploading and downloading the backups to S3, but this support just single key. That causes impossibility to remove the SSE-C encryption or changing the key once it was used. I'm proposing to add support for using multiple keys for reading (since there is no way to find out which key was used for upload, we would try one by one) and using the first key for writing. The proposal is to replace the original file with SSE-C key with json structure.

I'm working at Akamai (formerly Linode) on our internal implementation and I'll soon have a patch which we would be able to upstream, so I'd like to gather feedback if there would be a desire for that, since the SSE-C support is already there.

This is the example of encryption config file which would replace the sseCustomerKey:

{
  "algorithm": "AES256",
  "disableEncryptionForWriting": false,
  "keys": [
    {
      "id": "primary-key-2024",
      "value": "base64-encoded-32-byte-key"
    },
    {
      "id": "backup-key-2023",
      "value": "another-base64-encoded-32-byte-key"
    }
  ]
}

[renormalize]

Hi @akaokunc! The feature does sound interesting and might be useful for consumers of etcd-backup-restore using SSE-C with S3 compatible providers. The initial support for SSE-C was also contributed by folks from Akamai.

Like you said, feel free to raise a draft, and @gardener/etcd-druid-maintainers will take a look and provide feedback.

[akaokunc]

Hello @renormalize, I've forward ported the change to current master and provided the draft patch as #936 . I had some difficulties with backwards compatibility which we likely want to keep, so there is support for the original sse configuration options as well as having the secrets in the new format. I was not sure how it is meant with directory/<keys_as_files>, directory/credentials.json and just credentials json, so for now it should support all the means of configuration. The SSECustomerKeyConf (the new format) can be provided as single file within the credentials directory or it can be one key in the credentials.json.

[renormalize]

@akaokunc sorry this hasn't moved for a couple weeks. I will spend some time this week to go through the PR.

I'm also considering unifying the way credentials are passed right now, since it is more complicated than it needs to be.