fix: EdDSA to Ed25519 token migration

[Stellatsuu]

cc @aldbr
Closes: #718

Changes:

• changed token creation algorithm to be Ed25519 instead of EdDSA
• added Ed25519 to the authorized algorithm list. EdDSA is still listed as authorized algorithm so people have time to rotate keys before full migration

Documentation:

Token migration and keys rotation

The EdDSA token algorithm is being deprecated by RFC 9864, so it should be replaced with the Ed25519 token algorithm. To do this, users need to rotate their token keys using diracx rotate function.

Prerequisites:

• Existing EdDSA keys prior to the migration.
• Ensure joserfc >= 1.5.0, otherwise, it won't work.

Important note:
Once the migration from EdDSA to Ed25519 is complete, it is recommended to rotate the Ed25519 keys after a certain period of time.

Locally (using DiracX demo) - only used as tests for this PR

1. Create a demo instance using the main branch (or any branch without this PR implementation): {PATH}/run_demo {PATH}/diracx

• Note 1: you must create the demo instance from another branch due to the behavior of diracx-charts. If you create the demo instance from this PR branch, it will automatically generate an Ed25519 key, preventing you from properly testing the migration/rotation process.
• Note 2: ensure your pixi environment includes joserfc >= 1.5.0 before creating the demo instance. You will later need to switch branches, and without the correct joserfc version, the new Ed25519 implementation will not work properly.

2. Export Kube configuration and retrieve token secret: kubectl get secret diracx-jwks -o yaml, output:

apiVersion: v1
data:
  jwks.json: ewogICJrZXlzIjogWwogICAgewogICAgICAiY3J2IjogIkVkMjU1MTkiLAogICAgICAieCI6ICJhck9YQmhtejNzY2xtMXZZcm5KTEdLSzJ3WWNfLWcxaGYtRW9wcVV5VVFvIiwKICAgICAgImQiOiAiZHRQeEJUcVRsdmZkVmVlYzlBdnFuTTZQekJHelItNlRnTGZ0NE5rcVZpdyIsCiAgICAgICJrZXlfb3BzIjogWwogICAgICAgICJzaWduIiwKICAgICAgICAidmVyaWZ5IgogICAgICBdLAogICAgICAiYWxnIjogIkVkRFNBIiwKICAgICAgImtpZCI6ICIwMTljNmJmMzU2YjU3ZGMyYjg3MjY3ZTNkZmM2NmQ5OCIsCiAgICAgICJrdHkiOiAiT0tQIgogICAgfQogIF0KfQ==
...

3. Save old keys as jwks.json: echo "{jwks.json.VALUE}" | base64 --decode > jwks.json

• Note: values inside jwks.json should only contain "alg": "EdDSA" before migration.

4. Switch branch to this PR branch, keeping the same demo instance
5. Log-in: dirac login diracAdmin, it should work.
6. Rotate the keys using jwks.json: python -m diracx.logic rotate-jwk --jwks-path jwks.json

• Note 1: values inside jwks.json should now contain "alg": "EdDSA" and "alg": "Ed25519"
• Note 2: note all kid values in jwks.json for later use.

7. Update token secret value and restart the pods:

• kubectl create secret generic diracx-jwks --namespace="$namespace" --from-file=jwks.json --dry-run=client -o yaml | kubectl apply -f -
• kubectl rollout restart deployment diracx-demo

Production

The following procedure applies both to:

• Migrating from EdDSA to Ed25519 (one-time only)
• Periodically rotating Ed25519 keys

1. Retrieve your token secret id: kubectl get secret
2. Export Kube configuration and retrieve token secret: kubectl get secret {SECRET_TOKEN_ID} -o yaml, output:

apiVersion: v1
data:
  jwks.json: ewogICJrZXlzIjogWwogICAgewogICAgICAiY3J2IjogIkVkMjU1MTkiLAogICAgICAieCI6ICJhck9YQmhtejNzY2xtMXZZcm5KTEdLSzJ3WWNfLWcxaGYtRW9wcVV5VVFvIiwKICAgICAgImQiOiAiZHRQeEJUcVRsdmZkVmVlYzlBdnFuTTZQekJHelItNlRnTGZ0NE5rcVZpdyIsCiAgICAgICJrZXlfb3BzIjogWwogICAgICAgICJzaWduIiwKICAgICAgICAidmVyaWZ5IgogICAgICBdLAogICAgICAiYWxnIjogIkVkRFNBIiwKICAgICAgImtpZCI6ICIwMTljNmJmMzU2YjU3ZGMyYjg3MjY3ZTNkZmM2NmQ5OCIsCiAgICAgICJrdHkiOiAiT0tQIgogICAgfQogIF0KfQ==
...

3. Save old keys as jwks.json: echo "{jwks.json.VALUE}" | base64 --decode > jwks.json

• Note: if migrating, values inside jwks.json should only contain "alg": "EdDSA" before migration.

4. Try to log-in, it should work.
5. Rotate the keys using jwks.json and diracx rotate function: python -m diracx.logic rotate-jwk --jwks-path jwks.json

• Note 1: if migrating, values inside jwks.json should now contain "alg": "EdDSA" and "alg": "Ed25519"
• Note 2: if migrating, note the kid values in jwks.json for later use.

7. Update token secret value and restart the pods:

• kubectl create secret generic {SECRET_TOKEN_ID} --namespace="$namespace" --from-file=jwks.json --dry-run=client -o yaml | kubectl apply -f -
• kubectl rollout restart deployment {POD_ID}

Check-up after the rotation

Once migration and rotation are done:

• Log-in again: it should work
• Decode .cache/diracx/credentials.json and verify that it contains "alg": "Ed25519".
• If migrating, ensure old EdDSA key kid is different from new Ed25519 key kid (see step 6 (local) and step 5 (production), note 2)

[aldbr]

That looks good, thank you 🙂
I just have a few minor comments around the tests.
Next step: making sure there is a documentation page to rotate the keys (and delete old ones if needed). If it does not exist, then we need to create one based on #499 (comment)

[read-the-docs-community]

Documentation build overview

📚 diracx | 🛠️ Build #31400200 | 📁 Comparing 152795e against latest (11433a3)

🔍 Preview build

Show files changed (3 files in total): 📝 3 modified | ➕ 0 added | ➖ 0 deleted

File	Status
admin/reference/env-variables/index.html	📝 modified
dev/reference/coding-conventions/index.html	📝 modified
dev/explanations/components/db/index.html	📝 modified

[Stellatsuu]

Small documentation

How to rotate the keys:

1. Launch demo (+ export Kube config)
2. Retrieve secrets ids (optional): kubectl get secrets
3. Retrieve old keys: kubectl get secret diracx-jwks -o yaml
   Output:

data:
  jwks.json: ewogICJrZXlzIjogWwogICAgewogICAgICAiY3J2IjogIkVkMjU1MTkiLAogICAgICAieCI6ICJ0QXY5VmZDUy1ER0I0d3hkUlRScHpoRlBLOU1wbmNUOHZVRHdaMnBJclBVIiwKICAgICAgImQiOiAiVG5vSG0teWJDTGVOY2drbm1ma3lTMkJGSGl2enBEblV2OFl1S2xaMjNNTSIsCiAgICAgICJrZXlfb3BzIjogWwogICAgICAgICJzaWduIiwKICAgICAgICAidmVyaWZ5IgogICAgICBdLAogICAgICAiYWxnIjogIkVkMjU1MTkiLAogICAgICAia2lkIjogIjAxOWM1NjVhMWI1NzczMjE5MGQ2ZGVlYjVjY2MxYzAzIiwKICAgICAgImt0eSI6ICJPS1AiCiAgICB9CiAgXQp9
...

4. Save olds keys as jwks.json

echo "ewogICJrZXlzIjogWwogICAgewogICAgICAiY3J2IjogIkVkMjU1MTkiLAogICAgICAieCI6ICJ0QXY5VmZDUy1ER0I0d3hkUlRScHpoRlBLOU1wbmNUOHZVRHdaMnBJclBVIiwKICAgICAgImQiOiAiVG5vSG0teWJDTGVOY2drbm1ma3lTMkJGSGl2enBEblV2OFl1S2xaMjNNTSIsCiAgICAgICJrZXlfb3BzIjogWwogICAgICAgICJzaWduIiwKICAgICAgICAidmVyaWZ5IgogICAgICBdLAogICAgICAiYWxnIjogIkVkMjU1MTkiLAogICAgICAia2lkIjogIjAxOWM1NjVhMWI1NzczMjE5MGQ2ZGVlYjVjY2MxYzAzIiwKICAgICAgImt0eSI6ICJPS1AiCiAgICB9CiAgXQp9" | base64 --decode > jwks.json

5. Rotate the keys: python -m diracx.logic rotate-jwk --jwks-path jwks.json
6. Delete the old secret: kubectl delete secret diracx-jwks --namespace=$namespace
7. Upload the new secret (with the new keys):
   kubectl create secret generic diracx-jwks \ --namespace=$namespace \ --from-file=jwks.json \

How to check if rotation worked:

1. dirac login diracAdmin -> rotate key -> submit job -> check if ok
2. re-login -> /Users/{username}/.cache/diracx/credentials.json
   -> decode -> check if alg = Ed25519

[Stellatsuu]

joserfc error breaking the demo when trying to login as diracAdmin:

• HttpResponseError: Operation returned an invalid status 'Internal Server Error'
• kubectl logs diracx-demo-...

Full Logs:

File "/diracx_source/diracx/diracx-logic/src/diracx/logic/auth/token.py", line 380, in create_token
   return jwt.encode(
          ^^^^^^^^^^^
 File "/opt/conda/lib/python3.11/site-packages/joserfc/jwt.py", line 80, in encode
   return serialize_compact(_header, payload, key, algorithms, registry)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/conda/lib/python3.11/site-packages/joserfc/jws.py", line 105, in serialize_compact
   alg: JWSAlgModel = registry.get_alg(protected["alg"])
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/conda/lib/python3.11/site-packages/joserfc/_rfc7515/registry.py", line 81, in get_alg
   raise UnsupportedAlgorithmError(f"Algorithm of '{name}' is not supported")
joserfc.errors.UnsupportedAlgorithmError: unsupported_algorithm: Algorithm of 'Ed25519' is not supported

See: https://jose.authlib.org/en/guide/errors/#unsupportedalgorithmerror

Edit: error was due to wrong joserfc version in demo dependencies.

[aldbr]

I followed your steps on my side to check and I have a few minor comments 🙂

Small documentation

How to rotate the keys:

1. Launch demo (+ export Kube config)

Note: you launch the demo only when you want to test the key rotation with run_demo locally. In that case, you should use the current version of the rotate_key() (not your version within this PR). It is used by diracx-charts init-keystore to generate the initial key. If you use the new rotate_key(), then you will generate an Ed25519 key instead of an EdDSA key 😉

2. Retrieve secrets ids (optional): kubectl get secrets
3. Retrieve old keys: kubectl get secret diracx-jwks -o yaml
   Output:

data:
  jwks.json: ewogICJrZXlzIjogWwogICAgewogICAgICAiY3J2IjogIkVkMjU1MTkiLAogICAgICAieCI6ICJ0QXY5VmZDUy1ER0I0d3hkUlRScHpoRlBLOU1wbmNUOHZVRHdaMnBJclBVIiwKICAgICAgImQiOiAiVG5vSG0teWJDTGVOY2drbm1ma3lTMkJGSGl2enBEblV2OFl1S2xaMjNNTSIsCiAgICAgICJrZXlfb3BzIjogWwogICAgICAgICJzaWduIiwKICAgICAgICAidmVyaWZ5IgogICAgICBdLAogICAgICAiYWxnIjogIkVkMjU1MTkiLAogICAgICAia2lkIjogIjAxOWM1NjVhMWI1NzczMjE5MGQ2ZGVlYjVjY2MxYzAzIiwKICAgICAgImt0eSI6ICJPS1AiCiAgICB9CiAgXQp9
...

So here, if you try to decode the jwks.json content you will notice that the key is already Ed25519.

4. Save olds keys as jwks.json

echo "ewogICJrZXlzIjogWwogICAgewogICAgICAiY3J2IjogIkVkMjU1MTkiLAogICAgICAieCI6ICJ0QXY5VmZDUy1ER0I0d3hkUlRScHpoRlBLOU1wbmNUOHZVRHdaMnBJclBVIiwKICAgICAgImQiOiAiVG5vSG0teWJDTGVOY2drbm1ma3lTMkJGSGl2enBEblV2OFl1S2xaMjNNTSIsCiAgICAgICJrZXlfb3BzIjogWwogICAgICAgICJzaWduIiwKICAgICAgICAidmVyaWZ5IgogICAgICBdLAogICAgICAiYWxnIjogIkVkMjU1MTkiLAogICAgICAia2lkIjogIjAxOWM1NjVhMWI1NzczMjE5MGQ2ZGVlYjVjY2MxYzAzIiwKICAgICAgImt0eSI6ICJPS1AiCiAgICB9CiAgXQp9" | base64 --decode > jwks.json

5. Rotate the keys: python -m diracx.logic rotate-jwk --jwks-path jwks.json

Here you should use the updated script.

6. Delete the old secret: kubectl delete secret diracx-jwks --namespace=$namespace
7. Upload the new secret (with the new keys):
   kubectl create secret generic diracx-jwks \ --namespace=$namespace \ --from-file=jwks.json \

I think it's cleaner to update the existing secret with:

kubectl create secret generic diracx-jwks \
  --namespace="$namespace" \
  --from-file=jwks.json \
  --dry-run=client -o yaml \
| kubectl apply -f -

You need to restart the diracx-demo pod(s) with:

kubectl rollout restart deployment diracx-demo -n $namespace

How to check if rotation worked:

1. dirac login diracAdmin -> rotate key -> submit job -> check if ok
2. re-login -> /Users/{username}/.cache/diracx/credentials.json
   -> decode -> check if alg = Ed25519

In the context of the demo, you also need to check the kid, as you might have started with an Ed25519 key if you started the demo with the new rotate_key function.