justify Fernet

[chadwhitacre]

Could we unpack the decision to use Fernet as the basis for cryptography's symmetric encryption recipe? I learned about Fernet from cryptography. I expected to find that it is a well-vetted standard, but from what we've been able to dig up, Fernet seems rather to be a largely unknown protocol, extracted ad-hoc from a Ruby gem developed internally at Heroku. I searched the cryptography-dev archives, and didn't find any discussion of the rationale for trusting Fernet. I also don't find any explanation in the initial PR, the docs, or the source. I do find a ticket about writing a tutorial for "re-inventing most of fernet," but nothing about why Fernet was chosen in the first place.

Why did cryptography choose Fernet for its symmetric encryption recipe?

[collisdigital]

I'm also interested in this, we've been working with JWT and JWS/JWE (JSON Web Token/Signsture/Encryption) recently (with Python cryptography) as a good basis for standards based approaches for the web and wonder if this might be a good option for cryptography in future (I.e select a profile of JWT within the standard).

https://jwt.io
https://tools.ietf.org/html/rfc7519
https://tools.ietf.org/html/rfc7515
https://tools.ietf.org/html/rfc7516

[chadwhitacre]

we've been working with JWT and JWS/JWE (JSON Web Token/Signsture/Encryption) recently (with Python cryptography)

Interesting. Is your work publicly available?

[collisdigital]

https://github.com/ONSdigital/eq-survey-runner

[chadwhitacre]

Thanks! Looks like app/authentication/jwt_decoder.py is where the action is, ya?

I had wondered what alternatives exist to Fernet. Why was Fernet chosen over JWT/S/E, I wonder? Perhaps because Fernet is simpler?

[collisdigital]

Yeah also we have an encoder for a different part of the app: https://github.com/ONSdigital/eq-survey-runner/blob/master/app/submitter/encrypter.py

[collisdigital]

If this project wants to use JWT/S/E I'd be happy to collaborate on adding support but would like to get agreement that it is a winner first, starting with documenting a profile to use etc.

[reaperhulk]

Cryptography has a project goal of providing opinionated constructs that give reasonable security with a minimal of configuration "footguns". Fernet was chosen as our first recipe for several reasons, but in brief:

• JWT/JWE/JWS were not standardized when we chose to implement Fernet.
• Fernet has no configuration options and the design favors versioning agility over algorithmic configuration.

Cryptographic design by committee (TLS prior to 1.3, JW{E,T,S}) tends to favor large amounts of configuration options which drastically increases complexity and makes it very difficult for laypeople to know whether they've done the right thing. For example, jwcrypto, which uses cryptography and implements these standards, has the following list of supported algorithms for encryption:

['RSA1_5', 'RSA-OAEP', 'RSA-OAEP-256', 'A128KW', 'A192KW', 'A256KW', 'ECDH-ES', 'ECDH-ES+A128KW', 'ECDH-ES+A192KW', 'ECDH-ES+A256KW', 'A128GCMKW', 'A192GCMKW', 'A256GCMKW', 'PBES2-HS256+A128KW', 'PBES2-HS384+A192KW', 'PBES2-HS512+A256KW', 'A128CBC-HS256', 'A192CBC-HS384', 'A256CBC-HS512', 'A128GCM', 'A192GCM', 'A256GCM']

Forcing users to make choices like this is antithetical to our stated project goal for the recipes. To make use of JWE, then, we'd need to essentially pick one for you. That isn't necessarily a terrible approach, but since we've added Fernet already now there would be two symmetric encryption recipes. How do we document to users that you can pick either one and that the relative security of them is the same, but some users might prefer to use an RFC standardized construction (but where we've picked the algorithm for you).

These issues aren't insurmountable by any means, but I think we'd need a strong justification to add a secondary one-shot symmetric encryption recipe when we already have a solid one. Having JW{S,E,T} available in hazmat (with full configuration options, that's what hazmat is for) is perhaps a more sensible target, as our other stated goal of being a 'cryptographic standard library' would indicate that properly lives inside this project.

[collisdigital]

Agree with the above and that the project would select a cipher to use etc. rather than being open. Also could add a non-hazmat assymetric encryption and signing via JWT? The main reason for JWT over Fernet from my perspective is the stronger standardisation of JWT family and decent specification documents to follow via the RFCs.

[reaperhulk]

@collisdigital If we go ahead and add JW{S,E,T} to hazmat then we can have the conversation about whether we should hoist up a pre-picked algorithm as a recipe. I am supportive of getting those specs into our hazmat layer, although it'd be nice if we could avoid duplication of effort and potentially fold an existing project into our codebase (with whatever style/testing changes our project policies require).

[chadwhitacre]

JW{S,E,T}

I see what you did there. :-P

The main reason for JWT over Fernet from my perspective is the stronger standardisation of JWT family and decent specification documents to follow via the RFCs.

Ditto. In our audit we are converging on accepting Fernet, but a less boutique protocol would've been preferable.

Fernet was chosen as our first recipe for several reasons

Ah! I forgot to check the IRC logs ... oooh, but they only go back to February, 2014. :-/ Ftr, I would expect to find at least a summary on GitHub or the mailing list, even if the original discussion happened privately. A cross-link to the IRC logs from #203 would've been great, e.g., were it possible. Comments such as "decide whether we want fernet at all until we convince them to make some changes" inspire great curiosity. ;)

but in brief:

• JWT/JWE/JWS were not standardized when we chose to implement Fernet.
• Fernet has no configuration options and the design favors versioning agility over algorithmic configuration.

Fair enough. Thanks for the explanation.

Closing this out; I'll let yinz reticket pursuit of the JW{S,E,T} idea. Go team! :-)

[reaperhulk]

@whit537 Yeah our early project decision documentation is somewhat lacking unfortunately. Time has blunted my memory a bit, but the main concern we originally had (although credit goes to @dstufft for originally raising the issue) was around the fact that Fernet's wire format exposes a plaintext timestamp. For some use cases this is useful and it is possible to use this safely, but it represents a chink in the "bulletproof can't use it wrong" armor we desire.

[dstufft]

It's possible the decision was made in a different IRC channel too, a lot of stuff early on happened before we really had a dedicated one.

[chadwhitacre]

the main concern we originally had was around the fact that Fernet's wire format exposes a plaintext timestamp

Yeah, I did discover traces of that in my spelunking:

The Fernet recipe comes with a notice about leaking timestamps (first a sentence, then upgraded to a warning, finally downgraded to a note). Here's where it was requested, since timestamps "can be sensitive information" ("plus it feels dirty"). Having this warning erodes the sense that cryptography recipes are "safe and easy to use," though the whole idea of outsourcing crypto safety may be suspect.

I tend to agree with @dstufft's assessment that "it feels dirty" (as does Fernet's, ahem, "versioning agility" ;). In light of JWT and other reading, I'm seeing Fernet's timestamping as basically a stunted header. It makes sense in the context of Fernet's original design intention, to communicate secrets between applications in a distributed architecture. It's irrelevant in our own use-case, which is application-layer encryption prior to long-term storage (we're not going to try to teach Postgres to enforce TTLs on the Fernet tokens we're asking it to store ;). Fernet's timestamping is probably safe for us, but it is a "chink in the armor," as you say (could leaking the time of encryption interact negatively with key rotation, for instance?).

Fernet makes TTL enforcement on the decryption side optional, but not timestamping on the encryption side. A symmetric encryption recipe that made both sides optional would seem preferable—perhaps something to keep in mind when developing teh glorious JW{S,E,T} future.

[chadwhitacre]

we're not going to try to teach Postgres to enforce TTLs on the Fernet tokens we're asking it to store

(Although I guess that could be pretty easily done with a trigger ... hmm ... ;)

[chadwhitacre]

until we convince them to make some changes

Tickets that seemingly relate:

• "Spec Readme should state that b64 encoding uses the urlsafe variant"
• "Provide timestamps for vectors in unix time"
• "Document the need for a clock skew check"

[chadwhitacre]

Interesting! I just discovered #2714, a ticket about the timestamp issue, where @Ayrx shares a link to python-aead, an implementation of an IETF draft that also underlies RNCryptor. Hmm ...