Ca functionality

[robert-cronin]

This PR is to add CA functionality to polykey which includes the ability to act as a CA for peers but also the ability to respond to certificate requests from other services through a standard API.

Fixes #48
Fixes #105

[robert-cronin]

ready for review, all tests have been updated and are passing locally 👍

[CMCDragonkai]

What is this api/generated/*? What is making this and why do we need it?

[CMCDragonkai]

Tests are failing on gitlab too. Something to do with the agent simulation.

[robert-cronin]

What is this api/generated/*? What is making this and why do we need it?

the generated folder contains the API server and client code generated from swagger-codegen. I've kept the generated code outside the src folder to follow the precedent.

[robert-cronin]

Tests are failing on gitlab too. Something to do with the agent simulation.

I will fix these and re-commit 👍

[CMCDragonkai]

I'm wondering if we can just write our API directly without generating code. Like the way we do it in Ouroboros. It seems like a lot of code that is generated.

…

On 23 September 2020 12:22:17 pm AEST, Robbie Cronin ***@***.***> wrote: > What is this `api/generated/*`? What is making this and why do we need it? the generated folder contains the API server and client code generated from `swagger-codegen`. I've kept the generated code outside the src folder to follow the precedent. -- You are receiving this because your review was requested. Reply to this email directly or view it on GitHub: #94 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[robert-cronin]

We are already kind of doing this, I will get rid of the stuff we don't need from the generated folder. Also I guess we don't really need to generate the client? we're really only using it for testing.

To be honest, we could get away with just using the generated yaml file and write all the logic ourselves in the src directory.

[robert-cronin]

Here are the changes made in this PR:

1. Add CA functionality (generating root cert/keypair, responding to certificate signing request by other peers and services)
2. Add gRPC service to agent to replace the custom net.Socket we were using before (also uses the CA module to generate client and server certificates to secure the agent/client communications)
3. Add HTTP API (this was to serve the CA functionality as an API but also to get the openapi/swagger stuff started)

[CMCDragonkai]

Why does npm testing keep failing.

[CMCDragonkai]

Is there an issue for the openapi/swagger/HTTP Api integration. Seems like this PR brings that in as well.

[CMCDragonkai]

Can you review the workflow of the netboot system, and see if polykey solves netboot's requirements?

[CMCDragonkai]

We need to review 2 things.

Whether this solves the iPXE Netboot problem. And the mapping/relationship between this and the smallstep commands. Does pk now act as both the CA and the CA client where small step is separating this into 2 commands.

For the iPXE netboot problem, we need to have a meeting about this. But have a read of this: https://ipxe.org/crypto

Primary usecase of the CA functionality here, is to facilitate mTLS for third party applications.

Please address the requirements here and describe how the CA functionality/what GRPC calls/CLI commands address them?

• https://smallstep.com/blog/private-acme-server/
• https://smallstep.com/blog/use-ssh-certificates/

How does this CA functionality interact with DNS?

I need more detailed diagrams/documentation about this CA PR.

[CMCDragonkai]

What are all the files in test/lib/api/client for? There's alot of generated files there.

[CMCDragonkai]

Also why not remove webpack entirely at this point?

[CMCDragonkai]

Ever since we removed dist from typescript demo lib, does that mean here we no longer have dist anymore?

[robert-cronin]

We still have dist, but I should probably get rid of it in favour of our new approach and in light of it still being uploaded to npm if its in .gitignore (overridden with .npm_ignore)

[robert-cronin]

client folder in tests has been removed, we are now just using http.request

[robert-cronin]

I am just making some ascii diagrams for understanding so I will post them here as I go and put them into the readme once finished.
Here is how mutual TLS is supported by our polykey CA system:

1. Peer A requests a certificate (can act as both client and server certificate for mtls)

    Request Cert
    for domain   +---------+
    (gRPC)       |         |
    +----------->+ CA Peer |
    |            |         |
    |            +---+-----+
    |                |
+---+----+           |            +--------+
|        |           |            |        |
| Peer A <-----------+            | Peer B |
|        | Signed certificate     |        |
+--------+ if peer is trusted     +--------+
           (gRPC)

2. Peer B requests a certificate (can act as both client and server certificate for mtls)

                             Request Cert
                 +---------+ for domain
                 |         | (gRPC)
                 | CA Peer +<----------+
                 |         |           |
                 +----+----+           |
                      |                |
+--------+            |           +----+---+
|        |            |           |        |
| Peer A |            +---------->+ Peer B |
|        |     Signed certificate |        |
+--------+     if peer is trusted +--------+
               (gRPC)

3. mTLS connection is then established.

                +---------+
                |         |
                | CA Peer |
                |         |
                +---------+

+--------+                        +--------+
|        |  Mutual TLS Connetion  |        |
| Peer A +<-----------------------> Peer B |
|        |        (gRPC)          |        |
+--------+                        +--------+

[robert-cronin]

The following is how the CA functionality is exposed over HTTP for external services

1. external service (e.g. web server) asks for a signed certificate over HTTP.service is challenged and returned the signed certificate if successful:

        for domain   +---------+
        (HTTP)       |         |
        +----------->+ CA Peer |
        |            |         |
        |            +---+-----+
        |                |
+-------+----+           |            +-----------------------+
|            |           |            |                       |
| Web Server <-----------+            | Client (e.g. browser) |
|            | Signed certificate     |                       |
+------------+ after challenge passed +-----------------------+
               (HTTP)

2. client (e.g. browser) gets the root certificate from the CA peer via HTTP:

                     +---------+  Get Root
                     |         |  Certificate (HTTP)
                     | CA Peer +<-----------------+
                     |         |                  |
                     +---------+                  |
                                                  |
+------------+                        +-----------+-----------+
|            |                        |                       |
| Web Server +<-----------------------+ Client (e.g. browser) |
|            |                        |                       |
+------------+                        +-----------------------+

3. one way TLS connection is established:

                     +---------+
                     |         |
                     | CA Peer |
                     |         |
                     +---------+

+------------+                        +-----------------------+
|            |     TLS connection     |                       |
| Web Server +<-----------------------+ Client (e.g. browser) |
|            |                        |                       |
+------------+                        +-----------------------+

[CMCDragonkai]

Add in some design notes regarding code signing:

• Generate a Code Signing Certificate smallstep/certificates#147
• https://en.wikipedia.org/wiki/Non-repudiation

iPXE also supports code signing, which allows you to verify the authenticity and integrity of files downloaded by iPXE. To enable support for code signing, you must enable the IMAGE_TRUST_CMD build configuration option, and use the imgtrust command within an embedded script.
https://ipxe.org/crypto

Basically it means the x509 certificates can be used for lots of things. And it is up the CA to give these capabilities.

[CMCDragonkai]

Another thing we need to understand is that certificates have "capabilities".

These are encoded as Key Usage Constraints and Extended Key Usage Constraints.

Examples include "Code Signing" which enables the certificate to be used for signing digital artifacts.

Most commonly is the certificate is used for TLS Web Server Authentication, TLS Web Client Authentication. These 2 are what is necessary to do mTLS.

But there's a whole bunch of other capabilities as well.

[CMCDragonkai]

What is the role of CSR for Polykey?

The CA does not generate certificates.

It is the "leaf" nodes that generate certificates, and then ask the CA to sign them.

Whether the CA signs them or not depends on a CSR process.

Let's use an example. For Let's Encrypt, it just needs to know whether the requester/provisioner "owns" the domain that it's asking to get. This is why they have a bunch of different domain verification techniques like DNS check and HTTP/HTML check.

Smallstep has instead a token-based provisioner system. We can do something similar with our OAuth mechanism. But really let's investigate what this really means.

[CMCDragonkai]

• Every Keynode has a Root Keypair (this root keypair is used for vault keypairs, and also its public key represents the identity of the keynode)
• Every Keynode also as a Root x509 Certificate
• What is the relationship between the Root Keypair and the Root x509 Certificate?

There's a few options here:

1. Start with the root keypair and then use a KDF to generate the root x509 certificate pair.
2. Start with the root x509 certificate pair, and then use a KDF to generate the root keypair
3. Use the public-key of the root keypair as the actual public key of the x509 certificate
4. Use the x509 certificate and private key AS the root keypair

Ok so the problem is that the current root keypair are PGP keys.

I'm not sure which of the 4 options are viable.

But it appears that x509 certificates CAN be used as the root keypair because PGP (algorithm/protocol?) supports these keys.

One thing though, we should be using ECDSA algo rather than RSA... but this is something we might configure later.

The reason we should do this, is that keynodes now have a keypair to represent their identity. BUT by UNIFYING PGP and X509 together, it means the keynodes can be identified via PGP protocol OR x509 protocol.

In addition to this, it also means, every keynode can now be a CA.

[CMCDragonkai]

Once the keynodes have a root certificate. Instead of presenting the root public key around. They present the root certificate to other keynodes.

However for social sharing we still want to preserve the ability of using ECDSA (ed25519) keys since they very short. They can be easily read out and written down and published in text form wherever.

However during the "keynode discovery phase", the keynodes are actually exchanging the public certificate. This then has more information and capabilities encoded within it. The extra information can be used to identify/associate other pieces of information with keynodes (this was one of the original goals of x509, to associate physical identity with digital identity). And the ability to sign certificates... essentially gives us transitive trust.

Although PGP allows this too... I'm not too familiar with PGP certificates and who it exposes that transitive trust.

Now once every keynode has a root certificate. That's when every keynode is capable of being a CA!

[CMCDragonkai]

Once a keynode is a CA. We need to expose several features:

1. Signing CSRs given by token-authenticated clients (this requires the OAuth2 PR Add OAuth2 #64 to finish)
2. We can verify things
3. Do pretty what smallstep CA does.
4. It hosted and exposing the CA functionality...
5. Providing a GET endpoint for the root and intermediate certificate

[CMCDragonkai]

One benefit of doing this is that as soon as the keynodes have root public certificates. Then it is possible to use them for mTLS between Polykey keynodes for their GRPC connection.

This unifies peer to peer and agent to peer transit security with the identity system of keynodes. So that simplifies ALOT of things.