-
Notifications
You must be signed in to change notification settings - Fork 235
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Rewrite README to be more up-to-date/accurate (#145)
- Loading branch information
1 parent
9baf572
commit dca6ef1
Showing
1 changed file
with
132 additions
and
17 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,29 +1,144 @@ | ||
| This is the beginnings of a certificate transparency log | ||
| client written in Go, along with a log scanner tool. | ||
| # Certificate Transparency: Go Code | ||
|
|
||
| You'll need go v1.8 or higher to compile. | ||
| [](https://travis-ci.org/google/certificate-transparency-go) | ||
| [](https://goreportcard.com/report/github.com/google/certificate-transparency-go) | ||
| [](https://godoc.org/github.com/google/certificate-transparency-go) | ||
|
|
||
| # Installation | ||
| This repository holds Go code related to | ||
| [Certificate Transparency](https://www.certificate-transparency.org/) (CT). The | ||
| repository requires Go version 1.9. | ||
|
|
||
| This go code must be imported into your go workspace before you can | ||
| use it, which can be done with: | ||
| - [Repository Structure](#repository-structure) | ||
| - [Trillian CT Personality](#trillian-ct-personality) | ||
| - [Working on the Code](#working-on-the-code) | ||
| - [Rebuilding Generated Code](#rebuilding-generated-code) | ||
| - [Updating Vendor Code](#updating-vendor-code) | ||
| - [Running Codebase Checks](#running-codebase-checks) | ||
|
|
||
| ## Repository Structure | ||
|
|
||
| The main parts of the repository are: | ||
|
|
||
| - Encoding libraries: | ||
| - `asn1/` and `x509/` are forks of the upstream Go `encoding/asn1` and | ||
| `crypto/x509` libraries. We maintain separate forks of these packages | ||
| because CT is intended to act as an observatory of certificates across the | ||
| ecosystem; as such, we need to be able to process somewhat-malformed | ||
| certificates that the stricter upstream code would (correctly) reject. | ||
| Our `x509` fork also includes code for working with the | ||
| [pre-certificates defined in RFC 6962](https://tools.ietf.org/html/rfc6962#section-3.1). | ||
| - `tls` holds a library for processing TLS-encoded data as described in | ||
| [RFC 5246](https://tools.ietf.org/html/rfc5246). | ||
| - `x509util` provides additional utilities for dealing with | ||
| `x509.Certificate`s. | ||
| - CT client libraries: | ||
| - The top-level `ct` package (in `.`) holds types and utilities for working | ||
| with CT data structures defined in | ||
| [RFC 6962](https://tools.ietf.org/html/rfc6962). | ||
| - `client/` and `jsonclient/` hold libraries that allow access to CT Logs | ||
| via entrypoints described in | ||
| [section 4 of RFC 6962](https://tools.ietf.org/html/rfc6962#section-4). | ||
| - `scanner/` holds a library for scanning the entire contents of an existing | ||
| CT Log. | ||
| - Command line tools: | ||
| - `./client/ctclient` allows interaction with a CT Log | ||
| - `./scanner/scanlog` allows an existing CT Log to be scanned for certificates | ||
| of interest; please be polite when running this tool against a Log. | ||
| - `./x509util/certcheck` allows display and verification of certificates | ||
| - `./x509util/crlcheck` allows display and verification of certificate | ||
| revocation lists (CRLs). | ||
| - CT Personality for [Trillian](https://github.com/google/trillian): | ||
| - `trillian/` holds code that allows a Certificate Transparency Log to be | ||
| run using a Trillian Log as its back-end -- see | ||
| [below](#trillian-ct-personality). | ||
|
|
||
|
|
||
| ## Trillian CT Personality | ||
|
|
||
| The `trillian/` subdirectory holds code and scripts for running a CT Log based | ||
| on the [Trillian](https://github.com/google/trillian) general transparency Log. | ||
|
|
||
| The main code for the CT personality is held in `trillian/ctfe`; this code | ||
| responds to HTTP requests on the | ||
| [CT API paths](https://tools.ietf.org/html/rfc6962#section-4) and translates | ||
| them to the equivalent gRPC API requests to the Trillian Log. | ||
|
|
||
| This obviously relies on the gRPC API definitions at | ||
| `github.com/google/trillian`; the code also uses common libraries from the | ||
| Trillian project for: | ||
| - exposing monitoring and statistics via an `interface` and corresponding | ||
| Prometheus implementation (`github.com/google/trillian/monitoring/...`) | ||
| - dealing with cryptographic keys (`github.com/google/trillian/crypto/...`). | ||
|
|
||
| The `trillian/integration/` directory holds scripts and tests for running the whole | ||
| system locally. In particular: | ||
| - `trillian/integration/ct_integration_test.sh` brings up local processes | ||
| running a Trillian Log server, signer and a CT personality, and exercises the | ||
| complete set of RFC 6962 API entrypoints. | ||
| - `trillian/integration/ct_hammer_test.sh` brings up a complete system and runs | ||
| a continuous randomized test of the CT entrypoints. | ||
|
|
||
| These scripts require a local database instance to be configured as described | ||
| in the [Trillian instructions](https://github.com/google/trillian#mysql-setup). | ||
|
|
||
|
|
||
| ## Working on the Code | ||
|
|
||
| Developers who want to make changes to the codebase need some additional | ||
| dependencies and tools, described in the following sections. The | ||
| [Travis configuration](.travis.yml) for the codebase is also useful reference | ||
| for the required tools and scripts, as it may be more up-to-date than this | ||
| document. | ||
|
|
||
| ### Rebuilding Generated Code | ||
|
|
||
| Some of the CT Go code is autogenerated from other files: | ||
|
|
||
| - [Protocol buffer](https://developers.google.com/protocol-buffers/) message | ||
| definitions are converted to `.pb.go` implementations. | ||
| - A mock implementation of the Trillian gRPC API (in `trillian/mockclient`) is | ||
| created with [GoMock](https://github.com/golang/mock). | ||
|
|
||
| Re-generating mock or protobuffer files is only needed if you're changing | ||
| the original files; if you do, you'll need to install the prerequisites: | ||
|
|
||
| - `mockgen` tool from https://github.com/golang/mock | ||
| - `protoc`, [Go support for protoc](https://github.com/golang/protobuf) (see | ||
| documentation linked from the | ||
| [protobuf site](https://github.com/google/protobuf)) | ||
|
|
||
| and run the following: | ||
|
|
||
| ```bash | ||
| go get github.com/google/certificate-transparency-go/client | ||
| go get github.com/google/certificate-transparency-go/scanner | ||
| # etc. | ||
| go generate -x ./... # hunts for //go:generate comments and runs them | ||
| ``` | ||
|
|
||
| # Building the binaries | ||
| ### Updating Vendor Code | ||
|
|
||
| To compile the log scanner run: | ||
| The codebase includes a couple of external projects under the `vendor/` | ||
| subdirectory, to ensure that builds use a fixed version (typically because the | ||
| upstream repository does not guarantee back-compatibility between the tip | ||
| `master` branch and the current stable release). See | ||
| [instructions in the Trillian repo](https://github.com/google/trillian#updating-vendor-code) | ||
| for how to update vendored subtrees. | ||
|
|
||
|
|
||
| ### Running Codebase Checks | ||
|
|
||
| The [`scripts/presubmit.sh`](scripts/presubmit.sh) script runs various tools | ||
| and tests over the codebase. | ||
|
|
||
| ```bash | ||
| go get github.com/google/certificate-transparency-go/scanner/scanlog | ||
| ``` | ||
| # Install gometalinter and all linters | ||
| go get -u github.com/alecthomas/gometalinter | ||
| gometalinter --install | ||
|
|
||
| # Run code generation, build, test and linters | ||
| ./scripts/presubmit.sh | ||
|
|
||
| # Contributing | ||
| # Run build, test and linters but skip code generation | ||
| ./scripts/presubmit.sh --no-generate | ||
|
|
||
| When sending pull requests, please ensure that everything's been run | ||
| through ```gofmt``` beforehand so we can keep everything nice and | ||
| tidy. | ||
| # Or just run the linters alone: | ||
| gometalinter --config=gometalinter.json ./... | ||
| ``` |