The new version of the nitriding framework can proxy http traffic
to a separate server running inside the enclave. This allows us
to replace the ffi calls from golang with a pure-rust server
implementation, which is less error-prone.

Use the `info!` convenience macro with the default logger,
which reads the `RUST_LOG` environment variable. So run with:

    RUST_LOG=tower_http=trace,star_randsrv=debug cargo run

for reasonable output.

By default it's `ERROR` level only, so there's no convenience
info about requests or listening port; would be nice to default
to `star_randsrv=info` or something like that.

Define structs for the request and response and plumb them
through with a dummy response value.

FIXME: only processes the first point.

Note that the `State` argument must come *before* the request,
not after!

Needs better error handling.

Use the current version of the ppoprf which cleans up some
unneeded dependencies. Bump everything else to the latest
compatible version.

This allows multiple concurrent reads to the ppoprf struct for
evaluations. We only need exclusive access to reset or puncture.

Make the API docs slightly less sparse

Spawn a background task to advance the current epoch on a fixed
schedule. When it runs out of epochs it eventually panics the
main thread instead of re-initializing the ppoprf state.

Instead of leaving all epochs puctured and later panicking, create
and fresh OPRFServer instance with the same set of epochs and
continue with that.

Abstract the struct initialization into OPRFServer::new() so it
can be called from both locations, Move the `future_epochs`
initialization inside the background loop so the same code
can be used each time a new state is initialized.

Define a response type for errors and convert various unwrap()
and expect() calls to return that instead.

Replace the iterator chain in the randomness handler with an
explicit loop so we can return early if there's an error in
the request data.

Turns out enum variants with inner values don't just look like
functions, they actually work as a FnOnce!

Also stonger camelcasing.

The previous code reset the `future_epochs` vector before the
empty check that was supposed to trigger re-initialization.
So OprfServer.epoch was updated, but the actual set of viable
epochs was punctured, resulting in perpetual "No prefix found"
errors.

Instead, assume we'll use contiguous blocks of epoch numbers
and store them as a Range, which is smaller and more convenient.
`EpochRange` never needs to change and can be passed to
`OprfServer::new` each time the key is rotated.

Note that because `Range` and `InclusiveRange` are different
types, we can't both keep the range as `u8` and include epoch
255. I though code simplicity was more important here.

Instead of linking over ffi from the go nitriding framework,
build it as a separate proxy daemon which runs inside the
the same container as star-randsrv.

Having `nitriding` as a submodule complicates copying just
that subtree to the go-builder, so we copy everything, which
is slow if there are local build files.

Build and lint the local application. Assume the nitriding
repo takes care of itself.

Rather than installing golangci-lint so the default Makefile
target in nitriding/cmd can run it, just build the `nitriding`
target directly. This is a faster build and follows our earlier
policy of treating the submodule as a separate unit for testing.

Copy `star-randsrv` from the correct path so both executables
are present in the final container.

This is 5x larger than the alpine containers, but provides
the expected execution environment.

Go back to using alpine linux for the runtime, which requires
building with CGO disabled, and on the rust:alpine build
container to avoid dependencies on glibc.

This is about 25% the size of the debian:slim runtime container.

Which to a revision of `nitriding` that builds a static executable
by default so we can invoke the Makefile but still use it with
our alpine runtime container.

Doesn't return the timestamp at the end of the epoch.

We don't support returning the zk proofs, so returning the public
key isn't useful, but neither does the current golang version,
and I wanted to match its api. I expect that to change when we
implement batch proofs anyway.

Instead of accepting any unpunctured epoch, and letting the
ppoprf eval fail for punctured ones, filter out non-current
epochs when the request is processed.

Also enforce the MAX_POINTS limit on requests. By this point
the point vec has already been constructed, so this doesn't
affect the amount of memory consumed, but it still limits
the cpu cost of each request, making any denial-of-service
mitigation based on network flows more effective.

Set up a testing framework inside the main implementation to check
the handlers without the overhead of a network socket. Based on
the `testing` example from the axum repository.

Test that the `/` route returns some text.

`TryFrom` will be useful for making `ppoprf::Point` creation
properly fallible. Also makes future updates smoother.

Co-authored-by: Philipp Winter <[email protected]>

Co-authored-by: Philipp Winter <[email protected]>

Co-authored-by: Philipp Winter <[email protected]>

Co-authored-by: Philipp Winter <[email protected]>

Dependency updates on the obsolete go side.

Resolve Cargo.lock conflicts omitted in the previous
commit.

Previously I implemented the stringification manually to keep
things shallow for auditors, but this makes the source more
compact without adding much functionality.

This adds about 4k to the release binary size.

Use the `From` impl with the `?` operator to map errors implicitly.
This simplifies reading the code, but I'm not convinced it's better
for auditing, where all the paths need to be traced.

Reformat to fix layout nits.
Used `cargo fmt -- --config max_width=78`

Use the `clap` crate to derive a parser and help text for setting
the webservice config options through command-line switches.

> STAR randomness webservice
>
> Usage: star-randsrv [OPTIONS]
>
> Options:
>       --epoch-seconds <EPOCH_SECONDS>  Duration of each randomness epoch [default: 5]
>       --first-epoch <FIRST_EPOCH>      First epoch tag to make available [default: 0]
>       --last-epoch <LAST_EPOCH>        Last epoch tag to make available [default: 255]
>   -h, --help                           Print help
>   -V, --version                        Print version

Confirm we get valid json with the correct keys and types.

Undo the two layers of encoding to recover the original
struct, throwing if any of the steps fails. This gives
some basic confirmation that the `/info` endpoint returned
something plausible.

Submit a random RistrettoPoint and confirm we get one back
under the correct epoch.

Use the same config for each test and call a helper function
to construct the `app` axum router state for testing. This
removes duplication and makes the test code more compact.

Move the redundant parts of generating a request for an endpoint
into a helper function. If a payload is included, the request
is POSTed as application/json.

This doesn't save any code really, but makes the tests a bit
easier to read since we lose the conversion code to build the
request.

We want something human-readable there.

We don't have unit tests other than for the endpoints,
so might as well be brief.

Fill in the `nextEpochTime` field in the `/info` response with
the RFC 3339 timestamp of the next rotation. Make the field
optional so we can set it on one place inside the update loop.

Just to shorten the timestamp string, which doesn't need
nanosecond precision.

Fill in a dummy timestamp whem making a test OPRFServer state.
This is sufficient to pass the simple `is_string` check in the
endpoint unit test.

Fix typos and elaborate on more fields for clarity.

Merge upstream Cargo.lock changes by just re-running
`cargo update` on this branch as well.

Apply standard formatting with a 78-character line length.

Save a couple of lines by setting response values from short
expressions inside the struct constructor.

Check that the values we use in `test_app` make it into
the response.

These are no longer needed since we're building the two servers
separately.

Change the welcome endpoint test to match the others.
Addresses a clippy lint.

Abstract some of the test code for the /randomness endpoint into
helper functions and call it from additional tests to verify
multiple point submissions in the same request are processed.

The /randomness endpoint allows specifying a specific epoch tag
to support the same route offering multiple valid epochs at the
same time. For example this could be two series with different
cadences or an overlap to prevent failed queries close to the
rotation time.

Verify that the current epoch can be accepted, but that earlier
and later epochs are rejected.

Use the latest stable rust toolchain and alpine runtime release
for the linux container image build.

Run `cargo update` to bump Cargo.lock to the latest compatible
releases.

Have the nitriding proxy listen on standard port 443. The daemons
run as root inside the container so there's no reason to use a
non-privileged port.

Pick up recent fixes and depend on an actual tag rather than
a random commit.

Un-indent the part of the next_epoch_time calculation which doesn't
require the lock guard for better readability. Still prefer an
scope block around the lock section to an explicit drop since
it's harder for it to become disconnected from the following code.

This might make pulling the object out of the structured log
easier? Addresses a review comment.

Clarify where these messages are from by declaring a span for them.

Allow setting the server's listening ip address and port from
the command line, rather than hard-coding it. The default is
the same behaviour as before: ipv4 localhost port 8080.

Break things up so the individual files are shorter and save
an indent level.

Saving an indent level lets us expand a wrapped line.

Split server into smaller files. Addresses review feedback.

Split the implementation across multiple source files in response
to review feedback.

The `OPRF*` and `Error` types are directly-related to the implementation
of these particular handlers, so I kept those together. They need to
be public so the main application can call them, but otherwise are
just interfacing with the axum::route machinery.

The epoch test has assertion that the epoch test values it uses
are distinct from the accepted value. These compile to nothing
because they are const expressions. However, it's nice to have
the assertion in case the base constant changes in an incompatible
way.

`cargo clippy --all-targets` warns about this test, so silence
the lint for a clean report while keeping the assertion.

This fetches the Docker image from Amazon's container registry instead of Docker's, which prevents the frequent rate limits in our reproducible build pipeline. At the time of commit, at least, the `docker/library` path is a mirror of the docker.io registry with identical images hashes for these
three tags.

Co-authored-by: Philipp Winter <[email protected]>

/usr/local/bin isn't in path in the alpine runtime container,
so it needs to be run from the absolute path.

Adjust Makefile to build a versioned eif file in docker, which
shortcuts for deployment. Some of this is style changes, but
other parts are functional; the previous `ko` build didn't
work for the non-go parts of the container.

This reduces the memory cap to 512 MB to match the default
allocation limit. This simplifies testing, especially since
bumping the limit has been failing occasionaly in my testing.
It's certainly enough memory for verifying deployment, but
benchmarking is needed to see if it's sufficient to handle
a full network load.

This fixes an issue building the container image under kaniko.

This is a little hacky since we don't generate complete dependency
information, but use nitriding/cmd/Makefile, invoked by the
Dockerfile build, as a stamp for having the submodule available.

Assumes we're running out of a git repository.

Keep the state structs and their maintenance code together to
avoid cross-importing between `handler` and the old `update`
module. Export just the wrapped version through the top-level
to shorten references there and in the handlers.

Save a line by not declaring this and pass a `false` literal
directly. Requested in review.

Automatic image building was failing because the nitriding submodule
wasn't initialized before invoking kaniko. This option should make
the checkout action also checkout any submodules.

Check the build script is working before merging into main.

This commit makes two changes:

1. Invoke kaniko with the flag '--custom-platform linux/amd64'.  This is
   necessary when building star-randsrv on non-Linux, non-amd64
   platforms like macOS.

2. Use an intermediate build layer to add start.sh.  If we don't do
   this, we may end up with a build layer that contains inconsistent
   file permissions from the host operating system.

With the above two changes, it's now possible to arrive at identical
image IDs, even when building star-randsrv on Linux (amd64) and macOS
(arm64).

Make star-randsrv build reproducibly.

If this happens it indicates an internal problem with the state
management, and the server likely needs to be restarted. Signal
this so the more serious error case is obvious.

Other error variants should be the result of bad client input
and don't affect the usability of the service.

The ppoprf epoch tag is a `u8`. When using the full epoch range
0..=255, the final increment would overflow. In debug builds this
would panic. In release builds, it would roll over unchecked, then
panic at the end of the next interval when it tried to puncture
the already-punctured starting epoch value. In either case, the
panic happened while the write lock was held, poisoning it.

The tokio thread runner would catch the panic, but with the
`RwLock` poisoned, it couldn't respond to further queries.

Instead, check for overflow and use that to trigger key rotation,
just as we do when the epoch is out of the configure range. This
is more localized than promoting the epoch counter and range to
u16 and narrowing at the ppoprf calls.

Addresses a clippy lint.

The default makefile target just builds, tests, lints and
audits the rust application, so we no longer need a go
environment for this job.

Configure Renovate

This was for testing with the deployable container build pipeline
during development. Remove before merging into the default branch
since it will no longer be needed there.

I'm told this is no longer necessary as the pipeline is using
a different solution.