Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: refactor router and router inner #566

Merged
merged 8 commits into from
Jan 6, 2023
Merged

feat: refactor router and router inner #566

merged 8 commits into from
Jan 6, 2023

Conversation

oddgrd
Copy link
Contributor

@oddgrd oddgrd commented Jan 6, 2023 •
edited
Loading

Refactor the axum-wasm router, and implement some error handling.

I have some doubts about the error handling in handle_request, currently I propagate the errors, trace! their context and return a 500 from hyper. But I'm thinking we should panic on a lot of these errors, since if we can't connect/write to the module in one request, we probably won't be able to in the next.

The wasmtime API's return anyhow::Errors, so I opted to use that and not create an Error enum. However since this refactor is meant to ease future development, we might want to do it now anyway? Another thing I'm wondering is if the RouterBuilder is worthwhile, since I feel it might as well just be a function currently, but here again having the builder might be worthwhile for future development when we might do more configuration.

chesedo
chesedo reviewed Jan 6, 2023
View reviewed changes
Copy link
Contributor

@chesedo chesedo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, this is great, thanks @oddgrd!

I think going with anyhow is fine. This isn't a library and we can always upgrade in the future

common/src/wasm.rs Outdated Show resolved Hide resolved
runtime/src/axum/mod.rs Show resolved Hide resolved
runtime/src/axum/mod.rs Outdated Show resolved Hide resolved
runtime/src/axum/mod.rs Show resolved Hide resolved
@oddgrd oddgrd marked this pull request as ready for review January 6, 2023 14:14
@oddgrd oddgrd merged commit 8324824 into shuttle-hq:shuttle-next Jan 6, 2023
@oddgrd oddgrd deleted the feature/eng-361-refactor-router-and-router-inner branch January 6, 2023 17:18
chesedo added a commit that referenced this pull request Mar 20, 2023
@oddgrd @brokad
@chesedo @akrantz01 @trezm @XyLyXyRR @Nereuxofficial @biryukovmaxim @guerinoni @tsar-boomba @Procrat @stavares843 @HexPandaa @kierendavies
chore: merge main into prod for v0.12.0 (#730)
97c0d07 
* feat: shuttle-serenity initial commit poc (#429)

* feat: shuttle-serenity initial commit poc

* remove shuttle-service

* refactor: drop more shuttle_service stuff

* refactor: drop default serenity framework

* misc: add wasm32-wasi to nix shell

* refactor: cargo sort

* refactor: cargo fmt

Co-authored-by: chesedo <[email protected]>

* shuttle next wrapper POC (#431)

* feat: runtime skeleton

* feat: cleanup

* feat: update cargo.lock, sort deps

* fix: typo

* refactor: create runtimes workspace (#432)

* refactor: rename plugins to runtimes

* refactor: drop serenity folder

* refactor: rename runtime to shuttle-next

* refactor: prepare shuttle-legacy runtime

* feat: add   --provisioner-address arg to both runtimes (#433)

* refactor: fix up dependencies

* feat: add --provisioner-address to both runtimes

* feat: create a control plane interface (part 1) (#436)

* feat: add proto for runtimes

* refactor: legacy move main to lib

* feat: impl Runtime server for legacy

* feat: impl Runtime server for next

* refactor: combine runtimes into one (#438)

* feat: add --legacy flag

* refactor: merge next and legacy runtimes

* refactor: merge proto crates

* refactor: rename to runtime

* refactor: promote runtime

* refactor: update README.md

* refactor: cargo sort

* feat(deployer): add support for building wasm projects (#437)

* feat: move factory to runtime (#444)

* feat: initial commit of moving factory to runtime

* feat: replace deployer factory with dummy,
clean commented code

* feat: update legacy runtime readme

* WIP feat: start runtime from deployer (#450)

* WIP feat: start runtime from deployer

* feat: attempt to start runtime as a child process

* fix: add current dir

* feat: clean up, update readme

* feat: use cargo manifest dir for runtime path

* refactor: clean up, fmt

* refactor: clippy warnings for Eq

Co-authored-by: chesedo <[email protected]>

* feat: get logs from runtime (#459)

* feat: get logs from runtime

* refactor: trim down deployer

* refactor: start runtime earlier

* refactor: connect to client earlier

* refactor: hook runtime logs to persistence

* bug: associate deployment id with logs

* refactor: cleanup

* feat: make sure grpc connection stays open

* feat(shuttle-next): first edition of axum-wasm router (#472)

* feat(shuttle-next): POC of axum wasm router

* feat: use std mutex

* feat: serialize/deserialize http requests to rmp

this serializes/deserializes http requests across the ffi to the axum-wasm router, using the rust messagepack dataformat. sending the body is still a wip

* feat: serialize the full HTTP req/res to rmp

* fix: comment typo

* feat: start hyper with tonic and serve wasm router

* feat: clone inner router arc

* feat: send request body without serialization

* docs: todo comment

* feat: write response body

* fix: serialize response parts

* feat: deserialize parts directly from reader

this also adds a new fd to separate streaming of body and parts

* feat: add axum-wasm to runtime cli

* refactor: remove unused method

* refactor: typo

Co-authored-by: Pieter <[email protected]>

* refactor: comments, clean up wrappers

* refactor: move axum-wasm utils to shuttle-common

* refactor: fmt

* refactor: clippy

Co-authored-by: Pieter <[email protected]>

* Feat(shuttle-next): stop runtime services (#481)

* feat: add tonic endpoint to stop runtime services

* refactor: use tracing

* feat: implement stop method for serenity bot

* feat: install protoc 21.9 in dockerfile

* refactor: comment

* feat: convert uuid bytes to string then uuid

* docs: update readme

* docs: update readme for serenity bot

* feat: implement stop for axum-wasm and document it

* ci: install newer protoc in CI

* feat: remove axum-wasm feature in runtime for now

we will come back to this later and likely split the runtime by features

* docs: remove feature flags from readme

* fix: clippy

* fix: clippy

* fix: clippy

* feat: send kill signal without spawning task

* feat: return error response on kill_tx err

* feat(next): expand macro into axum routes (#488)

* feat: app codegen model

* refactor: qualify all namespaces

* feat: low-level wasi export fn

* refactor: restrict to supported axum methods

* Feat: parse shuttle::endpoint macro (#490)

* feat: parse params

* feat: parse app

* feat: add test for missing endpoint attribute

* test: invalid args, params and syntax

* feat: check for and test no params

* feat: don't emit error for missing annotation

this way users can create separate functions for their endpoint logic

* fix: remove missing attribute error, not param

* refactor: reword comment

Co-authored-by: Pieter <[email protected]>

* feat: duplicate param test, address review

* feat: only strip endpoint attributes

* feat: check for extra endpoint attributes

* refactor: clearer error span for extra endpoints

* feat: has_err variable to group param errors

* refactor: remove optional hint

Co-authored-by: Pieter <[email protected]>

* docs: add TODO for multiple endpoint attributes

Co-authored-by: Pieter <[email protected]>

* feat: merge main into shuttle-next (#543)

* feat(blog): add missing sqlx migration code to auth blog post (#408)

* feat(blog): add missing sqlx migration code

* fix: nit

* misc: 0.7.0 (#407)

* misc: drop patches

* infra: pin postgres to version 14

* refactor: fix thruster example app name

* infra: db_fqdn fix

* tests: warp hello world

* fix(cargo-shuttle): prevent crash when config owned by root (#409)

* fix: use correct timeout start point (#410)

* fix: use correct timeout start point

* tweak health check frequency

* fmt

* feat(deployer): implement container memory limits (#411)

* feat(deployer): implement container memory limits

* test: fix warp integration test

* bug: `transport error` when trying to connect to provisioner (#416)

* bug: keep provisioner connections alive

* refactor: reduce scope provisioner client is running

* feat: gateway admin revive (#412)

* feat: gateway admin command (revive)

* fmt

* clippy

* refactor: revive deployers using GatewayService

* tests: add ContextArgs

* refactor: simplify passing around of fqdn

* tests: update test archive

* refactor: remove stray exec.rs file

* refactor: unused is_error()

Co-authored-by: chesedo <[email protected]>

* bug: timeout curl health check on deployer (#415)

* Article/beta article (#420)

* updated docs url

* beta article w/ damiens corrections

* Feat(www): shuttle beta signup (#421)

* feat(www): add beta signup form

* feat: add formspree endpoint

* feat: remove socials footer from beta page

* feat: add dummy text above sign up form

* feat: placeholder gif

* feat: align dummy text left

* feat: remove gif, add text

* feat: update beta page text

* feat: update the rest of the site

* bug: Fix thruster postgres example (#414)

* feat: shell completions (#343)

* bug: package Secrets.toml (#422)

* bug: package Secrets.toml

* refactor: clippy suggestion

* bug: big archives being cut off at 32 768 bytes (#423)

* refactor: set chunked header

* refactor: don't send stream across threads

Doing this is causing us to loose everything after 32 768 bytes. Don't
know the reason why 🤷

* refactor: fix health check

* refactor: remove unused use

* chore: v0.7.1 (#424)

* chore: v0.7.1

* chore: resources v0.7.1

* docs: v0.7.1

* tests: v0.7.1

* tests: fix warp test

* tests: fix warp test

* tests: fix secrets e2e test

* ci: remove hard coded DD env

* misc: unintended shell changes

* refactor: cargo check suggestion

* refactor: provide better context for errors (#430)

* refactor: do our own health checks on deployer containers (#427)

* refactor: do our own health checks on deployer containers

We are doing this because the docker health checks eats up 25% of a
single CPU core and we are spinning up many of these containers.

https://www.reddit.com/r/docker/comments/b68r53/healtchecks_add_high_cpu_load/
moby/moby#39102

* refactor: clippy suggestion

* bug: clear build folder before extracting (#428)

* refactor: make sure extract directory is created

* tests: test build folder is cleared

* bug: clear the build directory

* added some images (#435)

* feat(www): beta blog updates (#434)

* feat: add captioned image component (#440)

* feat: add captioned image component

* feat: update images

* feat: add header caption

* fix: quotes in header

* www: post small tweaks (#439)

* a few tweaks to the article

* feat: update gif

Co-authored-by: oddgrd <[email protected]>

* feat: replace cursed gif (#441)

* Feat/update contributing (#426)

* feat(docs): update contributing.md

* feat(e2e): make e2e admin user unique

* feat(e2e): clean up e2e test projects on test complete

* feat(e2e): don't fail on superuser conflict

* feat: add section about contributing to docs

* feat: remove unwrap, add comment to drop impl

* tests: update e2e test key

Co-authored-by: chesedo <[email protected]>

* chore: v0.7.2 (#442)

* chore: v0.7.2

* chore: resources v0.7.2

* docs: v0.7.2

* misc: v0.7.2

* misc: script to deploy all examples

* misc: targets for publishing crates

* misc: remove old scripts

* misc: instructions for next steps

* misc: get Cargo.lock changes

* misc: update .PHONY

* misc: show usage

* feat: link the tracing spans between services (#445)

* ci: DD environments

* refactor: try lowering broadcast channel for quicker feedback

* refactor: propagate tracing across threads in deployer

* refactor: propagate tracing between gateway and deployer

* refactor: trace account name

* refactor: associate project with each container

* feat: trace and propagate proxy in gateway

* feat: gateway record project

* refactor: tracing use 'error' for errors

* refactor: fix comment

* refactor: drop recording unused field

* Add docker-compose extra flags param in Makefile (#446)

* Add docker-compose extra flags param

* rename to avoid confusion

* misc: restructure repo (#453)

* misc: only run e2e on production

* misc: move www to shuttle-hq/www repo

* misc: move examples to shuttle-hq/examples repo

* refactor: update links to examples repo

* ci: remove checking fmt of examples

* ci: checkout submodules

* refactor: tf files have been moved to shuttle-hq/terraform-aws-shuttle

* fix: wrap around common::ProjectName for parsing (#451)

* fix: gateway state drifts, health checks and project recreation (#447)

* misc: add more helpful flags to Makefile

* misc: remove old migrator (#463)

* feat: add account_tier column (#458)

* feat: prefetch shuttle-service crates (#461)

* feat: prefetch shuttle-service crates

* refactor: add comment to prepare.sh files

* Feat: revive via gateway endpoint (#460)

* feat: revive via gateway api initial commit

* feat: cleanup, document gateway testing

* refactor: use post for revive endpoint

Co-authored-by: Damien <[email protected]>

* feat: send task to global queue

Co-authored-by: Damien <[email protected]>

* feat: create a new admin cli binary crate (#462)

* feat: init shuttle-admin

* feat: add args

* feat: simple client

* feat: read api_key

* refactor: hook it all together

* refactor: switch to post

* fix: broken link (#467)

* fix: add timeout to health checks (#468)

* fix: add timeout to health checks

* Parameterise the timeout

* rollback on inadvertent circleci change

* misc: log out a warning when a task has been running for a long time

* longer timeouts

* Address review

* Fmt

* feat(gateway): add custom domains table and routing (#465)

* WIP feat: count recent start events before restart (#469)

* feat: count recent start events before restart

* feat: try_collect and handle error in start counter

* fix: clippy

* refactor: gateway clippy

* feat: make deployer only answer its own project (#466)

* feat: make deployer only answer its own project

* bug: use correct project name

* refactor: make backwards compatible

* Fixed Links in Readme (#477)

* refactor: base client error off response status code (#470)

* refactor: base client error off response status code

* Update common/src/models/error.rs

Co-authored-by: Oddbjørn Grødem <[email protected]>

* refactor: use tracing

Co-authored-by: Oddbjørn Grødem <[email protected]>

* feat: verify project exists before sending destroy task (#474)

* feat: add a custom domains admin route (#473)

Co-authored-by: Alex Krantz <[email protected]>

* bug: deployer freezes (#478)

* bug: reduce spawning to have deployer lock up less

* refactor: don't return cargo logs

* ci: green (#482)

* tests: allow longer time for build

* refactor: trim dependencies on persist

* refactor: clippy suggestion

* feat: TLS acceptor with SNI resolver (#471)

* fix: custom domain routing (#484)

* refactor: more metrics (#475)

* feat: add more metric dimensions to deployer

* feat: add more metric dimensions to gateway

* refactor: common metrics code

* refactor: forward account name

* refactor: add backend feature to deployer

* refactor: standardize naming

* refactor: cargo sort

* misc: configurable deployment tags (#486)

* feat: gateway restores removed containers (#485)

* fix: backend bumps and hot fixes (#487)

* Feature/support actix web (#491)

* Support actix-web: impl Service

* Support actix-web: cli + boilerplate code

* Support actix-web: add framework to doc comment

* Support actix-web: add e2e test

* Support actix-web: fmt

* Support actix-web: sorting toml deps

* Support actix-web: add actix-web to ci

* Support actix-web: formatted boilerplate code for actix-web hello world example

* Support actix-web: formatted boilerplate code for actix-web hello world example
Support actix-web: update cargo lock zstd dep

* Support actix-web: simplify example

* Support actix-web: add test, change example dependency

* fix: e2e test assert, reset example module

Co-authored-by: maksim <[email protected]>
Co-authored-by: oddgrd <[email protected]>

* Doc: Fix command to prime database with docker-compose (#502)

Signed-off-by: Federico Guerinoni <[email protected]>

Signed-off-by: Federico Guerinoni <[email protected]>

* Doc: Improve contributing documentation (#499)

In the doc it mentions to go to subfolder of examples but it was not
there :).

Signed-off-by: Federico Guerinoni <[email protected]>

Signed-off-by: Federico Guerinoni <[email protected]>

* feat: static file support for a single folder (#501)

* feat: static-folder resource

* refactor: make local client work

* refactor: make local deployer work

* feat: storage_manager

* refactor: comments

* refactor: remove unwraps and expects

* refactor: update tests

* refactor: clippy suggestion

* refactor: update version

* refactor: update readme

* refactor: update comment

* refactor: change public to static

* ci: add static-folder

* refactor: code fixes

* refactor: update tests

* refactor: update description

Co-authored-by: Damien <[email protected]>

Co-authored-by: Damien <[email protected]>

* ci: build binary (#483)

* ci: build binary

* ci: make tar archive

* ci: make GH draft release with artifacts

* ci: add aarch64

* ci: get version from tag

* ci: rename to linux builds

* ci: add windows

* ci: add macos

* ci: test all

* ci: put all the artifacts under a command

* ci: only on tagged

* ci: restore old

* ci: fix windows

* ci: put archive in separate folder

* ci: switch to tar

* ci: restore old

* refactor: better `cargo-binstall` support

Co-authored-by: Isaiah Gamble <[email protected]>

* refactor: binary name

Co-authored-by: Isaiah Gamble <[email protected]>

Co-authored-by: Isaiah Gamble <[email protected]>

* feat: bump rust to 1.64, bump dependencies (#495)

* feat: bump rust to 1.64, correct contributing.md bug

* feat: bump service deps, change example branch

* fix: pin chrono to 4.22

* chore: upgrade workspace dependencies

* feat: remove aws-smithy-types

ref: smithy-lang/smithy-rs#1760

* feat: upgrade axum to 0.6 in common

* feat: upgrade to axum 0.6.0 in deployer

* feat: upgrade gateway to axum 0.6.0

* feat: upgrde sqlx to 0.6.2 in gateway

* feat: bump chrono to 0.4.23

* feat: replace deprecated chrono functions in deployer

* feat: bump chrono in common

* feat: ignore actix integration test

* feat: implement new state extractor in gateway

* feat: interactive project initialization (#498)

* feat: bump pinned rust version to 1.65 (#504)

* feat: bump pinned rust version to 1.65

 lockfile update is from binary dist PR #483

* ci: install newer protoc

* fix: special module name warning

* fix: clippy

* feat: install newer protoc in dockerfile as well

* misc: env updates (#509)

* ci: update toolchain for binaries

* misc: update dev environment

* misc: remove patches from Cargo.lock

* feat: make the folder configurable (#508)

* feat: make the folder configurable

* Update resources/static-folder/README.md

Co-authored-by: Damien <[email protected]>

Co-authored-by: Damien <[email protected]>

* feat: implement workspace inheritance (#506)

* refactor: switch away from cargo package (#507)

* feat: add spinner wait for `project new` and `project status --follow` (#503)

Signed-off-by: Federico Guerinoni <[email protected]>
Co-authored-by: oddgrd <[email protected]>

* fix: capitalise correctly (#511)

* fix: make nice (#512)

* feat: find (soon to be) invalid project names (#479)

* feat: find (soon to be) invalid project names

* refactor: move logic to admin client

* refactor: missed axum 0.6 update (#513)

* feat: build tests in release profile, limit build workers (#514)

* feat: build tests in release profile, limit build workers

* fix: typo in manifests

* feat: create `init` project from correct dir (#518)

* misc: interactive init gif (#519)

* Feat/set examples submodule to main (#520)

* feat: revert examples submodule back to main

* feat: un-ignore actix tests

* chore: 0.8.0 (#521)

* chore: bump examples (#522)

* bug: hacking static folders (#524)

* fix: actix integration with state (#523)

* feat(gateway,deployer): add more tracing events (#500)

* feat(deployer): add more tracing events

* feat(gateway): add more tracing events

* feat: canonicalize before trace (#531)

* feat: 'clean' subcommand (#530)

* feat: 'clean' subcommand

* refactor: output cleaning is done

* Feat/set cpu limit (#529)

* feat: remove redundant actix worker limit

* feat: remove redundant build job limit

* feat: set cpu limit for deployer container

* fix: limit actix worker and build jobs to 4

* feat: add panamax for mirroring crates.io (#525)

* feat: build queue (#532)

* feat: per-project parallelism (#533)

* feat: temp validation of project name in gateway (#534)

* fix(deployer): keep Cargo.lock between deployments (#517)

* ci: remove build and push req on build binaries (#535)

* refactor: don't crash when failing to release slot (#536)

* refactor: release build slot parse type correctly (#538)

* refactor: remove prefetch (#539)

* feat: add cron job for syncing mirror (#537)

* chore: bump cargo-shuttle to 0.8.1 (#540)

* chore: bump cargo-shuttle to 0.8.1

* chore: cargo.lock

* refactor: update manifests and lockfile

implement workspace inheritance for runtime, and remove default-features from shuttle-service

* refactor: fmt

* fix: unused imports, add runtime to deploymentmanager

* feat: move storagemanager to common

* fix: storage manager imports

* fix: codegen test with outdated expected error

* fix: service tests after removing service default features

* fix: deployer tests after removing service default features

* fix: typo

* ci: attempt to fix protoc error

* fix: deployer runtime client connection

* refactor: remove redundant feature flags

* refactor: remove storage_manager from deployer crate

Signed-off-by: Federico Guerinoni <[email protected]>
Co-authored-by: Pieter <[email protected]>
Co-authored-by: Alexander Krantz <[email protected]>
Co-authored-by: Damien <[email protected]>
Co-authored-by: Ivan <[email protected]>
Co-authored-by: Peter Mertz <[email protected]>
Co-authored-by: XyLyXyRR <[email protected]>
Co-authored-by: Damien <[email protected]>
Co-authored-by: Nereuxofficial <[email protected]>
Co-authored-by: Maxim <[email protected]>
Co-authored-by: maksim <[email protected]>
Co-authored-by: Federico Guerinoni <[email protected]>
Co-authored-by: Isaiah Gamble <[email protected]>
Co-authored-by: Stijn ("stain") Seghers <[email protected]>

* refactor: one store per request (#510)

* refactor: read from file

* refactor: make store for every request

* refactor: remove the serenity runtime (#549)

* Feat: shuttle next bump deps (#551)

* fix: axum tests

* feat: bump wasmtime crates

* feat: return streaming body from wasm router (#558)

* feat: stream body to and from router

* fix: reader to stream hack didn't work in wasm

I added a post endpoint and test, which proved that my hacky stream from BufReader didn't work on the wasm side

* refactor: test string

* feat: update codegen with axum-wasm changes

* refactor: clean up

* refactor: typo

* feat: guard against large request bodies

* feat: embed runtime into client and deployer (#559)

* refactor: runtime startup code

* feat: hook cli to runtime

* feat: hook up logs

* feat: custom port

* feat: start a next project with the local runner

* feat: embed executable

* refactor: axum update

* refactor: tonic version from workspace

* refactor: kill runtime correctly

* feat: DB resources for local runs

* feat: static folders for local runs

* feat: secrets for local runner

* refactor: cleanup logs and errors

* refactor: rebuild runtime if it changed

* refactor: more comments

* feat: minimal axum logs

* refactor: unneeded default features

* refactor: fix rerun-if for runtime

* bug: codegen handle_request

* refactor: restore db error

* bug: expanded broken merge (#567)

* feat: refactor router and router inner (#566)

* refactor: clean up imports

* refactor: remove RouterInner

* feat: handle wasi errors

* feat: handle_request error handling

* docs: comments

* refactor: make hyper dev-dep, better error message

* fix: axum-wasm missed import and bracket

* feat: hook in runtime logs (#568)

* feat: simple wasm log to send over unix socket

* feat: hook up logs subscription

* refactor: extra comments

* refactor: simpler interface

* refactor: combine JsonVisitors

* refactor: clippy suggestions

* refactor: update tests

* refactor: extra comment

* refactor: increase channel for better performance

* refactor: move off stderr

* misc: merge main (#572)

* Merge remote-tracking branch 'upstream/main' into shuttle-next

* refactor: touch ups

* fix: missing feature flag in common dep (#573)

* feat: change log read to spawn blocking (#574)

* feat: DX (#577)

* feat: shuttle-next crate for better DX

* refactor: trim dependencies

* bug: fix next check

* ci: add shuttle-runtime

* refactor: activating missing features on dependencies

* refactor: update to shuttle_next::endpoint

* feat: get runtime binary from cargo install (#578)

* feat: get runtime binary from cargo install

* refactor: remove build.rs

* refactor: determine environment with debug assertions

* ci: comment out cargo-sort installation

* fix: clippy

* feat: use cargo home dir, install runtime in prepare.sh

* fix: unused import

* refactor: build from local version in prepare.sh

* fix: local debug run installing from incorrect path

* feat: canonicalize path to debug runtime

* feat: set release runtime install branch to prod

* refactor: move secondary protoc install to common stage

* feat: deployer next (#575)

* feat: propagate next runtime

* feat: store is_next in DB

* feat: runtime manager to allow deployer to start up both runtimes

* feat: make sure tests run

* refactor: better migration query

* refactor: handle runtime errors better

* feat: shutdown runtimes

* bug: missing so

* bug: stop services

* bug: ffi and runtime manager not living long enough

* bug: missing so error

* refactor: run cleanups

* refactor: clippy suggestions

* misc: remove unneeded codegen feature (#584)

* refactor: proto fixups (#585)

* refactor: comment

* refactor: restore binding to external address

* Feat: cleanup fds and tmp (#586)

* feat: remove redundant body fd and /tmp

* feat: update codegen

* refactor: remove tmp from workspace exclude

* ci: next patch and wasi target

* refactor: unwraps and mor (#587)

* refactor: better error handling

* refactor: trim mutexes

* refactor: remove abstract factory

* refactor: remove extension todo

* feat: build our images with the release profile (#583)

* feat: build our images with the release profile

Currently our containerfile builds images with the default debug profile, which means when `get_runtime_executable` is called it will try to install from a checked out local version of `shuttle-runtime`.

* feat: add build arg for cargo profile

* refactor: update comment

* fix: broken codegen/axum test

* feat: install wasm32-wasi target in deploy.sh

* feat: build deps in `CARGO_PROFILE`

* feat: use closure to determine shuttle-runtime path

* fix: path to runtime in docker deployer, clippy

* fix: incorrect function for cargo home

* refactor: always use cargo_home for path to runtime in deployer

* Feature/eng 378 axum wasm multiple handlers per endpoint (#588)

* refactor: clean up if let

* feat: move method validation, add test for it

* test: add test for chained endpoint handlers

* feat: chain endpoints with the same address

* feat: duplicate method error, handler chain bug fix

chaining handlers with a full type path caused a bug where a `,` was inserted, I think the reason is that `path` types can't be followed by `.`. Importing all the routing methods in `app` works, but it is not ideal

* refactor: remove some clones

* refactor: remove unwrap

Co-authored-by: Pieter <[email protected]>

* feat: use BTreeMap for building endpoint chains

* fix: only add namespace to first handler in chain

* fix: skip rustfmt for quote in app_to_token test

rustfmt will add a comma to the expected output, which will also be done in the expanded macro output, but not in the stringified output

* refactor: revert btreemap change

* docs: document why we sort the endpoint chains

---------

Co-authored-by: Pieter <[email protected]>

* refactor: find code to wrap legacy runtime (#675)

* refactor: find code to wrap legacy runtime

* misc: update README with test instructions

* feat: remove box self from services, remove syncwrapper from axum service (#677)

* feat: remove box self from all services

* feat: remove syncwrapper from axum integration

* Fix/move docker profiles to dev (#674)

* refactor: move docker profiles to dev.yml

* docs: update comment in makefile

* feat: extract next runtime into separate binary (#679)

* feat: extract next runtime into separate binary

* fix: remove startrequestsservice name

* Feature: remove tokio runtime from all resources (#680)

* feat: remove runtime from resources

* fix: compile errors, comments

* fix: remove comment

* feat: trim the service loader, unpin tokio (#681)

* feat: trim the serviceloader, unpin tokio

* fix: tokio sync is needed

* feat: unpin tokio in runtime as well

* fix: static folder tests need tokio rt

* feat: rename loader, build_crate return bin path

* fix: build_Crate invalid args

* chore(typos): fix typos (#682)

* chore(typos): fix typos

* Fix indentation for poise init example (#687)

* Feature/eng 477 make wasm dependencies optional (#688)

* feat: make wasm runtime dependencies optional

* feat: feature gate next lib, update readme

* ci: rename loader feature in ci

* refactor: hyper with server feature in next features

* Feature: eng 465 update all the codegens (#686)

* feat: initial commit of codegen rewrite

* refactor: wrapper to loader

* tests: update the unit tests

* fix: don't unwrap return type

* feat: setup tracing in loader codegen

* fix: pass inn logger to loader

* feat: mvp of shuttle-next local run

* refactor: move logger to runtime

* refactor: move storagemanager export to runtime

* docs: add note about git tags in contrib (#691)

* bug: idle static folder (#692)

* misc: add more logs

* bug: copy static folder

* bug: extra subdirectory

* chore: v0.11.3 (#695)

* Feature: eng 483 trim and fix the tests in shuttle-service (#693)

* tests: update build_crate tests

* tests: remove loader tests

* feat: cleanup service deps

* feat: setup integration tests in runtime

* feat: expected panic message for not_shuttle

* refactor: simplify dummy provisioner

* feat: re-export main and service from runtime

* fix: Respect `Cargo.lock` when building containers (#700)

Building the shuttle containers was failing during the `cargo chef cook`
step because of a cyclic dependency introduced by an upstream crate.
This did not happen when building outside of a container, because the
versions pinned by `Cargo.lock` did not contain any cycles.

See tkaitchuck/aHash#95

* ci: resolve CI errors in shuttle-next (#580)

* test: compile wasm module in axum runtime test setup

* ci: add next patch override to CI

* ci: include wasm32-wasi target in rust install

* fix: deployer tests where runtime fails to start

* fix: incorrect provisioner address

* feat: log service state changes in runtime

* feat: don't send stop req on startup failure

* refactor: unused imports

* refactor: handling legacy panics

* tests: deadlock less

* refactor: fixups

* refactor: clippy suggestions

* tests: mock provisioner

* refactor: restore capture from 'log' and colors

* refactor: clippy suggestions

* tests: longer wait

* tests: don't panic while holding lock

* tests: don't panic on stream closed

* tests: don't filter out state logs

* tests: bigger timeout

* ci: remove duplicate patch

* refactor: comments

---------

Co-authored-by: chesedo <[email protected]>

* Merge remote-tracking branch 'upstream/main' into shuttle-next

* Feature/eng 486 update deployer with runtime changes (#696)

* feat: update deployer to work with runtime changes

* feat: pass next feature flag to runtime installs

* fix: revert this change, the error is a rust-analyzer bug

* tests: make deployer test resources bins

* ci: add shuttle-runtime patch to CI

* refactor: consistent naming

* fix: redundant clone of runtime manager

* feat: delete rocket bin from runtime

* feat: make storagemanager a trait object

* docs: add comment about why we use arc

* refactor: use &str in storagemanager

* feat: add runtime patch to deployer

* fix: deployer logic bug, add debug logs

* feat: extract service integrations into separate crates (#702)

* feat: draft of extracting service integrations

* refactor: revert codegen changes

* refactor: depend on shuttle_runtime & service integration

* feat: remove rocket bin, comment out version check

* feat: impl from for integration wrapper

* feat: cleanup shuttle-axum docs, remove version check

* feat: actix-web service integration

* feat: extract poem service integration

* feat: extract poise service integration

* feat: extract rocket service integration

* feat: extract salvo service integration

* feat: extract the serenity service integration

* feat: extract the thruster service integration

* feat: extract warp service integration

* feat: extract the tower service integration

* feat: delete persist from service

* feat: extract tide service integration

* feat: update cargo.lock

* feat: make service integration inner pub

* fix: merge fixes

* refactor: rename integrations, remove comment

* ci: run check-standalone on services

* feat: update test resources

* ci: refactor workspace-clippy job

* fix: add tokio dev dep to services

* fix: remaining services tests

* [next] tests: CI go green (#704)

* tests: thread local tracing

* tests: bind panic in runtime

* tests: post create project params

* tests: fix cargo-shuttle

* misc: std missing from tracing

* tests: update main test for shuttle runtime

* ci: merge workspace jobs

* ci: move service to platform tests

* ci: better names

* refactor: trim dependencies

Trim dependencies to minimize the build times of user projects.

* refactor: catch error and panics in main

* tests: trim cargo-shuttle tests

Make sure there is only one main test for each major vector / feature.

* refactor: missing serde derive

* refactor: missing serde std

* ci: balance machines

* tests: give more time

* ci: bump images more

* ci: bump workspace image

* ci: 2xlarge not available '/

* feat: emit error when shuttle::main is named main (#707)

* Feature: eng 484 update init with codegen changes (#706)

* feat: update init boilerplate with shuttle-next changes

* tests: update shuttle_init_tests

* tests: ignore broken integration tests

* feat: refactor init to create bin

* fix: bug when passing path to init

* fix: codegen clippy (#709)

* fix: codegen clippy

* fix: clippy

* misc: v0.12.0-rc1 (#708)

* misc: v0.12.0-rc1

* misc: publish proto

* refactor: remove relative from build.rs

* refactor: add description to proto

* refactor: make runtime publishable

* refactor: make actix publishable

* feat: make shuttle-axum publishable

* feat: make poem publishable

* feat: make poise publishable

* feat: make shuttle-rocket publishable

* refactor: make warp publishable

* refactor: make tower publishable

* refactor: make tide publishable

* refactor: make thruster publishable

* refactor: make serenity publishable

* refactor: make salvo publishable

---------

Co-authored-by: oddgrd <[email protected]>

* [next] refactor: update runtime manager (#711)

* refactor: allow multiple runtimes to run at the same time

This is needed so that a new deployment can enter the loading state
without killing currently running runtimes.

* refactor: extra comment

* Feature: eng 497 update or remove the docs in shuttle (#710)

* docs: update service docs

* docs: move service crate-docs to runtime

* docs: update main readme

* tests: fix init test

* refactor: rename legacy runtime to alpha (#713)

* refactor: rename legacy runtime to alpha

* refactor: move next to services (#714)

* refactor: move next to services

* ci: correct next

* refactor: align folder

* [next] refactor: remove ids from runtime (#712)

* refactor: remove deployment id

* refactor: create proto LogItem directly

* refactor: more tracing for local binary

* misc: add kondo to env

* refactor: clippy suggestion

* tests: update

* feat: embed protoc in shuttle-proto (#715)

* feat: embed protoc in shuttle-proto

* fix: path to shuttle-next in runtime next-resource

* [next] bug: communicating resources (#716)

* refactor: send claim to provisioner

* feat: recording of provisioned resources

This returns provisioned resources from runtime so that they can be
recorded by deployer. Closes #ENG-252.

* refactor: restore resources for idle containers waking up

* tests: next tests using wrong folder

* tests: swap rocket for the more stable poem

* refactor: pass auth_uri to runtime

* misc: update all patches

* feat(local): don't install next-runtime from git (#718)

* fix: static folder local run clearing file contents, add missing tests in cargo-shuttle (#717)

* fix: static folder local run clearing file contents

* fix: clippy

* fix: cargo shuttle init tests not logging in

* fix: bail if non-interactive and --new without api-key

* fix: update codegen tests

* [next] refactor: tracing (#719)

* refactor: use tracing setup helper

* refactor: better trace propagation

* refactor: clippy suggestion

* fix: windows local run path bug (#721)

* refactor: pull out vendored protoc from shuttle-proto (#726)

* feat: use grpcio to avoid needing protoc

* refactor: remove vendored protoc, revert to tonic otlp

* [next] bug: misc fixes (#725)

* bug: thread local logging breaking serenity

* refactor: patch all crates

* refactor: extend tokens to 15 minutes

* bug: next runtime not sending stop signal (#728)

* bug: next runtime not sending stop signal

* refactor: better unwrap

* chore: v0.12.0 (#727)

* chore: v0.12.0

* chore: bump examples

---------

Signed-off-by: Federico Guerinoni <[email protected]>
Co-authored-by: Damien <[email protected]>
Co-authored-by: chesedo <[email protected]>
Co-authored-by: Alexander Krantz <[email protected]>
Co-authored-by: Ivan <[email protected]>
Co-authored-by: Peter Mertz <[email protected]>
Co-authored-by: XyLyXyRR <[email protected]>
Co-authored-by: Damien <[email protected]>
Co-authored-by: Nereuxofficial <[email protected]>
Co-authored-by: Maxim <[email protected]>
Co-authored-by: maksim <[email protected]>
Co-authored-by: Federico Guerinoni <[email protected]>
Co-authored-by: Isaiah Gamble <[email protected]>
Co-authored-by: Stijn ("stain") Seghers <[email protected]>
Co-authored-by: Sara Tavares <[email protected]>
Co-authored-by: HexPandaa <[email protected]>
Co-authored-by: Kieren Davies <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chesedo chesedo chesedo left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@oddgrd @chesedo