Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement rudimentary tokenserver route in syncstorage-rs #871

Merged
merged 14 commits into from
Nov 5, 2020
+366 −32
Merged

feat: Implement rudimentary tokenserver route in syncstorage-rs #871

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
72cf6c3
feature: Implement an initial tokenserver endpoint spike in the sync…
fzzzy Oct 26, 2020
ac5e7e3
Merge branch 'master' into tokenserver-experiments
fzzzy Oct 30, 2020
dcdca66
Re-add Cargo.lock
fzzzy Oct 30, 2020
80f481b
Clean up.
fzzzy Oct 30, 2020
2b18d7a
Return the urrl with the uid and no trailing slash
fzzzy Nov 2, 2020
bfe7a20
format such that flake8 shall not complain.
fzzzy Nov 2, 2020
20d4964
Pretty-format the sql.
fzzzy Nov 2, 2020
f92ad56
Missed an AND
fzzzy Nov 2, 2020
0aa7912
Use r# for the sql string
fzzzy Nov 2, 2020
e1585de
cargo fmt
fzzzy Nov 2, 2020
c542b35
merge from master
fzzzy Nov 2, 2020
b462131
resolve conflicts
fzzzy Nov 4, 2020
b784e99
resolve conflicts
fzzzy Nov 5, 2020
67d5c3e
Merge branch 'master' into tokenserver-experiments
fzzzy Nov 5, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,13 @@ commands:
rustup component add rustfmt
cargo install cargo-audit
rustup component add clippy
setup-python:
steps:
- run:
name: Setup python
command: |
sudo apt-get update && sudo apt-get install -y python3-dev python3-pip
pip3 install tokenlib
rust-check:
steps:
- run:
Expand Down Expand Up @@ -157,6 +164,7 @@ jobs:
fi
- checkout
- setup-rust
- setup-python
- setup-gcp-grpc
# XXX: currently the time needed to setup-sccache negates its savings
#- setup-sccache
Expand Down
192 changes: 192 additions & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ debug = 1
[dependencies]
actix-http = "2"
actix-web = "3"
actix-web-httpauth = "0.5"
actix-rt = "1"
actix-cors = "0.5"
async-trait = "0.1.40"
Expand All @@ -40,10 +41,12 @@ googleapis-raw = { version = "0", path = "vendor/mozilla-rust-sdk/googleapis-raw
# `cargo build --features grpcio/openssl ...`
grpcio = { version = "0.6.0" }
lazy_static = "1.4.0"
pyo3 = "0.12.1"
hawk = "3.2"
hostname = "0.3.1"
hkdf = "0.10"
hmac = "0.10"
jsonwebtoken = "7.2.0"
log = { version = "0.4", features = ["max_level_info", "release_max_level_info"] }
mime = "0.3"
num_cpus = "1"
Expand Down
6 changes: 4 additions & 2 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ ADD . /app
ENV PATH=$PATH:/root/.cargo/bin
# temp removed --no-install-recommends due to CI docker build issue
RUN apt-get -q update && \
apt-get -q install -y --no-install-recommends default-libmysqlclient-dev cmake golang-go && \
apt-get -q install -y --no-install-recommends default-libmysqlclient-dev cmake golang-go python3-dev python3-pip && \
pip3 install tokenlib && \
rm -rf /var/lib/apt/lists/* && \
cd /app && \
mkdir -m 755 bin
Expand All @@ -21,7 +22,8 @@ RUN \
groupadd --gid 10001 app && \
useradd --uid 10001 --gid 10001 --home /app --create-home app && \
apt-get -q update && \
apt-get -q install -y build-essential default-libmysqlclient-dev libssl-dev ca-certificates libcurl4 && \
apt-get -q install -y build-essential default-libmysqlclient-dev libssl-dev ca-certificates libcurl4 python3-dev python3-pip && \
pip3 install tokenlib && \
rm -rf /var/lib/apt/lists/*

COPY --from=builder /app/bin /app/bin
Expand Down
2 changes: 1 addition & 1 deletion src/main.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ extern crate slog_scope;
use std::{error::Error, sync::Arc};

use docopt::Docopt;
use serde_derive::Deserialize;
use serde::Deserialize;

use logging::init_logging;
use syncstorage::{logging, server, settings};
Expand Down
2 changes: 1 addition & 1 deletion src/server/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ macro_rules! build_app {
)
// Tokenserver
.service(
web::resource(&cfg_path("/1.0/sync/1.5")).route(web::get().to(tokenserver::get)),
web::resource("/1.0/sync/1.5".to_string()).route(web::get().to(tokenserver::get)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that this route does something other than nothing, we should probably disable this during development somehow on the live service. A setting or maybe something as simple as detecting a debug build (but note the docker-compose tests in ci run against release mode)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docker-compose tests don't hit this tokenserver yet, so that should be fine.

Copy link
Contributor Author

@fzzzy fzzzy Nov 2, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to go far out of your way to configure your firefox to use this new tokenserver route and it's not documented anywhere, so does this really buy anything? [edit] I'm not sure if ops has a whitelist of urls on the nginx frontend. If they do, presumably nobody would even be able to hit this route on production?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll admit I'm being pedantic but I think it makes sense at some point to disable the route when it's not needed anyway (thinking about it more, presence of TOKENSERVER_DATABASE_URL would be a good toggle).

Definitely a long shot anyone hits this endpoint but I was more concerned about it being untested/load tested, particularly w/ this handler potentially loading the Python runtime. So I'd rather just disable it sooner rather than later so there's one less thing on prod to worry about while it's still under development.

)
// Dockerflow
// Remember to update .::web::middleware::DOCKER_FLOW_ENDPOINTS
Expand Down
Loading