Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement rudimentary tokenserver route in syncstorage-rs #871

Merged
merged 14 commits into from
Nov 5, 2020

Conversation

fzzzy
Copy link
Contributor

@fzzzy fzzzy commented Oct 21, 2020

Description

Add the hacky initial tokenserver route. Uses an existing tokenserver mysql database.

Testing

First, you need python3 and to run pip3 install tokenlib.

Need a tokenserver mysql database:

create table services (id int(11) auto_increment not null, service varchar(30), pattern varchar(128), primary key(id));
create table nodes (
  id bigint(20) auto_increment not null, service int(11) not null,
  node varchar(64) not null, available int(11) not null, current_load int(11) not null,
  capacity int(11) not null, downed int(11) not null, backoff int(11) not null,
  primary key(id), foreign key(service) references services(id)
);
create table users (
  uid bigint(20) auto_increment not null, service int(11) not null,
  email varchar(255) not null, generation bigint(20) not null,
  client_state varchar(32) not null, created_at bigint(20) not null,
  replaced_at bigint(20), nodeid bigint(20) not null,
  keys_changed_at bigint(20), primary key(uid),
  foreign key(nodeid) references nodes(id)
);

Run syncstorage-rs.

In firefox, create a new profile. In about:config, set identity.sync.tokenserver.uri to http://localhost:8000/1.0/sync/1.5

Log in to sync with your normal prod fxa credentials. It should be able to put your stuff in your localhost syncstorage-rs.

Issue(s)

Fix #864

@fzzzy fzzzy changed the title Implement rudimentary tokenserver route in syncstorage-rs feat: Implement rudimentary tokenserver route in syncstorage-rs Oct 21, 2020
@fzzzy fzzzy requested a review from pjenvey October 21, 2020 20:29
@fzzzy fzzzy force-pushed the tokenserver-experiments branch from 25e95aa to 61908d0 Compare October 21, 2020 20:33
…storage-rs codebase. Fix #864

Requires an existing tokenserver mysql db.
@fzzzy fzzzy force-pushed the tokenserver-experiments branch from 19bfd6c to 72cf6c3 Compare October 26, 2020 19:45
@jrconlin jrconlin requested a review from a team October 28, 2020 17:38
Copy link
Member

@jrconlin jrconlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks ok, but still seems a bit WIP.

I'd feel better with less println! and hard coded values, particularly if we're going to land this in master.

src/web/tokenserver.rs Show resolved Hide resolved
src/web/tokenserver.rs Outdated Show resolved Hide resolved
src/web/tokenserver.rs Outdated Show resolved Hide resolved
src/web/tokenserver.rs Show resolved Hide resolved
src/web/tokenserver.rs Outdated Show resolved Hide resolved
src/web/tokenserver.rs Show resolved Hide resolved
@fzzzy
Copy link
Contributor Author

fzzzy commented Oct 29, 2020

There's obviously a tension between stuff being wip and landing on master. As the new tokenserver route is not used yet, I feel it is prudent to land rougher code since it breaks the larger task up into smaller chunks, which we have been having a difficult time doing.

src/web/tokenserver.rs Outdated Show resolved Hide resolved
@@ -128,7 +128,7 @@ macro_rules! build_app {
)
// Tokenserver
.service(
web::resource(&cfg_path("/1.0/sync/1.5")).route(web::get().to(tokenserver::get)),
web::resource("/1.0/sync/1.5".to_string()).route(web::get().to(tokenserver::get)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that this route does something other than nothing, we should probably disable this during development somehow on the live service. A setting or maybe something as simple as detecting a debug build (but note the docker-compose tests in ci run against release mode)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docker-compose tests don't hit this tokenserver yet, so that should be fine.

Copy link
Contributor Author

@fzzzy fzzzy Nov 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to go far out of your way to configure your firefox to use this new tokenserver route and it's not documented anywhere, so does this really buy anything? [edit] I'm not sure if ops has a whitelist of urls on the nginx frontend. If they do, presumably nobody would even be able to hit this route on production?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll admit I'm being pedantic but I think it makes sense at some point to disable the route when it's not needed anyway (thinking about it more, presence of TOKENSERVER_DATABASE_URL would be a good toggle).

Definitely a long shot anyone hits this endpoint but I was more concerned about it being untested/load tested, particularly w/ this handler potentially loading the Python runtime. So I'd rather just disable it sooner rather than later so there's one less thing on prod to worry about while it's still under development.

pjenvey
pjenvey previously approved these changes Nov 3, 2020
jrconlin
jrconlin previously approved these changes Nov 4, 2020
let email = format!("{:}@api.accounts.firefox.com", token_data.claims.sub);

// TODO pull out of settings instead
let shared_secret = env::var("SYNC_MASTER_SECRET").expect("SYNC_MASTER_SECRET must be set");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Tempted to move these up a bit in a future release. They'd be the trigger that @pjenvey is looking for and would save the cycles of building the token_data above.

@fzzzy fzzzy merged commit b74943e into master Nov 5, 2020
@fzzzy fzzzy deleted the tokenserver-experiments branch November 5, 2020 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement mysql backend for tokenserver route
3 participants