-
Notifications
You must be signed in to change notification settings - Fork 440
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rust_doc: add web server for docs #475
Conversation
The Zip archives emitted by `rust_doc` rules used to contain a long `/bazel-bin/k8-fastbuild/path/to/target` prefix before each archive entry name. This made it hard to work with the generated archive. This patch adds a transformation to This could be written in Starlark using the `DirectoryExapnder` API, but that was only introduced in Bazel 3.4.0. Since we have a minimum Bazel version of 0.17.1, we instead write the helper as a small Python script. Fixes bazelbuild#471. Test Plan: Unit tests included. As an end-to-end test, run ``` cd examples/ && bazel build //hello_world:hello_world_doc && unzip -l bazel-bin/hello_world/hello_world_doc.zip ``` to list the contents of the built `hello_world_doc.zip` archive. Before this patch, the result was like: ``` Archive: bazel-bin/hello_world/hello_world_doc.zip Length Date Time Name --------- ---------- ----- ---- 0 2010-01-01 00:00 bazel-out/k8-fastbuild/bin/hello_world/hello_world_doc/.lock 1792 2010-01-01 00:00 bazel-out/k8-fastbuild/bin/hello_world/hello_world_doc/COPYRIGHT.txt 4421 2010-01-01 00:00 bazel-out/k8-fastbuild/bin/hello_world/hello_world_doc/FiraSans-LICENSE.txt ... 2132 2010-01-01 00:00 bazel-out/k8-fastbuild/bin/hello_world/hello_world_doc/storage.js 1180 2010-01-01 00:00 bazel-out/k8-fastbuild/bin/hello_world/hello_world_doc/theme.js 3764 2010-01-01 00:00 bazel-out/k8-fastbuild/bin/hello_world/hello_world_doc/wheel.svg --------- ------- 890555 38 files ``` After this patch: ``` Archive: bazel-bin/hello_world/hello_world_doc.zip Length Date Time Name --------- ---------- ----- ---- 0 2010-01-01 00:00 .lock 1792 2010-01-01 00:00 COPYRIGHT.txt 4421 2010-01-01 00:00 FiraSans-LICENSE.txt ... 2132 2010-01-01 00:00 storage.js 1180 2010-01-01 00:00 theme.js 3764 2010-01-01 00:00 wheel.svg --------- ------- 890555 38 files ``` wchargin-branch: rustdoc-strip-prefix wchargin-source: 29978cf3bdb7de6cee154f8c11d0251574c2b104
wchargin-branch: rustdoc-strip-prefix wchargin-source: 69b2e78f1f52c88609aff631c908471c7521c5f7
A `rust_doc` rule emits its output into a zip archive so that the build graph can be statically known, but this makes it hard to actually view the documentation. This patch adds a `rust_doc_server` macro, a sibling to `rust_doc` that spins up a tiny web server that reads from the zip file. The server can run under `ibazel` and automatically restart when the documentation sources are updated. This is written in Python for a simple cross-platform solution that doesn’t require compiling a Rust web stack, which takes about 20 seconds on my workstation (Xeon W-2135). Fixes bazelbuild#472. Test Plan: From `examples/`, run `bazel run //hello_world:hello_world_doc_server` and note that the server responds properly. Run with `-- --port 0` (extra `--` for `bazel run`) and note that the server properly prints its actual port. wchargin-branch: rustdoc-server wchargin-source: 307dcd532d0f7880b1cceaf0421a51416607b44f
This PR’s diffbase is the #474 branch, so please ignore the extraneous Opening this to get @dfreese’s opinion re: Python usage, per discussion Thanks! Happy to incorporate any feedback. |
wchargin-branch: rustdoc-server wchargin-source: bdb753c552346c721fdf558a15e95777f8214827
From a high level design perspective, do we want to be generating a server target for every rustdoc target, or just a single binary target that serves as a tool that can handle any rustdoc output? |
wchargin-branch: rustdoc-strip-prefix wchargin-source: 40227180c7f910d9f76716cf42621e08e5f808a1
wchargin-branch: rustdoc-server wchargin-source: 1f90836672a1cc8467b6b11e3b045497b8b05903
wchargin-branch: rustdoc-server wchargin-source: 1f90836672a1cc8467b6b11e3b045497b8b05903
wchargin-branch: rustdoc-strip-prefix wchargin-source: 8c46290195fc0668f3126589fd07ee863df8749f
wchargin-branch: rustdoc-server wchargin-source: 62146a7c4b075d8567d2e92309c07b3c36bd5ea0
wchargin-branch: rustdoc-strip-prefix wchargin-source: 838287ca34eb69c27f62e4a0900dbb7ac6a3ebd0
wchargin-branch: rustdoc-server wchargin-source: 442c03cd289428a9f7f461159599daf6876686fd
Summary: The Rust Bazel rules provide a `rust_doc` rule for documentation, but it generates a zipfile with long file paths, which is inconvenient to consume. This patch adds a simple Werkzeug server to serve from the generated documentation. It’s compatible with `ibazel run`: just change a Rust source file and refresh your browser tab to see your changes. I have a [pending upstream patch for this][upstream], but I’m still working on it, and it’s on my back burner. In the meantime, this is helpful for both Rust authors and Rust reviewers on TensorBoard. [upstream]: bazelbuild/rules_rust#475 Test Plan: Run `bazel run //tensorboard/data/server:rustboard_core_doc` and navigate to port 6005: ![Screenshot of the docs for `rustboard_core::masked_crc::MaskedCrc` in a standard rustdoc output format][ss] [ss]: https://user-images.githubusercontent.com/4317806/98600750-e788bd80-2292-11eb-9f09-fd51ed5e5996.png wchargin-branch: rust-doc-server wchargin-source: ddc3596e08606fb52ab63157900df1b3741fbfaf
Summary: The Rust Bazel rules provide a `rust_doc` rule for documentation, but it generates a zipfile with long file paths, which is inconvenient to consume. This patch adds a simple Werkzeug server to serve from the generated documentation. It’s compatible with `ibazel run`: just change a Rust source file and refresh your browser tab to see your changes. I have a [pending upstream patch for this][upstream], but I’m still working on it, and it’s on my back burner. In the meantime, this is helpful for both Rust authors and Rust reviewers on TensorBoard. [upstream]: bazelbuild/rules_rust#475 Test Plan: Run `bazel run //tensorboard/data/server:rustboard_core_doc_server` and navigate to port 6005: ![Screenshot of the docs for `rustboard_core::masked_crc::MaskedCrc` in a standard rustdoc output format][ss] [ss]: https://user-images.githubusercontent.com/4317806/98600750-e788bd80-2292-11eb-9f09-fd51ed5e5996.png wchargin-branch: rust-doc-server
wchargin-branch: rustdoc-strip-prefix wchargin-source: ea88a0484df0608cb08521fb4cde2d4dc7904553
wchargin-branch: rustdoc-strip-prefix wchargin-source: ea88a0484df0608cb08521fb4cde2d4dc7904553
wchargin-branch: rustdoc-server wchargin-source: f1172908d26a8f6621f0ed0a894ec8541db6c345
wchargin-branch: rustdoc-server wchargin-source: f1172908d26a8f6621f0ed0a894ec8541db6c345
wchargin-branch: rustdoc-server wchargin-source: 8817e8fef2309159717320f672701c2fd8999517
If you want to be able to On the build side, I’m having trouble getting the docs sub-repo to work @dfreese, does this sound reasonable to you? |
A
rust_doc
rule emits its output into a zip archive so that the buildgraph can be statically known, but this makes it hard to actually view
the documentation. This patch adds a
rust_doc_server
macro, a siblingto
rust_doc
that spins up a tiny web server that reads from the zipfile. The server can run under
ibazel
and automatically restart whenthe documentation sources are updated.
This is written in Python for a simple cross-platform solution that
doesn’t require compiling a Rust web stack, which takes about 20 seconds
on my workstation (Xeon W-2135).
Fixes #472.
Test Plan:
From
examples/
, runbazel run //hello_world:hello_world_doc_server
and note that the server responds properly. Run with
-- --port 0
(extra
--
forbazel run
) and note that the server properly printsits actual port.
wchargin-branch: rustdoc-server