Documentation; User group; Announcement group; oncall.
This directory contains the code for the Buck2 build system - the successor to the original Buck build system.
For the setup guide and basics, continue reading below. For tips and techniques for debugging, see docs/developers/developers.md
.
Commands below assume the use of bash/zsh, modify accordingly for other shells.
If you use EdenFS backed repository (the default settings) we highly recommend setting $CARGO_TARGET_DIR
to $HOME/cargo-target-buck2
to make Cargo put its files elsewhere.
export CARGO_TARGET_DIR=$HOME/cargo-target-buck2
This dramatically reduces compilation times and improves cargo stability.
To set up a macOS machine run:
fbclone fbcode --eden [dest_dir] # dest_dir is optional
Next, install the Rust toolchain using:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Finally, you need to include Rust's toolchain in your PATH
. Depending on your shell, you should put the following directive in either ~/.profile
, ~/.bashrc
, ~/.zshrc
or ~/.zshenv
. Consult your shell documentation.
NB: macOS Catalina (10.15) changed the default shell from Bash to Zsh.
NB: The Rust toolchain probably already adjusted your shell environment (it says as part of the installation). If that's the case, you don't need to add the following directive.
source "$HOME/.cargo/env"
To set up a Linux devserver run:
fbclone fbcode --eden
sudo feature install ttls_fwdproxy
sudo dnf install elfutils-libelf-devel cmake
mkdir -p "$HOME/.cargo"
echo '[http]' >> "$HOME/.cargo/config"
echo 'proxy="fwdproxy:8080"' >> "$HOME/.cargo/config"
(
export HTTPS_PROXY=fwdproxy:8080
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain none
)
. "$HOME/.cargo/env"
If cargo complains about SSL peer certificate or SSH remote key error, then you can try adding proxy = http://fwdproxy:8080
under http in ~/.gitconfig
.
To set up a Windows machine run:
fbclone fbsource --eden [dest_dir] # dest_dir is optional
Install OpenSSL to have no warnings from rust-analyzer:
cd C:\open
git clone https://github.com/microsoft/vcpkg
cd vcpkg
.\bootstrap-vcpkg.bat
.\vcpkg.exe integrate install
.\vcpkg.exe install openssl:x64-windows-static-md
Then in Administrator terminal:
choco install cmake -s 'https://chocolatey.org/api/v2/'
Install Rust toolchain: https://win.rustup.rs/x86_64
To develop incrementally you should copy setup Rust on your path using . "$HOME/.cargo/env"
in a shell startup file (e.g. ~/.profile
). You can then use cargo build
or a Rust Analyser IDE. There are a few useful scripts:
./test.py
which checks the build for formatting, warnings, errors, lint violations and runs the unit tests. Running./test.py <project>
(where<project>
is something likegazebo
,starlark
,buck2_build_api
etc - see the rootCargo.toml
) will only test one specific project. Adding--lint-only
will only do linting (not compile or test), and--lint-rust-only
will not lint the Starlark code../test_e2e.sh
which runs end to end tests building test targets. Run./test_e2e.sh <testname>
(where<testname>
is something liketest_cached_build
,test_symlink_dir
etc - see thetests/e2e/*.py
files) to only run one test../buck2.sh
which compiles and runs a localbuck2
binary.
These scripts are run in CI (by scripts/ci.sh
). Do not merge your code unless the buck2
CI job is successful.
We recommend using Rust Analyzer project with either VS Code @ FB - Insiders or normal VS Code, running locally (not on a devserver), opening buck2
as the root folder of the workspace (File, Open). We recommend the following settings:
"[rust]": {
"editor.formatOnSave": true,
},
"rust-analyzer.diagnostics.disabled": ["unresolved-import"],
"rust-analyzer.procMacro.enable": true,
"rust-analyzer.assist.importPrefix": "crate",
"files.trimTrailingWhitespace": true,
"files.trimFinalNewlines": true,
To add the settings, show the Command Palette (Cmd-Shift-P on macOS), type "settings" and select "Preferences: Open Settings (JSON)". For more info, see here.
When using VSCode @ FB Insiders, you can open fbsource instead of buck2 as the root folder of the workspace. To do so, you need to add this line to the settings json:
"rust-analyzer.linkedProjects": [
"fbcode/buck2/Cargo.toml"
],
Note that Rust Analyzer is not supported by VS Code @ FB when using remote development (see).
To debug Rust code install the CodeLLDB extension. This extension is not supported by VS Code @ FB when using remote development (see). It does work when using VS Code @ FB locally. Be aware that starting a debugger can easily take 10-20 seconds, and may not show any feedback while it is starting, so be patient and don't start the debugger repeatedly.
- To debug a test, set whatever breakpoints you want, then click the
Debug
hover link above the#[test]
annotation. - To debug a binary, hit the
Run
icon (play icon with an insect beneath it) on the VS Code left menu and you should see a little green play icon at the top withDebug buck-build
. Set whatever breakpoints you want, then hit the green play icon. To modify the command or arguments (it defaults to buildingprocess_wrapper
withbuck-build
) see.vscode/launch.json
in thebuck2
directory (or just click the cog icon next to "Debug buck-build"). In most cases, don't commit your modifications. See this screenshot for visual guidance. - To debug a running buck2 daemon, see the instructions in
docs/developers/developers.md
.
Our team maintains a generic Starlark VS Code extension that gives IDE support for build files and .bzl files. It is under active development and will be gaining features regularly.
There are two copies of the VS Code Extension. One is for internal use, and is at fbsource/vscode/vscode-extensions/starlark
, and the other, for OSS use, is at starlark-rust/vscode
.
The key difference is that the internal version integrates with our internal VSCode infrastructure, and has some extra feature gating, interaction with the old buck extension, and uses our metrics framework. It also has some different setting defaults, like using buck2
as the LSP binary instead of starlark
This plugin works by talking to buck2 lsp
over stdin/stderr. As features are implemented in buck2's LSP server, they will automatically start working in the plugin.
If trying to build and install the OSS plugin on an eden FS, it can help to add a redirect to the vscode plugin's node_modules directory:
eden redirect add fbcode/buck2/starlark-rust/vscode/node_modules bind
To use the internal plugin for development, see the README in fbsource/vscode/vscode-extensions/starlark
.
To use the OSS plugin for development, follow the instructions at starlark-rust/vscode/README.md
, or the abbreviated ones here.
cd starlark-rust/vscode && npm install && npm install vsce && npm exec vsce package
- In VS Code, go to Extensions, click on the "..." button in the Extensions bar, select "Install from VSIX" and then select the
starlark-1.0.0.vsix
file at$FBCODE/buck2/starlark-rust/vscode/starlark-1.0.0.vsix
- Update the extension settings to point to
buck2
forstarlark.lspPath
, and["lsp"]
forstarlark.lspArguments
. This can be done in the VS Code UI by going to the extension's settings panel.
If you want to use a local build of buck2, just change the path to $FBCODE/buck2/scripts/buck2_lsp.sh
and unset starlark.lspArguments
.
NOTE: If you're seeing slightly confusing behavior, make sure that you're using "Starlark" as the language for files that you're editing.
To upgrade our dependencies:
cargo update
will rewrite theCargo.lock
file. This file is not checked in, so the impact will be to make your builds more like CI (which generated the file afresh each time).- Change the
rust-toolchain
file to use a newer nightly compiler. You can see what components are in which version. cargo outdated
will tell you what bounds could be relaxed.- Validate with
./test.py
.
- If Remote Execution throws errors, make sure you are on the VPN/Lighthouse, then try
kinit
,cc-certs
andfixmymac
. - If you get linker errors for missing
.so
files, try runningsudo dnf install elfutils-libelf-devel libseccomp-devel -y
. You can also find the missing package usingsudo dnf search
command. - If you get compile errors in generated Thrift code when building with Cargo, try updating the Thrift generator.
- If you get the output "error: could not download file from..." when building for the first time with Cargo on a devserver, make sure you've added the
ttls_proxy
feature and try adddingHTTPS_PROXY=fwdproxy:8080
right before the build command. You'll need to do this every time the toolchain is updated.
- A target, e.g.
fbcode//eden:eden
, is something a user defines that is an instance of a rule, which can be built. - A rule, e.g.
cxx_library
, is an implementation of how something is built. - Loading a
TARGETS
/BUCK
file involves evaluating the Starlark and doing attribute coercion/resolution. It can be done withbuck2 cquery fbcode//eden:eden
orbuck2 cquery 'deps(fbcode//eden:eden)'
to do it recursively. - Analysing a target involves running the associated rule to produce the providers. It can be done with
buck2 audit providers fbcode//eden:eden
. - Building a target involves demanding the artifacts from a provider (e.g.
DefaultInfo
). It can be done withbuck2 build fbcode//eden:eden
.
Beyond the obvious (well-tested, easy to read) we prefer guidelines that are automatically enforced, e.g. through rust fmt
, Clippy or the custom linter we have written. Some rules:
- Use the utilities from Gazebo where they are useful, in particular,
dupe
. - Prefer
to_owned
to convert&str
toString
. - Qualify
anyhow::Result
rather thanuse anyhow::Result
. - Most errors should be returned as
anyhow::Result
. Inspecting errors outside tests and the top-level error handler is strongly discouraged. - Most errors should be constructed with
thiserror
derivingenum
values, not rawanyhow!
. - We use the
derivative
library to derive thePartialEq
andHash
traits when some fields should be ignored. - Prefer
use crate::foo::bar
overuse super::bar
oruse crate::foo::*
, apart from test modules which often haveuse super::*
at the top. - Modules should either have submodules or types/functions/constants, but not both.
- Names (of variables, targets, files, etc) should be quoted with backticks,
e.g.
Variable `x` not defined
. - Lists should use square brackets, e.g.
Available targets: [`aa`, `bb`]
. - Error messages should start with an upper case letter. Error messages should not end with a period.
Buck2 is both MIT and Apache License, Version 2.0 licensed, as found in the LICENSE-MIT and LICENSE-APACHE files.