Update docs site for version 2.24.0.dev3 (#292)

Docs from
https://github.com/pantsbuild/pants/releases/tag/release_2.24.0.dev3

docs/docs/contributions/development/maintenance-tasks-and-scripts.mdx

@@ -62,6 +62,16 @@ Some tools use `NodeJSToolBase` to install executable npm packages. To update th
 
 Example: [#21007](https://github.com/pantsbuild/pants/pull/21007).
 
+
+### Python Build Standalone known versions
+
+The Python Build Standalone providers needs to be updated with new upstream releases.  There are *many* artifacts here, so the hashes are stored in a json file that is updated by running:
+
+```
+pants run src/python/pants/backend/python/providers/python_build_standalone/scripts/generate_urls.py
+```
+
+
 ## Update or create FaaS complete platforms files
 
 The function-as-a-service (FaaS) subsystems provide some built-in PEX complete platforms JSON files, for specific runtimes. To update or create these:

docs/docs/contributions/index.mdx

@@ -104,20 +104,6 @@ Bad titles:
 
 Then, include a description. You can use the default template if you'd like, or use a normal description instead. Link to any corresponding GitHub issues.
 
-Finally—if you have the permissions—add exactly one of the following labels to your PR. Otherwise, a maintainer will do this for you:
-
-- `category:new feature` for new features
-- `category:user api change` for changes that affect how end-users interact with Pants
-- `category:plugin api change` for changes that affect how plugin authors interact with Pants internals
-- `category:performance` for changes focused on improving performance
-- `category:bugfix` for bugfixes
-- `category:documentation` for documentation changes, including logging and help messages
-- `category:internal` for miscellaneous, internal-facing changes
-
-Pick the first of these that applies to your change. I.e., if you have modified the user API in a change that also improves performance, use `category:user api change`.
-
-These labels are used to generate the changelist for each release.
-
 :::note Tip: Review your own PR
 It is often helpful to other reviewers if you proactively review your own code. Specifically, add comments to parts where you want extra attention.
 
@@ -175,3 +161,5 @@ We have guidance to walk us through this, so it's not a problem to forget. Pull
 
 - the PR release notes, by having changes in `docs/notes/`
 - someone has opted out, by labelling the PR with `release-notes:not-required` or `category:internal` (the latter means that release notes are optional for all `category:internal` PRs).
+
+For minor releases, all GitHub release description will simply list all the commits save those with `release-notes:not-required` or `category:internal`.

docs/docs/contributions/releases/github-actions-linux-aarch64-runners.mdx

@@ -25,25 +25,30 @@ AMI. To do so manually:
 
 ### Create a temporary EC2 instance on the standard AMI
 
-In the AWS web console, initiate instance creation, and when prompted for an AMI, search for the
-latest standard AMI, as described in the Runs On [images repo](https://github.com/runs-on/runner-images-for-aws).
-In practice this means using `runs-on-v2.2-ubuntu22-full-arm64` as the search term, and checking the results
-in the Community AMIs tab.
+In the AWS web console, initiate creation of a temporary instance that we use just to build the custom AMI.
 
-After selecting the AMI, open the Advanced section and set the instance's User Data to:
+- Any instance type will do, t4g.nano for example.
+- To select the base AMI for the instance, click on "Browse more AMIs", then the "Community AMIs" tab,
+then search for `runs-on-v2.2-ubuntu22-full-arm64` and pick the latest standard AMI, as described in
+the Runs On [images repo](https://github.com/runs-on/runner-images-for-aws).
+- Select `pantsbuild.org.bastion` as the instance's SSH keypair.
+- Allow the wizard to create a security group.
+- The default storage settings are fine.
+- Open the Advanced section, scroll all the way down, and set the instance's User Data to:
 
 ```bash
 #!/bin/bash
 systemctl start ssh
 ```
 
-to ensure that its SSH daemon runs on startup. Then choose "pantsbuild.org.bastion" as the
-instance's SSH keypair.
+to ensure that its SSH daemon runs on startup.
+
+- Click "Launch instance"
 
 ### SSH into the temporary EC2 instance
 
-Once the instance is running, click on "Connect" to get SSH instructions. You will need the
-`pantsbuild.org.bastion` private key, which is in our 1Password account.
+Once the instance is running, navigate to its details page and click on "Connect" to get SSH instructions.
+You will need the `pantsbuild.org.bastion` private key, which is in our 1Password account.
 
 ### Install Pythons on the instance
 
@@ -56,19 +61,27 @@ sudo apt update
 sudo apt install -y \
   python3.7 python3.7-dev python3.7-venv \
   python3.8 python3.8-dev python3.8-venv \
-  python3.9 python3.9-dev python3.9-venv
+  python3.9 python3.9-dev python3.9-venv \
+  python3.10 python3.10-dev python3.10-venv \
+  python3.11 python3.11-dev python3.11-venv \
+  python3.12 python3.12-dev python3.12-venv \
+  python3.13 python3.13-dev python3.13-venv
 ```
 
 to install the necessary Pythons.
 
 ### Create an AMI from the instance
 
 From the instance's actions menu in the web console select "Images" and then "Create image".
-The image name doesn't strictly matter, but `ubuntu22-full-arm64-python3.7-python3.8-python3.9`
+The image name doesn't strictly matter, but `ubuntu22-full-arm64-python3.7-3.13`
 is a good name for consistency (multiple AMIs may have the same name).
 
-We hope to [automate the above via Packer](https://runs-on.com/guides/building-custom-ami-with-packer/)
-at some point.
+### Terminate the temporary instance
+
+Once the AMI status shows as "Available", terminate the temporary instance and delete its security group
+(whose name will match launch-wizard-*).
+
+We hope to [automate the above via Packer](https://runs-on.com/guides/building-custom-ami-with-packer/) at some point.
 
 ### Update the Runs On config
 

docs/docs/contributions/releases/release-process.mdx

@@ -186,5 +186,5 @@ values are added.
 
 From time to time, a release will fail. It's a complex process. The first thing to do after you've
 exhausted your knowledge and debugging skills or patience is to contact others. You might reach out
-to the development or maintainers channels on Pantbuild Slack in the absence of other ideas about
+to the development or maintainers channels on Pantsbuild Slack in the absence of other ideas about
 whom to ask for help.

docs/docs/using-pants/key-concepts/options.mdx

@@ -358,9 +358,8 @@ pants --scope-option="@path/to/file.json"
 :::caution Gotcha: If you modify the value file, you must manually restart pantsd
 Until we resolve [this issue](https://github.com/pantsbuild/pants/issues/10360), changing
 the value in a file used with the `@` syntax as described above will not invalidate the build.
-For now, if such a file changes you will have to restart the Pants daemon manually. You can
-do so by `kill`ing it (after using `ps -ef | grep pantsd` to find its pid), or by running
-Pants once with `--no-pantsd`.
+For now, if such a file changes you will have to stop pantsd so that it will be restarted on
+the next invocation of pants. To do so, run `rm -r .pants.d/pids/` in the build root.
 :::
 
 ## `.pants.rc` file

docs/docs/using-pants/setting-up-an-ide.mdx

@@ -51,6 +51,10 @@ The `--py-resolve-format=symlinked_immutable_virtualenv` option symlinks to an i
 
 `pants export` can also create a virtualenv for each of the Python tools you use via Pants, such as `black`, `isort`, `pytest`, `mypy`, `flake8` and so on. This allows you to configure your editor to use the same version of the tool as Pants does for workflows like formatting on save. To use a custom version of these tools, follow [the instructions for creating a tool lockfile](../python/overview/lockfiles#lockfiles-for-tools).
 
+### Binary tools
+
+`pants export` can export many tools fetched by Pants. For example, `pants export --bin=taplo`.
+
 ## Generated code
 
 If you're using [Protobuf and gRPC](../python/integrations/protobuf-and-grpc.mdx), you may want your editor to be able to index and navigate the generated source code.

docs/docs/using-pants/troubleshooting-common-issues.mdx

@@ -48,16 +48,11 @@ There is even a `__run.sh` script in the directory that will run the process usi
 
 If you are using the latest stable version of Pants and still experience a cache invalidation issue: we are sorry for the trouble. We have not yet added a comprehensive goal to "clear all caches", because we are very interested in coming up with coherent solutions to potential issues (see for more information). If you experience a cache issue, please absolutely [file a bug](https://github.com/pantsbuild/pants/issues/new) before proceeding to the following steps.
 
-To start with, first try using `--no-pantsd`. If that doesn't work, you can also try `--no-local-cache`.
-
-If `--no-pantsd` worked, you can restart `pantsd`, either by:
-
-- Killing the `pantsd` process associated with your workspace. You can use `ps aux | grep pants` to find the PID, the `kill -9 <pid>`.
-- Deleting the `<build root>/.pants.d/pids` directory.
+To start with, first try using `--no-pantsd`. If `--no-pantsd` worked, you can stop pantsd so that it will restarted on the next invocation of pants. To do so, run `rm -r .pants.d/pids/`in the build root.
 
 If this resolves the issue, please report that on the ticket and attach the recent content of the `.pants.d/workdir/pantsd/pantsd.log` file.
 
-If restarting `pantsd` is not sufficient, you can also use `--no-local-cache` to ignore the persistent caches. If this resolves the issue, then it is possible that the contents of the cache (at `~/.cache/pants`) will be useful for debugging the ticket that you filed: please try to preserve the cache contents until it can be resolved.
+If restarting pantsd is not sufficient, you can also use `--no-local-cache` to ignore the persistent caches. If this resolves the issue, then it is possible that the contents of the cache (at `~/.cache/pants`) will be useful for debugging the ticket that you filed: please try to preserve the cache contents until it can be resolved.
 
 ## Pants cannot find a file in your project
 
@@ -74,6 +69,20 @@ pants_ignore.add = ["!folder/"]
 
 Alternatively, you can stop populating `pants_ignore` from your `.gitignore` by setting `pants_ignore_use_gitignore = false` in the `[GLOBAL]` scope.
 
+## Pants cannot find a required binary
+
+Pants may be unable to locate a required binary to execute a specified goal. This can manifest in an error message such as:
+
+```
+BinaryNotFoundError: Cannot find `required_binary` on `['/usr/bin/', '/bin', '/usr/local/bin', '/opt/homebrew/bin']`. Please ensure that it is installed so that Pants can interact with the required_binary.
+```
+
+Helpfully, the error message already contains the list of directories that pants searches for binaries. This list is controlled by the [`system_binary_paths`](../../reference/subsystems/system-binaries#system_binary_paths) setting in the system-binaries subsystem. Ensure that the directory that contains your binary is on the list.
+
+If `system_binary_paths` contains the special string `<PATH>`, note that the PATH variable can also be changed after the invocation of pants in [`.pants.bootstrap`](../..//docs/using-pants/key-concepts/options#pantsbootstrap-file).
+
+Pantsd caches the location of binaries and also negative hits. If you recently installed the binary or changed its location, make sure to stop pantsd so that it will be restarted on the next invocation of pants. To do so, run `rm -r .pants.d/pids/` in the build root.
+
 ## Import errors and missing dependencies
 
 Because Pants runs processes in hermetic sandboxes (temporary directories), Pants must properly know about your [dependencies](../introduction/how-does-pants-work#dependency-inference) to avoid import errors.

docs/docs/writing-plugins/common-plugin-tasks/add-a-formatter.mdx

@@ -178,3 +178,8 @@ Now, when you run `pants fmt ::` or `pants lint ::`, your new formatter should r
 ## 4. Add tests (optional)
 
 Refer to [Testing rules](../the-rules-api/testing-plugins.mdx).
+
+
+## 5. Make the tool exportable (optional)
+
+Refer to [Allowing tool export](allowing-tool-export.mdx) to allow users to export the tool for use in external programs.

docs/docs/writing-plugins/common-plugin-tasks/add-a-linter.mdx

@@ -211,6 +211,7 @@ def rules():
     return [
       	*collect_rules(),
         ShellcheckRequest.rules(partitioner_type=PartitionerType.DEFAULT_SINGLE_PARTITION),
+        UnionRule(ExportableTool, Shellcheck),  # allows exporting the `shellcheck` binary
     ]
 ```
 
@@ -229,3 +230,7 @@ Now, when you run `pants lint ::`, your new linter should run.
 ## 4. Add tests (optional)
 
 Refer to [Testing rules](../the-rules-api/testing-plugins.mdx).
+
+## 5. Make the tool exportable (optional)
+
+Refer to [Allowing tool export](allowing-tool-export.mdx) to allow users to export the tool for use in external programs.

docs/docs/writing-plugins/common-plugin-tasks/add-codegen.mdx

@@ -19,9 +19,7 @@ This guide assumes that you are running a code generator that already exists out
 
 If you are instead writing your own code generation logic inline, you can skip Step 2. In Step 4, rather than running a `Process`, use [`CreateDigest`](../the-rules-api/file-system.mdx).
 
-1. Create a target type for the protocol
-
----
+## 1. Create a target type for the protocol
 
 You will need to define a new target type to allow users to provide metadata for their protocol files, e.g. their `.proto` files. See [Creating new targets](../the-target-api/creating-new-targets.mdx) for a guide on how to do this.
 
@@ -91,9 +89,7 @@ def rules():
 
 This example hardcodes the injected address. You can instead add logic to your rule to make this dynamic. For example, in Pants's Protobuf implementation, Pants looks for a `python_requirement` target with `protobuf`. See [protobuf/python/python_protobuf_subsystem.py](https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/codegen/protobuf/python/python_protobuf_subsystem.py).
 
-2. Install your code generator
-
----
+## 2. Install your code generator
 
 There are several ways for Pants to install your tool. See [Installing tools](../the-rules-api/installing-tools.mdx). This example will use `ExternalTool` because there is already a pre-compiled binary for Protoc.
 
@@ -130,9 +126,7 @@ class Protoc(ExternalTool):
         return "./bin/protoc"
 ```
 
-3. Create a `GenerateSourcesRequest`
-
----
+## 3. Create a `GenerateSourcesRequest`
 
 `GenerateSourcesRequest` tells Pants the `input` and the `output` of your code generator, such as going from `ProtobufSourceField -> PythonSourceField`. Pants will use this to determine when to use your code generation implementation.
 
@@ -165,9 +159,7 @@ def rules():
     ]
 ```
 
-4. Create a rule for your codegen logic
-
----
+## 4. Create a rule for your codegen logic
 
 Your rule should take as a parameter the `GenerateSourcesRequest` from Step 3 and the `Subsystem` (or `ExternalTool`) from Step 2. It should return `GeneratedSources`.
 
@@ -286,18 +278,18 @@ def rules():
 Run `pants export-codegen path/to/file.ext` to ensure Pants is correctly generating the file. This will write the generated file(s) under the `dist/` directory, using the same path that will be used during Pants runs.
 :::
 
-5. Audit call sites to ensure they've enabled codegen
-
----
+## 5. Audit call sites to ensure they've enabled codegen
 
 Call sites must opt into using codegen, and they must also specify what types of sources they're expecting. See [Rules and the Target API](../the-rules-api/rules-and-the-target-api.mdx) about `SourcesField`.
 
 For example, if you added a code generator that goes from `ProtobufSourceField -> JavaSourceField`, then Pants's Python backend would not use your new implementation because it ignores `JavaSourceField`.
 
 You should check that everywhere you're expecting is using your new codegen implementation by manually testing it out. Create a new protocol target, add it to the `dependencies` field of a target, and then run goals like `pants package` and `pants test` to make sure that the generated file works correctly.
 
-6. Add tests (optional)
-
----
+## 6. Add tests (optional)
 
 Refer to [Testing rules](../the-rules-api/testing-plugins.mdx).
+
+## 7. Make the tool exportable (optional)
+
+Refer to [Allowing tool export](allowing-tool-export.mdx) to allow users to export the tool for use in external programs.

docs/docs/writing-plugins/common-plugin-tasks/allowing-tool-export.mdx

@@ -0,0 +1,61 @@
+---
+    title: Making a tool exportable
+    sidebar_position: 10
+---
+
+How to make a tool exportable with the `export` goal.
+
+---
+
+Backends that implement the `export` goal can indicate binaries that should be exported. These will have their contents exported to a subfolder in the `dist/bins` directory, and the binary itself will be linked in `dist/bin`.
+
+## Downloadable Tools
+
+Subclasses of `ExternalTool` (including `TemplatedExternalTool`) have the logic for exporting implemented. Tools are marked for export as follows:
+
+1. Implement `ExternalTool.generate_exe` if the default is not correct. For instance, a tool downloaded might include a binary, a readme, and a license. This method will point to the binary within the downloaded files.
+
+2. Register a `UnionRule` with `ExportableTool`. For example, `UnionRule(ExportableTool, FortranLint)`
+
+## Implementing for new backends
+
+Backends need to implement:
+
+1. A subclass of `ExportRequest`
+
+```python
+@dataclass(frozen=True)
+class ExportExternalToolRequest(ExportRequest):
+    pass
+```
+
+2. A rule from this subclass to `ExportResults`
+
+```python
+@rule
+async def export_external_tools(
+    request: ExportExternalToolRequest, export: ExportSubsystem
+) -> ExportResults:
+```
+
+3. Inside of that rule, fill the `ExportResult.exported_binaries` field.
+
+```python
+ExportResult(
+    description=f"Export tool {req.resolve}",
+    reldir=dest,
+    digest=downloaded_tool.digest,
+    resolve=req.resolve,
+    exported_binaries=(ExportedBinary(name=Path(exe).name, path_in_export=exe),),
+)
+```
+
+4. For every tool, mark it for export registering a `UnionRule` with `ExportableTool`.
+
+```python
+def rules():
+    return [
+        ...,
+        `UnionRule(ExportableTool, FortranLint)`,
+    ]
+```