Skip to content

Commit

Permalink
Create documentation for Starlark code with stardoc
Browse files Browse the repository at this point in the history
We want a single source of truth and less work by maintaining
the docstring as well as the README.
  • Loading branch information
martis42 committed Aug 7, 2024
1 parent 66fe4a6 commit 84a91eb
Show file tree
Hide file tree
Showing 14 changed files with 280 additions and 154 deletions.
7 changes: 7 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
define: &generated_doc_files "^docs/(cc_info_mapping|dwyu_aspect).md"

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
Expand All @@ -11,15 +13,19 @@ repos:
- id: check-json
# Code style
- id: end-of-file-fixer
exclude: *generated_doc_files
- id: trailing-whitespace
# Python code quality
- id: debug-statements

- repo: https://github.com/executablebooks/mdformat
rev: 0.7.17
hooks:
- id: mdformat
exclude: *generated_doc_files
additional_dependencies:
- mdformat-gfm

- repo: https://github.com/keith/pre-commit-buildifier
rev: 7.1.2
hooks:
Expand All @@ -30,6 +36,7 @@ repos:
"--diff_command='diff'",
"--warnings=-module-docstring,-function-docstring,-function-docstring-header,-print"
]

- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.5.5
Expand Down
5 changes: 3 additions & 2 deletions MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ module(

bazel_dep(name = "rules_cc", version = "0.0.8")
bazel_dep(name = "rules_python", version = "0.27.1")
bazel_dep(name = "bazel_skylib", version = "1.7.1")

non_module_dependencies = use_extension("//third_party:extensions.bzl", "non_module_dependencies")
use_repo(non_module_dependencies, "dwyu_pcpp")
Expand All @@ -14,8 +15,8 @@ use_repo(non_module_dependencies, "dwyu_pcpp")
### Development Dependencies
###

# Keep in sync with third_party/dependencies.bzl
bazel_dep(name = "bazel_skylib", version = "1.7.1", dev_dependency = True)
bazel_dep(name = "aspect_bazel_lib", version = "2.7.9", dev_dependency = True)
bazel_dep(name = "stardoc", version = "0.7.0", dev_dependency = True)

python = use_extension(
"@rules_python//python/extensions:python.bzl",
Expand Down
127 changes: 8 additions & 119 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,6 @@
- [Get a release](#get-a-release)
- [Get a specific commit](#get-a-specific-commit)
- [Use DWYU](#use-dwyu)
- [Configuring DWYU](#configuring-dwyu)
- [Custom header ignore list](#custom-header-ignore-list)
- [Skipping Targets](#skipping-targets)
- [Recursion](#recursion)
- [Implementation_deps](#Implementation_deps)
- [Target mapping](#target-mapping)
- [Verbosity](#verbosity)
- [Applying automatic fixes](#applying-automatic-fixes)
- [Assumptions of use](#assumptions-of-use)
- [Supported Platforms](#supported-platforms)
Expand Down Expand Up @@ -82,24 +75,25 @@ dwyu_setup_step_2()

### Configure the aspect

The features which can be configured through the aspect factory attributes are documented at [Configuring DWYU](#configuring-dwyu).
Put the following inside a `.bzl` file:
The DWYU aspect is created in your project by a [factory function offering various options](docs/dwyu_aspect.md) to configure the aspect.
Various illustrations for configuring and using the DWYU aspect can be seen in the [examples](/examples).

Example `.bzl` file creating a DWYU aspect with default configuration:

```starlark
load("@depend_on_what_you_use//:defs.bzl", "dwyu_aspect_factory")

# Provide no arguments for the default behavior
# Or set a custom value for the various attributes
your_dwyu_aspect = dwyu_aspect_factory()
```

### Use the aspect

Invoke the aspect through the command line on a target:<br>
`bazel build <target_pattern> --aspects=//:aspect.bzl%your_dwyu_aspect --output_groups=dwyu`
Assuming you created the DWYU aspect in file `//:aspect.bzl`, execute it on a target pattern:<br>
`bazel build --aspects=//:aspect.bzl%your_dwyu_aspect --output_groups=dwyu <target_pattern>`

If no problem is found, the command will exit with `INFO: Build completed successfully`.<br>
If a problem is detected, the build command will fail with an error and a description of the problem will be printed in the terminal. For example:
If a problem is detected, the build command will fail with an error and a description of the problem will be printed in the terminal.
For example:

```
================================================================================
Expand All @@ -121,111 +115,6 @@ The Bazel documentation for invoking an aspect from within a rule can be found [

This is demonstrated in the [rule_using_dwyu example](/examples/rule_using_dwyu).

# Configuring DWYU

## Custom header ignore list

By default, DWYU ignores all header from the standard library when comparing include statements to the dependencies.
This list of headers can be seen in [std_header.py](src/analyze_includes/std_header.py).

You can exclude a custom set of header files by providing a config file in json format to the aspect:

```starlark
your_aspect = dwyu_aspect_factory(ignored_includes = "//<your_config_file>.json")
```

The config file can contain these fields which should be lists of strings.
All fields are optional:

| Field | Description |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ignore_include_paths` | List of include paths which are ignored by the analysis. Setting this **disables ignoring the standard library include paths**. |
| `extra_ignore_include_paths` | List of include paths which are ignored by the analysis. If `ignore_include_paths` is specified as well, both list are combined. If `ignore_include_paths` is not set, the default list of standard library headers is extended. |
| `ignore_include_patterns` | List of patterns for include paths which are ignored by the analysis. Patterns have to be compatible to Python [regex syntax](https://docs.python.org/3/library/re.html#regular-expression-syntax). The [match](https://docs.python.org/3/library/re.html#re.match) function is used to process the patterns. |

This is demonstrated in the [ignoring_includes example](/examples/ignoring_includes).

## Skipping targets

If you want the DWYU aspect to skip certain targets and negative target patterns are not an option you can do so by setting the `no-dwyu` tag on those.
You can also configure the aspect to skip targets based on a custom list of tags:

```starlark
your_aspect = dwyu_aspect_factory(skipped_tags = ["tag1_marking_skipping", "tag2_marking_skipping"])
```

Another possibility is skipping all targets from external workspaces.
Often external dependencies are not our under control and thus analyzing them is of little value.
This option is mostly useful in combination with the recursive analysis.
You configure it like this:

```starlark
your_aspect = dwyu_aspect_factory(skip_external_targets = True)
```

Both options are demonstrated in the [skipping_targets example](/examples/skipping_targets).

## Recursion

By default, DWYU analyzes only the target it is being applied to.

You can also activate recursive analysis.
Meaning the aspect analyzes recursively all dependencies of the target it is being applied to:

```starlark
your_aspect = dwyu_aspect_factory(recursive = True)
```

This is demonstrated in the [recursion example](/examples/recursion).

## Implementation_deps

Bazel offers the experimental feature [`implementation_deps`](https://bazel.build/reference/be/c-cpp#cc_library.implementation_deps) to distinguish between public (aka interface) and private (aka implementation) dependencies for `cc_library`.
Headers from the private dependencies are not made available to users of the library.

DWYU analyzes the usage of headers from the dependencies and can raise an error if a dependency is used only in private files, but not put into the private dependency attribute.
Meaning, it can find dependencies which should be moved from `deps` to `implementation_deps`.

Activate this behavior via:

```starlark
your_aspect = dwyu_aspect_factory(use_implementation_deps = True)
```

Usage of this can be seen in the [basic example](examples/basic_usage).

## Target mapping

Sometimes users don't want to follow the DWYU rules for all targets or have to work with external dependencies not following the DWYU principles.
For such cases DWYU allows creating a mapping which for a chosen target makes more headers available as the target actually provides.
In other words, one can combine virtually multiple targets for the DWYU analysis.
Doing so allows using headers from transitive dependencies without DWYU raising an error for select cases.

Such a mapping is created with the [dwyu_make_cc_info_mapping](src/cc_info_mapping/cc_info_mapping.bzl) rule.
This offers multiple ways of mapping targets:

1. Explicitly providing a list of targets which are mapped into a single target.
1. Specifying that all direct dependencies of a given target are mapped into the target.
1. Specifying that all transitive dependencies of a given target are mapped into the target.

Activate this behavior via:

```starlark
your_aspect = dwyu_aspect_factory(target_mapping = "<mapping_target_created_by_the_user>")
```

This is demonstrated in the [target_mapping example](/examples/target_mapping).

## Verbosity

One can configure the DWYU aspect to print debugging information.

Activate this behavior via:

```starlark
your_aspect = dwyu_aspect_factory(verbose = True)
```

# Applying automatic fixes

> \[!WARNING\]
Expand Down
13 changes: 13 additions & 0 deletions docs/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
load("@aspect_bazel_lib//lib:docs.bzl", "stardoc_with_diff_test", "update_docs")

stardoc_with_diff_test(
name = "cc_info_mapping",
bzl_library_target = "//src/cc_info_mapping:cc_info_mapping",
)

stardoc_with_diff_test(
name = "dwyu_aspect",
bzl_library_target = "//src/aspect:factory",
)

update_docs()
52 changes: 52 additions & 0 deletions docs/cc_info_mapping.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
<!-- Generated with Stardoc: http://skydoc.bazel.build -->

# Motivation

Sometimes users don't want to follow the DWYU rules for all targets or have to work with external dependencies not following the DWYU principles.
While one can completely exclude targets from the DWYU analysis (e.g. via tags), one might not want to disable DWYU completely, but define custom rules for specific dependencies.
One can do so by defining exceptions where includes can be provided by selected transitive dependencies instead of direct dependencies.
In other words, one can virtually change which header files are treated as being available from direct dependencies.

One example use case for this are unit tests based on gtest.
Following strictly the DWYU principles each test using a gtest header should depend both on the gtest library and the gtest main:
```starlark
cc_test(
name = "my_test",
srcs = ["my_test.cc"],
deps = [
"@com_google_googletest//:gtest",
"@com_google_googletest//:gtest_main",
],
)
```
This can be considered superfluous noise without a significant benefit.
The mapping feature described here allows defining that `@com_google_googletest//:gtest_main` offers the header files from `@com_google_googletest//:gtest`.
Then a test can specify only the dependency to `@com_google_googletest//:gtest_main` without DWYU raising an error while analysing the test.

<a id="dwyu_make_cc_info_mapping"></a>

## dwyu_make_cc_info_mapping

<pre>
load("@depend_on_what_you_use//src/cc_info_mapping:cc_info_mapping.bzl", "dwyu_make_cc_info_mapping")

dwyu_make_cc_info_mapping(<a href="#dwyu_make_cc_info_mapping-name">name</a>, <a href="#dwyu_make_cc_info_mapping-mapping">mapping</a>)
</pre>

Map include paths available from one or several targets to another target.

Create a mapping allowing treating targets as if they themselves would offer header files, which in fact are coming from their dependencies.
This enables the DWYU analysis to skip over some usage of headers provided by transitive dependencies without raising an error.

Using this rule and the various mapping techniques is demonstrated in the [target_mapping example](/examples/target_mapping).


**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="dwyu_make_cc_info_mapping-name"></a>name | Unique name for this target. Will be the prefix for all private intermediate targets. | none |
| <a id="dwyu_make_cc_info_mapping-mapping"></a>mapping | Dictionary containing various targets and how they should be mapped. Possible mappings are:<br> - An explicit list of targets which are mapped to the main target. Be careful only to choose targets which are dependencies of the main target! <br> - The `MAP_DIRECT_DEPS` token which tells the rule to map all direct dependencies to the main target. <br> - The `MAP_TRANSITIVE_DEPS` token which tells the rule to map recursively all transitive dependencies to the main target. | none |


43 changes: 43 additions & 0 deletions docs/dwyu_aspect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
<!-- Generated with Stardoc: http://skydoc.bazel.build -->



<a id="dwyu_aspect_factory"></a>

## dwyu_aspect_factory

<pre>
load("@depend_on_what_you_use//src/aspect:factory.bzl", "dwyu_aspect_factory")

dwyu_aspect_factory(<a href="#dwyu_aspect_factory-ignored_includes">ignored_includes</a>, <a href="#dwyu_aspect_factory-recursive">recursive</a>, <a href="#dwyu_aspect_factory-skip_external_targets">skip_external_targets</a>, <a href="#dwyu_aspect_factory-skipped_tags">skipped_tags</a>,
<a href="#dwyu_aspect_factory-target_mapping">target_mapping</a>, <a href="#dwyu_aspect_factory-use_implementation_deps">use_implementation_deps</a>, <a href="#dwyu_aspect_factory-verbose">verbose</a>)
</pre>

Create a "Depend on What You Use" (DWYU) aspect.

Use the factory in a `.bzl` file to instantiate a DWYU aspect:
```starlark
load("@depend_on_what_you_use//:defs.bzl", "dwyu_aspect_factory")

your_dwyu_aspect = dwyu_aspect_factory(<aspect_options>)
```


**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="dwyu_aspect_factory-ignored_includes"></a>ignored_includes | By default, DWYU ignores all headers from the standard library when comparing include statements to the dependencies. This list of headers can be seen in [std_header.py](/src/analyze_includes/std_header.py).<br> You can extend this list of ignored headers or replace it with a custom one by providing a json file with the information to this attribute.<br> Specification of possible files in the json file: <ul><li> `ignore_include_paths` : List of include paths which are ignored by the analysis. Setting this **disables ignoring the standard library include paths**. </li><li> `extra_ignore_include_paths` : List of concrete include paths which are ignored by the analysis. Those are always ignored, no matter what other fields you provide. </li><li> `ignore_include_patterns` : List of patterns for include paths which are ignored by the analysis. Patterns have to be compatible to Python [regex syntax](https://docs.python.org/3/library/re.html#regular-expression-syntax). The [match](https://docs.python.org/3/library/re.html#re.match) function is used to process the patterns. </li></ul> This feature is demonstrated in the [ignoring_includes example](/examples/ignoring_includes). | `None` |
| <a id="dwyu_aspect_factory-recursive"></a>recursive | By default, the DWYU aspect analyzes only the target it is being applied to. You can change this to recursively analyzing dependencies following the `deps` and `implementation_deps` attributes by setting this to True.<br> This feature is demonstrated in the [recursion example](/examples/recursion). | `False` |
| <a id="dwyu_aspect_factory-skip_external_targets"></a>skip_external_targets | Sometimes external dependencies are not our under control and thus analyzing them is of little value. If this flag is True, DWYU will automatically skip all targets from external workspaces. This can be useful in combination with the recursive analysis mode.<br> This feature is demonstrated in the [skipping_targets example](/examples/skipping_targets). | `False` |
| <a id="dwyu_aspect_factory-skipped_tags"></a>skipped_tags | Do not execute the DWYU analysis on targets with at least one of those tags. By default skips the analysis for targets tagged with 'no-dwyu'.<br> This feature is demonstrated in the [skipping_targets example](/examples/skipping_targets). | `["no-dwyu"]` |
| <a id="dwyu_aspect_factory-target_mapping"></a>target_mapping | Accepts a [dwyu_make_cc_info_mapping](/docs/cc_info_mapping.md) target. Allows virtually combining targets regarding which header can be provided by which dependency. For the full details see the `dwyu_make_cc_info_mapping` documentation.<br> This feature is demonstrated in the [target_mapping example](/examples/target_mapping). | `None` |
| <a id="dwyu_aspect_factory-use_implementation_deps"></a>use_implementation_deps | `cc_library` offers the attribute [`implementation_deps`](https://bazel.build/reference/be/c-cpp#cc_library.implementation_deps) to distinguish between public (aka interface) and private (aka implementation) dependencies. Headers from the private dependencies are not made available to users of the library.<br> Setting this to True allows DWYU to raise an error if headers from a `deps` dependency are used only in private files. In such a cease the dependency should be moved from `deps` to `implementation_deps`.<br> This feature is demonstrated in the [basic_usage example](/examples/basic_usage). | `False` |
| <a id="dwyu_aspect_factory-verbose"></a>verbose | If True, print debugging information about what DWYU does. | `False` |

**RETURNS**

Configured DWYU aspect


19 changes: 19 additions & 0 deletions src/aspect/BUILD
Original file line number Diff line number Diff line change
@@ -1,7 +1,26 @@
load("@bazel_skylib//:bzl_library.bzl", "bzl_library")
load("@rules_python//python:defs.bzl", "py_binary")

py_binary(
name = "process_target",
srcs = ["process_target.py"],
visibility = ["//visibility:public"],
)

bzl_library(
name = "factory",
srcs = [
"dwyu.bzl",
"factory.bzl",
"@bazel_tools//tools/build_defs/cc:action_names.bzl",
],
visibility = ["//visibility:public"],
deps = [
"//src/cc_info_mapping",
],
)

exports_files(
["factory.bzl"],
visibility = ["//docs:__pkg__"],
)
2 changes: 1 addition & 1 deletion src/aspect/dwyu.bzl
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "CPP_COMPILE_ACTION_NAME")
load("@bazel_tools//tools/cpp:toolchain_utils.bzl", "find_cpp_toolchain")
load("@depend_on_what_you_use//src/cc_info_mapping:cc_info_mapping.bzl", "DwyuCcInfoRemappingsInfo")
load("@depend_on_what_you_use//src/cc_info_mapping:providers.bzl", "DwyuCcInfoRemappingsInfo")
load("@rules_cc//cc:defs.bzl", "CcInfo", "cc_common")

def _is_external(ctx):
Expand Down
Loading

0 comments on commit 84a91eb

Please sign in to comment.