Individual versioning using semver/commitizen

[IsmaelMartinez]

Is your feature request related to a problem? Please describe.
When packaging multiple libraries, I don't find a way to increase their version number (patch, minor, major) automatically. Currently it requires to remember to edit the pyproject.toml file

Describe the solution you'd like
It would be ideal if the package command can automatically increase the version number depending on the commits by using commitizen or others.

Describe alternatives you've considered
Adding a plugin (probably better than a macro) might be able to archive this, but haven't had the time to explore that option.

The idea is that it will run just before the package command and increase the version number of any libraries that has changed.

Appreciate if you can provide more info in other ways and/or examples to archive this.

Additional context
We use poetry for our libraries, and have a mono repo that generates multiple libraries (for external systems to consume).

An example. If I got 3 pyproject.toml that then I publish as libraries, called A,B and C.
Assuming dependencies goes like this C -> B -> A. Meaning, B depends on A and C depends on B.

• If A changes, I would like the version number for A, B and C to increase.
• If B changes, I would like the version of B and C to change.
• If C changes, Then I would like only the version in C to change.
  Does this make sense?
  A bonus extra would be to detect what number to increase depending on the commit (using git commitizen or similar) but I can live without that

Our libraries have a poetry file like this:

[tool.poetry]
name = "library_1"
version = "2.0.0"
description = "my internal library"
authors = ["[email protected]"]
readme = "README.md"
repository = "https://ohhh-this-url-is-amazing/but-is-not-real"

[tool.poetry.dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Got a wee publish macro that looks like this:

# flake8: noqa: F821
def poetry_distribution(name, **kwargs):
    resources(name="package_data", sources=["pyproject.toml", "README.md"])

    python_distribution(
        name="dist",
        dependencies=[":package_data", f"src/python/{name}/src", "//:root"],
        provides=python_artifact(name=name),
        generate_setup=False,
    )

Then in the folder for each library I have this in the BUILD file

poetry_distribution(
    name="name-of-my-library",
)

The project structure is like this:

├── BUILD
├── dependencies.lock
├── dist
│   ├── library-1-2.0.0-py2.py3-none-any.whl
│   ├── library-1-2.0.0.tar.gz
│   ├── library-2-5.0.0-py2.py3-none-any.whl
│   ├── library-2-5.0.0.tar.gz
│   ├── library-3-3.0.0-py2.py3-none-any.whl
│   ├── library-3-3.0.0.tar.gz
│   ├── library-4-2.0.0-py2.py3-none-any.whl
│   └── library-4-2.0.0.tar.gz
├── pants
├── pants.toml
├── pants_plugins
│   ├── BUILD
│   └── macros.py <- This is where I have put the macro
├── pyproject.toml
├── setup.cfg
└── src
    └── python
        ├── library-1
        │   ├── BUILD
        │   ├── README.md
        │   ├── pyproject.toml
        │   ├── src
        │   │   ├── BUILD
        │   │   └── library-1.py
        │   └── test
        │       ├── BUILD
        │       └── test_library-1.py
        ├── library-2
        │   ├── BUILD
        │   ├── README.md
        │   ├── pyproject.toml
        │   └── src
        │       ├── BUILD
        │       ├── library-2.py
        │       ├── test_something_else_library-2.py
        │       └── test_library-2.py
        ├── library-3
        │   ├── BUILD
        │   ├── README.md
        │   ├── pyproject.toml
        │   └── src
        │       ├── BUILD
        │       ├── __init__.py
        │       ├── library-3.py
        │       └── test_library-3.py
        └── ppl_library-4
            ├── BUILD
            ├── README.md
            ├── pyproject.toml
            └── src
                ├── BUILD
                ├── ppl_library-4.py
                └── test_library-4.py

The only root source config I got in the pants.toml is

[source]
root_patterns = ['/src/python/*/src', '/']

And added also this like to activate the macros

build_file_prelude_globs = ["pants_plugins/macros.py"]

Thanks in advance!!

From this slack message

[IsmaelMartinez]

This is similar to #15374 , whoever a simpler version that updates the version number of pyproject.toml files (or similar) with whatever version the user decides to pass. maybe there is a way with the experimental shell command?? Thanks in advance!

[IsmaelMartinez]

Ok, moving a wee bit but I think I need a bit of guidance.

I can update the version number of my dependent libraries providing this experimental_run_shell_command:

experimental_run_shell_command(
        name="bump_version",
        tags=["bump_version"],
        workdir=f"src/python/{name}/",
        command=f"poetry version patch",
    )

Whoever, I need to run this for each individual library.

If library-1 and library-3 have changed. I need to run:

./pants run src/python/library-1::bump_version
./pants run src/python/library-3::bump_version

Ideally I just want to do:

./pants run --changed-since=main --changed-dependees=transitive ::bump_version 

I will try to make it a plugin (and play around that area) but would appreciate some guidance to see if I am going in the right direction.

Thanks in advance!!

[IsmaelMartinez]

Ok, got something close enough. I have created a plugin in the folder pants_extras/internal_plugins/bump_version:

BUILD

python_sources()

register.py

from typing import Iterable
from pants.engine.target import Target

from internal_plugins.bump_version.target_types import ProjectVersionTarget
import internal_plugins.bump_version.rules as bump_version_rules


def target_types() -> Iterable[type[Target]]:
    return [ProjectVersionTarget]


def rules():
    return [*bump_version_rules.rules()]

target_types.py

from pants.engine.target import (
    COMMON_TARGET_FIELDS,
    SingleSourceField,
    Target,
)

import logging

logger = logging.getLogger(__name__)


class PyprojectSourceField(SingleSourceField):
    help = "pyproject.toml source field"
    alias = "pyproject_file_name"
    default = "pyproject.toml"
    expected_file_extensions = [".toml"]
    required = False


class ProjectVersionTarget(Target):
    alias = "versioned_project"
    description = "A versioned project target where the pyproject.toml version is updated via the `poetry version ..` command"
    core_fields = (*COMMON_TARGET_FIELDS, PyprojectSourceField)
    tags = ["bump", "versioned"]
    help = """ This project will update the pyproject.toml version using the `poetry version $(type)` command.
            Type can be `major`, `minor` or `patch` (default)"""

rules.py

from pants.engine.target import (
    Targets,
)
from pants.engine.fs import AddPrefix, Digest, MergeDigests, Workspace
from pants.engine.target import HydratedSources, HydrateSourcesRequest, SourcesField
from pants.util.logging import LogLevel
from pants.engine.rules import collect_rules, goal_rule, MultiGet, Get, rule
from pants.backend.python.target_types import ConsoleScript
from pants.backend.python.util_rules.interpreter_constraints import (
    InterpreterConstraints,
)
from pants.backend.python.util_rules.pex import (
    Pex,
    PexProcess,
    PexRequest,
    PexRequirements,
)
from pants.engine.process import ProcessResult
from pants.engine.goal import Goal, GoalSubsystem
from pants.option.option_types import StrOption
import logging

from internal_plugins.bump_version.target_types import ProjectVersionTarget

logger = logging.getLogger(__name__)


class ProjectVersionSubsystem(GoalSubsystem):
    name = "bump-version"
    help = "Manage pyproject.toml version using poetry."

    bump_type = StrOption(
        default="patch",
        advanced=True,
        help="Type of version increase. Can be major, minor or patch (default)",
    )


class ProjectVersionGoal(Goal):
    subsystem_cls = ProjectVersionSubsystem


@goal_rule
async def goal_show_project_version(
    targets: Targets, workspace: Workspace
) -> ProjectVersionGoal:
    projectVersionTargets = [
        tgt for tgt in targets if tgt.alias == ProjectVersionTarget.alias
    ]
    results = await MultiGet(
        Get(Digest, ProjectVersionTarget, tgt) for tgt in projectVersionTargets
    )
    output_digest = await Get(Digest, MergeDigests([r for r in results]))
    workspace.write_digest(output_digest)
    return ProjectVersionGoal(exit_code=0)


@rule
async def get_project_version_file_view(
    target: ProjectVersionTarget, options: ProjectVersionSubsystem
) -> Digest:
    sources = await Get(HydratedSources, HydrateSourcesRequest(target[SourcesField]))
    pex = await Get(
        Pex,
        PexRequest(
            output_filename="poetry.pex",
            internal_only=True,
            interpreter_constraints=InterpreterConstraints(["==3.9.*"]),
            requirements=PexRequirements(["poetry"]),
            main=ConsoleScript("poetry"),
        ),
    )
    result = await Get(
        ProcessResult,
        PexProcess(
            pex,
            argv=["version", options.bump_type],
            input_digest=sources.snapshot.digest,
            description=f"Applying {options.bump_type} package version bump",
            working_directory=target.residence_dir,
            level=LogLevel.DEBUG,
            output_files=["pyproject.toml"],
        ),
    )

    digest = await Get(Digest, AddPrefix(result.output_digest, target.residence_dir))

    if len(result.stdout) > 0:
        logger.info(f"{result.stdout.decode()} File {target.residence_dir}.toml")
    if len(result.stderr) > 0:
        logger.error(
            f"Failed to bump version for file {target.residence_dir}.toml. {result.stderr.decode()}"
        )

    return digest


def rules():
    return collect_rules()

I have a macro in the pants-plugin folder with the following:

# flake8: noqa: F821
def poetry_distribution(name, **kwargs):
    resources(name="package_data", sources=["pyproject.toml", "README.md"])

    versioned_project(name=f"{name}_version", tags=["version", "bump"])

    python_distribution(
        name="dist",
        dependencies=[":package_data", f"src/python/{name}/src", "//:root"],
        repositories=["https://nexus3.tl.pplaws.com/repository/pypi-ppl/"],
        provides=python_artifact(name=f"{name}"),
        generate_setup=False,
    )

I can run the following commands no bother:

./pants bump-version ::

Whoever it will be ideal if I can somehow hook it with the python source files so that it bumps the pyproject.toml file when the code has changed. Currently the following command understandably only bumps the version of the files if they have changed.

./pants bump-version --changed-since=main --changed-dependees=transitive

Any ideas or examples on how to archive this? Should I have extended the pythonsource instead of creating a new target? (aka, have I gone the wrong route??)

Thanks in advance!