Consider adding a composer.lock file in the repository

[drupol]

Hi,

I'm opening this PR to try to get the file composer.lock live in the repository and be part of the Psysh sources.

Summary

The composer.lock file, though sometimes overlooked, is key for project reproducibility, performance, and security.
Currently, when we want to ensure reproducibility, we need to create the composer.lock ourselves, like in here. This issue is to open a discussion with the author of Psysh about letting the file composer.lock live in the sources of the project.

Rationale

Composer is the dependency manager for PHP. It allows you to declare the libraries your project depends on, and it will manage (install/update) them for you. In a composer.json file, you specify the dependencies for your project. For example, you might need a certain library, and that library might need another library, and so on.

When you run composer install for the first time, it resolves dependencies and then downloads all of them and writes all the packages and the exact versions of them that it downloaded to the composer.lock file, locking the project to those specific versions.

Committing this file in this repo is important for many reasons:

1. Reproducibility: Reproducibility is one of the foundational aspects of software engineering. It ensures that all team members and the deployment system are using the exact same versions of the dependencies. This eliminates the potential issues that may arise from differences in versions used, such as function deprecations, changes in behavior, etc. Without the composer.lock file, running composer install would fetch the latest versions of the dependencies you defined in composer.json, which could cause inconsistencies between environments.

2. Security: It helps ensure that you're using the versions of dependencies that you have tested and reviewed. Without a composer.lock file, there's a risk that a malicious person could find a vulnerability in one of your dependencies, fix that vulnerability, bump the version number, and if you run composer install or composer update, you could be pulling in un-reviewed/un-tested code since you do not control anymore the dependency resolution and delegate it to the host system.

3. Speed: Having a composer.lock file speeds up the composer installation process. Composer no longer needs to resolve dependencies (which can take some time) and can just download the version of each dependency specified in the composer.lock.

Inclusion of composer.lock in this repository also promotes the following advantages (but I won't go into the details, this post is already too long):

1. Less time spending in debugging,
2. Increased reliability,
3. Enhanced collaboration and scalability,
4. Effective version control,
5. Streamlined Continuous Integration and Continuous Deployment,
6. Efficient auditing and compliance,
7. Reliable scientific and data-driven applications.

Conclusion

While the composer.lock file may appear non-essential and is sometimes overlooked, its inclusion is critical for ensuring the reproducibility, speed, and security of your project. By including the composer.lock file in your project's repository, you are effectively shifting the responsibility of specifying correct dependencies from the end-users to the project's author.

Consequently, the author takes charge of determining the precise dependencies to install, sparing the users from this task. This approach streamlines the installation process and fosters a consistent project environment.

By including the composer.lock file in the repository, not only does it improve the development experience for contributors, but it also provides users with an assurance of quality, security, and reliability. The presence of the composer.lock file signifies that the project's dependencies are carefully managed and tested. It confirms that the project operates effectively with the locked versions of dependencies, thus reducing the risk of compatibility issues, unexpected behaviour and bugs.

The composer.lock file ultimately serves as a testament to the project's stability. It safeguards users against potential risks associated with untested updates or patches in dependencies. Therefore, both contributors and users can benefit from the project author's meticulous dependency management reflected in the committed composer.lock file. This can enhance the overall confidence in the project's robustness and reliability.

Further Considerations

While Psysh is developed in PHP, it can also be viewed as a binary, not solely a PHP library. Given this, it's imperative that it retains reproducibility across various environments. composer.lock is our most effective asset to ensure this within the PHP ecosystem. Considering the reasons outlined here, I strongly believe it deserves a place in the project.

What others are doing ?

It's often beneficial to consider practices in other projects. Composer, the PHP package manager itself, follows this practice. There are many more such examples in the ecosystem.

Application, binaries, libraries

A PHP application or binary is a standalone piece of software intended to be installed and run as is, and the composer.lock file should be included. This is because the application or binary needs to ensure that it is using the exact versions of its dependencies that it was developed and tested with. If the dependencies were to change unpredictably, the application could break. Therefore, the composer.lock file, which specifies the exact versions of the dependencies, is essential.

On the other hand, a PHP library is a piece of software intended to be included as a dependency of other projects. In this case, the composer.lock file is typically not included in version control. This is because the library should be compatible with a range of versions of its own dependencies. The library doesn't control the exact versions of its dependencies - the parent project that uses the library does. As such, the dependency resolving mechanism should indeed be handled by the project that requires the library, not the library itself.

Relation with Nix

Reproducibility is a hard requirement when bundling packages in Nix. To build a reproducible version of Psysh, we need to ensure all inputs are accurately provided to Composer. We currently provide the composer.lock ourselves to ensure consistent use of the same dependencies, and I believe we should not do that. Making developers aware of the significance of this file would be beneficial.

Practical, hands on

To ensure reproducibility with a Psysh build, containing composer.lock:

1. git clone https://github.com/bobthecow/psysh
2. cd psysh

Let's install the required dependencies only:

3. composer install --no-dev

The next command will compute a checksum of the psysh directory containing now the vendor directory, excluding the .git directory:

4. cd -; tar --exclude-vcs --mtime='1970-01-01' -c psysh | sha256sum; cd -

Since Composer 2.6.4 and when the composer.lock is available, there is a guarantee of reproducibility, you could run this on any platform, and you will always get the same checksum.

When the composer.lock is not available, the checksum might be different from a day to another, there is absolutely no guarantee of reproducibility.

[bobthecow]

PsySH isn't a binary, it is a library which happens to distribute a binary in its releases. It should not have a Composer lockfile, any more than Symfony or Laravel should.

[drupol]

Hi Justin,

That's precisely why I open this issue. The aim is to ensure that anyone can reproduce the exact same build, down to the last bit, of the PHAR that you're releasing. It's even more important if you're shipping a binary.

At the moment, without the composer.lock file, it's impossible for me, or anyone else, to replicate the artifacts that you provide in your GitHub releases. The absence of that file could potentially introduces variability in the dependencies, which defeats the purpose of being able to precisely reproduce builds.

Would you please reconsider this issue?

If you have reservations about including it in the main repo, could you perhaps include it as part of the release artifacts on GitHub? This would go a long way in making builds reproducible and ensuring everyone is working with the same set of dependencies.

Thank you for (re)considering this request, and I look forward to your response.

[bobthecow]

Yeah, that's fair.

Besides the library/binary justification, it's also a bit complicated by the fact that there are multiple binaries per release, each with their own dependency versions. So even if it made sense to add it to the repo, it wouldn't cover both binaries in each release.

Maybe the right answer is to bundle the lock file up in the tarball with each released asset? Or maybe we could include it in the Phar itself? 🤔

[drupol]

Glad to read this :)

If I had to choose, I would add it to the release, not in the PHAR.

Thanks for reconsidering my request ! Much appreciated!

[bobthecow]

Just dumping it in the tarfile feels like a backwards compatibility break … ish. It'd break workflows if people depend exactly on the structure of the dist bundles.

[drupol]

How about adding it as artifact in the Github release so you don't change the structure of the produced artifacts?

[bobthecow]

it needs one per binary though.

[drupol]

That's fine, no?

We could have 2 files added to the artifacts:

• composer.lock.php7 (for PHP 7)
• composer.lock.php8 (for PHP 8)

WDYT?

[drupol]

Excellent, looking forward for the next tag to update it in Nix!

[bobthecow]

Great! This will be available in the upcoming v0.12.x release, but not in v0.11.x releases (not that I'm expecting any ¯\_(ツ)_/¯)

[drupol]

Excellent :) Thanks!!!