ex-conda user: How pixi resolves situations when a new package requires a downgrade of one of already added packages?

[ishipachev]

Hi!

I work mostly with Computer Vision package stack. And in many of my applications installing one package with conda required me to downgrade some other packages for compatibility reasons. Usually conda offers such options if it will satisfy the dependency for the requested package.

Does it work like this in pixi? Sadly, I can't bring the case I was experiencing yesterday.

If I want to update a package to a newer version what should I do? Update? Can I try to update add packages in manifesto to the latest compatible combination? How to do that? If package requires me to downgrade a python -> what is the correct way to do? Can it be done in a semi-automatic mode as conda does when it tells you that to satisfy this requirement these packages has to be updated or downgraded? What is the default resolve behavior? For now I saw that pixi is able to find dependencies or say you that it can not satisfy them. And this is it. Maybe I missed some of it functionality?

Also, what is the best way to downgrade packages in manifesto? Remove them and add them again with a new >= and <= limits?

Sorry for a lot of questions! For me it is the only missing link I have in my brain to fully enjoy moving my projects to pixi (which I already did :))
If anyone can point me to the specific part of the doc which covers that it would be great and close this topic for me.

[ruben-arts]

Hi @ishipachev Nice to hear that you enjoying pixi!

TL;DR: Yes pixi add "package_name<=1.2.3" will only update that package version and all packages that we're also needed to be updated as a result of the change.

If your project uses the pixi.lock pixi will only try to do minimal updates of the environment when changing requirements.
This means that it will consume the lockfile as input for the resolving algorithm and only change packages if it requires them to be changed, not updating them by default.

To update a package that was already installed you can use pixi update this will find the next best version that is within your requirement specification.

To downgrade you can pixi add the constraint. e.g. pixi add "package_name<=1.2.3" This will do the reinstall of only that package with a new constraint. Modifying the manifest by hand and running pixi install will have the same result.

I hope this answers your questions 😄

[ishipachev]

Hi @ruben-arts,

Nice to hear that you enjoying pixi!
Yep, I guess I finally have a way to manage environments and packages I actually enjoy and share with others who asks for recommendations.

Thank you for your answer. I think in general it makes it more sense now.

I still have a question, maybe I will formulate it in a bit more practical way. Conda potentially can ask you to downgrade packages. Like here, on this example:

And usually I allow it to do so because most of the time I need to have a consistent environment only with several packages out of many locked down to their specific major and minor versions.

Let me add one package for an example, let's consider pandas. I will add it with pixi add pandas. In pyproject.toml this line will be added:

pandas = ">=2.2.3,<3"

What will happen if the other package I want to install with pixi add dundas which will require pandas lower than 2.2.3? I didn't specify initially when I installed that? (sorry, I can't bring other example, otherwise I would test it on my own)

What is the "right" pixi way to handle such situations? I'm trying to build a mental model of the way pixi resolves conflicts like this one and where manual interogation is expected.

I've enctountered a situation when I wasn't able to get a working environment compatiblve with any opencv packages from pypi. But when I tried from scratch and started to install it in a different order, starting from python and opencv the environment was finally resolved. I wish I could bring a concrete example, but I was too tired of having it working and when it was resolved I had a relied and didn't think about writing down about this specific case. (Maybe I will have some time to duplicate it later!)

[ishipachev]

I would like to get back to this with the specific situation when I need to downgrade python, but it doesn't happening either automatically or even in manual mode.

This is my pixi.toml file:

[project]
authors = ["Ilia Shipachev <[email protected]>"]
channels = ["conda-forge"]
description = "Data processing tools and scripts"
name = "data_processing"
platforms = ["win-64","linux-64"]
version = "0.1.0"

[tasks]

[dependencies]
opencv = ">=4.10"
numpy = ">=2.0"
scipy = ">=1.14.1,<2"
matplotlib = ">=3.9.2,<4"
ipython = ">=8.28.0,<9"
scikit-learn = ">=1.5.2,<2"
scikit-image = ">=0.24.0,<0.25"
pyyaml = ">=6.0.2,<7"
python = ">=3.12.7,<4"

[pypi-dependencies]
rawpy = ">=0.23.1, <0.24"

I would like to have open3d installed from pypi, but I have python=3.12 in the current environment. So when I try to install it with command pixi add --pypi open3d it complains that can't find compatible with ABI version:

  × failed to solve the pypi requirements of 'default' 'win-64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because only the following versions of open3d are available:
          open3d==0.8.0.0
          open3d==0.9.0.0
          open3d==0.10.0.0
          open3d==0.10.0.1
          open3d==0.11.0
          open3d==0.11.1
          open3d==0.11.2
          open3d==0.12.0
          open3d==0.13.0
          open3d==0.14.1
          open3d==0.15.1
          open3d==0.15.2
          open3d==0.16.0
          open3d==0.16.1
          open3d==0.17.0
          open3d==0.18.0
      and open3d<=0.9.0.0 has no wheels with a matching Python implementation tag, we can conclude that open3d<0.9.0.0 cannot be used.
      And because open3d>=0.10.0.0 has no wheels with a matching Python ABI tag and you require open3d, we can conclude that your requirements are unsatisfiable.

I see that for open3d from pypi I need to downgrade python to 3.11. And I try to install it with

pixi add python=3.11.*

And I get this:

╭─ pwsh  C:\dev\data_processing   main ?5 ~6                                                                                                                        11:29:01 ╰─❯ pixi add python==3.11.*    
  ⠄ default:linux-64     [00:00:01] resolving conda
    ⠁ validate cache       [00:00:00] [━━╾─────────────────] python (+28)
    ⠁ download & extract   [00:00:00] [────────────────────]      0 B @ 0 B/s    pillow (+18)
    ⠁ installing packages  [00:00:00] [╾───────────────────]     1/110 pillow (+53)
  × failed to instantiate a prefix for 'default'
  ├─▶ failed to unlink python-3.12.7-hce54a09_0_cpython.conda
  ╰─▶ failed to delete file: DLLs/_asyncio.pyd

I've tried to remove python with pixi remove python and install again the right version with pixi add python==3.11.* and it didn't work still.
When I run pixi add python it adds python >= 3.12.7, <4

So the same question: how do I downgrade packages when I need it? It is not happening automatically? Is there something hidden under which locks the whole configuration in the way that I can't even modify it?