Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

noarch/black-19.10b0-py_2.tar.bz2 contains files from other packages #13

Closed
hugobuddel opened this issue Jul 21, 2020 · 10 comments · Fixed by #14
Closed

noarch/black-19.10b0-py_2.tar.bz2 contains files from other packages #13

hugobuddel opened this issue Jul 21, 2020 · 10 comments · Fixed by #14

Comments

@hugobuddel
Copy link

There are many files in noarch/black-19.10b0-py_2.tar.bz2 that perhaps should not be there because they are from different packages:

$ tar --list -f black-19.10b0-py_2.tar.bz2 
info/hash_input.json
info/link.json
info/index.json
info/about.json
info/files
info/paths.json
info/git
site-packages/typed_ast-1.4.1.dist-info/INSTALLER
site-packages/typed_ast-1.4.1.dist-info/WHEEL
site-packages/typed_ast-1.4.1.dist-info/METADATA
site-packages/typed_ast-1.4.1.dist-info/RECORD
info/recipe/conda_build_config.yaml
info/recipe/meta.yaml
info/test/run_test.bat
site-packages/attr/py.typed
site-packages/attr/_version_info.pyi
site-packages/attr/filters.pyi
site-packages/attr/converters.pyi
site-packages/attr/exceptions.pyi
site-packages/attr/validators.pyi
site-packages/attr/__init__.pyi
info/licenses/LICENSE
site-packages/blib2to3/__init__.py
site-packages/_black_version.py
site-packages/typed_ast/__init__.py
site-packages/regex/__init__.py
site-packages/blib2to3/pgen2/__init__.py
site-packages/pathspec/patterns/__init__.py
site-packages/attr/_config.py
site-packages/pathspec/compat.py
site-packages/attr/filters.py
site-packages/click/_textwrap.py
site-packages/blib2to3/pgen2/token.py
site-packages/attr/__init__.py
site-packages/click/globals.py
site-packages/blib2to3/pgen2/literals.py
site-packages/attr/exceptions.py
site-packages/blib2to3/pygram.py
site-packages/attr/converters.py
site-packages/attr/_version_info.py
site-packages/pathspec/__init__.py
site-packages/click/__init__.py
site-packages/pathspec/tests/test_pathspec.py
site-packages/click/_unicodefun.py
site-packages/pathspec/pattern.py
site-packages/blib2to3/pgen2/grammar.py
site-packages/pathspec/pathspec.py
site-packages/blackd.py
site-packages/attr/_compat.py
site-packages/typed_ast/tests/test_basics.py
site-packages/blib2to3/pgen2/driver.py
site-packages/pathspec/tests/test_util.py
site-packages/blib2to3/pgen2/parse.py
site-packages/click/exceptions.py
site-packages/typed_ast/conversions.py
site-packages/click/formatting.py
site-packages/blib2to3/pgen2/conv.py
site-packages/attr/_funcs.py
site-packages/pathspec/patterns/gitwildmatch.py
site-packages/click/_winconsole.py
site-packages/click/decorators.py
site-packages/attr/validators.py
site-packages/click/_bashcomplete.py
site-packages/typed_ast/ast27.py
site-packages/pathspec/tests/test_gitwildmatch.py
site-packages/click/testing.py
site-packages/typed_ast/ast3.py
site-packages/blib2to3/pgen2/pgen.py
site-packages/click/parser.py
site-packages/click/utils.py
site-packages/pathspec/util.py
site-packages/click/_termui_impl.py
site-packages/blib2to3/pgen2/tokenize.py
site-packages/click/termui.py
site-packages/click/_compat.py
site-packages/appdirs.py
site-packages/click/types.py
site-packages/blib2to3/pytree.py
site-packages/regex/regex.py
site-packages/attr/_make.py
site-packages/click/core.py
site-packages/pathspec/tests/__init__.py
site-packages/regex/_regex_core.py
site-packages/black.py
site-packages/regex/test_regex.py
site-packages/click-7.1.2.dist-info/LICENSE.rst
info/recipe/meta.yaml.template
site-packages/pathspec-0.8.0.dist-info/INSTALLER
site-packages/pathspec-0.8.0.dist-info/WHEEL
site-packages/pathspec-0.8.0.dist-info/RECORD
site-packages/pathspec-0.8.0.dist-info/METADATA
site-packages/pathspec-0.8.0.dist-info/LICENSE
site-packages/appdirs-1.4.4.dist-info/INSTALLER
site-packages/appdirs-1.4.4.dist-info/WHEEL
site-packages/appdirs-1.4.4.dist-info/RECORD
site-packages/appdirs-1.4.4.dist-info/METADATA
site-packages/attrs-19.3.0.dist-info/top_level.txt
site-packages/regex-2020.7.14.dist-info/top_level.txt
site-packages/click-7.1.2.dist-info/top_level.txt
site-packages/appdirs-1.4.4.dist-info/top_level.txt
site-packages/pathspec-0.8.0.dist-info/top_level.txt
site-packages/typed_ast-1.4.1.dist-info/top_level.txt
site-packages/black-19.10b0.dist-info/top_level.txt
site-packages/black-19.10b0.dist-info/entry_points.txt
site-packages/blib2to3/PatternGrammar.txt
site-packages/appdirs-1.4.4.dist-info/LICENSE.txt
site-packages/blib2to3/Grammar.txt
site-packages/black-19.10b0.dist-info/INSTALLER
site-packages/black-19.10b0.dist-info/WHEEL
site-packages/black-19.10b0.dist-info/LICENSE
site-packages/black-19.10b0.dist-info/RECORD
site-packages/black-19.10b0.dist-info/METADATA
site-packages/typed_ast/_ast27.cpython-38-x86_64-linux-gnu.so
site-packages/typed_ast/_ast3.cpython-38-x86_64-linux-gnu.so
site-packages/regex/_regex.cpython-38-x86_64-linux-gnu.so
site-packages/attrs-19.3.0.dist-info/INSTALLER
site-packages/attrs-19.3.0.dist-info/WHEEL
site-packages/attrs-19.3.0.dist-info/LICENSE
site-packages/attrs-19.3.0.dist-info/RECORD
site-packages/attrs-19.3.0.dist-info/METADATA
site-packages/black-19.10b0.dist-info/direct_url.json
site-packages/click-7.1.2.dist-info/INSTALLER
site-packages/click-7.1.2.dist-info/WHEEL
site-packages/click-7.1.2.dist-info/RECORD
site-packages/click-7.1.2.dist-info/METADATA
site-packages/regex-2020.7.14.dist-info/INSTALLER
site-packages/regex-2020.7.14.dist-info/WHEEL
site-packages/regex-2020.7.14.dist-info/RECORD
site-packages/regex-2020.7.14.dist-info/METADATA
info/recipe/build_base.sh
info/test/run_test.sh

As a result, conda complains:

...
ERROR conda.core.link:_execute(700): An error occurred while installing package 'conda-forge::black-19.10b0-py_2'.
...
conda.CondaMultiError: Conda was asked to clobber an existing path.
...
Conda no longer clobbers existing paths without the use of the --clobber option
...

I can submit more details about my specific environment if this is useful.

@hugobuddel
Copy link
Author

Attached is a screenshot of the black package on conda-forge. It seems that a new version of the noarch package has been uploaded months after the other 19.10b0 packages are uploaded. It is uploaded by cf-staging instead of conda-forge. This all looks a bit fishy.

blackcondaforgewebsite

hugobuddel referenced this issue Jul 21, 2020
* add blackd recipe output

* MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.3, and conda-forge-pinning 2020.06.24.16.31.11

* fix: test section indent

* fix: indent

* fix: put back noarch

* MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.3, and conda-forge-pinning 2020.06.24.16.31.11

* try using a metapackage

* remove pin?

* MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.3, and conda-forge-pinning 2020.06.24.16.31.11

* remove jinja template python

* try moving noarch section

* nope, try something else

* MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.3, and conda-forge-pinning 2020.06.24.16.31.11

* add pin version info

* fix pin

Co-authored-by: Uwe L. Korn <[email protected]>

* remove dup output

Co-authored-by: Uwe L. Korn <[email protected]>

* fix noarch

* MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.3, and conda-forge-pinning 2020.06.24.16.31.11

* try moving entry point

* put back noarch in blackd

* make blackd available in black

* separate build script

* MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.4, and conda-forge-pinning 2020.07.19.19.30.49

* no build in outputs

* try moving entry points to general build section

* MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.4, and conda-forge-pinning 2020.07.19.19.30.49

* add back noarch in output

* MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.4, and conda-forge-pinning 2020.07.19.19.30.49

* try this

* MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.4, and conda-forge-pinning 2020.07.19.19.30.49

* back

* check tests

Co-authored-by: Carlo Dri <[email protected]>
Co-authored-by: conda-forge-linter <[email protected]>
Co-authored-by: Uwe L. Korn <[email protected]>
@hugobuddel
Copy link
Author

Is it possible to install the previous version of this package?

I'm installing black in our CI environment, which is now broken, so I'd like to install the previous build.

Earlier it installed 19.10b0-py37_0 (with a trailing 0), now it tries to install 19.10b0-py_2 (build number 2) and fails due to conflicts with other packages.

However, the file corresponding to build 0 does not seem to exist: https://anaconda.org/conda-forge/black/19.10b0/download/noarch/black-19.10b0-py_0.tar.bz2

So I don't know how to get the previous build back.

@hugobuddel
Copy link
Author

I'm trying to learn what is going on.

First off, it is not really clear what the reason behind #12 change is. It just says "add blackd recipe output". My guess is that the idea is to split the package into just 'black' and 'blackd'. This is corroborated by the availability of the blackd package.

However, there is almost nothing in noarch/blackd-19.10b0-2.tar.bz2:

$ tar -xvf ../blackd-19.10b0-2.tar.bz2
info/hash_input.json
info/paths.json
info/link.json
info/index.json
info/about.json
info/files
info/git
info/recipe/conda_build_config.yaml
info/recipe/meta.yaml
info/recipe/parent/meta.yaml
info/test/run_test.bat
info/licenses/LICENSE
info/recipe/parent/build_base.sh
info/test/run_test.sh
$ cat info/files
$ cat info/hash_input.json
{}⏎
$
$ cat info/paths.json
{
  "paths": [],
  "paths_version": 1
}⏎

This makes sense, because there is no files section in the blackd outputs section in the recipe. As in, no attempt has been made to actually separate the files for 'black' and the files for 'blackd'. E.g. both 'black' and 'blackd' are tested only in the 'blackd' package.

The build_base.sh that is currently in the files section of the black section seems to be wrong, but I have not yet figured out what should be there.

@hugobuddel
Copy link
Author

Ah, the idea was to have separate build steps, but instead the build script has been used to specify the files to include in the output.

And it is okay that the blackd package is empty, as it is a metapackage that only has the dependencies for blackd.

@xhochy
Copy link
Member

xhochy commented Jul 21, 2020

@carlodri can you take care of this?

@carlodri
Copy link
Contributor

@hugobuddel yes, you are right, things are definitely wrong in there, sorry for messing up your CI... The goal was to have a metapackage for blackd without polluting the requirements of black if someone did not need the server part.

@xhochy yes, I can try, and I might need your help there! 🙏

@carlodri
Copy link
Contributor

actually, I have just tested on my windows machine that in a fresh environment the installation of black and blackd work perfectly as they should...

@carlodri
Copy link
Contributor

probably the presence of spurious files in the archive comes from the fact that the build_base.sh script is missing the --no-deps flag (just an idea judging from other feedstocks)

@carlodri
Copy link
Contributor

you can pin to the latest good package temporarily with

conda install black=19.10b0=py38_0

or whatever python version you need

@hugobuddel
Copy link
Author

@carlodri NP for breaking my build! I'm using this as an opportunity to learn:

  • A single conda recipe can produce multiple packages.
  • It is possible to specify a specific build number for a dependency.

My direct problem is solved by pinning the version to =19.10b0=py_0. This means I'm using the anaconda black, but to be honest it is not clear to me what benefit the conda-forge version brings.

tadeu added a commit to tadeu/admin-requests that referenced this issue Jul 22, 2020
because it includes "vendored" packages (such as attr, regex and click)
(it has been fixed in build number 3)

See conda-forge/black-feedstock#13
beckermr pushed a commit to conda-forge/admin-requests that referenced this issue Jul 22, 2020
because it includes "vendored" packages (such as attr, regex and click)
(it has been fixed in build number 3)

See conda-forge/black-feedstock#13
ammattita6i added a commit to ammattita6i/admin-requests that referenced this issue Aug 14, 2024
because it includes "vendored" packages (such as attr, regex and click)
(it has been fixed in build number 3)

See conda-forge/black-feedstock#13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants