Include exclude globs with whitelist pattern

[dpgaspar]

Hi,

Making a POC for using poetry with apache/superset#8079 but I'm having some issues

First superset has a complex directory structure, I'm struggling with the include, exclude parameters in glob, using MANIFEST.in we could whitelist for example:

recursive-exclude superset/static *
recursive-include superset/static/assets/branding *
recursive-include superset/static/assets/dist *

But using poetry toml we can't, how can I implement the same?

# excluding all assets possibilities ( dotted directories and files )
exclude = [
    "superset/assets/*",
    "superset/assets/.*",
    "superset/assets/**/*",
    "superset/assets/**/.*",
    "superset/assets/**/.*/*",
    "superset/assets/**/.*/**/*",
    "superset/assets/**/.*/**/.*",
    "tests/**"
]

include = [
    "superset/assets/branding/**",
    "superset/assets/dist/**",
]

With this pattern all superset/assets get excluded. Any advice?

[MarcusSorealheis]

for the includes, it looks like you might need to add file extensions at the end @dpgaspar like: { include = "superset/assets/dist/**/*.css", include = "superset/assets/dist/**/*.js", } untested as discussed

[stale]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[rschmied]

This is also an issue for me. Like the @dpgaspar I want to exclude the superset of a directory and including specific files from that directory using

exclude = [
    "assets/sample/**/*"
]
include = [
    "assets/sample/subdir/one.yaml",
    "assets/sample/subdir/two.yaml"
]

This did work until recently and I suspect 12f981c has something to do with it. basically, since i upgraded from a previous 1.0.0 pre-release build, this was working as expected.

Not anymore with 1.0.0. Will downgrade for now.

[finswimmer]

The reason for this is, that there is no check whether a file is explicit included or not. poetry builds a list of all files it can found and remove those from this list, that are excluded by .gitignore or via exclude in pyproject.toml.

[rschmied]

Thanks. So, apparently this worked in previous builds and according to @dpgaspar this is also something that the MANIFEST supports.

How would this be done now with Poetry? I have lots of files in that sample directory but only want to include a minimal subset.

[finswimmer]

Could you please tell me in which version this works?

[finswimmer]

Sorry, I was wrong. poetry checks for explicit included files and doesn't exclude them. What might have changed is, that you have to start with the package name or a wildcard when giving the paths:

exclude = [
    "my_package/assets/sample/**/*"
]
include = [
    "my_package/assets/sample/subdir/one.yaml",
    "my_package/assets/sample/subdir/two.yaml"
]

[rschmied]

in terms of what version works: i'm not 100% sure as there is unfortunately no indication what i had before :(

in terms of your comment regarding the package name: That's what I already do. I have the package name as first path element

this is what i currently use (slight changed paths)

exclude = [
    "my_package/**/tests/**/*",
    "my_package/dir1/csrc/**/*",
    "my_package/dir1/automation/**",
    "my_package/dir1/automation/lib/.gitignore",
    "my_package/sample/**/*"
]
include = [                                                                                                                                         
    "my_package/sample/file1.yaml",
    "my_package/sample/file2.yaml",
    "my_package/sample/file3.yaml",

    # these are python files in the sample dir that must not be excluded
    "my_package/sample/__init__.py",
    "my_package/sample/sample_data.py",
    "my_package/sample/some_other_sample_data.py",

    "my_package/subdir2/schema/*.yaml",
    "my_package/**/*.so"
]

And just to be sure:

$ poetry -V
Poetry version 1.0.0
$ python -V
Python 3.6.8

The result of a poetry build -vvv is that none of the files in sample are showing up in the wheel.

[finswimmer]

Ah, the problem only exists for wheel and not for sdist packages (at least in my testings). Fix (#1750) is on the way.

[rschmied]

cool... also... fwiw:

Wheel-Version: 1.0
Generator: poetry 1.0.0b3
Root-Is-Purelib: true
Tag: py3-none-any

was working OK before.

[Umesh-JNU]

I am unable to exclude the pycache folder. Please tell me..

[iamishalkin]

I am unable to exclude the pycache folder. Please tell me..

Same issue.
I add
exclude = ["*/__pycache__/*"] to pyproject.toml, but it still appears in the tar.gz when build

[finswimmer]

Hello @iamishalkin,

please open a new issue with a minimal reproducible example included.

fin swimmer

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.