name temporary files with a pattern which can be globbed

[l0b0]

When nbQA runs it extracts a foo[randomness].py file from foo.ipynb, and puts this file in the same directory as the notebook. This confuses my IDE, which is set to add any new files to VCS. It would probably be better to put this in a temporary directory.

[MarcoGorelli]

It would probably be better to put this in a temporary directory.

It used to be like this, but then that caused issues for importing modules

The temporary files should be automatically cleaned up after nbqa finishes running though, is that not what you're seeing?

[MarcoGorelli]

can't reproduce this, even on windows - my only guess is that you repeatedly ran ctrl+C while nbQA was still running and someone intercepted the finally block which cleans up the temporary files?

closing for now then, but thanks for the report

[l0b0]

No, I didn't Ctrl-C. nbQA just ran long enough for the IDE to see the file and stage it.

[MarcoGorelli]

I see - which IDE was it, just out of interest?

Either way, I'd suggest reporting to them (or seeing if they have a way of increasing the time for temporary files)

[l0b0]

JetBrains IDEA. It would be closed as not an issue, because it's a feature. I've set up my IDE to add all files to VCS when they are created.

[MarcoGorelli]

doesn't this cause issues for you when using other tools that create temporary files, such as pytest or yesqa?

[l0b0]

Those are in .gitignore, so the IDE ignores them. nbQA is special since it creates a file in the same directory and with almost the same name, without any easy way to glob just for the temporary files.

[MarcoGorelli]

so if the temporary file names all started with the same pattern, such as .nbqa.foo[randomness].py, then that would work as a solution for you?

if so, then sure, pull requests welcome

[l0b0]

so if the temporary file names all started with the same pattern, such as .nbqa.foo[randomness].py, then that would work as a solution for you?

Yeah, something like that would be great!

[MarcoGorelli]

cool, here's the contributing guide if you're interested in submitting a PR https://nbqa.readthedocs.io/en/latest/contributing.html

[jpmckinney]

How does nbqa generate the temporary file?

Here's a sample script, let's call it x.py:

import sys


if __name__ == "__main__":
    print(sys.argv[-1])
    print(list(sys.argv[-1]))

Running it:

$ nbqa x component_check_quality.ipynb
component_check_quality.ipynb
['c', 'o', 'm', 'p', 'o', 'n', 'e', 'n', 't', '_', 'c', 'h', 'e', 'c', 'k', '_', 'q', 'u', 'a', 'l', 'i', 't', 'y', '1', 'f', 'c', 'o', 'w', '4', '5', '_', '.', 'p', 'y']

As you can see, the first print matches the ipynb path. But, if it's broken into characters, it matches the temporary file path (component_check_quality1fcow45_.py).

This, of course, makes it difficult to match filenames in the command. Not sure how print() yields a different value than print(list()).

[jpmckinney]

Ok, figured out my issue. the value that the subprocess receives is the temporary file name. However, nbqa captures and rewrites output, such that temporary file names are rewritten as original file names. That's why list() shows the temporary file name.

I tried using breakpoint() in my command, but not sure it works in subprocesses.

[minrk]

I have another use case for predictable names: #823

I have directories containing notebooks and .py files, and want to apply a few different rules only in the notebooks (mainly because of failing to handle weird namespace shenanigans, which I don't expect the linter to deal with), so relying on directory prefix globs as suggested in #823 won't work without tracking the individual names of every notebook in the repo in the per-file-ignores.

[minrk]

#870 addresses this, I believe.