For a cloned environment: UnicodeDecodeError: Invalid start byte in METADATA file

[SRJaffry]

Description

Hi,

pip commands, such as 'pip list' is giving errors such that they are unable to read certain files.
When I dug into the issue, the files had gibberish data into them. The same command is working fine the root environment.

Following is the error when I ran pip list (for example):

> 
> ERROR: Exception:
> Traceback (most recent call last):
>   File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\cli\base_command.py", line 173, in _main
>     status = self.run(options, args)
>   File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\commands\list.py", line 179, in run
>     self.output_package_listing(packages, options)
>   File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\commands\list.py", line 255, in output_package_listing
>     data, header = format_for_columns(packages, options)
>   File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\commands\list.py", line 307, in format_for_columns
>     row = [proj.raw_name, str(proj.version)]
>   File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\metadata\base.py", line 163, in raw_name
>     return self.metadata.get("Name", self.canonical_name)
>   File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\metadata\pkg_resources.py", line 96, in metadata
>     return get_metadata(self._dist)
>   File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\utils\packaging.py", line 48, in get_metadata
>     metadata = dist.get_metadata(metadata_name)
>   File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1424, in get_metadata
>     return value.decode('utf-8')
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 14097: invalid start byte in METADATA file at path: c:\users\shan_jaffry\miniconda3\lib\site-packages\hupper-1.10.2.dist-info\METADATA
> 

When I open the file, it actually had the gibberish at the very end.

0.2 (2016-10-26)

• Windows support!

• Added support for watchdog <https://pypi.org/project/watchdog/>_ if it's
  installed to do inotify-style file monitoring. This is an optional dependency
  and hupper will fallback to using polling if it's not available.

0.1 (2016-10-21)

• Initial rele9ýl�·øA

Expected behavior

commands with pip should run smoothly, such as 'pip list'.
Commands such as pip --version are working fine.

pip version

21.2.4

Python version

3.9

OS

Windows

How to Reproduce

pip list

Output

ERROR: Exception:
Traceback (most recent call last):
  File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\cli\base_command.py", line 173, in _main
    status = self.run(options, args)
  File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\commands\list.py", line 179, in run
    self.output_package_listing(packages, options)
  File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\commands\list.py", line 255, in output_package_listing
    data, header = format_for_columns(packages, options)
  File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\commands\list.py", line 307, in format_for_columns
    row = [proj.raw_name, str(proj.version)]
  File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\metadata\base.py", line 163, in raw_name
    return self.metadata.get("Name", self.canonical_name)
  File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\metadata\pkg_resources.py", line 96, in metadata
    return get_metadata(self._dist)
  File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_internal\utils\packaging.py", line 48, in get_metadata
    metadata = dist.get_metadata(metadata_name)
  File "C:\Users\shan_jaffry\Miniconda3\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1424, in get_metadata
    return value.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 14097: invalid start byte in METADATA file at path: c:\users\shan_jaffry\miniconda3\lib\site-packages\hupper-1.10.2.dist-info\METADATA

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pradyunsg]

Sadly, in situations like this, pip can't really "just work" like you're hoping since there's no real way to read this file for pip. And reading this file is necessary for parsing the version.

What we can do is provide a clearer error message, so I'll mark this issue for that.

[Shivansh-007]

Hey, @SRJaffry could you provide a better reproducer, I tried reproducing the "bug" with pip==21.2.4 and hupper=1.10.2 but I couldn't reproduce it, there were no undecodable bytes at the end of the METADATA file of hupper.

I tried injecting them manually at the end of the file, but that didn't work either.

[pradyunsg]

Closing due to lack fo a response here.