PEP 541 Request: install
#451
Comments
|
I acknowledge I might have been a bit adamant in my last reply. I am a bit frustrated, especially since given the owner's comments on how he believes my project is wrong, even after I provided links to discussions which explore the issue I am trying to solve in depth. This is not an excuse, I should have worded it in a more friendly way. |
|
Hi @FFY00, there's a few elements at play here. Firstly, at the moment of this writing, Secondly, if he does confirm he refuses the transfer, the PEP is clear about that:
Thirdly, in those emails you've posted the owner does make a case that Finally, you argue that your project is a better fit to the name
However, there's no sign of involvement of the PyPA in your project. I would encourage you to reach out to the PyPA (I suggest the official Python forum) and discuss the project with them, including any potential names and subsquent actions. I would very much like for other @pypa/pypi-moderators to comment on this. Personally, I think the best course forward is to place |
This change was made after I created the request. The owner admitted to namesquatting and refused to collaborate, and as such, I do not believe this change was made in good faith. I have been very patient and cordial when dealing with this, up to the penultimate email, I have explained why the project is needed and why other alternatives are not usable, @eugenekolo simply dismisses it as "I don't agree" providing no arguments. I do not think this action is correct. The package had no downloads until today, which now has 2. And after I checked out the contents of the package, I cannot help to feel like this is a personal attack. #!/usr/bin/python
import subprocess
import sys
import shlex
import os
import tempfile
import urllib.request
def _get_pip():
fd, path = tempfile.mkstemp('_get-pip.py')
urllib.request.urlretrieve("https://bootstrap.pypa.io/get-pip.py", path)
subprocess.check_call([sys.executable, path])
def _check_pip():
try:
subprocess.check_call([sys.executable, '-m', 'pip'], stdout=subprocess.DEVNULL)
return True
except subprocess.CalledProcessError as exc:
return False
def install(pkg, use_pep517=None, requirements=None, options=None):
"""Install packages dynamically in your code
Args:
pkg: Name of the package as a string, you can also use version specifiers like requests==1.2.3
use_pep517: Optionally set to True or False to force --use-pep517/--no-use-pep517
options: Arbitary list of options to pass to pip for installation
"""
if not _check_pip(): _get_pip()
cmd = [sys.executable, '-m', 'pip', 'install']
if options and isinstance(options, list):
cmd.extend(options)
if use_pep517 is True:
cmd.append('--use-pep517')
elif use_pep517 is False:
cmd.append('--no-use-pep517')
pkg = shlex.quote(pkg)
cmd.append(pkg)
subprocess.check_call(cmd)The package downloads "I believe package installation should be delegated to pip solely." "No package manager is perfect, but I do not believe creating additional build/installation tools is the solution." I can't...... About the PyPA involvement, I talked with @pganssle (sorry for tagging you like this, please let me know if you wish me to not do that in the future) and the outcome of the conversation was that we could have a standalone tool to build wheels and one to install them. |
|
Thanks for reaching out to me @yeraydiazdiaz. I refuse the transfer. I am okay with either the package name being blocked, or for me to continue maintaining ownership of the package. The current functionality of the package is being used in a few projects of mine and may be useful to others too. |
|
@eugenekolo that doesn't make much sense. If we were to place the project name in the blocklist you wouldn't be able to use that code with that name thus breaking your own usage of the project. Regardless you uploaded some code after the request was raised and now you're claiming you're actively using the code. PEP 541 does not completely cover every possible claim case so we rely on the good will of the community to work together to resolve these issues. |
|
If I may, I would like to advocate against placing the name on the blocklist. The I just wanted to state this before any further discussion. I will now refrain from making any other comments (until I am directly addressed) and let you proceed with the conflict resolution. Thank you :) |
|
@yeraydiazdiaz Sorry for the misunderstanding. The code I've placed into the package is code I am using on other projects. It was added to the install package and will continue to be maintained there. Thanks for looking into this. |
|
@eugenekolo You registered the project in 2016 with no functionality and then you pushed a release with some code after the request was made in an attempt to keep the project name. I understand you probably had plans for that project name in 2016 but the fact is you have not used it in four years. By pushing some code now, immediately after the PEP 541 request, and giving some vague reasons for it you're not showing a lot support for a process that's driven by volunteers such as myself. From my point of view as a moderator |
|
I am in favor of install being on the blocklist. |
|
Tagging @pypa/warehouse-admins. |
|
As an admin, I'm +1 to adding to prohibited project name list. |
|
What is the argument for putting it onto the blocked list, particularly if Filipe is going to put a legitimate package there? It's possible someone will typo it and it will accidentally install the extra package, but that's true of many things. That's really only dangerous if something malicious lives there. I don't really think it matters very much, I think Filipe's package could easily be named I am just not sure what the argument is for |
It is currently only a wheel installer, but I wanted to keep the possibility of other types in the future. Much like https://github.com/FFY00/python-build. It is not crazy to think that in the future other package types could arise. Similarly, I don't really understand the argument. I feel like this is being a little too conservative. We are preemptively trying to solve an issue that has no real data backing it up -- by this I mean no data supports that people are mistakenly running |
|
puts on pip maintainer hat
@FFY00 had also discussed this with me (and I'm a PyPA member, but don't represent all of PyPA). We'd talked about his project and https://github.com/pradyunsg/installer. The latter is certainly a project that I will be putting forward for being accepted into PyPA once it's "ready". He also expressed interest in doing so, if puts on PyPI moderator hat I'm definitely a +1 for putting this on the prohibited project name list. When @FFY00 and I discussed about installers for Python packages, I'd advised him that
Typos like |
Yeah, I get that it is something you might install accidentally, but I might also mindlessly type anything in there. The only problem with "typo squatting" like this is if someone puts something malicious there (or has the potential to put something malicious there), and accidental installs lead to harmful outcomes. This is just a situation where someone might accidentally install more packages than they wanted to, which is not really a problem and happens constantly. |
|
Assuming that `install` never gets compromised, ofc.
…On Tue, Jun 16, 2020, 1:10 PM Paul Ganssle ***@***.***> wrote:
Typos like pip install install <list of packages>
Yeah, I get that it is something you might install accidentally, but I
might also mindlessly type *anything* in there. The only problem with
"typo squatting" like this is if someone puts something *malicious* there
(or has the potential to put something malicious there), and accidental
installs lead to harmful outcomes.
This is just a situation where someone might accidentally install more
packages than they wanted to, which is not really a problem and happens
constantly.
—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
<#451 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAB5I42MQ6WPEACHPIFRQJ3RW7GVFANCNFSM4N5GAF6A>
.
|
Sure, but if the threat model is "package compromise", then it's no different whether it is deliberately installed or accidentally installed. In fact, things that are primarily (or even very frequently) accidentally installed are probably a particularly bad target for compromise, because that doesn't happen terribly often compared to deliberate installs and when someone accidentally installs something they're much more likely to scrutinize it de novo than if they deliberately installed it. Like I said, I don't really care that much, and I don't even really think |
|
Hello, I'd like to bring up a few facts and statements to this conversation. Unfortunately, FFY00 has blocked me on GitHub, and that means I cannot respond to any threads created by them with my main account, so please excuse this secondary account.
I believe that in accordance with PEP-541, I am in the right to continue maintaining this project as I am a reachable and legitimate maintainer of a project that is receiving active installations. |
My bad, this was to prevent you to interact with my personal repositories, as I do not think the way you handled this was correct. I did not think it would prevent you from commenting on this issue. I have temporarily removed it until this gets resolved.
I would really like to believe this but it does not track with our email communication. After I contacted you, you did not pushed anything, that only happened after I opened this request. Furthermore, what you said in our emails goes directly against this statement.
Respectfully, all this seems like damage control... I always try to assume best intentions, unless I have proof against it. In this case, I find it difficult. But if you are truly being genuine, I apologize for all the drama/trouble. This would have been simply resolved by you replying "Hi Filipe, I do have plans to publish something under that namespace.".
Well, I actually think this specific case is not defined in PEP 541. This was name squatting, you only pushed something after this request was opened. |
|
I think we have enough information from involved parties for the time being, and it looks like things could get unnecessarily heated. I'm going to lock this to maintainers/admins for the time being until we can get a decision made. |
|
As long as this is a valid and non-malicious project, I'm also not really seeing a reason to put this on the prohibited project name list. Looking at the source for https://pypi.org/project/install/ (see below) it seems to be both, basically just a thin importable wrapper around I'm wondering if there's a potential for collaboration here if both parties are willing. Based on the discussions in https://discuss.python.org/t/moving-python-build-to-pypa/4390/ about @FFY00's I imagine it would require an importable API as well, so https://pypi.org/project/install/ could have roughly the existing behavior (importable API to invoke Ultimately if we can't find a compromise, I agree with @pganssle that https://github.com/FFY00/python-install doesn't really need a short, memorable name as it's not likely to be widely used by end-users. In addition, PEP 541 is quite clear that a name should not be reassigned against the wishes of a reachable owner. PS: I couldn't find a source repository for https://pypi.org/project/install/, so I extracted the source from the latest distribution and included it below to make it easier to understand the goal of this project. #!/usr/bin/python
import subprocess
import sys
import shlex
import os
import tempfile
import urllib.request
def _get_pip():
fd, path = tempfile.mkstemp('_get-pip.py')
urllib.request.urlretrieve("https://bootstrap.pypa.io/get-pip.py", path)
subprocess.check_call([sys.executable, path])
def _check_pip():
try:
subprocess.check_call([sys.executable, '-m', 'pip'], stdout=subprocess.DEVNULL)
return True
except subprocess.CalledProcessError as exc:
return False
def install(pkg, use_pep517=None, requirements=None, pip_options=None, install_options=None):
"""Install packages dynamically in your code
Args:
pkg: Name of the package or requirements.txt file as a string, you can also use version specifiers like requests==1.2.3
use_pep517: Optional boolean to force --use-pep517/--no-use-pep517
requirements: Optional boolean if a requirements.txt was specified
pip_options: Optional arbitary list of global options to pass to pip
install_options: Optional arbitary list of install options to pass to pip install
"""
if not _check_pip(): _get_pip()
cmd = [sys.executable, '-m', 'pip']
if pip_options:
if isinstance(pip_options, list):
options = [shlex.quote(option) for option in pip_options]
cmd.extend(options)
else:
raise TypeError('pip_options passed to install must be a list')
cmd.append('install')
if install_options:
if isinstance(install_options, list):
options = [shlex.quote(option) for option in install_options]
cmd.extend(options)
else:
raise TypeError('install_options passed to install must be a list')
if use_pep517 is True:
cmd.append('--use-pep517')
elif use_pep517 is False:
cmd.append('--no-use-pep517')
if requirements:
cmd.append('-r')
pkg = shlex.quote(pkg)
cmd.append(pkg)
subprocess.check_call(cmd) |
|
(I'll give this a little more time to cool off and for other admins to add their thoughts before unlocking.) |
|
I'm going to motion based on @di's research that this project be placed on the prohibited project list, and that the existing project is removed. Especially given that there have been a few reports regarding the package name to the security list. It voices a number of similar concerns around confusion and potential for abuse. Given that, I propose a 30 day window beginning with admin approval to provide @eugenekolo time to release the project with a new name before the existing one is removed. |
|
Per https://deps.dev/pypi/install/1.3.5/dependents, this project has 69 direct dependents and 19 indirect dependents. I haven't looked at all of them but I'm going to assume at least some of them are legitimately using the project for its intended purpose, so in the interest of not breaking those downstream consumers, I still think this should be left as-is. |
|
I have created an overview of all direct dependents. Usage of
|
| Author | Packages | Used in code | downloads/month | comment |
|---|---|---|---|---|
| ffreemt | 28 | 5 | 1107 | contributor to install |
| gkostass | 4 | 62 | ||
| miklevin | 2 | 480 | ||
| waza-ari | 1 | 4969 | removed with waza-ari/fastapi-keycloak-middleware#47 | |
| atherashraf | 1 | 550 | ||
| whaly-w | 1 | 489 | ||
| acivitillo | 1 | 446 | ||
| deepkeep | 1 | 431 | ||
| HamzaFarhan | 1 | 367 | ||
| blueboxdev | 1 | 267 | ||
| group_projecty | 1 | 208 | ||
| krisapostolov | 1 | 194 | ||
| rkrishnasanka | 1 | 141 | ||
| matteocacciola | 1 | 113 | ||
| tobiadefami | 1 | 113 | ||
| ghm2189 | 1 | 65 | ||
| AkalankaSakalasooriya | 1 | 57 | ||
| thomktz | 1 | 52 | ||
| alexanderlozovoy | 1 | 40 | ||
| kforti | 1 | 40 | ||
| sayantan013 | 1 | 38 | ||
| xenups | 1 | 34 | ||
| tinvo0908 | 1 | 33 | ||
| jshossein | 1 | 24 | ||
| Luttik | 1 | 22 | ||
| willembressers | 1 | 22 | ||
| qren | 1 | 21 | ||
| oalami | 1 | 19 | ||
| cspielvogel | 1 | 17 | ||
| sagara | 1 | 15 | ||
| Rishabh20539011 | 1 | 15 | ||
| codesapienbe | 1 | 15 | ||
| amarco | 1 | 10 | ||
| summary | 64 | 5 | 10476 |
|
I'm not entirely sure how this process usually works, but as I've been tagged and specifically have been reached out to, I spent some time trying to investigate what actually happened. I was able to track down the presence of the Straight from bash history: poetry add install python-keycloakThe following is just a potential explanation, I do not have clear memory anymore. The documentation of a dependent project python-keycloak has installation instructions using regular pip install python-keycloakI probably typed |
|
This seems to support the argument that we should remove this package and add it to the block list. |
|
Petr's and mine recommendation here seconds what Pradyun is saying in the message above. |
|
Project |
|
@ewdurbin Hello there o/ Is there a way where the web page for the package could be redirected to the new package, generally for any Prohibited Project Names? Context: my team was shipping an app couple days ago when the Github Action choked on the dependency installation check. Looking at the logs pip reported that package for supported version not found. So I tried to dig up the package on PyPI and found a package not found page. Since the dependency was setup before my time on the team, I didn't know what it did. So I tried googling for the package and the error before landing on the github project. Thankfully there was a github issue that clearly articulated the issue and linked to this thread. The fix was easy for us then, just point to the new package name. But going through the process I figured it might be helpful for other developers if PyPI had some information on the deprecated package page that showed some context and path to resolution. Especially if the original package author was engaged and has a preferred migration path. So, appreciate the documentation here which I was able to find through Google. But I wanted to share my experience and offer a recommendation to make it easier for any future deprecations. And thanks for all your hard work! |
Same thing happened to us! 😅 |
|
Seconding what @kamikaz1k said. It was especially uncomforting not knowing if this was a security issue which we should respond to in any way. IMHO it would be really useful if PyPi had a transparency log which shows why packages have been retracted / removed / blocked. |
I could be wrong, but I believe this kind of project deletion should show up in the project journal/XML-RPC events. But I fully agree that a transparency log would be a good solution here; there's been some early design thought around that 🙂 |
|
Given installation by poetry is (apologies if this is the wrong place for this comment) |
|
@fredkingham That might be a better fit for a new issue along similar lines 🙂 (With that being said Poetry isn't a PyPA tool, so they could also reject |
|
Turns out (sorry for the time waste!) |
|
No problem -- it looks like it fails because |
This line crashed requirements installing process (see below). No code is using the package, so removing the dependency looks acceptable. FYI: `install` is `pip-install` now, see: pypi/support#451
and others
Project to be claimed
PROJECT_NAME: https://pypi.org/project/installYour PyPI username
USER_NAME: https://pypi.org/user/FFY00Reasons for the request
Name squatting.
Maintenance or replacement?
Replacement.
Upstream: https://github.com/FFY00/python-install
Contact and additional research
I contacted the owner, he confirmed he was name squatting. His main argument was that he is trying to prevent the abuse of the namespace to distribute malware, however I have a valid use for it.
The full details:
I lurked google and the owner's github page for the project but I couldn't find it, I contacted him on the 28th of May:
He reached out on the 10th of June:
I replied the same day:
And he finally replied today (14th of June):
I sent my final reply and decided to open this request since the owner confirmed the name squatting and doesn't seem to be collaborative:
The text was updated successfully, but these errors were encountered: