[v2][proposal] Installing AWS CLI v2 from source #6352
Conversation
Codecov Report
@@ Coverage Diff @@
## v2 #6352 +/- ##
==========================================
+ Coverage 93.68% 93.69% +0.01%
==========================================
Files 350 350
Lines 35690 35690
Branches 5105 5105
==========================================
+ Hits 33436 33440 +4
+ Misses 1652 1648 -4
Partials 602 602
Continue to review full report at Codecov.
|
|
I was able to get this to successfully build on Windows 10 using git-bash, MinGW, and a manually installed CMake. The configure script didn't check for the presence of CMake, which is needed to build Still, great success! C:\Users\theto>"c:\Program Files\AWSCLI\bin\aws.cmd" --version
aws-cli/2.2.1 Python/3.9.6 Windows/10 source/AMD64 prompt/off |
|
Can "community contributions to the AWS CLI" be included in the motivations? Without the ability to build from source, it's impossible for a potential contributor to test their changes. I know that's not an immediate need but this is a building block for it. |
proposals/source-install.md
Outdated
| ## Motivation | ||
|
|
||
| The AWS CLI v2 is available as pre-built executables for macOS, | ||
| Linux x86_64, Linux aarch64, and Windows (64-bit). The AWS CLI v2 is also |
There was a problem hiding this comment.
The linux should probably also specify glibc.
There was a problem hiding this comment.
I have a bullet point for it later on, but I'll see if I can include a note earlier.
|
I can confirm the poc-sdist branch works on alpine 3.14: It takes quite long time but I guess that is expected. |
Sorry for jumping in this thread, but this is very good news 👍 |
|
so happy this works on alpine now, can't wait for it to be merged |
|
Hi @kdaily - I've played with this branch a bit from the distro packaging point of view. Thanks very much for your effort here! The debian cloud team has had requests for packages for awscli v2, and we'd like to be able to provide them. I haven't gotten very far, but already have some questions. Some of these things are probably bugs on other repos, and I'm sure others are misunderstandings. Please pardon my ignorance, I'm new to the awslabs tools used in this repo. Build orderIs there a reasonable build order known? I started to recurse through git modules, but it was hard to be confidant I had things correct. s2n and libcryptoWould the team react negatively to awscliv2 packages that used s2n's openssl libcrypto support instead of aws-lc? In this config, s2n was easy to get going on, while aws-lc would be more work. static vs dynamic linkingSo far, most of the modules I've looked at default to building statically. Some (e.g. Generally, dynamic libs are preferred in distro packages. But if that's not really/well/fully supported by the upstream projects, that'd be really useful to know. litani & aws-templates-for-cbmc-proofsNeither one defines the usual kind of python package. Both look like they are just designed to be included in the source of a new project. Am I understanding their intended usage correctly? Debian's packages are only built from components in debian, not gitmodules. So we'll need to work a bit differently with these tools. Here's one way this could work. During the build of e.g.
Build instructions/docsNo question here, just some feedback - some modules do not have build instructions. Some assume you're building on Ubuntu in an environment emulating CodeBuild. Generally, distro folks will have an easier time with direct build instructions. Thanks again for this effort! |
proposals/source-install.md
Outdated
| * Pulling in a new Python library dependency | ||
| * Requiring a new system dependency | ||
|
|
||
| Furthermore, the `--with-downloaded-deps` option does not guarantee that all |
There was a problem hiding this comment.
| Furthermore, the `--with-downloaded-deps` option does not guarantee that all | |
| Furthermore, the `--with-download-deps` option does not guarantee that all |
| both the sdist and wheel | ||
| (see [PEP 517](https://www.python.org/dev/peps/pep-0517/#in-tree-build-backends)). | ||
|
|
||
| * Pull in and maintain the unreleased `botocore` `v2` branch along with |
There was a problem hiding this comment.
This makes me so sad. Just release/publish botocore v2! So much easier for the entire community.
There was a problem hiding this comment.
So much easier for the entire community.
Could you elaborate on what specifically will make things easier? If we were to release a new major version of botocore using what is currently on the v2 branch, it would have the following implications:
-
Right now the changes on the v2 branch is only scoped to changing defaults and removing functionality. The major version release would not address many of the popular feature requests that would only be feasible in a major version bump (e.g., Async support, modularization, out of the box typing support) and likely require larger rewrites of botocore (or even replace botocore outright). Currently, the maintainers of the Python SDK are exploring how to go about making these changes such as in their work writing the Amazon Transcribe Streaming SDK.
-
It would require a major version bump of
boto3and force many downstream Python packages to account for new major versions ofbotocore/boto3(e.g., by updating their version range onbotocore/boto3or making major version bumps themselves). If appropriate adjustments aren't made, then these downstream packages would not be usable alongside the latest version ofbotocore.
In the end, given the current state of changes in the v2 branch, a major version bump does not seem worth the churn it would cause to the community, especially because it would be missing key features that would require another major version bump down the road.
There was a problem hiding this comment.
That makes sense. I didn't realize the botocore v2 dependency of the released v2 awscli was so incomplete! That's a bit of an ugly dependency tree for the v2 cli. Sorry I'm so bitter. Appreciate the detailed reply. I'll try to be more kind.
So, would it not work to publish the dev releases of botocore v2? There's less of a compatibility promise that way (except to yourselves). And you could still include "normal" python dependency references and installation that way. Pip won't install dev releases unless you tell it to explicitly.
There was a problem hiding this comment.
Sorry I'm so bitter. Appreciate the detailed reply. I'll try to be more kind.
No worries. The frustration is understandable given our inconsistent communication with the community about the conversations/decisions happening around issue prioritization and project direction (which we are trying to improve).
I didn't realize the botocore v2 dependency of the released v2 awscli was so incomplete! That's a bit of an ugly dependency tree for the v2 cli.
I completely agree. Originally the intention was to fast-follow with a major version bump of botocore/boto3 so we would not be sitting on this unreleased branch of botocore for long, but the direction has changed as we realized it was going to require much larger breaking changes to implement the larger features that the community wants.
So, would it not work to publish the dev releases of botocore v2?
These are the main concerns:
-
If you are going to try to install the AWS CLI v2 side-by-side with
boto3as a Python package installation, it would not be feasible becauseboto3relies on v1 ofbotocoreand the CLI v2 would require this dev version. For Python developers, they should be savvy enough to install the CLI into a separatevirtualenvor usepipx. However, this becomes really problematic for Linux distro maintainers where they only package Python packages at the system level without a virtual environment (because virtualenvs work against their packaging principles). As alternative, distro maintainers should be able package using theportable-exeinstall type, but if we were to rely on a public dev version forbotocorev2, it would block their ability to treat packaging the AWS CLI v2 as a typical Python package. And we want to make sure we are minimizing roadblocks for distro maintainers to pull in v2. -
Because a larger rewrite will be needed for the larger changes that the Python SDK maintainers want to make, there is a reasonable chance it warrants creating a new package name (e.g.
botocore2or even depart from thebotocorename entirely). If we start releasing public dev versions ofbotocorev2 and decide to use a new package name for v2, it creates an awkward story where we've been releasing dev versions ofbotocorev2 but the actual GA version is a completely separate package on PyPI.
In the end, I do not like that we'd be pulling in the unreleased botocore branch into the codebase, but at the current moment, seems like the most attainable option that unblocks issues installing the AWS CLI v2 from source and is also a two-way door. Long term, I would like to see us move a way from this approach through any of these options below:
- Integrate with whatever the new major version of the Python SDK would be.
- Figure out if there is a path forward to re-integrate with
botocorev1 if the new major version of the Python SDK is not a viable path forward. Theoretically it seems feasible? But there may be some issues that make it not straightforward. - Rely on another language SDK or more directly rely on the Python bindings for the AWS Common Runtime as that project expands.
Thoughts?
There was a problem hiding this comment.
Ahh, so if a botocore v2.dev version is installed in the botocore namespace, I can see how that could conflict with other local python development. Practices in this area of python have come a long way the last few years, and not using a virtualenv for that python development, and not using pipx for cli installs...yeah that just ends in pain.
In that case, a new namespace, e.g. botocore2, and publishing only dev versions for now makes a lot of sense to me. It would also allow other python libraries to start integrating with it without jumping through hoops, to explore new features or any interface changes they may need to make.
I don't think I have enough experience with the distro packaging world to understand this piece:
if we were to rely on a public dev version for botocore v2, it would block [distro maintainers] ability to treat packaging the AWS CLI v2 as a typical Python package
Why wouldn't they be able to use the public dev version? I'm probably off base, but I'd think they would also package "botocore2" and establish the dependency the same way they would for other packages?
There was a problem hiding this comment.
Also, thanks for thinking about the challenges facing distros, it's definitely appreciated.
@rvandegrift Yep! Removing these challenges is important to us. While our official installers (PKG, MSI, Linux installer) are not going away, long term it is going to be a better experience for everyone if they can install the AWS CLI v2 through their standard package manager.
There was a problem hiding this comment.
In that case, a new namespace, e.g.
botocore2, and publishing only dev versions for now makes a lot of sense to me. It would also allow other python libraries to start integrating with it without jumping through hoops, to explore new features or any interface changes they may need to make.
@lorengordon Yeah I think is the strategy that we would like to see if we went down the route of integrating with the new major version of the Python SDK. Specifically, the maintainers of the AWS Python SDK would publish 0.x.y versions under the new namespace and the AWS CLI would try to keep up with any interface changes they made while progress was still being made. This approach would also allow other Python developers to give feedback on the changes being made.
If we were to try to integrate with the package before it was GA'd, we'd definitely need to be mindful of a couple things as a GA'd project relying on a non-GA'd project:
-
We should make sure to integrate once we are confident that there would be no significant changes to the client/session interfaces. If there were significant changes to those interfaces, the hope would be that there is some back compat interface to ease the migration as it could be potentially an enormous effort to update all client usage and we would only have to make the change once.
-
The moment we do integrate it makes it more difficult the new project to make larger changes quickly as a large GA'd project is relying on it. So if we just put whatever was on the
v2branch published under abotocore2namespace and immediately started relying on it, it will be impede/slow down progress on many of the broader, more invasive changes the maintainers of the Python SDK team do want to make. This was similar to the situation when AWS CLI v1 GA'd butbotocorewas still a non-GA'd project. While it was still feasible to make larger changes in thebotocorecodebase, making changes was not straightforward, and at times, design choices were made with emphasis on the AWS CLI usage, resulting in tight coupling and interfaces that could be better generalized/defined. With this new major version, it would be ideal to see the larger changes have a stronger foundation before on-boarding the AWS CLI to not too strongly influence the approach.
There was a problem hiding this comment.
The moment we do integrate it makes it more difficult the new project to make larger changes quickly as a large GA'd project is relying on it. So if we just put whatever was on the v2 branch published under abotocore2 namespace and immediately started relying on it, it will be impede/slow down progress on many of the broader, more invasive changes the maintainers of the Python SDK team do want to make.
I'm not sure I understand how this is true. It is already the case that you pin to a specific botocore v2 tag, and increment that for each release, testing as you go, right? So, already a large GA'd project is relying on the botocore v2 implementation. Are you saying the botocore project wants to start v2 over again, basically? Or that you want to relax that strict pin in awscli v2?
There was a problem hiding this comment.
So, already a large GA'd project is relying on the botocore v2 implementation. Are you saying the botocore project wants to start v2 over again, basically?
I'd say it is more along the lines of starting over. The repository is not public yet, but the maintainers are exploring a rewrite in a separate repository where they are looking to add support for all of the big asks (e.g. async, modularization, static classes + typing). It is still in early development, which is why it is not public yet. Their plan is to make it public sooner as opposed to later in order to get early feedback (well before it goes public GA), but the repository needs to get into a spot where it is conducive for the community to provide feedback.
There was a problem hiding this comment.
That makes sense. Many of the other AWS SDKs also seem to take that approach, releasing new major versions as a new repo with a new namespace for the package. Makes a lot more sense, with that in mind, to vendor your usage of the current botocore-v2 branch!
I really appreciate the time and thought you've put into this. I may be a curmudgeon a lot of the time, but I do recognize the effort and difficulty of what you're doing. This is a big ship to steer. Very glad to see the team more actively engaging here also!
| the number of cycles required to figure out how to install the AWS CLI from | ||
| source. | ||
|
|
||
| 3. The source installation process is language agnostic. While the AWS CLI v2 |
There was a problem hiding this comment.
This feels like such an odd goal and the result of overthinking and overcomplicating things. The installation process can be language agnostic, but the build process never is. It is the responsibility of users choosing to build from source or install using language-specific tooling to understand how that works. You don't have to promote language-specific tooling as an official method and you can guide users to your own packaged and hosted installers, but language-specific methods should be supported and maintained (not "provisional"). As a python project implementing a cli utility, users should be able to simply pipx install awscliv2.
If you switch to Go, for example, users will want to be able to use go install ... and if this were a Go project that should just work! Switching languages like this would almost certainly qualify as a major release anyway, and so have no backwards combability guarantees, not of the user interface and not of the build process!
There was a problem hiding this comment.
It is the responsibility of users choosing to build from source or install using language-specific tooling to understand how that works.
What if building from source is their only choice? A large number of the users that will be leveraging this proposal will be the ones that have no other option to install the AWS CLI v2 (i.e., they are running on platforms where there is no official installer). Furthermore, these users are not all going to be experienced in any particular language, and nor should they need to be experienced in order to get a language-agnostic tool installed on their system. For example with v1, using Python tooling was often the only way to get the CLI installed on their system and this often bit non-Python developers whether they had to spend an significant amount of cycles trying to install the CLI or broke other tools installed on their system. I think if there were other valid alternatives to install the AWS CLI v2 for this group of users, the goal could be relaxed/removed.
language-specific methods should be supported and maintained (not "provisional"). As a python project implementing a cli utility, users should be able to simply
pipx install awscliv2.
Maybe provisional has the wrong connotations and there's a better word or something needs to be more explicit in the definition? But the idea is that as long as the AWS CLI is a Python project, we will comply to Python standards (e.g., PEP517) to enable building/installing of the AWS CLI using Python tooling. For example with pipx (assuming it complies with PEP517 in building a wheel), you should be able to just run pipx install . on the repository. In the end, if a specific tool supports PEP517 and only requires support for that standard, we are not going to prevent users from using that specific tool on the CLI codebase; we'd likely just not document/promote using the tool on the codebase.
The idea with labeling it as provisional is that allows the project to adapt to whatever language-tooling is appropriate based on the repository and language community. For example, it does not make sense to support PEP517 if the codebase was entirely written in Go. And yes while if we were to rewrite it in Go, it would likely warrant a major version bump, breaking PEP517 compliance would not necessarily require a complete rewrite. For example:
-
We could introduce a new separate executable to the AWS CLI package (similar to the separate
aws_completerwe ship in the package today) that could be written in a different language. -
A new PEP in the future could replace PEP517 (i.e., come out with a new format to replace wheels). Ideally the CLI project would just support this new PEP while dropping support for PEP517 in order to follow what is recommended for a Python project.
There was a problem hiding this comment.
What if building from source is their only choice? A large number of the users that will be leveraging this proposal will be the ones that have no other option to install the AWS CLI v2 (i.e., they are running on platforms where there is no official installer).
This really seems to be taking on an awful lot yourselves and introducing a lot of complexity as a result. At some point, there has to be a boundary between this project and the user's environment. I might suggest focusing on creating official installers for those platforms?
And, the approach detailed here is still building from source and requires a healthy python install to make that work! So I don't really see much difference, I guess. Users that, before, had trouble managing python runtime isolation and installing packages into a specific python runtime will still have trouble, won't they? It is certainly nice to provide the logic for creating the portable and sandbox packages, though. Certainly do appreciate that!
this often bit non-Python developers whether they had to spend an significant amount of cycles trying to install the CLI or broke other tools installed on their system.
Publishing to PyPi and constraining the recommendation for a python-based cli installation to use pipx instead of pip might go a long way to reducing these problems, I think.
There was a problem hiding this comment.
This really seems to be taking on an awful lot yourselves and introducing a lot of complexity as a result. At some point, there has to be a boundary between this project and the user's environment.
Yeah it's definitely not an insignificant amount of effort to maintain this. The end approach is actually fairly similar to the bundled installer for v1. So we have experience with the approach/boundary and this new approach tries to incorporate lessons learned with that installer. For example, the main differences between the bundled installer and the build process in this proposal are:
-
The bundled installer is a zip containing the sdists for all of the AWS CLI v1's dependencies. This was fairly limiting because users could not control what version of a dependency they wanted to install and could not leverage already built wheels (and would have to rebuild the wheel as part of installing the bundled installer). With this build process it just reuses the Python packages already on the system or just downloads them into a virtualenv using pip (which has more flexibility in that it can better choose what artifact to download and install depending on the system).
-
The bundled installer had backwards compatibility guarantees that all you need was a Python interpreter and any subsequent upgrade should continue to pass without any new software being installed on the system. This has prevented us from pulling in dependencies that require a compiler or even upgrade some dependencies. With the proposal, there is no backwards compatibility guarantee on needing additional system dependencies.
I might suggest focusing on creating official installers for those platforms?
Yep we should or even better yet make the AWS CLI v2 installable through standard package managers for those platforms. The primary purpose for this approach is for the development team to stop being the bottleneck for all potential users that currently have no path forward to installing the AWS CLI v2 as it would take a considerable amount of time to reach full coverage for all of the platforms people want to run the AWS CLI. Ideally, in the future, less users would need to use the source distribution, but there is no timetable for when that future would take hold.
And, the approach detailed here is still building from source and requires a healthy python install to make that work! So I don't really see much difference, I guess. Users that, before, had trouble managing python runtime isolation and installing packages into a specific python runtime will still have trouble, won't they?
Not necessarily. The idea is that for the user that has no experience in Python nor cares about reusing existing Python dependencies (i.e., just get the AWS CLI v2 installed in the quickest way possible) they should be able to blindly just run the following commands and the build/install just works (because it is abstracting over the common issues such as installing/isolating the packages to a virtualenv):
./configure --with-download-deps
make
make install
Especially, with v1, non-Python users would just blindly run pip install awscli because they would not know any better and not realize there is best practices to installing Python packages.
I do think the Python community has improved in the area of tools enforcing best practices and making them more accessible (e.g., pipx), but the concern is these tools are still Python specific and would force non-Python users to install and learn a new tool/terminology outside of their standard package manager or configure/make workflow.
There was a problem hiding this comment.
Not necessarily. The idea is that for the user that has no experience in Python nor cares about reusing existing Python dependencies (i.e., just get the AWS CLI v2 installed in the quickest way possible) they should be able to blindly just run the following commands and the build/install just works (because it is abstracting over the common issues such as installing/isolating the packages to a virtualenv)
Well ok. It seems like a nice idea. But isn't it just trading expertise with python for expertise with make? And probably particular versions of make and the shell (MacOS problems). And some number of other system packages, and the inscrutable errors from make when an assumption in the script/Makefile gets violated.
There are totally valid use cases where users need to build from source, and providing utilities like this does make it easier (which is awesome!). But it doesn't really seem like this is making anything any easier for the type of user that is going to be having problems managing their system and Python runtimes. This is still a very advanced type of install requiring specialized knowledge and tools. These are orthogonal use cases.
these tools are still Python specific and would force non-Python users to install and learn a new tool/terminology
... When there is no official installer for their platform. Or when their environment restrictions are such that they are prevented from using an official installer (e.g. non-admin user on Windows with awscli v2)...
But even then, it feels much lower effort to lean into the packaging tools and practices and expectations for the programming language, and to actively support that as a lowest common denominator (see above, trading one bit of expertise for another). Publishing to PyPi is trivially easy, and addresses a large swath of the user and developer community (~100mil/month!). Why put so much thought and effort into making things harder for those users?
There was a problem hiding this comment.
Oh wow, over the last month, botocore and s3transfer are in the top 5, and awscli v1 is 10! People really love these on pypi! https://pypistats.org/top
There was a problem hiding this comment.
I think this is actually evidence the AWS CLI has a significantly wider userbase than Python users.
There was a problem hiding this comment.
@benkehoe. I agree. One of the most common use patterns is to just install the CLI in your CI/CD/automation workflow and run whatever CLI command you need for your workflow. The large download numbers is a reflection of this usage pattern, especially when you keep in mind v1 is limited in that pip install is the only official installation mechanism that is relatively mainstream and users have a tendency of always installing the latest AWS CLI as part of each automation run (instead of relying on some pre-built image that already has the CLI installed).
@benkehoe Yep I can add that. As of right now there are dev instructions in order to be able to work on feature development, but this proposal will make it easier to get up and running in order to make contributions. |
@graywolf-at-work Yeah depending on the environment it is being built on, most of the time it is going to be spent building |
proposals/source-install.md
Outdated
| --with-install-type=[portable-exe|system-sandbox] | ||
| Specify type of AWS CLI installation. Options are: | ||
| "portable-exe", "system-sandbox" (default is | ||
| "frozen-exe") |
There was a problem hiding this comment.
If the options are portable-exe or system-sandbox, what is the default of frozen-exe?
There was a problem hiding this comment.
Oh that is a typo in the POC. The default should be system-sandbox. The frozen-exe option does not exist and I believe that was just the name that I originally gave portable-exe.
|
@rvandegrift all of these questions seem specific to the
There is no public documentation on this, but they have a visualization on how they all depend on each other in order to dictate build order:
It's fine to use openssl with the caveat about dynamic linking in the next reply
All of the C libraries can be built as shared, but the AWS Common Runtime (CRT) itself assumes (as in has never been tested otherwise) that the native part is a single embedded shared object. They don't see why it wouldn't work (installing all the C libraries as .sos and then the crt just having the tiny bindings in its embedded .so) but there are no guarantees on binary compatibility between releases of the C libraries - they're intended to be a point-in-time snapshot.
These support CI-time correctness proofs that do not need to be bundled I hope this helps! Let us know if you have any follow up questions. |
Yes, you're right- sorry about that, but many thanks for the response.
That's great, thanks!
As I've worked through more of the C SDK dependencies, I've run into some which don't actually support this. In particular, the docs for
Thanks, that's really good to know. Since they don't intend on ensuring binary compatibility, I think that undermines all of the value we'd get from shared builds. So I'll continue as if this wasn't an option.
Ah great - that'll substantially simplifies my first pass. |
|
I've just followed the steps to install in a docker image for Alpine 3.13.6 and it works great! My hat off to you sir @kyleknap Looking forward to when this is merged in. I did have one question. The installed version of the CLI from your poc-sdist branch was 2.2.1, how would I go about updating that to say 2.2.47? From what I can tell the DL links e.g. Thanks |
|
@kyleknap I completed a working set of Debian packages from this branch, plus it's requirements. The details, including packaging repos, are in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=966573 This was mostly straightforward after your initial help - thanks! I used a then-current snapshot of Two issues came up:
|
|
I'm interested in getting v2 into conda-forge: conda-forge/awscli-feedstock#828 I second @rvandegrift's comments about loosening the python version requirements though. When testing the v2 conda-forge package I changed some dependencies to allow newer versions of packages since, and I've been using the resulting installation day-to-day with no problems. |
@mr-davidc. I pushed up a refresh of the POC its version is 2.3.0. Yeah the links (e.g. |
Nice! Another thing to point out that I've already completed parts of the proposal to enable better support around PEP517 and begin the process of unblocking people. As of version 2.3.0 we have updated to embed our implementation of botocore + s3transfer and automatically build the auto-complete index as part of building the wheel. Moving forward, the main bulk of changes is around implementing the Autotools interface. I do not see any larger changes being made with regards to how it is built/installed as a Python package other than:
Since you are using the PEP517 option, you should be able to rely on building the Debian package off of the official
We would be open. At least for prompt-toolkit we should upgrade it to version 3. I opened a GitHub issue to track this: #6507. For how open we are with regards to version pinning, I'd like us to find an equilibrium where we do have some version range pin to make sure a dependency does not introduce a breaking change (outside of a major version) but still have some flexibility in which version of a library a user wants installed. In the past, we have gravitated to having a version ceiling to the latest minor version (assuming they are following Semver) that we update with each new vetted minor version. However, in the past, we have not been great at making timely updates to that ceiling and is something we need to get better at especially as we support installing from source.
It was originally intentional because we do not ship it in the official pre-built installers and wanted to clean up any pre-existing artifacts that was not removed (e.g. |
Awesome! That is great to hear.
Makes sense. Other than |
|
@kyleknap I increased the upper version bounds for:
|
|
Does building the AWS Cli v2 for Alpine according to this proposal will still enforce TLS 1.2+ as it's written in the AWS documentation? See https://docs.aws.amazon.com/cli/latest/userguide/cli-security-enforcing-tls.html |
| during the bootloading process of the frozen `aws` executable. | ||
|
|
||
| * Users want to be able to install the AWS CLI using the package manager | ||
| standard to their environment (e.g., `brew`, `yum`, `apt`) instead of having |
There was a problem hiding this comment.
Add chocolatey to the list?
There was a problem hiding this comment.
Chocolatey is already using our MSIs and stays up to date on v2: https://community.chocolatey.org/packages/awscli. I'm not planning to add it to this list.
proposals/source-install.md
Outdated
| to run Autotools generated files. For more information, see the | ||
| [Appendix on Windows usage](#windows). | ||
|
|
||
| * Python 3.8+ interpreter |
There was a problem hiding this comment.
How often are we planning to bump this? It would be good to set expectations of version support so we are not trapped supporting this version long past the community has moved on.
There was a problem hiding this comment.
Our expected version support is going to follow suit with what it is planned for v1, boto3/botocore: https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/. Basically, we will support the Python runtime up to 6 months after the PSF ends support. I'll update this bullet.
The proposal specifies an official mechanism to install the AWS CLI v2 from source along with an official source distibution artifact.
Specifically: * Added a bullet for community members wanting to install from source in order to test local patches that they can contribute back to the project. * Explicitly call out glibc in available Linux environments * Add support for DESTDIR * Add explicit wording around increasing the minimum required Python version * Fix documentation and typos in sample interfaces
The accepted state indicates that the initial proposal has been approved and official implementation is in-progress. As progress is made on the official implementation, amendments can be added to the accepted proposal.
kyleknap
force-pushed
the
source-proposal
branch
from
c425338 to
647b0e1
Compare
|
I moved the proposal to accepted and merged the PR. For those following the PR, the accepted state indicates that the initial proposal has been approved and progress will be started toward releasing an official implementation. As progress is made on the official implementation, amendments may be added to the accepted proposal. |
|
@kyleknap That's great news! Is there a single GitHub issue we can subscribe to track the implementation? |
Python3.7 was dropped in accordance with the accepted installing from source proposal: aws#6352. Furthermore, none of the installers currently run on Python3.7. Python3.8 will be the minimal Python version required once installing from source is officially supported. The codebase will be updated to support Python 3.10 in a subsequent update.
The proposal specifies an official mechanism to install the AWS CLI v2 from source along with an official source distribution artifact. For MarkDown rendered version of this proposal, view it here.
The AWS CLI development team is trying out a new process where proposals for new features are opened and reviewed as pull requests on GitHub, and this is the first proposal we are trying with this new process. This allows us to be more open on our thought process and solicit design feedback from community members. Currently, we do not recommend community members from opening pull requests with new proposals until we have fully solidified and documented the proposal process. However, we do recommend and encourage community engagement by:
Reading the proposal and providing feedback. If you are in general agreement with the proposal, please add the 👍 reaction to this top-level comment. If you have any questions or suggestions, feel free to leave a comment in the pull request.
Trying out the sample implementation (if provided) and leaving any feedback with regards to the approach. In general, the sample implementation is a proof of concept so there will be some refactoring and polishing prior to merging the implementation.
Sample implementation
I also have a sample implementation of this proposal hosted at the
poc-sdistbranch on my fork. While reviewing this proposal, I'd recommend checking out this branch and trying to install the AWS CLI v2 from source yourself. After checking out the branch, the simplest set of commands to get an up and running installation from source (without clobbering your existing v2 installation) is to run the following checked out branch:$ ./configure --prefix=/tmp/install --with-download-deps $ make $ make install $ /tmp/install/bin/aws --version # verify the CLI is installed correctlyIf you run into any functional issues with the proof of concept feel free to comment on this pull request.