Amazonka 2.0 #718
Comments
|
Status update: We won't hit the the planned release date, and with Christmas nearly upon us that's going to slow everything down more. Mid-January seems more realistic. There are things I want to test by porting my work codebase to |
|
Hi folks, how are things looking for a 2.0 release? There's no rush from my side, I'm just looking to plan when we can move our work codebase to |
|
About time for a status update, thanks for a poke:
I would love to say April for final 2.0 release, but we might drag out longer than that. Any chance you can pitch in a PR for any of the milestone items? P.S.: Glad to hear you're using 2.0 with |
Unfortunately, I'm not sure we actually use |
I recommend to move some of the open milestones into post 2.0 releases. Some, to me: Appear not in the category of "needs to be done in a breaking change so should not delay 2.0". These would IMO not cause a breaking change post 2.0 once they are addressed, to me they look like pending additions which could go into amazonka-2.0.x, or amazonka-2.1:
These milestone items, to me look more like bugfixes which have a minor breaking change character, but would from my (downstream perspective) not be counted as a breaking change, despite the fact I may have to adjust some imports: Consequence of not considering these as blocking may be a bit earlier release, which allows downstream libraries to adapt much earlier, increasing the total value provided by amazonka-2.0. This is my 2¢, not in any case authoritative. Just another amazonka downstream consumers perspective. |
|
Yeah, perhaps. It's something I'll look at once I put a chunk of time into things and see what sort of momentum I can achieve. |
|
I just thought I'd report that I've spent some time porting one of the services at work to use 2.0.0rc1. So far it works well. I'm particularly happy about the support for A few observations:
All in all it was rather straight forward to move from 1.6 to 2.0. Very nice work! Hopefully you'll find the time to make the official release soon. |
|
Thanks for the feedback. The lenses are in their own module because Line 53 in f73a957 If you don't like that, you can also use |
|
I have also patched a codebase from the now-deleted
Any chance a new estimate could be made? Because it seems like it is certain to drag on longer now. I ask because Amazonka is keeping us on Aeson 1.5, and I want to know how much effort I should put into maybe getting a new Aeson 1.5 release. There is no rush of course, I thank you for you work. Just thought it would be useful with a new estimate if it is easy to do. |
|
@magthe If you're using the generated (legacy) lenses, the usage for your example is intended to be: import Amazonka.SQS
( newDeleteMessageBatch,
newDeleteMessageBatchRequestEntry,
newReceiveMessage,
)
import Amazonka.SQS.Lens
( deleteMessageBatch_entries,
receiveMessageResponse_httpStatus,
receiveMessageResponse_messages,
receiveMessage_maxNumberOfMessages,
message_body,
message_receiptHandle
) |
|
April is no longer realistic - you are right. The work on tidying up service definitions and adding the new AWS services is stalled because we currently cannot regenerate services, and I'd like @brendanhay to review #761 before I charge off and merge it. Once that's unblocked, I'll see if I can get a little momentum up. However, my life has also become significantly busier of late, which makes evening/weekend contributions more difficult. Work does let me do some Amazonka work on company time, but it's usually in the two-week blocks between six-week projects, and the next one of those isn't for another ~four weeks. |
|
@brendanhay: aha, I used the links in the docs to guide me to find the functions/lenses I needed. I guess that's not the way, even for legacy lenses then. I didn't find any mention of Your comments make me think that maybe this is something that is difficult for users to just discover on their own. Especially when having used 1.6 somewhat casually, like I have. Well, at least it was for me 😁 Not even the first example anyone is likely to bump into, the example in I hope I'm not coming across as confrontational. It's not my intention. Just trying to express how I approached porting to 2.0 as someone who has a complete lack of context beyond what I found in the documentation. |
|
This is helpful feedback, @magthe, thanks! The documentation for much of the work that has come along with 2.0 still needs improvement, and feedback like this lets us know what we should focus on. |
|
We've ported a fair bit of 1.6.1 code to 2.0.0 and for the modules we are using amazonka is running fine. Your work has been great, and while I know you're wanting everything to be perfect before you upload 2.0.0 to Hackage we'd encourage you to iterate fixes and documentation as you go after the milestone release. Nothing wrong with lots of micro releases afterwards. You've done a really amazing job. |
|
I'm looking forward to this release as well! |
|
So am I. This week's objective is to regenerate bindings to existing services using latest botocore, and from there start generating bindings for the services we don't have yet. Unfortunately, four services generate code that doesn't compile and the reasons all appear to be different. So I'll dig into that tomorrow. |
|
We've been using the following libraries in production w/ no issue for the last month or so: - git: https://github.com/brendanhay/amazonka/
commit: 0ccede621e56fb6f240e4850e205cde82d0e4a4b
subdirs:
- lib/amazonka
- lib/amazonka-core
- lib/services/amazonka-sso
- lib/services/amazonka-sts
- lib/services/amazonka-cloudwatch
- lib/services/amazonka-ecs
- lib/services/amazonka-s3
- lib/services/amazonka-ses
- lib/services/amazonka-sns
- lib/services/amazonka-sqs
- lib/services/amazonka-ssm |
|
Thanks for the updates everyone. #806 is the PR where all current services are regenerated against latest botocore - I would appreciate it if people can test against it and report back. #807 Adds the configuration for a bunch of new services. After this goes through, I'll generate them. That will get us up to botocore minus five or six services that don't compile (usually because of circular module imports). |
|
Had some down time so bumped our codebase from We only use: It went smoothly, I only had to fix one thing which was removing dealing with a old: https://github.com/brendanhay/amazonka/blob/51408b16b4/lib/services/amazonka-sts/gen/Amazonka/STS/AssumeRole.hs#L925 then it all compiled and all our tests passed. |
|
Does anyone have a recipe for how I might build nix-based projects until this is in hackage? This fails with recent symbols not being present in nix builds and is making it a bit difficult to use my projects across multiple machines. (alternatively, just releasing it or even releasing it as amazonka2 would be awesome) |
|
We've been using this at work with |
|
Can you give an example of how you do this? I don't think there's much of a point in filing a bug if I'm just doing it entirely wrong. I'm using stack (with nix) and have the following in my stack.yaml: extra-deps:
- github: brendanhay/amazonka
commit: e64638ae16dc6d549d6e7c21262245e29761ede7
subdirs:
- lib/amazonka
- lib/amazonka-core
- lib/services/amazonka-s3
- lib/services/amazonka-sts
- lib/services/amazonka-sqs
- lib/services/amazonka-lambdanix complains that it can't find that revision (same with current |
|
It would help keep threads of discussion from becoming confused with each other (this issue is a bit of a mess already), and there have been similar requests for help in issues before: #750 I need to know how you are using nix (haskell.nix, nixpkgs, some other mechanism?) and a minimal reproduction repo would help a lot, as that snippet looks correct. I haven't used stack for years, as I've found it caused me more problems than it solved, so I cannot infer what exactly you are doing. Please help me so I can try and help you. |
|
Understood. I filed #815, but had mentioned it here specifically because closing this bug would be the least-effort way to fix this (from my side, anyway -- obviously if it were easy on your side, the bug wouldn't be open). |
|
I'd love for it to be on Hackage too, but there are still quite a few blockers. |
The specifics of how you use a generic lens library are up to you (which is why only some of the examples use them), but there's actually a recommendation in each package description: As well as a recommendation in each request's haddock: I think this is a funny edge case where it's harder for experienced users to notice things. Once we're back on Hackage I think it will be a lot more obvious, as these recommendations will be rendered in a much more visible way. (FWIW, I didn't actually notice these recommendations until I was hacking on the generator and actually had to change the templates.) |
|
About time I gave another update. Amazonka now has bindings to every service in botocore except for two ( In a couple of weeks I should get another couple of blocks of work-sponsored time to work on things; the goal will be to tick off a bunch of the smaller tasks that have been hanging around in the milestone for some time. |
|
Thanks for the update @endgame! Sorry if you've mentioned this elsewhere but is there a list of what specifically is blocking a Hackage release? Do you think those "couple of blocks" you'll get in a couple of weeks could get the project to a release, or is there still more? I'm interested in pitching in if I can, and would love some kind of "blocks 2.0 on Hackage" label to work against. Or is it safe to say any Issue that doesn't have the "post 2.0" label is that list? |
|
There are a lot of people using this repo, and it's being actively worked on. Nobody is doing crazy things like merging half-built stuff to I know it's been a long road, but please bear with us, we're making steady progress. There are a few more annoying chore-type changes that really need to go into 2.0 so we don't cause gratuitous breaking with 2.1. I have the rest of this week as work-sponsored time to dedicate to Amazonka, then a couple of weeks' leave, then another couple of weeks of work-sponsored Amazonka time, so we're getting there. |
|
@endgame Thank you for the quick answer, and the great and often thankless work ❤️ |
|
@NorfairKing, @endgame One reasons I thought I wanted to have a release on hackage was the download time of this repo when using a git source. And than I found this neat trick (thanks @dkubb) which reduces the initial download time significantly: From here hackage vs git is less of an issue for me than it was. Just use the commit hash you are interested in, and on large repos such as |
|
This download time problem is a missing feature in stack. |
yes it does not pass
same with passing the archive URL as I did. |
There was a bug pre Stack v.2.7.5 where it'd clone multiple times: see changelog After this fix, it only clones a repo once, even if you have multiple subdirs. It's still gonna download the whole history though, so there is still a place for using the tarball directly (as @mbj shows) or using the Were you referring to this bug, @NorfairKing ? |
I think the |
|
@mbj I don't think that's true, if I am reading the BuildPlanSpec test right. And it also doesn't explain how Amazonka is sped up in Stack 2.7.3 by switching to the |
|
@ysangkok you are right, I was to infected by other package managers I use in my assumption! Yes its equivalent and my explicit URL is just a more verbose encoding of the same semantics. |
|
Is there anything I can do to help get a release on Hackage? We've been using various workarounds (Git source, vendored tarballs, etc) at work. We can continue to use those workarounds, but it would be much nicer to simply use a version from Hackage. |
|
@tfausak I'd love to get this on Hackage too, but I found a pretty serious blocker this week: in the current repository state, I cannot regenerate all of amazonka any more. I've tried changing the botocore pin and pinning This is a blocker because I really want to ship 2.0 as a single Hackage package containing multiple public named libraries. I want to do this because we are nearly up to 350 distinct AWS services, and about half of those are unknown to Hackage. I really don't want to camp another 100+ new names on Hackage with 2.0. So that means I'll probably have to bring the work in #888 into the 2.0 milestone, so that a) we can keep up with botocore at all and b) so that I can rework the packaging. Something that would be extremely valuable: making sure whatever build tool you use is ready for multiple public libraries, whether that's |
|
This is my unsolicited opinion, so take it for what it's worth (which is nothing): Please just make a release. The last release was more than four years ago. I've been following this issue and it feels like there's always "one more thing" to do before the release can go out. Package names are cheap, and multiple public libraries are only barely supported by some tools. I'd massively prefer hundreds of Thanks for the work that you (and other collaborators) do on this repo. I hugely appreciate it! As I said before, I'm able to use Amazonka regardless, so this basically boils down to a quality of life improvement for me. |
|
I'd rather ship multiple libraries (edit: packages) despite the presentational issues on Hackage. As mentioned before the build tool support for multiple public libs is quite shite. I also think regenerating to produce new service bindings (as opposed to updating existing ones) should halt until the shape/pointer related code is improved, rather than generating and being blocked on a new service not building that noone as-of-yet depends on. |
|
@endgame, @brendanhay is there a place where I can sponsor your work? EDIT: I'm, blind, just found the sponsor button of @brendanhay, @endgame where is yours? |
|
@tfausak I sympathise, believe me. We have to do a bunch of hacks at work to make git packages work well with our tooling, and I'd rather not have to do them. Multiple public libs came from a 2018 GSoC project, also co-incidentally about four years ago. It's not just the presentation issues on Hackage, but also that it adds more to the index for every single user to download and complicates maintenance (I still have to go through and add those bounds to the 0.0.0 packages — #915). Some pretty fundamental fixes to multiple public libs came through in @mbj Thanks for the thought, but at the moment work is semi-sponsoring me to do this, so taking sponsorship in that way probably wouldn't be appropriate. I get to do some amount of maintenance on amazonka as part of my normal duties, and a few times a year I can block out larger chunks of time (with their blessing and project manager support) to work on meatier stuff. Haven't had the chance to do that for a while, which is why things have stalled. Soon™. @brendanhay : I was afraid that the generator was actually broken for the current botocore pin despite yesterday's So, given that we have a working generator for this pin and have service bindings for almost everything up to a point earlier this year, here is my proposal:
@brendanhay what do you say to this five-step plan? Spamming hundreds of new packages onto Hackage makes me sad, because I think the developer experience of adding |
Me too, but: Most people only use a small subset of the AWS services API, and from a pure user perspective: I'll not really notice the difference. Adjacent, I maintain stratosphere now that faces the same issue but not as massive (Cloudformation is not supported by all AWS services). And still plan to go with the package spam for that exact reason. The amount of inconvenience is "to small" to invest into subpackages. |
@endgame The five-step plan sounds good to me. |
|
Regenerated (reformatted) all services, pushed, merged, then rebased the @mbj The other awkward situation we have here, which might not affect stratosphere so much: if we make a fundamental update to @tfausak Once we cut RC2, a big callout in HWN would be extremely valuable to get eyes onto it before we go to Hackage. |
|
@endgame Thanks for that additional color, makes sense. |
|
Thanks @endgame! That all sounds great to me. I'd be happy to include Amazonka's RC2 in Haskell Weekly. |
|
An aside, re: having a single package with public libraries:
It does add to the Hackage index which everyone has to download, while a unified package with sublibraries increases the download size for amazonka users only, and they will still only have to compile a subset of the code. |
|
@brendanhay I've merged the I think a three-to-four-week final RC period is probably good here. I have some stuff on at the moment which will make a large demand of my time with little warning, but I can't say exactly when that will be. |
|
We will definitely want to pick up the new version of |
|
Great - I'll write up the announcement and do a tag/release shortly, providing StarLink decides to grace me with coverage today/tomorrow. |
|
Please see the 2.0.0-rc2 release and related discussion. |
Per the release announcement, Amazonka 2.0.0-rc1 is available for testing. The release announcement also explains how to change your
cabal.projectorstack.yamlfile to test Amazonka in your own projects.Because of the magnitude of the accumulated changes, we are now in a stabilisation period before we release to Hackage. During this period, we'd greatly appreciate people testing
amazonkaagainst their projects, and reporting issues against the 2.0 milestone. We are aiming for a four-week stabilisation period (which would end on 2021-12-16), but may shorten or lengthen this depending on the volume of issues raised and any impact from the holiday season.The text was updated successfully, but these errors were encountered: