Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce snapshots to distribute advisories #179

Merged
merged 23 commits into from
Jul 18, 2024

Conversation

blackheaven
Copy link
Collaborator

@blackheaven blackheaven commented Apr 1, 2024


hsec-tools

  • Previous advisories are still valid

TBD

@blackheaven
Copy link
Collaborator Author

@frasertweedale I think the failure is expected as the file is oob (not tracked in Git history)

@blackheaven blackheaven force-pushed the snapshots/introduction branch 2 times, most recently from 2feae57 to 24bd053 Compare April 1, 2024 17:15
Copy link
Contributor

@MangoIV MangoIV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider being more careful in pulling in new dependencies, either depends on a whole load of things, admittedly the dependencies are quite low in the dependency tree, usually, but I don't think it's justified here.

A few more informative comments/ docstrings wouldn't hurt either, to increase readability.

I don't think switching the toml parsing library is a good idea at all. e.g. because it doesn't even support toml 1.0.0 or it actually does wrong parses in our case (see my review).

I also think treewide formatting changes in files that get heavily changed should be done in a separate PR.

code/hsec-tools/hsec-tools.cabal Outdated Show resolved Hide resolved
Comment on lines 21 to 22
resultE <- try $ get $ repoUrl </> "commits" </> branch </> "advisories.atom"
resultE <- try $ get $ mkUrl [repoUrl, "commits", branch, "advisories.atom"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filepath is OS-dependant, here we are dealing with URL, I did not want to pull a new library just for that.

Copy link
Contributor

@MangoIV MangoIV Apr 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, this is an URL, sure; I think with these error prone things, it would actually be useful to pull a new library, in contrast to extra or either which provide trivial combinators

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a proper library, do you have any suggestions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hasufell: I believe you had to deal with URLs/URIs in the recent past. What would you recommend?

code/hsec-tools/src/Security/Advisories/Format.hs Outdated Show resolved Hide resolved
code/hsec-tools/src/Security/Advisories/Format.hs Outdated Show resolved Hide resolved
code/hsec-tools/test/golden/MISSING_AFFECTED.md.golden Outdated Show resolved Hide resolved
code/hsec-sync/src/Security/Advisories/Sync.hs Outdated Show resolved Hide resolved
@blackheaven blackheaven force-pushed the snapshots/introduction branch 4 times, most recently from 534c3ce to 5f0cef1 Compare April 3, 2024 22:27
@blackheaven
Copy link
Collaborator Author

@frasertweedale FYI, I have revert back to toml-parser and properly tested FrontMatter rendering

@blackheaven
Copy link
Collaborator Author

@frasertweedale can you review it again, so I can move forward (see items in the description)

@blackheaven blackheaven force-pushed the snapshots/introduction branch 2 times, most recently from 8a24576 to 28fd3cc Compare June 10, 2024 19:58
@blackheaven
Copy link
Collaborator Author

I won't lie, it was one of the most painful rebase I had to do.

@frasertweedale good luck :)

@frasertweedale
Copy link
Collaborator

Seven hour train ride on Wednesday. Should be enough time, I hope :)

@frasertweedale
Copy link
Collaborator

Why so much reformatting? 😭

@blackheaven did you reformat using some formatter? If so, it would be better to tackle that as the first commit, and keep the signal-to-noise ratio high for the subsequent commits.

Also, should we consolidate around a particular formatting tool? (I'm not really a fan, but I'm even more not a fan of what happened in e72abb5 ^_^)

@Kleidukos
Copy link
Member

(I would advise fourmolu, as it is diff-friendly and reduces the noise of each commit)

@frasertweedale
Copy link
Collaborator

@blackheaven some of the error formatting undoes Mango's Exception instance work. I can fix it up if you like.

@blackheaven
Copy link
Collaborator Author

Actually it was a mistake, @TristanCacqueray enforced fourmolu, but hls does not take le configuration file in account when formatting.

I'm afraid that fixing reformatting during rebase might be a nightmare.

@frasertweedale either you have some time to revert back Mango's work, or highlight where I have overwritten them.

@@ -51,6 +58,7 @@ library
, commonmark-pandoc >=0.2 && <0.3
, containers >=0.6 && <0.7
, cvss >= 0.1 && < 0.2
, data-default >=0.7 && <0.8
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Text.Pandoc.Options re-exports def (ref). Perhaps we can import from there and avoid this direct dependency?

@frasertweedale
Copy link
Collaborator

@frasertweedale either you have some time to revert back Mango's work, or highlight where I have overwritten them.

Yes, I'll make the changes and then we can compare and review. Expect it tomorrow. Apart from that, there was just one comment so far. I've read most of the diffs and the new code LGTM at a high level. Still need to test, also.

@TristanCacqueray TristanCacqueray changed the title Introduce snapshots to distribute advisories (#170) Introduce snapshots to distribute advisories Jun 12, 2024
@frasertweedale
Copy link
Collaborator

@frasertweedale either you have some time to revert back Mango's work, or highlight where I have overwritten them.

Yes, I'll make the changes and then we can compare and review. Expect it tomorrow. Apart from that, there was just one comment so far. I've read most of the diffs and the new code LGTM at a high level. Still need to test, also.

I see that later commits in the series put everything back the way it was. The commit history is a winding road but all is well at the destination.

I will do some testing. Unless I find bugs / regressions, I will merge it :)

@blackheaven
Copy link
Collaborator Author

I have done the changes (fixings root directory and using etag), let me know what you think.

@frasertweedale
Copy link
Collaborator

@blackheaven thanks mate. I fly home to Australia tomorrow so I will have time to review it!

@blackheaven
Copy link
Collaborator Author

Have a nice flight back!

@frasertweedale
Copy link
Collaborator

uh @blackheaven, did you push your changes? I don't see them 😕

Comment on lines +140 to +141
maybe
(Right p)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This causes the top-level directory/ies and other top-level paths to be created. It does work for our use case, so I'll merge it as is. But I will probably circle back later to tidy it up.

@frasertweedale
Copy link
Collaborator

I've reviewed and tested it. Works as expected. There are some improvements we can make but no show-stoppers. Sorry for the delay and thanks for your patience!

I pushed one new commit that adds more detail to the HTTP error messages. I'll merge when green.

@frasertweedale
Copy link
Collaborator

CI is busted. Dealing with it in PR #220.

@frasertweedale frasertweedale merged commit d71b4c8 into haskell:main Jul 18, 2024
8 checks passed
@frasertweedale
Copy link
Collaborator

Finally merged. Thanks for implementing this feature, @blackheaven!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants