Feature/384-generate-checksums-for-component-files

[reglim]

Fixes #384

This is basically a rebase of #473, which was really out of date, so refer back to that PR for more information.

Requirements

All new code should be covered with tests, documentation should be updated. CI should pass.

Description of the Change

• Adds a skeleton (skel) folder at .aptly/skel that can store files you want to be in the repository
• On publish, aptly will look in that folder and write the files to the repository, and release files
• I did not add any API or CLI tags to change the skel directory, as that would be bad security
• All files will be copied AS IS. No compressing or signing

The skel folder mirrors the public folder, so on publish .aptly/skel/dists/maverick/main/dep11/Components-i386.yml.gz will create .aptly/public/dists/maverick/main/dep11/Components-i386.yml.gz or equivalent.

Checklist

• unit-test added (if change is algorithm)
• functional test added/updated (if change is functional)
• man page updated (if applicable)
• bash completion updated (if applicable)
• documentation updated
• author name in AUTHORS

[codecov]

Codecov Report

Attention: Patch coverage is 70.83333% with 21 lines in your changes missing coverage. Please review.

Project coverage is 74.78%. Comparing base (8d09c20) to head (632c8e7).
Report is 39 commits behind head on master.

Files	Patch %	Lines
deb/publish.go	55.31%	15 Missing and 6 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1186      +/-   ##
==========================================
- Coverage   74.79%   74.78%   -0.01%     
==========================================
  Files         144      144              
  Lines       16256    16314      +58     
==========================================
+ Hits        12158    12201      +43     
- Misses       3156     3167      +11     
- Partials      942      946       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[randombenj]

@jhonny-oliveira would you mind testing the changes before we merge?

[jhonny-oliveira]

Given this:

xd@dep11:~/.aptly/public/appstream$ tree -d -I pool -I appstream ~/.aptly/addon
/home/xd/.aptly/addon
└── ubuntu
    └── dists
        └── lunar
            ├── apps
            │   └── dep11 -> ../../../../../public/appstream/ubuntu/export/data/lunar/apps
            └── play
                └── dep11 -> ../../../../../public/appstream/ubuntu/export/data/lunar/play

After running:

aptly publish snapshot -component=apps,play xtradeb_apps_20230526 xtradeb_play_20230526 ubuntu

I got this:

xd@dep11:~/.aptly/public/appstream$ tree -d -I pool -I appstream ~/.aptly/public/
/home/xd/.aptly/public/
└── ubuntu
    └── dists
        └── lunar
            ├── apps
            │   └── binary-amd64
            └── play
                └── binary-amd64

So, the dep11 directory is still missing (target should be similar to: http://nl.archive.ubuntu.com/ubuntu/dists/lunar/main/). Any ideas? Did I miss something?

I'm also having trouble building the latest code with the previous debian recipe: https://launchpad.net/~xtradeb/+archive/ubuntu/apps/+sourcefiles/aptly/1.5.0-1~xtradeb1/aptly_1.5.0-1~xtradeb1.debian.tar.xz. But, I will ask you about it in a different thread.

[jhonny-oliveira]

Any update on this?

[reglim]

Not really. I hope @btkostner can take a look at this, since I don't actually understand the underlying issue.

[btkostner]

@jhonny-oliveira try using “.aptly/skel” instead of “.aptly/addon”.

PR looks good to me.

[jhonny-oliveira]

Dear @btkostner ,

I have tried your recommendation without success. Is there anyway we can troubleshoot this in a more interactive way? Discord or any other IM?

Thank you!

[r4co0n]

Is there some merit in always looking for the relevant publication's skeleton files in a path aquired by joining skelDir, p.Prefix, "dists", p.Distribution, component?

I would prefer if this was configurable, even if I won't ever touch it. It could be an option you pass to publish repo|snapshot|switch, or am I mistaken?