Support SHA512 for buildpack dependencies

[mgorsk1]

Describe the Enhancement

Make it possible to define checksum = "sha512:..." in the buildpack.toml metadata.dependencies (instead of just {sha256,md5})

Possible Solution

Motivation

Some dependency providers (in our case Apache Spark collected from archive FTP) publish sha512 only, in such case it's not a trival task to use it in buidlpacks.

[ryanmoran]

There should already be support for sha512. In fact, the dotnet-core-sdk buildpack uses sha512 today: https://github.com/paketo-buildpacks/dotnet-core-sdk/blob/e6bea630bfb5c7e6e91f26088ed93b6ed5cdd5d2/buildpack.toml#L22.

I am not sure that we do support md5 though, if that is what you were asking for. Can you provide some more details about any error messages or failures you are seeing?

[mgorsk1]

Hey @ryanmoran thanks for replying. Firstly, I assumed sha512 is not supported (and md5 is) after going through these unit tests: https://github.com/paketo-buildpacks/packit/blob/v2/cargo/checksum_test.go

Secondly, my issue is that when using Cheksum syntax 'sha512:...' (same way you shared) my logs say:

Spark Buildpack 0.0.1
  
  Creating package in ./packages/spark
  Caching Spark 3.3.1
  Warning: Dependency has no SHA256. Skipping cache.
    Downloading from https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz
    Adding bin/build
    Adding bin/detect
    Adding buildpack.toml
    Adding dependencies/.toml
unable to write file /home/vsts/carton-cache.toml to packages/spark/dependencies/.toml
unable to stat file /home/vsts/carton-cache.toml

so SHA512 doesn't seem to be used for caching (nor for validating if the collected package is safe).

[robdimsdale]

I think the issue is that the buildpack in question isn't using packit - it's using libpak. Searching for the line Dependency has no SHA256. Skipping cache returns the following section of libpak:

	if dependency.SHA256 == "" {
		d.Logger.Headerf("%s Dependency has no SHA256. Skipping cache.",
			color.New(color.FgYellow, color.Bold).Sprint("Warning:"))


		d.Logger.Bodyf("%s from %s", color.YellowString("Downloading"), uri)
		artifact = filepath.Join(d.DownloadPath, filepath.Base(uri))
		if err := d.download(uri, artifact, mods...); err != nil {
			return nil, fmt.Errorf("unable to download %s\n%w", uri, err)
		}


		return os.Open(artifact)
	}

So I think this issue should be moved to the libpak repo.

[mgorsk1]

thanks @robdimsdale you are probably right, nice catch!

[robdimsdale]

@dmikusa do you have permissions to transfer this issue to https://github.com/paketo-buildpacks/libpak ?

[dmikusa]

Transferred. I haven't looked at what it will take to implement this for libpak. My guess is that we'll do this as part of v2 (hopefully libcnb v2 is to be released soon, so then we can get working on libpack v2). That said, if someone wants to look into this and it can be done in a non-breaking way, we could add it to v1 as well.