dune subst fails on missing opam file

[zapashcanon]

Hi,

Running dune subst will fail with Error: No <package>.opam files found. when no opam file is here.

[ghost]

Indeed. At the time dune subst was introduced, having <package>.opam files was mandatory. It is not anymore, so we should indeed update dune subst to cope with this case.

[zapashcanon]

Sorry, I didn't checked properly, it has nothing to do with the name of the package. So yes indeed, it shouldn't fail on missing opam file now, I updated the issue accordingly.

[ghost]

I added a reproduction case in #2931. If you are willing to spend a bit of time on this, I'd be happy to help you write a fix. Otherwse, one of us will fix it at some point.

[fangyi-zhou]

I'm encountering the same problem here.
I'm using dune to generate opam file, but received an error when running dune subst. This is rather undesirable since the generated file uses dune subst on the 1st step of the build.

If anyone is happy to give some code pointers, I'm happy to submit a pull request

[ghost]

@fangyi-zhou a PR would be welcome!

If you look into src/dune/watermarks.ml, there is a function make_watermark_map. This function is responsible for reading the opam file and assigning values to variables such as NAME, VERSION, ... based on the contents of the opam file.

In order to make dune subst when no opam file is present, you need to modify this function to gather the information from the dune-project file instead. To do that, add an argument ~dune_project of type Dune_project.t option to this function. It has to be an option for backward compatibility reason. This value is readily available at the only call site of this function.

If this value is Some p, then you should lookup the package name (argument name of make_watermark_map) in the map Dune_project.packages p. Using both p and the Package.t value looked up in the map, you should have enough to produce the string map at the end of this function.

If dune_project is None, we should keep the current code that reads the opam file for backward compatibility.

[fangyi-zhou]

@diml Thanks for the guideline.
I'm wondering if it is necessary to check the consistency of opam file and dune-project file? If that's the case, which one should take precedence?

I'll try to submit a pull request this week to implement this feature

[ghost]

We shouldn't need to; if there is no package stanza in the dune-project file, dune will look at opam files already. If there are both package stanzas and opam files, dune will make sure that both define the same set of packages. So if there is a dune-project file, then all the information we need is readily available in Dune_project.packages.

One case we could provide special support for is the following:

• there are (package ...) stanzas in the dune-project file, but not maintainers, license, ... fields
• there is no (generate_opam_files true) field, so dune doesn't control the opam files
• there are opam files with the maintainers, `license, ... fields

In this case, Dune_project.packages p would be missing the maintainers, license, etc... We could handle this case by merging both source of information, but that doesn't seem necessary to me. This is a niche case and moving forward we are headed towards no longer having opam files in repositories anyway.

[fangyi-zhou]

@diml I made some preliminary changes in #2955, could you have a look?

[ghost]

Sure!

[k4rtik]

Is there a workaround for this issue? Such as downgrade to a certain version of dune?

I am stuck at dune subst step for a package I am trying to create.

[ghost]

Downgrading won't help here. The only workaround until #2955 is merged is to add an opam file, or ask dune to generate them.

[k4rtik]

When I ask dune to generate them using dune build @install, I still see the following error:

$ ls
_build  dune-project  lib
$ dune build @install
$ ls
_build  dune-project  lib  <pkg>-<name>.opam
$ dune-release distrib
[-] Building source archive
Error: No <package>.opam files found.
dune-release: [ERROR] run ['dune' 'subst']: exited with 1

(redacted package name, but it is there and can be read by dune-release lint for example)

[fangyi-zhou]

I committed the generated opam file from dune project in the repo as a workaround Fangyi — sent from a mobile device, apologies for typos

…

On 15 Jan 2020, at 16:38, Kartik Singhal ***@***.***> wrote:  When I ask dune to generate them using dune build @install, I still see the following error: $ ls _build dune-project lib $ dune build @install $ ls _build dune-project lib <pkg>-<name>.opam $ dune-release distrib [-] Building source archive Error: No <package>.opam files found. dune-release: [ERROR] run ['dune' 'subst']: exited with 1 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ghost]

Fixed by #2955