idf.py fails when IDF_TOOLS_PATH is not $HOME/.espressif (IDFGH-11401)

[amrsoll]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.1 and v5.1.1

Operating System used.

Linux

How did you build your project?

Other (please specify in More Information)

If you are using Windows, please specify command line type.

None

What is the expected behavior?

idf.py

< displays help >

What is the actual behavior?

idf.py

ERROR: /home/mt/.espressif/espidf.constraints.v5.1.txt doesn't exist. Perhaps you've forgotten to run the install scripts. Please check the installation guide for more information.

ESP-IDF v5.1.1

Steps to reproduce.

mkdir esp
cd esp/
idf_release=v5.1.1
idf_path="$HOME/esp/esp-idf-$idf_release"
# Clone to folder $idf_path
git clone --recursive --depth 1 --branch "$idf_release" https://github.com/espressif/esp-idf.git "$idf_path"
cd "$idf_path"
IDF_TOOLS_PATH="$HOME/esp/esp-idf-install-$idf_release" ./install.sh
IDF_TOOLS_PATH="$HOME/esp/esp-idf-install-$idf_release" . ./export.sh

Build or installation Logs.

No errors during install.sh and . export.sh

More Information.

This is contrary to the documentation.

I would like to install them in different paths because the different versions of esp idf do not seem to play nicely when all installed in the ~/.espressif/ folder.

I installed it on ARM aarch64 ( RPi4 ) but that shouldn't matter I imagine

[igrr]

Question, are you passing the same IDF_TOOLS_PATH variable when running idf.py?

because the different versions of esp idf do not seem to play nicely when all installed in the ~/.espressif/ folder.

And, could you please give some hint, what kind of issues did you face? Each tool is installed into a separate versioned directory and there should normally be no interaction between them. (Of course, there may always be a bug, in which case we'd like to figure out the root cause and fix it.)

[amrsoll]

Question, are you passing the same IDF_TOOLS_PATH variable when running idf.py?

I had tried that as well, with the same result. However, if that should be the case, I presume it would have to be added to the documentation?

And, could you please give some hint, what kind of issues did you face?

If I remember correctly, I had faced issues when running esp-idf on the master branch. Usually in the form of build errors because of some inconsistency in dependencies. Of course it might be because I need to run setup.sh again in between pulls.

If it doesn't affect pinned versions, then I will default to using $HOME/.espressif/ for now, thank you 😄

[igrr]

I had tried that as well, with the same result.

I see now, I have missed when reading your initial post that your are running

IDF_TOOLS_PATH="$HOME/esp/esp-idf-install-$idf_release" . ./export.sh

We haven't tested the IDF_TOOLS_PATH overriding functionality the same way, instead we used export IDF_TOOLS_PATH=.... Indeed, right now it doesn't work because the variable gets set only in the script context, and the script doesn't re-export it IDF_TOOLS_PATH, making it available to child processes. Hence the Python tools launched from install/export.sh don't see the same variable value.

You can work around this on your side by doing export IDF_TOOLS_PATH="$HOME/esp/esp-idf-install-$idf_release" and then running install.sh, export.sh and idf.py with IDF_TOOLS_PATH already set.

However, if that should be the case, I presume it would have to be added to the documentation?

Yes, will update the docs section you have linked to with a mention of idf.py, in addition to the install and export scripts.

If I remember correctly, I had faced issues when running esp-idf on the master branch. Usually in the form of build errors because of some inconsistency in dependencies.

This could be because of a number of reasons, not necessarily due to a tools version conflict. For example, the submodules might not have been up-to-date. It's difficult to say more without seeing the specific error log, though. In general, having the tools for all the releases in the same $HOME/.espressif directory should work. Most developers working on ESP-IDF switch between master and different release branches on a daily basis, and usually it's sufficient to do git checkout <rev> && git submodule update && $IDF_PATH/install.sh and then source export.sh. If you run into some specific issue with this workflow, though, please do report it so that we can figure out the root cause!

[amrsoll]

Thank you for your time @igrr and for the clarification. I will write an issue in case I find a problem with the default install path :)