Need access to esptool sources/non-bin esptool in espressif platform

[seclorum]

Evaluating platform.io for my needs (MagicShifter3000) this evening, I ran into the problem that the onboarded esptool needs to be tweaked for my hardware (prototypes/etc.) Our non-platform.io build system currently uses our locally tweaked esptool.py. Is the 'best' way to overload this with the platformio_extra_script?

[seclorum]

 seclorum@hackerspaceshop-dev/:~/Documents/MagicShifter3000 {(platformio)}
$ ls -l ~/Documents/esptool/esptool.py 
 -rwxrwxr-x 1 seclorum seclorum 28766 Nov 25 15:44 /home/seclorum/Documents/esptool/esptool.py

(It works, we use it on other branches.)

$ cat platformio_extra_script.py

from SCons.Script import DefaultEnvironment

env = DefaultEnvironment()

LOCAL_UPLOADER="~/Documents/esptool/esptool.py"

env.Replace(
    LOCAL_UPLOADERFLAGS=[
        "-p", "$UPLOAD_PORT",
        "-b", "921600",
        "-ff", "40m", 
        "-fm", "dio",
        "write_flash", "0x100000",
    ],
    UPLOADER=LOCAL_UPLOADER,
    UPLOADCMD='$UPLOADER $LOCAL_UPLOADERFLAGS $SOURCE',
)

Output:

text       data     bss     dec     hex filename
268514     8229   34992  311735   4c1b7 .pioenvs/esp12e/firmware.elf
"~/Documents/esptool/esptool.py" -eo "/home/ibisum/.platformio/packages/framework-arduinoespressif/bootloaders/eboot/eboot.elf" -bo .pioenvs/esp12e/firmware.bin -bm dio -bf 80 -bz 4M -bs .text -bp 4096 -ec -eo .pioenvs/esp12e/firmware.elf -bs .irom0.text -bs .text -bs .data -bs .rodata -bc -ec
sh: 1: ~/Documents/esptool/esptool.py: not found
scons: *** [.pioenvs/esp12e/firmware.bin] Error 127
============================= [ ERROR ] Took 4.86 seconds =============================

Hmmm.

[ivankravets]

Evaluating platform.io for my needs

I hope you meant PlatformIO. We are not "Open Real-Time Advertising Platform"

Is the 'best' way to overload this with the platformio_extra_script?

You are moving in the right direction. The problem here is linked with quotes. PlatformIO escapes all paths by default to avoid problems with paths under Windows which contain spaces. See

• https://github.com/platformio/platformio/blob/develop/platformio/builder/scripts/espressif.py#L140
• https://github.com/platformio/platformio/blob/develop/platformio/builder/scripts/espressif.py#L122

PlatformIO uses esptool to make binary from elf. In your case uploading will work, but converting from elf -> bin will not. What is more, sh doesn't handle ~ and etc within quotes. And this answer is true:

• "~/Documents/esptool/esptool.py" == sh: 1: ~/Documents/esptool/esptool.py: not found
• ~/Documents/esptool/esptool.py == FOUND

Conclusion:

• LOCAL_UPLOADER="~/Documents/esptool/esptool.py" - specify as full path or use https://docs.python.org/2/library/os.path.html#os.path.expanduser
• Use your modified esptool only for uploading: UPLOADCMD='$LOCAL_UPLOADER $LOCAL_UPLOADERFLAGS $SOURCE' and don't override base UPLOADER.

You need to decide: if you want to support all platforms then need to specify full path to esptool because it will be placed into double quotes.

[ivankravets]

What is more, please don't call Python script in direct way. This works on Unix, but doesn't work on Windows if user linked Python with IDE. See correct using https://github.com/platformio/platformio/blob/develop/platformio/builder/scripts/espressif.py#L123

[seclorum]

Thanks for the help - truly a simple fix.. I'd say that UPLOADER is a bit of a misnomer, though? Because it doesn't just upload but also touches the build product.. I guess its a question of wider topic of toolchain staging.

What is more, please don't call Python script in direct way.

Roger that, in truth the python load is one negative in my eval of PlatformIO(*), but only for explicitly local reasons ..

(*- (Sorry about that, it is confusing..) )

[ivankravets]

I'd say that UPLOADER is a bit of a misnomer, though? Because it doesn't just upload but also touches the build product..

UPLOADER is used only in Espressif development platform to make binary from elf. In other (14) development platforms uploader has only the one purpose - upload firmware.

Roger that, in truth the python load is one negative in my eval of PlatformIO(*), but only for explicitly local reasons ..

Sorry, I don't understand what do you mean :( I said, that you can have multiple version of Python interpreters. You have a choice to run script with specified Python interpreter. If don't specify interpreter directly, then OS will use default python from the current environment. On the Windows the situation is more complicated.

Nevertheless, if user runs PlatformIO with Python X.X.X, need to use it to call other nested scripts.

[seclorum]

UPLOADER is used only in Espressif development platform to make binary from elf. In other (14) development platforms uploader has only the one purpose - upload firmware.

Understood. Anyway, its of no consequence ..

(What I meant was that I have my own reasons for eschewing the use of Python to manage things. :)

[ivankravets]

(What I meant was that I have my own reasons for eschewing the use of Python to manage things. :)

It's your decision. We don't force users to use PlatformIO or Python. We don't have anything from it. We just propose different tools/instruments within @platformio ecosystem and each developer decides what is interesting for him.

As for Python, I don't understand your negative position to this interpreter. Each language has own aim/purpose. I know a lot of developers with have negative position to Java or even to C.

Nevertheless, you should not examine PlatformIO from "in which language it is written". If it works on the most popular OS including ARM-based PC without requirements to system software and requires 10Mb of RAM and 700Kb for source code, then need take to account.

Happy New Year! If you have any questions, please feel free to reopen this issue.

[ivankravets]

Ok, thanks, I'll split these env vars

[ivankravets]

@seclorum $UPOADER and $OBJCOPY have been split up. See 917c0e3

[seclorum]

Thanks for the headsup @ivankravets ..