collect2.exe: fatal error: CreateProcess: No such file or directory #231
Comments
|
Hi @TD-er ! On Windows the maximum length of the string (including all arguments) passed to |
|
Hmm that sounds plausible. I first made a normal failing build and then a verbose build as it would then just fail on the linker part, right? This is less than 6k, so I am not entirely sure it will be the true error. If you like, I can also make a full verbose build, but I guess the log will not entirely fit in the terminal window. |
|
The linker also expands |
|
Just made a new test build on this laptop to check. For example: |
|
Right now, quite a lot of function still have a global scope (left over from converting from .ino files) |
|
Does it make sense to move some common functionality to standalone libraries? PlatformIO will be able to link them as static archives so it should help with the long linker command. |
|
I can try, to see if this file will be smaller. |
|
Thanks to the |
|
Also if they are part of a library? |
|
Doesn't matter, the linker is smart enough to throw out unnecessary symbols even from libraries. |
|
Good to know. |
|
I now was looking again into this issue and thus also did look at the Just a small section of this: (It is all in one line, but for readability, I placed each on a single line) So I don't think it will help here to move parts to their own library, or am I missing something here? |
|
Just as an idea... Could this file be used with some relative path? |
|
I just found a bit more room by going over the libraries mentioned in the Edit: |
|
OK, dug a bit deeper into what is happening here and I found the To explain this, you need to know the structure of my project.
These plugins and controllers are now coded in .ino files, but some already use objects coded in .cpp/.h files The only thing I can think of right now, will be a true nightmare to maintain as it means every plugin and every controller will end up in a separate library. So I really hope there is a better solution for this. Edit: It is something I really don't like to do. |
Could you please explain in a few words what exactly is hard to maintain it that case? In my opinion, isolated modules (the granularity of that modules may vary as your project evolves) should be much easier to develop and implicitly lead to better software design. The main issue here is that all project files are located in the |
Well it depends on what PlatformIO may find acceptable as a library. Right now I am still in the process of refining the interface between the core and the plugins, so the interface between the plugins and core is still a bit in flux. I wonder how building it in Arduino IDE will be affected by such changes. |
You can group your libs as you like and keep them in your main repository, that's what the
Arduino IDE (as any other app) is exposed to that limitation since it's imposed by the OS. |
|
I do understand why it may fail, if it indeed tries to generate a similar line. |
If I'm not mistaken, Arduino IDE links only framework files and libraries as static archives. Project source files are linked as objects. |
|
OK, that would make sense. |
|
I was just thinking about this a bit more and I realized this (which I hope isn't correct). Imagine I will create a library of every single plugin and controller (e.g. just about any .ino file left in my project) I just got the feeling it is heading in a completely wrong direction. |
|
Assuming I grasp the problem, perhaps Platformio's linker could copy all the files to a temporary root folder, then link the files there. Afterwords, cleanup by deleting the temp folder. It seems to me that would reduce the command line length in a typical build environment. And this alternate linking method could be an option enabled by an entry in platformio.ini, where the drive root folder could be specified.
|
|
Just to note, this might also affect lib_deps=... entries, which was noticed some time ago here: Re: above, it might be a good idea to launch the linker with .pio/build/<env> as pwd? If this is the main problem here. (I also believe this could be migrated to the platformio-core issue tracker?) |
|
Yep it seems it is indeed a PIO core issue. And stripping the shared path of all .cpp.o file entries might save roughly 12k in my project's longcmd file. I will have a look at what was discussed in the topics you linked. |
|
@TD-er i reorganized the libs in Tasmota in subfolders. Reason was not a compile fail. |
|
@Jason2866 This means I have to transfer my build defines to the Python layer, which may not be a bad thing. |
|
@TD-er Had the idea with dynamic disabling libs too. After working on it i trashed the idea. |
|
Yep as soon as I found the best fix for ESPEasy, I will let you know. Still it would also be nice if we didn't hit this limit and it is fixed in PlatformIO :) |
|
Hi all, Could you show an example of |
|
Right now we do hit a limit in the On the other hand, if we could somehow determine what library is used, it does not have to be compiled (thus saving time and an entry in the So there either needs to be a practical way to populate the Edit: |
|
"Golden solution" |
|
In this case, you need to do the next:
lm = LibraryPackageManager(
env.subst(os.path.join("$PROJECT_LIBDEPS_DIR", "$PIOENV"))
)
for spec in custom_specs:
if lm.get_package(spec):
lm.install(spec)
|
|
In Tasmota we do NOT use |
|
If you use |
|
Idk. Without the libs are missing. With it it compiles every lib found in folder if used in code or not |
|
Even with Just like Tasmota, the ESPEasy libs folder is filled with adapted versions of libraries. I'm not sure what config the ldf scanner uses to determine whether a library is needed? |
Yes, it follows macros but is not so smart. It does not have information about toolchain's macros. We do not use native GCC preprocessor and use soft version written in Python (part of SCons). We have a feature request to use native GCC Preprocessor but this way will significantly increase compilation time. |
|
That explains it. We hope the feature request will be done (soon)😍 |
|
Would it be possible to make a filter on files to include/exclude in a build? if os.path.isfile('src/Custom.h'):
env.Append(CPPDEFINES=["USE_CUSTOM_H"])
else:
env.Append(CPPDEFINES=[
"CONTROLLER_SET_ALL",
"NOTIFIER_SET_NONE",
# "PLUGIN_BUILD_NORMAL",
"USES_P001", # Switch
"USES_P002", # ADC
"USES_P004", # Dallas DS18b20
"USES_P100", # Pulse Counter - DS2423
"USES_C016", # Cache Controller
"USES_C018", # TTN/RN2483
"FEATURE_MDNS",
"FEATURE_SD",
"FEATURE_I2CMULTIPLEXER",
"USE_SETTINGS_ARCHIVE"
])As you can see, there is quite a simple structure, which can also be found in the .cpp and .h files (at this moment still .ino files, but I can transform them into .h/.cpp files) |
|
It's possible to tune your build process following Thank you too for using PlatformIO! |
|
If you put dependency includes under |
|
I think they are all used in such a structure. And about using PIO... I truly have no idea how to work without it anymore. |
|
I am again running into this nasty command line limit and am a bit out of options now to think of new work arounds for it. Another thing I can do is:
This way I still have the files separate so a change on a single file is still usable in Git blame. |
|
I tried to make some Python scripts to concatenate .cpp files in a specific directory and exclude the original dir. Problem here is that the source filter not appears to be working to exclude the orginal .cpp files. Import("env")
import os
import glob
cpp_files = []
cpp_path = './src/src/PluginStructs'
cpp_path_out = './src/src/PluginStructs_tmp'
if not os.path.exists(cpp_path_out):
os.makedirs(cpp_path_out)
tmp_cpp_file = os.path.join(cpp_path_out, '__tmpfile.cpp')
if os.path.exists(tmp_cpp_file):
os.remove(tmp_cpp_file)
for root, dirs, files in os.walk(cpp_path):
for file in files:
if file.endswith('.cpp'):
fullname = os.path.join(root, file)
cpp_files.append(fullname)
print("Add {}".format(fullname))
with open(tmp_cpp_file,'wb') as newf:
for filename in cpp_files:
with open(filename,'rb') as hf:
newf.write(hf.read())I tried several filters for the Specifically excluding the files that are concatenated: Or excluding the original path: The output of the linker still suggests the old .cpp files are being compiled: @ivankravets |
|
Ah.... this one appears to be working: |
|
OK, I now have managed to limit the size of the e.g. Measures taken:
See: |
|
I'm hitting these issues again and now I'm running out of my own files to concatenate in the Python script mentioned above. I also found this PR, which should fix this issue as far as I could understand, but apparently it doesn't for me. (issue it tried to fix: platformio/platformio-core#3792 ) But maybe it isn't the exact same problem as my linker error is: I really would like to see a proper fix as my patch to concatenate files in the build process does have some side-effects like Intellisense often pointing me to the wrong file and I only find out I was editing the wrong file after all my edits were lost when building. (happened a lot, really frustrating) So please, can we somehow make a proper fix for this? |
|
Probably this will be the solution when the toolchain is available and the option used. |
|
That would be great if it is fixed in the tool chain. |
|
Apparently there is something merged which should (hopefully) fix this.... |
I've been getting this error when building on VScode in Windows.
If I build the exact same code in Linux (using command line) it just works out fine.
It happens with PIO 5.0.1 and the latest 5.0.2 dev version. (reinstalled PIO and all its cache and downloaded packages a few times already)
The strange thing is, it only happens with some of my PIO environments and also depends on some commits I make.
I have been working very hard on converting all my .ino files to .cpp/.h (see: platformio/platformio-vscode-ide#2067 ) and in the process I do see this happen on some of my PIO environments and after a new commit (changing again a few dozen files) it happens on even more environments.
If I checkout an older commit it can be built again in a few environments that failed before.
So it just seems like I'm hitting some resource limit when building in VScode?
In my changes from .ino to .cpp/.h I do try to include as little as possible in .h files an try to only include in the .cpp file.
Only exception is files like
ESPEasy_common.h, which are included almost everywhere.N.B. my environments differ in the used ESP8266/Arduino core library and also the number of "plugins" of my code included in the build.
I created a branch of which the last commit made compiling impossible in VS code for the
custom_ESP8266_4M1Menvironment. https://github.com/TD-er/ESPEasy/tree/build/failing_custom_windows_buildsWhen going back 1 commit in that branch, it compiles again.
Even building in verbose mode is not giving any clue why it failed to generate the .elf file.
So to summarize:
The text was updated successfully, but these errors were encountered: