Out of memory when compiling 1.0.2 on Raspberry

[andimik]

Dear Oona,

I had to reinstall my Raspberry Pi 3B+ but could not install latest 1.0.2 with meson due to out of memory during compilation. Using a GUI, after some minutes the complete x-server crashes and restarts.

I tried it around 20 times, but even booting the RPi without GUI did not help here, the system looks like frozen, the cursor blinks but I even cannot open another logon shell (ctrl + alt + F2 will blink as well). And I also tried to add some parameters like -Ddebug=false or -Dbuildtype=minsize from https://mesonbuild.com/Builtin-options.html

BTW: Building 0.21 (the last one without meson) was successful.

The following log below is after around 30 minutes (when I killed the process by pressing ctrl + c) still trying to compile 0/15.

Do you have an idea or advice me to log anything else?
Is there any chance that I can use cmake like before?

Unfortunately, it is not possible to log everything with script, so I needed to take these two pictures.

meson compile
�[?2004l
�[1;32mINFO:�[0m autodetecting backend as ninja
�[1;32mINFO:�[0m calculating backend command to run: /usr/bin/ninja

[0/15] Compiling C++ object redsea.p/src_block_sync.cc.o�[K
[0/15] Compiling C++ object redsea.p/src_channel.cc.o�[K
[0/15] Compiling C++ object redsea.p/src_dsp_liquid_wrappers.cc.o�[K
[0/15] Compiling C++ object redsea.p/src_dsp_subcarrier.cc.o�[K
[0/15] Compiling C++ object redsea.p/src_groups.cc.o�[K
[0/15] Compiling C++ object redsea.p/src_input.cc.o�[K^C^C^C
ninja: build stopped: interrupted by user.
Traceback (most recent call last):
  File "/usr/lib/python3.11/subprocess.py", line 1262, in wait
    return self._wait(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1997, in _wait
    (pid, sts) = self._try_wait(0)
                 ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1955, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line 194, in run
    return options.run_func(options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mcompile.py", line 359, in run
    p, *_ = mesonlib.Popen_safe(cmd, stdout=sys.stdout.buffer, stderr=sys.stderr.buffer, env=env)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/utils/universal.py", line 1460, in Popen_safe
    o, e = p.communicate(write)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1199, in communicate
    self.wait()
  File "/usr/lib/python3.11/subprocess.py", line 1275, in wait
    self._wait(timeout=sigint_timeout)
  File "/usr/lib/python3.11/subprocess.py", line 1965, in _wait
    def _wait(self, timeout):

KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/meson", line 40, in <module>
    sys.exit(mesonmain.main())
             ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line 287, in main
    return run(sys.argv[1:], launcher)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line 278, in run
    return CommandLineParser().run(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line 204, in run
    mlog.shutdown()
  File "/usr/lib/python3/dist-packages/mesonbuild/mlog.py", line 118, in shutdown
    def shutdown() -> T.Optional[str]:
    ```

![DSC_6786_new](https://github.com/user-attachments/assets/fb14adca-193f-424b-a86a-e399bf980d22)
![DSC_6787_new](https://github.com/user-attachments/assets/52f8c793-6883-4363-a164-24aba27b801e)

[KubaPro010]

looks like a meson problem to me, so redsea is not the guilty one here

[KubaPro010]

i guess you could make it use cmake, just ask chatgpt?

[KubaPro010]

try this CMakeLists.txt : (note that that was generated by chatgpt)

# Set project name and version
cmake_minimum_required(VERSION 3.16)
project(redsea VERSION 1.0.2-SNAPSHOT LANGUAGES CXX)

# Set default build options
set(CMAKE_BUILD_TYPE Release CACHE STRING "Choose the type of build." FORCE)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_INSTALL_PREFIX "/usr/local")

# Set optimization level
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O3")

# Set warning level
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -Wextra -Wpedantic")

# Compiler specific settings
if(MSVC)
  add_compile_definitions(_USE_MATH_DEFINES=1)
elseif(CYGWIN)
  # Use gnu++14 for cygwin
  set(CMAKE_CXX_STANDARD 14)
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=gnu++14")
endif()

# Handle configuration file generation for version
configure_file(
  "${CMAKE_SOURCE_DIR}/config.h.in" 
  "${CMAKE_BINARY_DIR}/config.h"
)

######################
### Dependencies #####
######################

# Find libsndfile
find_package(SndFile REQUIRED)

# Find nlohmann JSON library
find_package(nlohmann_json 3.9.0 REQUIRED)

# Find iconv
find_package(Iconv REQUIRED)

# Find liquid-dsp library
find_package(Liquid REQUIRED)

##########################
### Sources & Executable ###
##########################

# Add source files
set(SOURCES
  src/block_sync.cc
  src/channel.cc
  src/dsp/liquid_wrappers.cc
  src/dsp/subcarrier.cc
  src/groups.cc
  src/input.cc
  src/options.cc
  src/rdsstring.cc
  src/tables.cc
  src/tmc/csv.cc
  src/tmc/tmc.cc
  src/tmc/locationdb.cc
  src/util.cc
)

# Define the main executable
add_executable(redsea ${SOURCES} src/redsea.cc)
target_link_libraries(redsea PRIVATE iconv::iconv nlohmann_json::nlohmann_json sndfile::sndfile liquid::liquid)
install(TARGETS redsea DESTINATION bin)

####################
### Unit tests ######
####################

# Find Catch2 (with main)
find_package(Catch2 REQUIRED)

if(Catch2_FOUND)
  # Unit tests executable
  add_executable(redsea-test-unit ${SOURCES} test/unit.cc)
  target_link_libraries(redsea-test-unit PRIVATE iconv::iconv nlohmann_json::nlohmann_json sndfile::sndfile liquid::liquid Catch2::Catch2)

  # MPX tests executable
  add_executable(redsea-test-mpx ${SOURCES} test/mpx.cc)
  target_link_libraries(redsea-test-mpx PRIVATE iconv::iconv nlohmann_json::nlohmann_json sndfile::sndfile liquid::liquid Catch2::Catch2)

  # Add tests to CTest
  include(CTest)
  include(Catch2)
  catch_discover_tests(redsea-test-unit)
  catch_discover_tests(redsea-test-mpx)
endif()

[andimik]

Thanks for your reply. I have no experience in asking machines and do not trust them.

The syntax has some errors as some words (like Iconv::Iconv need to be case sensitive in capital letter in order to get work.

But I have path problems (for example sndfile not found) and will find out in the next days.

CMake Error at CMakeLists.txt:40 (find_package):
  By not providing "Findsndfile.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "sndfile", but
  CMake did not find one.

  Could not find a package configuration file provided by "sndfile" with any
  of the following names:

    sndfileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "sndfile" to CMAKE_PREFIX_PATH or set
  "sndfile_DIR" to a directory containing one of the above files.  If
  "sndfile" provides a separate development package or SDK, be sure it has
  been installed.

[windytan]

Thank you for the report. It might be because ninja (used by meson) by default starts a highly parallel run (other discussions: vscode-meson#152 and ninja#1441), which is not good for systems with low RAM.

Can you try and replace the compile command with this command:

taskset -c 0 meson compile

It limits ninja to only use 1 CPU. The build will take a little longer as a downside. Hope it helps.

-O.

[andimik]

Well, after several attempts I could build it finally.

Seems the default installation of https://dietpi.com/ on my Raspberry did not make use of a swap file.

After creating it, I could build it (without your hint) in two steps as it crashed after some files.

I don't know if we should close the issue or try to improve the installation procedure.

Oona, is there (still) an option to disable TMC?

[windytan]

No, I haven't put a TMC disabling option in the current system.

I tried the above command on my RasPi 3 B v1.2 and it compiled okay, although I don't have a windowing system there. We could add it to the Readme, I suggest this addition:

If your Linux system has very little RAM, please use the below line instead of 
`meson compile` to limit the number of build processes ninja will spawn:

        $ taskset -c 0 meson compile

Some other ideas that could be turned into tickets (I haven't tested their memory impact):

• Reinstating the TMC build option is not impossible and would require these steps:
  • add a build option and document it
  • add #ifdefs that disable all TMC code (including tests) from building if the build option is set
  • instead output a non-crashing error message into the JSON stream like here
  • add a non-tmc build action to workflows (perhaps for ubuntu-latest only, to keep it simpler)
• Another suggestion I have is to refactor the TMC decoder so that this file would not be compiled-in, but th strings would be loaded from an external TXT file
  • Having external config files will bring its own horde of problems (where to install them, do you have permissions to copy the files there, what if it's not found, what if it's malicious, ...)

Edit, still one option is to do a cross-compilation for Raspberry Pi. It's common in the embedded world to compile code on a more resourceful machine, and then just copy the binaries to the target; for instance, on modern M1 arm64 redsea would compile in around 10 seconds.

But it's something I haven't researched yet, what kind of setup it would require to cross-compile in meson. For sure at least a cross-compilation toolchain is required on the compiler machine, something like this.

-O.

[andimik]

Well, the only main solution is more RAM or a swap file.

It also seems to be depending on the Debian version. Because I remember I could compile it on the same Raspberry on older Debian (Buster or Strech, don't remember).

taskset will not help here, BTW. I still get memory crashes.

[windytan]

You might want to exit any graphical software that is taking up a lot of RAM before compilation. Currently it requires around 500 MB of memory with taskset -c 0.

You can check the amount of free memory with free -m but sometimes memory requirements are not an exact science of addition and subtraction, due to the use of shared areas, caches, etc.

Just for curiosity's sake, I made this Excel graph of how the redsea compilation uses memory on my RasPi 3B on Debian Buster.

The Debian system on the RasPi uses 38 MB in itself if the desktop is not running. Then we start the compile. Pink text, which is roughly what is being compiled at that time (pink lines are placed artistically in Y, does not correspond to the MB scale), show one of the the 500 MB peaks is during locationdb compilation So if a TMC disablement flag would be implemented the peak could probably go down to ~350 MB (guessing).

[andimik]

I have good news, I could compile the new 1.1.0 on the same machine (Raspberry Pi 3B+), but downloaded and installed a different OS in 32bit than before (which was dietpi 64bit distribution, which I do not recommend to others).

Have used the taskset -c 0 meson compile trick, which worked in some minutes.

Therefore, closing.

[windytan]

Great stuff, congrats!