Creating RGBMatrix with Python bindings while running as root hijacks/disables file system IO

[BenjaminPelletier]

I have a basic Python program modeled off of the runtext.py binding example, and I've copied one of the fonts (7x13.bdf) into a fonts subfolder relative to my script (display_driver.py). It works when the font is owned by pi:

pi@poolpi:~/poolmonitor/display_driver $ sudo chown -R pi:pi fonts
pi@poolpi:~/poolmonitor/display_driver $ python3 display_driver.py
Press CTRL-C to stop
FYI: not running as root which means we can't properly control timing unless this is a real-time kernel. Expect color degradation. Consider running as root with sudo.
Can't set realtime thread priority=99: Operation not permitted.
        You are probably not running as root ?
        This will seriously mess with color stability and flicker
        of the matrix. Please run as `root` (e.g. by invoking this
        program with `sudo`), or setting the capability on this
        binary by calling
        sudo setcap 'cap_sys_nice=eip' $THIS_BINARY
^CExiting

(the RGB matrix is displaying text properly until I hit ctrl-c)

However, my problem is that LoadFont has an error when I run as root:

pi@poolpi:~/poolmonitor/display_driver $ sudo python3 display_driver.py
Press CTRL-C to stop
Traceback (most recent call last):
  File "display_driver.py", line 51, in <module>
  File "display_driver.py", line 34, in loop
SystemError: <method 'LoadFont' of 'rgbmatrix.graphics.Font' objects> returned NULL without setting an error

I thought this was likely due to the font file being owned by pi, so I tried changing ownership to root:

pi@poolpi:~/poolmonitor/display_driver $ sudo chown -R root:root fonts
pi@poolpi:~/poolmonitor/display_driver $ python3 display_driver.py
Press CTRL-C to stop
FYI: not running as root which means we can't properly control timing unless this is a real-time kernel. Expect color degradation. Consider running as root with sudo.
Can't set realtime thread priority=99: Operation not permitted.
        You are probably not running as root ?
        This will seriously mess with color stability and flicker
        of the matrix. Please run as `root` (e.g. by invoking this
        program with `sudo`), or setting the capability on this
        binary by calling
        sudo setcap 'cap_sys_nice=eip' $THIS_BINARY
Traceback (most recent call last):
  File "display_driver.py", line 51, in <module>
    loop()
  File "display_driver.py", line 34, in loop
    font.LoadFont("./fonts/7x13.bdf")
SystemError: <method 'LoadFont' of 'rgbmatrix.graphics.Font' objects> returned NULL without setting an error
pi@poolpi:~/poolmonitor/display_driver $ sudo python3 display_driver.py
Press CTRL-C to stop
Traceback (most recent call last):
  File "display_driver.py", line 51, in <module>
  File "display_driver.py", line 34, in loop
SystemError: <method 'LoadFont' of 'rgbmatrix.graphics.Font' objects> returned NULL without setting an error

...however, as you can see above, changing ownership to root breaks even what was working previously. How can I fix this LoadFont error?

Here is the content of display_driver.py:

#!/usr/bin/env python

import sys
import time

# Requires rgbmatrix installed via special procedure from
# https://github.com/hzeller/rpi-rgb-led-matrix
from rgbmatrix import RGBMatrix, RGBMatrixOptions, graphics


def make_matrix() -> RGBMatrix:
  options = RGBMatrixOptions()

  options.rows = 32
  options.cols = 64
  options.chain_length = 1
  options.parallel = 1
  options.row_address_type = 0
  options.multiplexing = 0
  options.pwm_bits = 11
  options.brightness = 100
  options.pwm_lsb_nanoseconds = 130
  options.led_rgb_sequence = 'RGB'
  options.pixel_mapper_config = ''
  options.gpio_slowdown = 1

  return RGBMatrix(options=options)


def loop():
  matrix = make_matrix()
  offscreen_canvas = matrix.CreateFrameCanvas()
  font = graphics.Font()
  font.LoadFont("./fonts/7x13.bdf")
  textColor = graphics.Color(255, 255, 0)
  my_text = 'Hello world'

  while True:
    offscreen_canvas.Clear()
    graphics.DrawText(offscreen_canvas, font, 0, 10, textColor, my_text)

    time.sleep(0.05)
    offscreen_canvas = matrix.SwapOnVSync(offscreen_canvas)


# Main function
if __name__ == "__main__":
  try:
    # Start loop
    print("Press CTRL-C to stop")
    loop()
  except KeyboardInterrupt:
    print("Exiting\n")
    sys.exit(0)

[BenjaminPelletier]

sudo -s yields the same results. I've added

print(os.getcwd())
print(os.path.abspath("./fonts/7x13.bdf"))

...and this confirms that the same path/file is being requested regardless of whether I'm running with as a user, with sudo, or with sudo -s. If font.LoadFont does not support relative paths, it should throw an exception or similar for my request. If font.LoadFont uses a different base path for its relative paths than the current working directory, that seems like unexpected behavior and I'd expect that to be documented somewhere.

When I change the LoadFont line to

font.LoadFont('/home/pi/poolmonitor/display_driver/fonts/7x13.bdf')

...running as pi continues to work normally, running as sudo continues to fail as originally described, and running with sudo -s fails in the same way as with sudo.

In what way is this issue off-topic?

[BenjaminPelletier]

The text you have quoted does not relate to this issue; it is displayed only when I run not as root, which I am doing only to demonstrate that the issue is not related to any basic configuration -- the program works correctly when not run as root (despite the message about lower performance that you have quoted from, which I assume is true, which is why I want to run as root, which is why this issue is a problem).

Perhaps I am not being clear; let me try to be more succinct: When I attempt to run my Python program as root as suggested ( Please run as 'root' (e.g. by invoking this program with 'sudo'), font.LoadFont has a SystemError. This error does not appear to be due to an incorrect script on my side or installation because that same script works fine when I don't run as root. What could be causing this SystemError when I run as root, but not when I run not as root?

One of those being a new working/current directory.

I would like to investigate whether this possible difference between root and non-root is the cause of the SystemError. What do you suggest I do to check this beyond confirming that the SystemError still occurs when I use an absolute path to the font as I described in my most recent comment?

Some flags can work differently also depending on different contexts.

Sounds like an avenue that can be explored. What flags might cause a SystemError in this repo's code as root, but work properly as not-root?

Sometimes this happens where you get a new set of environment variables.

I would also be interested in exploring this possibility. What environment variables might cause a SystemError in this repo's code as root, but work properly as not-root? Here are my environment variables both as root and not as root:

pi@poolpi:~/poolmonitor/display_driver $ printenv
SHELL=/bin/bash
NO_AT_BRIDGE=1
PWD=/home/pi/poolmonitor/display_driver
LOGNAME=pi
XDG_SESSION_TYPE=tty
HOME=/home/pi
LANG=en_GB.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
SSH_CONNECTION=10.0.0.15 58974 192.168.1.82 22
XDG_SESSION_CLASS=user
TERM=xterm
USER=pi
SHLVL=1
XDG_SESSION_ID=c2
XDG_RUNTIME_DIR=/run/user/1000
PS1=\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w \$\[\033[00m\]
SSH_CLIENT=10.0.0.15 58974 22
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
MAIL=/var/mail/pi
SSH_TTY=/dev/pts/0
OLDPWD=/home/pi/poolmonitor/display_driver/lib
TEXTDOMAIN=Linux-PAM
_=/usr/bin/printenv
pi@poolpi:~/poolmonitor/display_driver $ sudo printenv
NO_AT_BRIDGE=1
LANG=en_GB.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
TERM=xterm
PS1=\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w \$\[\033[00m\]
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAIL=/var/mail/root
LOGNAME=root
USER=root
HOME=/root
SHELL=/bin/bash
SUDO_COMMAND=/usr/bin/printenv
SUDO_USER=pi
SUDO_UID=1000
SUDO_GID=1000

It is off topic because you are have issue from running things as root with python.

The things I am running are the Python bindings defined in this repository, and the documentation suggests that I should run as root (which is the condition under which the error occurs). So I am encountering an error when running the code from this repository in the manner suggested by this repository. Where do you suggest is an appropriate forum for me to address this issue?

[BenjaminPelletier]

I've distilled this issue to a smaller set of repro steps. Here is bad_font.py:

from rgbmatrix import RGBMatrix, RGBMatrixOptions, graphics

options = RGBMatrixOptions()
options.rows = 32
options.cols = 64
matrix = RGBMatrix(options=options)

font = graphics.Font()
font.LoadFont('/home/pi/poolmonitor/display_driver/fonts/7x13.bdf')

print('Exited successfully')

That script produces the following results:

pi@poolpi:~/poolmonitor/display_driver $ python3 bad_font.py
FYI: not running as root which means we can't properly control timing unless this is a real-time kernel. Expect color degradation. Consider running as root with sudo.
Can't set realtime thread priority=99: Operation not permitted.
        You are probably not running as root ?
        This will seriously mess with color stability and flicker
        of the matrix. Please run as `root` (e.g. by invoking this
        program with `sudo`), or setting the capability on this
        binary by calling
        sudo setcap 'cap_sys_nice=eip' $THIS_BINARY
Exited successfully
pi@poolpi:~/poolmonitor/display_driver $ sudo python3 bad_font.py
Traceback (most recent call last):
  File "bad_font.py", line 9, in <module>
SystemError: <method 'LoadFont' of 'rgbmatrix.graphics.Font' objects> returned NULL without setting an error
pi@poolpi:~/poolmonitor/display_driver $

Summary: bad_font.py completes successfully (no SystemError) when run as pi. bad_font.py produces a SystemError when run as root (via sudo).

But, now I comment out just one line:

from rgbmatrix import RGBMatrix, RGBMatrixOptions, graphics

options = RGBMatrixOptions()
options.rows = 32
options.cols = 64
#matrix = RGBMatrix(options=options)

font = graphics.Font()
font.LoadFont('/home/pi/poolmonitor/display_driver/fonts/7x13.bdf')

print('Exited successfully')

Now, everything works fine:

pi@poolpi:~/poolmonitor/display_driver $ python3 bad_font.py
Exited successfully
pi@poolpi:~/poolmonitor/display_driver $ sudo python3 bad_font.py
Exited successfully
pi@poolpi:~/poolmonitor/display_driver $

So, it seems like there must be some kind of static initialization when an RGBMatrix is created that breaks font creation with root. Ok, so let's create the font before creating the RGBMatrix:

from rgbmatrix import RGBMatrix, RGBMatrixOptions, graphics

font = graphics.Font()
font.LoadFont('/home/pi/poolmonitor/display_driver/fonts/7x13.bdf')

options = RGBMatrixOptions()
options.rows = 32
options.cols = 64
matrix = RGBMatrix(options=options)

print('Exited successfully')

And indeed, this works properly:

pi@poolpi:~/poolmonitor/display_driver $ python3 bad_font.py
FYI: not running as root which means we can't properly control timing unless this is a real-time kernel. Expect color degradation. Consider running as root with sudo.
Can't set realtime thread priority=99: Operation not permitted.
        You are probably not running as root ?
        This will seriously mess with color stability and flicker
        of the matrix. Please run as `root` (e.g. by invoking this
        program with `sudo`), or setting the capability on this
        binary by calling
        sudo setcap 'cap_sys_nice=eip' $THIS_BINARY
Exited successfully
pi@poolpi:~/poolmonitor/display_driver $ sudo python3 bad_font.py
Exited successfully
pi@poolpi:~/poolmonitor/display_driver $

So, I'm retitling this bug to better reflect the unexpected behavior: it seems that fonts cannot be created after creating an RGBMatrix, but this deficiency is only present when running as root. Very weird.

[BenjaminPelletier]

It looks like the original behavior observed is actually a special case of a general problem: this library seems to completely hijack/disable all IO once an RGBMatrix is created if run as root. Consider hijack.py:

from rgbmatrix import RGBMatrix, RGBMatrixOptions, graphics

options = RGBMatrixOptions()
options.rows = 32
options.cols = 64

print('Writing test1')
with open('test1.txt', 'w') as f:
  f.write('ok')
print('Wrote test1')

matrix = RGBMatrix(options=options)

print('Writing test2')
with open('test2.txt', 'w') as f:
  f.write('ok')
print('Wrote test2')

The result of running this file as a normal user and as root is:

pi@poolpi:~/poolmonitor/display_driver $ rm test1.txt test2.txt
pi@poolpi:~/poolmonitor/display_driver $ python3 hijack.py
Writing test1
Wrote test1
FYI: not running as root which means we can't properly control timing unless this is a real-time kernel. Expect color degradation. Consider running as root with sudo.
Can't set realtime thread priority=99: Operation not permitted.
        You are probably not running as root ?
        This will seriously mess with color stability and flicker
        of the matrix. Please run as `root` (e.g. by invoking this
        program with `sudo`), or setting the capability on this
        binary by calling
        sudo setcap 'cap_sys_nice=eip' $THIS_BINARY
Writing test2
Wrote test2
pi@poolpi:~/poolmonitor/display_driver $ rm test1.txt test2.txt
pi@poolpi:~/poolmonitor/display_driver $ sudo python3 hijack.py
Writing test1
Wrote test1
Writing test2
Traceback (most recent call last):
  File "hijack.py", line 15, in <module>
PermissionError: [Errno 13] Permission denied: 'test2.txt'
pi@poolpi:~/poolmonitor/display_driver $

Creating an RGBMatrix while running as root appears to disable all ability to interact with the file system. WTF?

[hzeller]

By default, the matrix initialization drops root privileges to daemon to reduce security attack surface (you can see that with ps what user the program runs. You start it as root, but after matrix initialization the user should be daemon). You have to keep that in mind when accessing files: make them readable to this user or writable to it if needed. Also, use absolute paths.
The file permission errors you see are the expected behavior.

So if you don't want to have the privileges dropped or to the priv dropping yourself to another user, set the drop_privileges value to false.

[BenjaminPelletier]

Thanks, that makes sense and I see it is intended and reasonable, though I wouldn't say that a demotion in privileges as a result of constructing an object (but only when root) is the behavior most users would expect. For anyone else who finds this issue in the future, run sudo python3 verify_user.py with the following verify_user.py to see the above explanation in action:

import os
import pwd
from rgbmatrix import RGBMatrix, RGBMatrixOptions, graphics

options = RGBMatrixOptions()
options.rows = 32
options.cols = 64

print('User before RGBMatrix: ' + pwd.getpwuid(os.getuid()).pw_name)
matrix = RGBMatrix(options=options)
print('User after RGBMatrix: ' + pwd.getpwuid(os.getuid()).pw_name)

Result:

pi@poolpi:~/poolmonitor/display_driver $ sudo python3 verify_user.py
User before RGBMatrix: root
User after RGBMatrix: daemon
pi@poolpi:~/poolmonitor/display_driver $

The drop_privileges option is defined here and exposed to Python here (note that the preceding links may point to an older code version). To change behavior, modify the example above to:

import os
import pwd
from rgbmatrix import RGBMatrix, RGBMatrixOptions, graphics

options = RGBMatrixOptions()
options.rows = 32
options.cols = 64
options.drop_privileges = False

print('User before RGBMatrix: ' + pwd.getpwuid(os.getuid()).pw_name)
matrix = RGBMatrix(options=options)
print('User after RGBMatrix: ' + pwd.getpwuid(os.getuid()).pw_name)

Result:

pi@poolpi:~/poolmonitor/display_driver $ sudo python3 verify_user.py
User before RGBMatrix: root
User after RGBMatrix: root
pi@poolpi:~/poolmonitor/display_driver $