Distro and distro setup for robots

[jgvictores]

Thoughts arising from roboticslab-uc3m/yarp-devices#118.

1. Debian vs Ubuntu. This is a nasty fight, don't want to get too dirty LOL. It was actually the start of the original issue stated above. @munozyanez commented on Debian and @PeterBowman and I were a bit more into Ubuntu. Nothing against Debian, which is usually considered more stable. Simple fact is Ubuntu 16.04 has a newer CMake version (hence newer versions in general) than Debian 9.0. Having newer versions is usually interesting for us. In sum, we usually leaned towards Ubuntu due to its newer package versions.

2. RT or non-RT. Currently, RT is at driver level, the Technosoft IPos PT Mode. We'd have to go years back to ASIBOT's pre-empt kernel, or more recently TEO's Meka head which was formatted due to lack of compatibility of drivers (e.g. its own wifi module could only be used via ndiswrapper, with poor communication quality). This is why we've been leaning towards non-RT.

3. Graphical desktop environments or not. I was originally against installing anything graphical on a robot. However, this is something like c++ vs python: "Is it worth to program on an ultra-efficient platform, if this means that you will not be as efficient/productive?". @smorante usually preferred graphical desktop environments and was much more productive than me at times, and I've seen this with @rsantos88 too. This, together with the fact that we're non-RT anyway, is why we've been leaning towards graphical desktop environments. Note: I believe installing a server version of Ubuntu is a bit more straightforward than Debian, but that may apply to full distros too. 😅
   Edit: Graphical distro tools can also help for settings such as wifi or sound configuration, which need to be configured quick even in working scenarios, such as an imminent demo. 😄 @David-Estevez and I propose, see below, some CLI/curses alternatives.
   Edit2: @David-Estevez Proposed, see below, the possibility of having a graphical desktop installed, but not autostarted.

@roboticslab-uc3m/roboticslab It would be great to have your feedback. Obviously there are more considerations such as which graphical desktop in case there is any (I'm in favor of using defaults per distro), please feel free to comment! Am I missing any more relevant considerations?

[David-Estevez]

About (3), I think that there is always the possibility of having a graphical desktop installed, but not autostarted. I think that solution provides a better use of resources when the GUI is not needed and the PC is running headless, but allows to start a graphical session if some work needs to be done.

[jgvictores]

About (3), I think that there is always the possibility of having a graphical desktop installed, but not autostarted. I think that solution provides a better use of resources when the GUI is not needed and the PC is running headless, but allows to start a graphical session if some work needs to be done.

3.1. I'm sure in practice, there is always work to be done! 😄
3.2. I'm updating my original comment to mention how graphical tools can help for settings such as wifi or sound configuration.

[jgvictores]

About (3), I think that there is always the possibility of having a graphical desktop installed, but not autostarted.

PS: This is a sane proposal, and would by no means discard it as an option. Updating above.

[David-Estevez]

3.2 If configuration is a concern, there are always alternatives to GUIs. For instance, nmtui for configuring wifi.

[jgvictores]

3.2 If configuration is a concern, there are always alternatives to GUIs. For instance, nmtui for configuring wifi.

That's interesting. I'll try it out if I ever find myself in need. I usually fumble around with ifconfig, iwconfig, and all the family, which later get overwritten by Network Manager. 😅

[David-Estevez]

Since I discovered nmtui I'm in love with it for configuring connections in Raspberry Pies and other SBCs. 😍

It would be interesting to discover similar programs for audio configuration, etc.

[jgvictores]

Nice. Regarding sound, CLI/curses alsamixer is sometimes "the only way". But perhaps there is an alternative which interacts better with modern distro tools.

[RaulFdzbis]

In my opinion using a non-graphical desktop cant take many learning/get-used to time at the beginning, and not all the people are comfortable working with it, so unless the graphical environment is a big resource problem for the robot, i would say stick with the graphical environment.

Anyway if @David-Estevez idea is easy to implement i think is the best option. Everyone happy!

[munozyanez]

I don't see the problem of having a graphic desktop, and maybe the easiest (even more than @David-Estevez idea) is to install as-is, and turn down graphics for specific situations.
About 3, I don't feel that leaning towards non RT is a good idea. Things like this ( roboticslab-uc3m/kinematics-dynamics#86 ) can be related.
Maybe we can forget about RT at the cpus, and transfer some local control to low level devices. AFAIK, the mbed boards were bought in that direction. Also, I have been checking, and banana pi could be an option (as it has canbus hardware).

[jgvictores]

Maybe we can forget about RT at the cpus, and transfer some local control to low level devices. AFAIK, the mbed boards were bought in that direction. Also, I have been checking, and banana pi could be an option (as it has canbus hardware).

Yes, this is precisely what I meant above with "RT is at driver level, the Technosoft IPos PT Mode" which is implemented on roboticslab/yarp-devices (and thus TEO and ASIBOT) and accesible via yarp::dev::IPositionDirect.

[munozyanez]

Yes, if you want to control position, driver is enough, but if you want to control velocity, you will need to set velocities according your timing. That makes necessary to reach the low level control parameters in order to set velocities in time.
The thing with ipos is that we are not going to be able to access the low level control but trough canOpen (and obviously trough can).
This makes the need for a low level controller that can send can messages to the ipos fast enough to serve the control needs.
If we had an open platform instead ipos, we could move control codes inside it and completely custom the interface, but this is, at the moment only a daydream.

[jgvictores]

Later on, @munozyanez and I had a relaxing cup of ☕ and agreed Technosoft PT Mode (and also PVT Mode) seem to have the potential to have some RT on the driver side (e.g. using only one position of the FIFO buffer, to command a position that actually is used to indicate a desired velocity at a fixed Ts). While it is a sort of hack, alternatives would not necessarily provide harder RT (TBD).

[munozyanez]

Also, if we are to use this mode, we can not forget about the canbus communication limits.
Making some rough numbers, we can find that:
Updating a position for one motor will take at least 32 bit (single precission), so we can say that one limb takes 32*6=192 bits per target.
Thinking in setting a target for every ms, it will lead us to 1000 target/s * 192 bits/target = 192 kbit/s, so it seems to fit in 250 kbps or more.
Of course, bandwidth will be greater because of packaging, etc...
This numbers can be doubled if we need also to read data from encoders for control tasks. Then we reach a rough 400 kbps or more.
It is not on the limits, but we would take care of keeping the bus tidy in order to avoid errors.
Of course, if we relax RT from 1 ms to 10 or 20 ms we can drastically reduce the bandwidth.
For me, a 0,01 second refresh is more than ok.

[jgvictores]

Also, if we are to use this mode, we can not forget about the canbus communication limits.
Making some rough numbers, we can find that:
Updating a position for one motor will take at least 32 bit (single precission), so we can say that one limb takes 32*6=192 bits per target.
Thinking in setting a target for every ms, it will lead us to 1000 target/s * 192 bits/target = 192 kbit/s, so it seems to fit in 250 kbps or more.
Of course, bandwidth will be greater because of packaging, etc...
This numbers can be doubled if we need also to read data from encoders for control tasks. Then we reach a rough 400 kbps or more.

@munozyanez Thanks for those numbers!

BTW, this reminded me of yet another approach for which Technosoft drivers can be used. The concept is to use external micro-controllers (uC) near the drivers, and instead of communicating via CAN, just directly output a PWM from the uC to the driver as a control reference. Comms between the uC could be CAN, USB, or Ethernet (the later was what was once planned for iCub, not sure if this was finally ever implemented). Again, no hurry and not necessarily a perfect solution, just documenting possibilities.

Of course, if we relax RT from 1 ms to 10 or 20 ms we can drastically reduce the bandwidth.
For me, a 0,01 second refresh is more than ok.

I've asked at many places what a decent control rate is... Not many people willing to answer. Out of non-statistically relevant surveys, let's say ~0.1 ms for drones, 1-10 ms for humanoid gaits, and maybe up to 50-100 ms for basic arm manipulation tasks. Invert for frequency [Hz] 😄. Suggestions and references are welcome!

[PeterBowman]

I wonder if we could stick with different distros in different robots as far as it concerns system performance, and choose the one that suits better in each scenario. Please remember that ASIBOT runs on a quite low-end (if compared with TEO) RoBoard CPU on which it takes over an hour (!) to compile YARP, thus I would change my mind and consider picking the newest Debian in case it proves to run more efficiently than an Ubuntu. Note that we are currently using a Debian 6, ASIBOT was probably never tested with Ubuntu. Productivity and graphic environments are not a concern since we rarely need to open the robot casing and have no access to video output pins either (we talk with it via WiFi/Ethernet all the time). Having a possibility to remove the autostart sounds nice.

[munozyanez]

I had no idea about that. Then It can make a difference even just to install graphics packages, but the difference could be not so big as it seems. Maybe some test and benchmarking can shed some light on it. Do you have any spare part boards?

[munozyanez]

BTW, I will be very happy if we decide to have a non graphic system as a base, then each platform installs the packages as needed.

[PeterBowman]

Do you have any spare part boards?

Yes, but I think we'll wait for the next round of hardware tweaks in the robot instead of risking that the circuitry is not correctly arranged during the tests. I'd just load a Debian 9 image with all necessary components installed and check it right away.

[jgvictores]

Update: roboticslab-uc3m/teo-main#32 (Migration from Debian 6.0 to Ubuntu 14.04 on TEO)

[jgvictores]

Removed "Debian was only used for it's old Linux kernel as commented here " as a consequence of roboticslab-uc3m/yarp-devices#118 and roboticslab-uc3m/teo-main#32.

[PeterBowman]

I'll be working on improving ASIBOT's SW in the next few days. Let me remind that its internal memory is a micro SD card, currently using a 8 GB one (we're not sure whether it's possible to go beyond that capacity due to memory addressing limitations). The full specs are here and here. In short:

• CPU: DM&P Vortex86DX- 1000MHz
• Memory: 256MB DDR2 onboard

Since this is way below Ubuntu Desktop minimum system requirements (2 GHz, 2 GB RAM, 25 GB HDD, ref), and I couldn't even manage to begin the installation (this step requires at least 8.5 GB), I've switched to the Ubuntu Server version and doing fine so far with just 1.5 GB on a VirtualBox image (ref).

Provided that we'll stay on Ubuntu Server, I can't stop noticing that Debian's system requirements are better suited for low-end HW (ref1, ref2):

You must have at least 135MB of memory and 780MB of hard disk space to perform a normal installation. Note that these are fairly minimal numbers.

Install Type	RAM (minimum)	RAM (recommended)	Hard Drive
No desktop	128 megabytes	512 megabytes	2 gigabytes
With Desktop	256 megabytes	1 gigabyte	10 gigabytes

[PeterBowman]

I had to enable PAE-NX support in my VirtualBox image in order to use Ubuntu. This was not compulsory on the previous Debian 6.0 installation. In order to learn whether the processor allows PAE-NX mode, look for the pae flag in cat /proc/cpuinfo.

This CPU nuance might render the Ubuntu system useless on a RB110 board. More info: Wikipedia.

More refs: https://help.ubuntu.com/community/PAE.

[munozyanez]

Ubuntu is not so different from debian, mostly if release dates are close. My personal experience is that everything has worked fine under debian testing, although I have not tested all codes. Anyway, I suppose that problems of using debian, if any, will be minor.

[PeterBowman]

Didn't test it yet, but the integrated CPU does not seem to support PAE:

asibot2@asibot:~$ cat /proc/cpuinfo 
processor	: 0
vendor_id	: Vortex86 SoC
cpu family	: 5
model		: 2
model name	: 05/02
stepping	: 2
cpu MHz		: 999.889
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu tsc cx8
bogomips	: 2000.10
clflush size	: 32
cache_alignment	: 32
address sizes	: 32 bits physical, 32 bits virtual

Debian is the way to go, then.

Edit:

asibot2@asibot:~$ uname -a
Linux asibot 2.6.34.10-vortex86-sg #3 Tue Jul 12 02:05:36 CST 2011 i586 GNU/Linux

[jgvictores]

AFAIK, PAE is for running a 32-bit OS on a 64-bit machine, but enabling tha OS to access the 64-bit memory addresses.
As Vortex86 is 32-bit, it's not a relevant feature. However, I'm not sure it will make the board choke. Maybe the OS is able to determine the amount of available memory and let it be?

[PeterBowman]

I fail to see any relation to 64-bit machines. Both the Ubuntu help page and the Wikipedia article refer to a feature found on 32-bit CPUs. Also, I could not install a 32-bit Ubuntu on a 32-bit VirtualBox image without enabling the PAE-NX option.

[jgvictores]

Right.

• PAE is 32-bit specific, enabling it to access 64-bit addesses, thus allowing addresses in memories larger than 4 GB.
• From here, PAE (...) on capable versions of Windows is a prerequisite for NX.; NX allows the processor to help guard the PC from attacks by malicious software; means No Execute, It means that your CPU knows how to make parts of memory safer by provent programs from executing code in that part

Also relevant: https://www.virtualbox.org/manual/ch03.html

Anyway, again... not sure it will make the board choke. Might be tedious, If any help, I think I have an extra roboard (from ASROB/ECRO).

[munozyanez]

From wikipedia: https://en.wikipedia.org/wiki/Physical_Address_Extension#Linux
Linux distributions now commonly use a PAE-enabled kernel as the default, a trend that began in 2009. As of 2012 many, including Ubuntu (and derivatives like Xubuntu and Linux Mint), Red Hat Enterprise Linux 6.0, and CentOS, have stopped distributing non-PAE kernels, thus making PAE-supporting hardware mandatory.

[PeterBowman]

Anyway, again... not sure it will make the board choke. Might be tedious, If any help, I think I have an extra roboard (from ASROB/ECRO).

I do have a RoBoard replacement in the lab (DMP eBox 3310MX-AP x86, specs) which runs on a Vortex86MX+ CPU. However, I'm unable to boot into Ubuntu (no lack-of-PAE support notice as in my VirtualBox test runs, though).

Relevant: Debian Dropping Support For Older CPUs .

The Debian project supports a wide range of hardware architectures, including 32-bit x86 CPUs. Changes are happening in Debian's development branches which will make older versions of the 32-bit architecture obsolete. Ben Hutchings provides the details:

"Last year it was decided to increase the minimum CPU features for the i386 architecture to 686-class in the Stretch release cycle. This means dropping support for 586-class and hybrid 586/686 processors. (Support for 486-class processors was dropped, somewhat accidentally, in Squeeze.) This was implemented in the Linux kernel packages starting with Linux 4.3, which was uploaded to Unstable in December last year. In case you missed that change, GCC for i386 has recently been changed to target 686-class processors and is generating code that will crash on other processors. Any such systems still running Testing or Unstable will need to be switched to run Stable (Jessie)."

And https://lists.debian.org/debian-devel-announce/2016/05/msg00001.html:

The following processors, supported in jessie, are now unspported:

• AMD K5, K6, K6-2 (aka K6 3D), K6-3
• DM&P/SiS Vortex86, Vortex86SX
• Cyrix III, MediaGX, MediaGXm
• IDT Winchip C6, Winchip 2
• Intel Pentium, Pentium with MMX
• Rise mP6
• VIA C3 'Samuel 2', C3 'Ezra'

Edit: https://www.debian.org/releases/stretch/i386/ch02s01.html.en (2.1.2.1. CPU)

Nearly all x86-based (IA-32) processors still in use in personal computers are supported. This also includes 32-bit AMD and VIA (former Cyrix) processors, and processors like the Athlon XP and Intel P4 Xeon.

However, Debian GNU/Linux stretch will not run on 586 (Pentium) or earlier processors.

Edit 2: compare with same docs for Jessie - https://www.debian.org/releases/jessie/i386/ch02s01.html.en

[PeterBowman]

From #20 (comment):

cpu family	: 5

I guess this means that ASIBOT's Vortex CPU is a i586. Alternatively, try uname -a (ref1, ref2). According to my previous comment, Debian Stretch (9.x) will not work on this robot.

The older processors will continue to be supported in jessie until at
least 2018, and until 2020 if i386 is included in jessie LTS.

Debian Jessie, then!

Edit: ...but Jessie provides CMake 3.0 via apt (ref).

Edit 2: Linux kernel 3.16 (ref) and GCC 4.19 (ref).

Edit 3: all kernels available on Jessie

[PeterBowman]

I guess this means that ASIBOT's Vortex CPU is a i586. Alternatively, try uname -a (ref). According to my previous comment, Debian Stretch (9.x) will not work on this robot.

Well, I'm not sure anymore. The Wikipedia article for Vortex86 gives a comprehensive explanation (marked as OR, though):

The CPU implements the i586 architecture, but the early versions Vortex86 original and Vortex86SX do not have a floating point unit (FPU). Any code that runs on an i486SX CPU without a 487 will run on Vortex86, as will any code that runs on i586 but does not use floating point instructions. Any i586 code will run on Vortex86DX and later. Some Linux kernels (by build-time option) emulate the FPU on any CPU that is missing one, so that even a program that uses floating point instructions, if it runs under Linux, works on any Vortex86 CPU, albeit slowly.

Code intended for i686 may fail because the CPU lacks a Conditional Move (CMOV) instruction. This is an instruction that combines the effect of a conditional branch and a move instruction. Compilers asked to optimize code for a more advanced CPU (for example the GNU Compiler with its -march=i686 option) generate code that uses CMOV. Linux systems intended to run on i686 are generally not runnable on Vortex86 because the GNU C Library, when built for i686, uses a CMOV instruction in its assembly language strcmp function, which its dynamic loader (ld.so) uses. Hence, no program that uses shared libraries can even start up.

Technically, CMOV is optional in the i686 architecture. But Intel's i686 product, the Pentium Pro, had it, and consequently things that generate code typically consider CMOV to be available when you ask them to generate code for i686.

Moreover, Linux images for i686 are actually installed and run on the current ASIBOT's SD image:

asibot2@asibot:~$ dpkg --get-selections | grep -i linux-image
linux-image-2.6-686				install
linux-image-2.6.32-5-686			install
linux-image-2.6.34.10-vortex86-sg		install

Notabene I'm unable to use linux-image-2.6.34.10-vortex86-sg on VirtualBox, I have to select linux-image-2.6.32-5-686 instead on the GRUB init screen. However, the former is selected as default...

[PeterBowman]

Apparently, it's not possible to make an .iso file out of a .vdi (as managed by VirtualBox) and expect it to boot when burnt into a SD card: https://superuser.com/a/114452. I'll go with the mini-PC mentioned in #20 (comment).

Edit: had no luck with writing back the original (non-modified via VirtualBox) .iso with Rufus on Windows, same lack of boot device on init.

[PeterBowman]

Debian Jessie, then!

Done, headless Debian Jessie (aka Debian 8) was successfully launched on ASIBOT (full Ethernet/WiFi/CAN comms) and it came here to stay.

Sorry for hijacking the thread. Is there anything else we should discuss here? Can we give it a final rest and close this issue?

[jgvictores]

Yes, time to close this issue. In line with #10 I've opened roboticslab-uc3m/developer-manual#31

[PeterBowman]

https://ubunlog.com/llega-la-nueva-version-de-puppy-linux-8-0-una-distro-para-equipos-obsoletos/