Building Shoes on Pi2

The pi is just a Linux box. It's a bit slow for big compiles but it's just Linux. In this case, its Raspbian (Stretch) from June 2018. Things like package names and library names do change and you have to find them if and when these instructions fail. I have a Pi3B that boots off of a USB SSD so its about as fast as I can buy.

Overview

1. Get RVM working
2. Build another Ruby
3. Shoes source and gems
4. Build Shoes

Get RVM working

I like RVM - it's smart and it installs Ruby dependencies and that alone is worth the effort to install RVM.

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable

Logout and log back in and make sure ~/.rvm is in your PATH. Modify .bashrc and .bash_profile if needed. Now we install an RVM ruby, our way.

$ rvm install 2.3.7  -C --enable-load-relative --enable-shared
Updating system...
Installing required packages: gawk, autoconf, automake, bison, libffi-dev, libgdbm-dev, libncurses5-dev, libsqlite3-dev, libtool, libyaml-dev, sqlite3, libgmp-dev, libreadline-dev, libssl1.0-dev
Requirements installation successful.
Installing Ruby from source to: /home/pi/.rvm/rubies/ruby-2.3.7, this may take a while depending on your cpu(s)...
ruby-2.3.7 - #downloading ruby-2.3.7, this may take a while depending on your connection...

     a long time later ...

ruby-2.3.7 - #making binaries executable..
ruby-2.3.7 - #downloading rubygems-2.7.7
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  894k  100  894k    0     0  1416k      0 --:--:-- --:--:-- --:--:-- 1415k
No checksum for downloaded archive, recording checksum in user configuration.
ruby-2.3.7 - #extracting rubygems-2.7.7....................................................................................................................................................................................................................................
ruby-2.3.7 - #removing old rubygems........
ruby-2.3.7 - #installing rubygems-2.7.7................................
ruby-2.3.7 - #gemset created /home/pi/.rvm/gems/ruby-2.3.7@global
ruby-2.3.7 - #importing gemset /home/pi/.rvm/gemsets/global.gems...................................................
ruby-2.3.7 - #generating global wrappers.......
ruby-2.3.7 - #gemset created /home/pi/.rvm/gems/ruby-2.3.7
ruby-2.3.7 - #importing gemsetfile /home/pi/.rvm/gemsets/default.gems evaluated to empty gem list
ruby-2.3.7 - #generating default wrappers.......
ruby-2.3.7 - #adjusting #shebangs for (gem irb erb ri rdoc testrb rake).
Install of ruby-2.3.7 - #complete 
Ruby was built without documentation, to build it run: rvm docs generate-ri

Build our Ruby

The way Shoes is built for distribution, strongly suggests (aka requires) that you use a second ruby for Shoes. Yes, it is slow to build Ruby on the Pi. But, it's not hard. We need a place to store the new ruby and a place for the source code. I used ~/shoesdeps/pi3 and ~/shoesdeps/src Download Ruby 2.3.7ruby 2.3.7 (not one of the 2.4 or 2.5 - Shoes can't use them). Move the tar ball into that `src' and untar it. Get the latest rubygems too. You should have:

pi@pi3:~ $ ls -l shoesdeps/src
total 18368
-rwxr-xr-x  1 pi pi      197 Jul  6 00:48 pi3-ruby.sh
drwxr-xr-x 26 pi pi    12288 Jul  6 02:49 ruby-2.3.7
-rw-r--r--  1 pi pi 17859100 Jul  6 01:20 ruby-2.3.7.tar.gz
drwxr-xr-x  8 pi pi     4096 May 17 19:18 rubygems-2.7.7
-rw-r--r--  1 pi pi   915847 Jul  6 01:22 rubygems-2.7.7.tgz

The contents of pi3-ruby.sh is:

#! /bin/bash
export dest=~/shoesdeps/pi3
./configure \
  --enable-shared \
  --enable-load-relative \
  --disable-install-doc \
  --without-tk --without-tcllib --without-tcltk \
  --prefix=${dest}

cd into ruby-2.3.7 and do ../pi3-ruby.sh - that runs the configure step for ruby. It takes a while. Now do make and after a very long while, make install That puts the new ruby in ~/shoesdeps/pi3

pi@pi3:~ $ ls shoesdeps/pi3
bin  include  lib  share

Gems are a bit trickier. cd into shoesdeps/src/rubygems-2.7.7 (or the version you downloaded). We're going to use our new ruby to update its gem version.

~/shoesdeps/pi3/bin/ruby setup.rb --no-rdoc --no-ri

You just tested the new ruby! --help works too ;-)

Get Shoes source and gems

Now we need a place for Shoes and the Gems that Shoes needs. I do this in ~/Projects/shoes3 and ~/Projects/gems/rb23. Note that rb23 is for gems compiled with and for ruby 2.3.x. One could image there is a rb22 in there from older versions of Shoes.

Git clone

git clone https://github.com/shoes/shoes3.git

NFS mount

I mount the ~/Projects directory with NFS which is some additional work and has cautions. Obviously you need and NFS server that exports that directory.

$ sudo apt-get install nfs-common -y
$ sudo mount -t nfs 192.168.1.2:/Projects Projects

Works for me - you would use your IP number and names. For the long term, we need to modify /etc/fstab to mount the NSF directory at boot time. This requires systemd on my version of raspbian. At the end of /etc/fstab I have these two lines.

# nfs mount
192.168.1.2:/Projects /home/pi/Projects nfs     noauto,x-systemd.automount,noatime,nolock,bg,nfsvers=4,intr,tcp,actimeo=1800    0       0

That x-systemd.automount parameter is important.

Build Gems

There are many options when dealing with Gems included with Shoes. The very easiest is to get them from someone who has done the work already. Pick rpigems.tgz, move it to ~/Projects/gems/ and untar it.

The harder way is to use our ruby to install and build them and then extract them into the ~Projects/gems/rb23 space That is documented later

Build Shoes.

cd into shoes3 and create a rpi-custom.yaml file. That file tells rake where you put all the bits and pieces. Mine looks like:

Ruby: /home/pi/shoesdeps/pi3 
Deps: 
Gemloc: /home/pi/Projects/gems/rb23
Extloc: /home/pi/Projects/gems/rb23
Gems:
InclGems:
  - chipmunk-6.1.3.4
  - sqlite3-1.3.13
  - nokogiri-1.8.0
  - mini_portile2-2.2.0
  - byebug-9.0.6
  - rb-readline-0.5.4
  - ffi-1.9.18
  # picky needs:
  - activesupport-5.1.2
  - concurrent-ruby-1.0.5
  - i18n-0.8.6
  - multi_json-1.12.1
  - picky-4.31.3
  - rack_fast_escape-2009.06.24
  - thread_safe-0.3.6
  - tzinfo-1.2.3
  - url_escape-2009.06.24
  - yajl-ruby-1.3.0
  # typhoeus
  - ethon-0.10.1
  - typhoeus-1.1.2
Debug: false 
Deprecations: true 

Note that those gem names include a version number. You will get errors if the yaml doesn't match your gems. Don't worry, it will tell you it can't find it.

$ rake -T
$ rake linux:setup:rpi
$ rake

Gems the hard way.

Basically we use Ruby to install gems, then we copy the gems. Then we tell Ruby to un-install them. This tedious on any system but the pi more tedious. Decide which Ruby. I like to use the RVM ruby for this because it's less likely to cause a problem with the ruby in ~/shoesdeps/pi3 Its possible that your $PATH is not setup correctly by RVM - mine isn't.

Try a simple gem first, chipmunk and lets see where things are installed.

$ ~/.rvm/rubies/ruby-2.3.7/bin/gem install chipmunk --no-doc
Fetching: chipmunk-6.1.3.4.gem (100%)
ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the /var/lib/gems/2.3.0 directory.

Hmmm. sudo is a pain for what we need to do. Doesn't work either - Raspbian has the header files in the wrong place or didn't include them. We can use our Ruby.

pi@pi3:~ $ ~/shoesdeps/pi3/bin/gem install chipmunk --no-doc
Fetching: chipmunk-6.1.3.4.gem (100%)
Building native extensions. This could take a while...
Successfully installed chipmunk-6.1.3.4
1 gem installed

That works - and compiling is still slow on the pi. Do not forget the --no-doc unless you really like slow. Repeat for the other gems - some will drag in the dependent gems. Write down all the version numbers because they may be different. ~/shoesdeps/pi3/bin/gem list -l can help. Repeat for sqlite3, nokogiri, rb-readline, byebug,

Since this version of Raspbian ships with ruby 2.3.3 that's close enough. Lets use that.

First you want a symbolic link

sudo gem install nokogiri

Version 1.6.7.1. It takes a lot of time to build from source on any system and even longer on a pi2 - unfortunately it's tied to the libxml2 version it was built with and it install the damn docs.

Turns out the gemrc should be /usr/local/etc/ and look like this
--- 
:sources: 
- http://gems.rubyforge.org/
- http://gems.github.com
:benchmark: false
:update_sources: true
:bulk_threshold: 1000
:backtrace: false
:verbose: true
gem: --no-document

So we need to undo some things. Uninstall nokogiri and any other gem we installed by hand. Then we blow away /usr/local/lib/ruby and /usr/local/bin/ruby. repeat the sudo make install in ruby and the gem update --system Now repeat the nokogiri install.

sudo gem install byebug (8.2.1) which is way past the 5.0.0 version so expect some trouble. sudo gem install sqlite3

Note that byebug didn't bring in columnize. We might want sudo gem install rb-readline (0.5.3 at this time)

Build Tight Shoes

apt-get install makeself Create rakefile target 'pi2' and env.rb and task.rb and pi2-custom.yaml. I'm going to clone the x86_64 and the new rake command will be rake linux:setup:pi2

We have an old, old, old problem: libgif and libungif and I failed to apg-get install libjpeg8-dev and libgif-dev. Eventually you'll get Shoes to link and then fail trying to copy prebuilt gems. Because we haven't built chipmunk, ftsearch, hpricot and sqlite3. rake shoesgems does that and puts them where ever the Gemloc: and ExtLoc: paths you set in pi2-custom.yaml. Note It is assumed you have the source for those gems in that directory. I'm going to use the NSF mount because you only have to build the damn things once.

Now we have to get those other gems out of the Ruby we built and into the Gemloc: location and we copy them without the tests and doc and other fluff that might be in the gem. There is a gemutils/extract1gem script included with Shoes. The first arg is the path to the gemspec file in the Ruby you want to copy from. The second command line arg is where to put it. Sqlite is just weird and we don't have to copy it. In fact we want to gem uninstall it from the Ruby. sudo gem uninstall sqlite3

Lets do the bad boy, nokogiri.

 ./gemutils/extract1gem.rb /usr/local/lib/ruby/gems/2.1.0/specifications/nokogiri-1.6.7.1.gemspec ../gems/shoes/built/pi2/

Make sure the names in the Inclgem: section match what you copied. Repeat for mini_portile byebug, and rb-readline. And any other gem you want in your Shoes copy. And then rake and hopefully you'll get a shoes built on your pi2 that you can distribute to other pi2 users. Will it work on a pi-B? Probably.

The build process is slow and slower. I believe it's NFS (writing). The osx build also has that problem which was helped by the ALTP env variable to point to local disk (that usb drive on the pi2) plus rakefile changes.