INSTALL

The INSTALL document for 42ity
==============================

This INSTALL document is about building and installing prerequisites
from source and/or as resulting packages for the Eaton 42ity core project.

Also keep in mind that the 42ity core project auto-configuration procedure
relies on `pkg-config` information (including the library versions), so
it should be available (as the installed subsystem of the OS as well as
the '*.pc' files as part of packaged or privately-built prerequisite
software).

To ease copy-pasting, the example commands in this document are prefixed
with a colon+semicolon prompt `:;` which is an empty command for shell
interpreters (unlike the dollar `$` or hash `#` signs which have their
meaning for shells and can adversely influence copy-pasted commands).



Get the source to build and/or develop
--------------------------------------
If you are reading this file, you probably have _some_ version of the
42ity core project source code already. It might not be the _most fresh_
version, however, so to stay on the bleeding edge of development you
might want to clone the source-code repository onto your development
system.

Sources for building
~~~~~~~~~~~~~~~~~~~~
To just build the project, get a hold of its common 'upstream' repository.
The 'upstream' 42ity core project Git repository can be cloned with the
command below over SSH (your keys should be uploaded to the user's GitHub
account first):
----
:; git clone git@github.com:42ity/fty-core.git
----
...or over HTTPS:
----
:; git clone https://github.com/42ity/fty-core.git
----

Sources for collaboration
~~~~~~~~~~~~~~~~~~~~~~~~~
If you also contribute to the project, the checkout procedure is similar,
but there are more nuances to know and some general rules to follow.

An overview of the source-code tracking in the 42ity project and deeper
details on code checkout are given in the 'CONTRIBUTING' document
('link:CONTRIBUTING.html[]'). The short example is below:
----
:; git clone git@github.com:user/fty-core-myfork.git fty-core-myfork && \
   cd fty-core-myfork/ && \
   git remote add upstream git@github.com:42ity/fty-core.git && \
   git fetch --all --tags
----

Verify the local workspace (list the checked-out and available branches):
----
:; git branch -a

* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/upstream/master
----

Then the developer may create branches for working on individual issues
and when some work is done -- push the code back to his personal fork
on GitHub for common review like this:
----
:; git add changedfiles* && git commit -m 'Meaningful text' && git push
----
...and afterwards use the GitHub web-interface to initiate a pull-request
for code review and ultimate merge of his or her changes into the common
codebase by the gate-keepers.



Configure package repositories of pre-built prerequisites for the project
-------------------------------------------------------------------------
Currently 42ity relies on some Linux-specific features, so the code is
not expected to compile and work in other operating system platforms,
at least not in its entirety. It is not currently a goal of the core
development team to support deployment on more platforms than we can
handle and validate ourselves. Current target devices will be ARM boxes
running an embedded Linux distribution, and the developer workstations
run an assortment of recent PC Linux distributions. The project team's
infrastructure services include an Open Build Service (OBS)-based server
to automatically check out and rebuild the packages with some of our own
and third-party code relevant for our project for some platforms that
are of interest to ourselves.

Alternately, it is possible to rebuild and install the needed packages
from latest sources, as will be detailed below. In particular, many of
these are either absent or critically too-old in the general OS package
repositories, and some projects are being developed in cooperation with
our 42ity project, so it is essential to use their bleeding-edge as well.

The software packages pre-built for the project can be located in our OBS
build-results repositories, including the following Linux OS releases:

 * http://obs.mbt.lab.etn.com:82/Pool:/master/Debian_8.0/
 * http://obs.roz53.lab.etn.com:82/Pool:/master/Debian_8.0/
 * http://obs.roz53.lab.etn.com:82/Pool:/master/CentOS_7.0/
 * http://obs.roz53.lab.etn.com:82/Pool:/master/openSUSE_13.1/

These packages include such build and/or run-time dependencies as
libvariant, iproute2, mariadb, zeromq and so on.

[NOTE]
*NOTE*: See the document 'INSTALL-referenceOS-debian8' (or its derivative
product 'link:INSTALL-referenceOS-debian8.html[]') for a how-to detailing
the reference installation of a Debian 8 x86_64 (VM) to build this project.

Installing an APT package source for Debian 8.0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
One of the Linux OSes supported automatically by our OBS service is
Debian 8.0 for "x86_64" and "armhf" architectures (and "all" for some
platform-independent code). This is currently a "testing" variant of
Debian and a moving target in terms of formal support (although seems
recently "frozen" and heading to official stable supported release).
However, automatic dependency handling in OBS should handle refreshed
source code coming from some project's upstream for us.

* *Keys to trust and repositories to use:*
Automatic setup of an OBS instance as a package source for a supported
Debian 8.0 platform with APT package management should work like this
(execute as 'root'):
----
:; wget -O - \
     http://obs.roz53.lab.etn.com:82/Pool:/master/Debian_8.0/Release.key | \
     apt-key add - && \
   echo 'deb http://obs.roz53.lab.etn.com:82/Pool:/master/Debian_8.0 /' > \
     /etc/apt/sources.list.d/95bios-obs-roz-deb8.list

:; wget -O - \
     http://obs.mbt.lab.etn.com:82/Pool:/master/Debian_8.0/Release.key | \
     apt-key add - && \
   echo 'deb http://obs.mbt.lab.etn.com:82/Pool:/master/Debian_8.0 /' > \
     /etc/apt/sources.list.d/95bios-obs-mbt-deb8.list
----
* *Repository preference:*
It is also encouraged to make sure that your package manager always prefers
the project's supported packages over any other repositories (i.e. in the
upstream OS distribution) that might be named the same:
----
:; cat > /etc/apt/preferences.d/bios <<EOF
Package: *
Pin: origin "obs.roz53.lab.etn.com"
Pin-Priority: 9999

Package: *
Pin: origin "obs.mbt.lab.etn.com"
Pin-Priority: 8999
EOF
----
See https://wiki.debian.org/AptPreferences for more details.

NOTE: for some experimental purposes, you might want to put just the
specific name or a pattern to prefer just some 'Package'(s) from our
OBS; keep in mind that such a list should at least include 'iproute*',
because the upstream Debian deprecated 'iproute2-dev' (claiming that
the interface is private to the kernel and should not be used directly)
and provides an empty package just for legacy compatibility purposes.

* *Before any installations, update the packaging metadata* known to
your packaging client: in order to calculate the dependencies, `apt`
keeps a cache of downloaded lists of packages provided by each of the
configured repositories. In a dynamic world, these should be updated
before starting any installations or upgrades:
----
:; apt-get update
----

NOTE: with a bleeding-edge distribution used in our case, formats can
change so once in a year a fallback trick may be required to get `apt`
to actually work (the error in our particular case was that upstream
repositories changed compressed metadata file extensions and this
baffled old cached metadata updates):
----
:; apt-get update || { rm -rf /var/lib/apt/lists/*; apt-get update; }
----

Afterwards the usual installation and update actions with `apt-get install`,
search with `apt-cache search` and so on should work.

To verify which variant (version/repo) of the package is installed and
what such variants are known to `apt`, you can use `apt-cache policy`:
----
:; apt-cache policy iproute2-dev
iproute2-dev:
  Installed: 3.16.0-0
  Candidate: 3.16.0-0
  Package pin: 3.16.0-0
  Version table:
 *** 3.16.0-0 9999
        500 http://obs.roz53.lab.etn.com:82/Pool:/master/Debian_8.0/  Packages
        500 http://obs.mbt.lab.etn.com:82/Pool:/master/Debian_8.0/  Packages
        100 /var/lib/dpkg/status

:; apt-cache policy mariadb-server-10.0
mariadb-server-10.0:
  Installed: 10.0.14-2
  Candidate: 10.0.14-2
  Version table:
 *** 10.0.14-2 0
        500 http://localhost:3142/debian/ jessie/main amd64 Packages
        500 http://gate:3142/debian/ jessie/main amd64 Packages
        500 http://ftp.debian.org/debian/ jessie/main amd64 Packages
        100 /var/lib/dpkg/status
     10.0.10-1 0
        500 http://obs.roz53.lab.etn.com:82/Pool:/master/Debian_8.0/  Packages
     10.0.10-1 0
        500 http://obs.mbt.lab.etn.com:82/Pool:/master/Debian_8.0/  Packages
----
In the example above, 'iproute2-dev' was installed from OBS, but for MariaDB
a version provided by the distribution and newer than the one in OBS was
picked (as permitted by the "pinning" settings on that OS installation).


Manually fetching and installing packages built for Debian 8.0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For weird manual installations of our pre-built packages from OBS to
a system similar to Debian-8.0, consider downloading the current
package files into some temporary storage with this script:
----
#!/bin/sh
for DIR in \
    http://obs.roz53.lab.etn.com:82/Pool:/master/Debian_8.0/{amd64,all}/ \
; do
    wget -l1 -np -nd -N -c -r --remove-listing --accept=.deb "$DIR"
done
rm -f *.html*
----
Then you can install the packages with `dpkg -i filename.deb`

To fetch the sources used by OBS for a local rebuild, check this out:
----
#!/bin/sh
wget -l1 -np -nd -N -c -r --remove-listing \
    --accept='.dsc,.tar.xz,.tar.gz,.tar.bz2,.tgz,.tbz,.tbz2,.txz' \
    http://obs.roz53.lab.etn.com:82/Pool:/master/Debian_8.0/
----



Installation of the build dependencies from packages
----------------------------------------------------
While most of the prerequisite software needed to build or run the 42ity
core can be compiled more or less easily, the project servers provide
binary packages for most of the needed software which is either absent
or obsolete in original OS repositories.

When installing Debian 8 "testing" packages from OBS as well as from
the upstream OS itself, don't forget to update the metadata index first
(new builds may be available):
----
:; apt-get update
----

The compiler and interpreter suites
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The project is developed and built with GCC 4.8 or newer, and GNU make.
The GNU auto-tools (autoconf, automake) and m4 packages are needed to
regenerate the `configure` script. Further tools that may be required
might include the linker, binutils and similar common C/C++ development
packages.

Scripts in the project are developed in POSIX shells (should work with
`dash` and similar constrained shells as a `/bin/sh`); some shell scripts
do require a `/bin/bash` explicitly. Further script environments include
Python.

We hope to minimize run-time dependencies and thus the total installation
footprint and RAM usage, so other languages (Java, JS, PERL, PHP, Ruby,
etc.) should have an undeniable reason to be added and used in the project.


iproute (may be named iproute2)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 * Install iproute-dev development files, version "20120521" or newer
(later versions are linked with the Linux kernel release like "3.16"),
using your configured system repositories, i.e. with Debian APT:
----
:; apt-get install -y iproute2-dev

### Note that this should fetch the non-empty variant of the package
### from OBS with uotput like this:

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  iproute2-dev
0 upgraded, 1 newly installed, 0 to remove and 310 not upgraded.
Need to get 9,662 B of archives.
After this operation, 64.5 kB of additional disk space will be used.
Get:1 http://obs.roz53.lab.etn.com:82/Pool:/master/Debian_8.0/  iproute2-dev 3.16.0-0 [9,662 B]
Fetched 9,662 B in 5s (1,921 B/s)
Selecting previously unselected package iproute2-dev.
(Reading database ... 49826 files and directories currently installed.)
Preparing to unpack .../iproute2-dev_3.16.0-0_amd64.deb ...
Unpacking iproute2-dev (3.16.0-0) ...
Setting up iproute2-dev (3.16.0-0) ...
----
...or with Suse Zypper:
----
:; zypper install libnetlink-devel
----

[NOTE]
======
*TODO*: Which version of iproute *should* be used if building from source --
same release as the current Linux kernel on a given machine? Specific fixed
release? Latest release?)

It seems currently that recent Debian, unlike other distributions, renamed
the 'iproute' package to 'iproute2' and intentionally stripped away its
variant of the 'netlink' library (which our project currently uses, and
which was previously part of 'iproute-dev'). Thus a private build of the
'iproute' software from the sources is needed, and for supported systems
our project provides it in a packaged form. We only need the library (not
the Linux kernel-intimate components), and the API is deemed stable, so
any recent version of 'iproute' should suffice -- the project has been
tested with "20120521" (which is available in Debian wheezy) and that
version is known to work and suffice for us.
======

MariaDB
~~~~~~~
 * Install MariaDB development and product packages from OBS:
----
:; apt-get install -y libmariadbclient-dev libmariadbd-dev \
   mariadb-server mariadb-client mariadb-test

### Pick the version needed to match the rest of the database,
### the OBS serves 'mariadb-10.0' at the moment of this writing:
:; apt-get install -y mariadb-connect-engine'*'
----

 ** The 'mariadb-server' installer can ask for the database's 'root'
user new password (otherwise, you'll enter one per documentation
for setup of the database engine). You might benefit from saving it
into a standard configuration file used by the MySQL-related tools:
----
:; [ -s ~/.my.cnf ] || { \
	touch ~/.my.cnf; \
	chmod 600 ~/.my.cnf; \
	cat <<EOF> ~/.my.cnf
[client]
user=root
password=biosroot
EOF
----

 ** To enable the database service run (note: legacy-named 'mysql'):
----
:; systemctl enable mysql
:; systemctl restart mysql
----

libCIDR and CIDRcalc
~~~~~~~~~~~~~~~~~~~~
 * Install libcidr and related stuff:
----
:; apt-get install -y cidrcalc libcidr'*'
----
 ** Note that the package-name version is numbered ('libcidr0' and
'libcidr0-dev') at the moment of this writing, which is a bit
inconvenient in the long term documentation support. Things may be
different on your newer system.

TNTNET, TNTDB, CXXTOOLS and friends
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 * Install the TNT project parts:
----
:; apt-get install -y tntnet tntnet-runtime libcxxtools-dev \
   libtntdb-dev libtntnet-dev libtnt-dev libsasl2-dev
----
 ** Notes: the `cxxtools` or `libcxxtools` package may be absent or differently
named in other OS releases, and may need to be rebuilt from source completely.
 ** The non-"dev" base packages tend to have a version number embedded,
so it is a safer bet to request the "-dev" packages since we need them
anyway, and they will pull in their modern base packages as needed.

Network UPS Tools (NUT)
~~~~~~~~~~~~~~~~~~~~~~~
 * Install NUT project parts needed for compilation and linking:
----
:; apt-get install -y libupsclient-dev libnutclient-dev libnutscan-dev
----
 ** Note: specifically the 42ity core project needs 'libnutclient' version
0.27 or newer, which is provided either by recent source of the Network
UPS Tools, or by its very recent packaging (numbered 2.7.2-2 on OBS for
Debian).
 ** Note: These packages should be delivered from OBS at the moment because
they include patches not served by the upstream projects and distro builds.
 ** If you need more of NUT (actual programs -- drivers, server and client
daemons and utilities), consider adding these packages:
----
:; apt-get install -y nut-server nut-client nut
----
 ** Installation of 'nut-client' can complain on a `systemd`-controlled OS
like Debian 8, because it tries to start all daemons regardless of their
default configuration files which block startup on SysV-based systems.
As a result, the not-yet-confiugred daemons fail and the package may end
up not-installed. In this case review the errors following hints from the
packaging program and fix the offending files such as '/etc/nut/upsmon.conf'
(set 'MINSUPPLIES 0' where appropriate). The 'nut-server' package can
complain about no configured UPS-monitoring drivers, but this is not fatal.
See https://github.com/networkupstools/nut/issues/156 for more details.

libVariant
~~~~~~~~~~
 * Install libVariant:
----
:; apt-get install -y libvariant-dev
----

ZeroMQ and friends
~~~~~~~~~~~~~~~~~~
 * Install ZeroMQ and related project parts:
----
:; apt-get install -y libsodium-dev libzmq3-dev libczmq-dev gsl-bin
----

 * On a rare system used to develop the ZMQ-based protocols involved 
in the 42ity code project (i.e. almost never) one might need 'zproto'
and 'gsl' programs used to generate the C sources and headers further
used by an application project like 42ity. Both of these are available
on Git and are easy to compile followint their READMEs, but binary
packages are not generally needed and are not provided. If required,
refer to these projects directly for more details:
 ** https://github.com/zeromq/zproto
 ** https://github.com/imatix/gsl


Install run-time dependencies
-----------------------------
Some third-party projects are required at run-time (to validate or
actually use various components of the 42ity project) rather than
to just compile its code.

Group and user accounts
~~~~~~~~~~~~~~~~~~~~~~~
For added security, the daemons run as unprivileged users. For those few
known operations which require more privileges than are normally available,
elevation is configured as detailed below.

To create the local group and user accounts needed for 42ity, run the script
`tools/init-os-accounts.sh` as 'root':
----
:; CREATE_HOME=yes tools/init-os-accounts.sh
----

The script allows much customization intended for image construction and 
other advanced non-default applications; for now see the comments in its
code for details on further configuration.

NOTE: This script defines also a group named 'bios', to which the elevated
permissions below are assigned. If you build and test the project under
a different account, consider adding 'bios' as its secondary group:
----
:; adduser builder bios
----


The SUDO program
~~~~~~~~~~~~~~~~
The standard Unix `sudo` program is used to allow elevations for the 42ity
project accounts to execute many specific tasks.

* Make sure it is available:
----
:; apt-get install -y sudo
----

* Inspect and copy the sample $BIOS project configuration file into your
'sudo' installation which defines a number of sudo'able commands needed
for the 42ity project daemons at run-time:
----
:; cp docs/examples/sudoers.d/bios_00_base /etc/sudoers.d/
----
(Consult that file regarding the matters of what it sets, why and how).

* To run the automated testing as the 'bios' user or a member of the 'bios'
group in your OS, a few more commands are needed, defined in an optional
`sudo` configuration file. In this case, install it also:
----
:; cp docs/examples/sudoers.d/bios_01_citest /etc/sudoers.d/
----


SASL2 daemon and PAM
~~~~~~~~~~~~~~~~~~~~
The SASL daemon should be running in order for `test_web.sh` (and other cases
of the REST API) to work with the 42ity unprivileged user account(s).

 * Install the SASL daemon and additional PAM modules:
----
:; apt-get install -y sasl2-bin libpam-runtime libpam-modules-bin \
    libpam-modules libpam-cracklib
----

 * Enable it to auto-start by editing '/etc/default/saslauthd' and setting:
----
START=yes
----
there.

 * Ensure that it works now by running:
----
:; /etc/init.d/saslauthd restart
----

 * If SASL was installed after the initial creation of the 42ity project
user accounts, you need to explicitly add the 'bios' user account to the
'sasl' group so it has access to the SASL configuration and database (it
is needed for the client programs to work, and should be done as part of
the `init-os-accounts.sh` script's job in favorable conditions):
----
:; adduser bios sasl
----

 * Likewise, if you run the project daemons and execure automated testing
from an OS account different from 'bios', it should be added to the 'sasl'
group as well.

 * Also note that the path to the SASL MUX socket is (currently) compiled
into the binary code, so the building user must either have access to the
socket (`adduser builder sasl`) or an explicit '--with-saslauthd-mux=...'
option must be passed to `configure`.

 * Finally, add the PAM configuration file which defines and constrains
the 'bios' service, allowing only members of the 'bios' group to pass the
authentication for this service (i.e. when doing web-logins in REST API).
Two files need to be installed -- a PAM snippet and a list of groups
which this snippet references:
----
:; cp docs/examples/pam.d/bios /etc/pam.d/
:; cp docs/examples/security/bios_groups.conf /etc/security
----

NOTE: the default PAM configurations in the OS are somewhat relaxed by
allowing authentication into service names which are not actually defined
in the current OS security setup. This can apparently be forbidden with 
changes to the file '/etc/pam.d/other', but the matter is out of scope
of our project. For more details, see:

 * http://tldp.org/HOWTO/User-Authentication-HOWTO/x263.html

Beside the setup discussed above, no extra configuration for SASL and PAM
is needed on the target Debian 8 platform -- everything is pre-integrated
by the OS.



Configure and build the 42ity project sources
---------------------------------------------
(Getting the project sources is covered above in this document).

NOTE: While the section below goes into greater detail about managing the
build following a generic GNU auto-tools procedure, some typical building
management scenarios are automated in the `./tools/builder.sh` script (and
our `autogen.sh` invokes it if any command-line parameters are provided),
so that most activity can be done by typing one relatively short command.
Details on its usage can be seen in the help output of that script (see
`autogen.sh --help`), or in its documentation (see
'link:README-builder.html[docs/develop/README-builder.txt]').
----
# The common and simple case of a clean+config+quiet_parmake+verbose_seqmake
# for an in-tree build would be:
:; ./autogen.sh build

# Follow-up remakes with the same settings (during development) can be:
:; ./autogen.sh make
----

NOTE: We are obliged to stress that the *supported* method of compilation
is the common `./autogen.sh && configure && make` pattern (and with the
sequential `make` while we are at it), possibly followed by `make install`.
While our scripted extensions are made for developers' personal convenience
as well as simplification and unification of automated testing scripts, and
are provided with the project sources to help others, they are not currently
endorsed nor prioritized by the official support of the 42ity project.

If no `configure` script is available yet (or if it needs to be rebuilt):

 * Create or update the `configure` script as deemed necessary by automated
logic:
----
:; ./autogen.sh
----
 * ...or if you want to most certainly enforce an update of the `configure`
script:
----
:; autoreconf -fiv
----

If the `configure` script exists and is acceptable, you can proceed with
the compilation and installation:

 * Configure the build system, default options should generally suffice:
----
:; ./configure
----
 ** An "out-of-tree" or "relocated" build (useful for example to produce
    the binaries and other build results for different platforms from the
    same copy of the source codebase) can be triggered in a way similar
    to this:
----
:; mkdir build
:; cd build && \
   ../configure && \
   make
----
 * Build the sources:
----
:; make
----
 * Install the binaries (under a chosen destination root directory
   '/some/path', such as under a temporary filesystem or your homedir):
----
:; make DESTDIR=/some/path install
----


About parallel `make`
~~~~~~~~~~~~~~~~~~~~
As noted above, the sequential `make` routine, which does one task at
a time, in order dictated by dependencies, is the most reliable way to
ensure that the project is compiled and linked and installed correctly.
It is also the slowest way to achieve this goal, and with the development
cycle involving lots of rebuilds and other automated `make` jobs, the
time loss of everyone waiting for the build to complete adds up.

Even on a single-CPU computer, the mathematical load of the compiler
can generally be parallelized with blocking disk I/O to read and write
the files, so parallel jobs are often requested as two per processor.

However, the parallel `make` has its traps and pitfalls: it uses more CPU
concurrently, so individual jobs may take longer to complete (especially
when there are several jobs per CPU and/or swapping becomes involved),
and coupled with filesystems or their specific mount options that for
some reasons do not update metadata atomically, the dependency order can
get broken (things expected to exist are not yet seen); also there may
be some third-party tools that use the same filename for output and so
parallel invokations of that tool are bound to fail; the memory resources
are used more heavily so jobs can fail due to out-of-memory errors rather
than coding mistakes. In case of GNU make, not specifying the maximum
number of jobs to run causes unlimited parallelicity, so hundreds of
processes can be spawned and might fail due to forking issues (`ulimit`
constraints or OS resource depletion).

In short, while parallel `make` can fail with higher probability than a
sequential `make`, and its failure alone does not mean bad quality of
the built project's source code. In certain cases, its success also does
not guarantee correctness.

When the project becomes large enough, it may make sense to speed up
the builds using parallelization, followed by a pass of sequential
`make` to reduce the impact of intemittent failures, such as those
caused by filesystem lags, out-of-memory issues, or by breaking some
poorly specified dependency ordering. This approach benefits from both
the substantial speed improvement of a parallel `make` and the higher
dependability of the sequential one. Also, linear and less clobbered
output of the sequential `make` makes spotting and debugging of the
remaining fatal errors during development (including development of
the 'Makefile' itself) much easier.

 * For a quick solution, assuming a GNU make or compatible, in place
of the singular `make` commands, run this:
----
:; { make -j 4 -k ; make; }
----
 * Thus the one-liner to configure and compile the project becomes
similar to this:
----
:; ./autogen.sh && ./configure && { make -j 4 -k; make; } && make install
----



About quieter `make`
~~~~~~~~~~~~~~~~~~~~
Output of the `make` program is by default moderately verbose, including
the command lines executed by the process. This adds up to a lot of text,
in which the possible compilation errors are getting somewhat lost.
The `make V=0` output format which only prints a short summary of what
file is being processed with what program, half a line per source file,
is often much more convenient.

 *  A quieter build can be triggered for GNU `make` with 'V=0' parameter
    (which means that means only 'CC', 'CXX', 'CXXLD' lines will appear -
    which is good at least until you hit some build errors and need more
    verbosity), thus suggesting this command line:
----
:; { make V=0 -j 4 -k ; make V=0; }
----



Testing the build
-----------------

Testing REST API
~~~~~~~~~~~~~~~~
Before testing the REST API with `test_web.sh`, make sure to configure
the OS properly (including the user accounts and run-time dependencies
described above).

See also:

 * 'link:README-web-tests-generalInfo.html[src/web/README.txt]'
 * 'link:README-web-tests-ciAutomation.html[tests/CI/web/README.txt]'


Address, Leak and Thread Sanitizers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The 42ity project supports builds under GCC4.8+ with included Google
Address Sanitizer and Leak Sanitizer (ASan+LSan) functionality. These
are enabled by default if the currently produced build is a `DEBUG`
one (currently this is the default mode; generally it is manageable
by `./configure --enable-debug-build`. Likewise, one can produce a
"production" (non-DEBUG) build still instrumented with ASan+LSan
by explicit `./configure --enable-address-sanitizer`. This is not
a default mode for non-DEBUG builds because the produced code is a
little slower and bigger (approx 2x), and it is forced to crash when
errors are detected.

GCC4.8+ also supports a Thread Sanitizer, which allows to detect race
conditions. These builds are much bigger and slower (approx 15x), so
this mode is off by default and can be enabled only explicitly with 
`./configure --enable-thread-sanitizer`. Depending on the compiler,
this may be incompatible with Address Sanitizer, so ASan defaults
to disabled if TSan is requested. If both features are requested
explicitly, the `configure` script will try to check if they can
be enabled simultaneously, and should exit if not.

NOTE: Thread Sanitizer support in 42ity project is currently only
formally supported by the autoconf scaffolding. The produced files
do not run well, claiming some internal conflicts, maybe with some
of the other compiler options. This is a matter for some further
research work, if need arises.

It is recommended to run ASan-instrumented binaries along with an
`LD_PRELOAD=/path/to/libasan.so` so that the third-party dynamic
libraries used by our project would employ the same `new`/`delete`
and `malloc`/`free` ASan-instrumented routines to reduce false
positive detections. Otherwise it is possible, for example, that
an exit from a function due to an exception would trigger the
`__asan_handle_no_return` error handler and crash the program.


Generic notes on testing the 42ity project
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The 'Makefile' includes some targets for automated testing, such as:

 * `make check` -- this would trigger the compilation of all tests
   (including 'db-tests'), and run all tests except 'db-tests'.
 * `make test-db` -- would trigger the compilation only for database
   tests
 * `./test-db` -- would run the database tests

Note: as part of the project's automated validation, the `make distcheck`
must succeed in building and testing all targets, including a subdirectory
build. An `./autogen.sh distcheck` action exists to automate this into a
single-command routine to clean-configure-build+check.