dart is a C++ library for tracking arbitrary articulated models with an RGB-D camera. It achieves real-time performance with the aid of a highly parallel CUDA implementation and state-of-the-art GPUs.
CUDA: https://developer.nvidia.com/cuda-zone
Eigen 3: sudo apt-get install libeigen3-dev
GNU libmatheval: sudo apt-get install libmatheval-dev
libjpeg: sudo apt-get install libjpeg-dev
libpng: sudo apt-get install libpng-dev
tinyxml: sudo apt-get install libtinyxml-dev
Pangolin [GUI for the examples]: https://github.com/stevenlovegrove/Pangolin
OpenNI [for PrimeSense sensors]: https://github.com/OpenNI/OpenNI2 or http://structure.io/openni
DepthSense SDK [for Intel sensor]: sudo apt-get install depthsensesdk
Open Asset Import Library: sudo apt-get install libassimp-dev [for mesh models]
gtest [for testing]: sudo apt-get install libgtest-dev; cd /usr/src/gtest; sudo mkdir build; cd build; sudo cmake ..; sudo make; sudo mv libgtest* /usr/lib/;
cd [dart directory] mkdir build cd build cmake .. make
An example demonstrating the use of DART to track robot hands manipulating objects, including the depth and color video, can be downloaded from here.
-
OpenGL context: dart is a very visual library and therefore assumes that it will be used in collaboration with a GUI tool. If an instance of dart::Tracker is instantiated with no active OpenGL context, it will not work. If you are not using the library with a GUI, you will have to create an OpenGL context (e.g. by using glutCreateWindow to create a 1x1 window).
-
Frames vs. Joints vs. SDFs: These are three separate but related ways in which parts of a DART model can be referenced. A frame is a frame of reference in the kinematic chain, a joint is a connection between two frames with a single degree of freedom, and a signed distance function (SDF) implicitly stores all geometry attached to a single frame. Every model has at least one frame (the root) but need not have any joints or signed distance functions. Because loops are not allowed, a model with N joints will have N+1 frames (and N+6 degrees of freedom). The number of SDFs is at most equal to the number of frames, but may be less if there are frames with no geometry attached to them. Functions in the Model class and subclasses that require indexing part of the model will indicate in the parameter name whether the index is by joint, by frame, or by SDF.
DART models are stored as XML files which define the kinematic and geometric structure, optionally reference other mesh files to further describe the geometry. All models open with the "model" tag which has a single attribute, "version" describing the version of the DART XML format (current 1), like so:
<model version ="1">
[model here]
</model>
The model can then optionally specify a number of parameters using the "param" tag, with attributes "name" (string) and "value" (floating point), like so:
<param name="armLength" value="1.5"/>
These parameters can then be referenced when defining sizes, positions, or orientations, as described below. Parameters are useful if the same value appears multiple times in your specification (as is often the case) or if you would like to set the parameter values programatically.
After defining parameters, the model may contain a number of hierachically nested "frame" and "geom" tags, which specify new rigid body frames of reference or geometric objects, respectively. The "frame" tag requires four attributes, "jointName" (string), "jointType" (currently accepts either "rotational" or "prismatic", "jointMin" (floating point), and "jointMax" (floating point), the last two of which define the joint limits. Additionally, the frame tag requires three nested tags, "position", "orientation", and "axis", each of which require three floating point attributes, "x", "y", and "z". An example might look like this:
<frame jointName="leftElbow" jointType="rotational" jointMin="0" jointMax="3.1416">
<position x="0" y="0" z="1.5" />
<orientation x="0" y="0" z="1.5708" />
<axis x="1" y="0" z="0"/>
[frame children here]
</frame>
This snippet defines a new frame of reference relative to its parent (the XML node directly above it in the hierarchy, or the root if the parent is the "model" tag). The transform from this frame of reference to the world is given by:
T_w,f = T_w,pTransR_zR_yR_x*R_axis(theta)
where T_w,p gives the transform from the parent to the world, Trans is a translation-only transform given by the "position" tag, R_z, R_y, and R_x are rotations about the z, y, and x axes (i.e. Euler angles) given by the corresponding entries in the "orientation" tag, and R_axis is a rotation by theta around the axis defined by the "axis" tag, with theta being given by the articulated pose of the model.
Finally, geometry can be rigidly attached to any frame of reference in the model by nesting a "geom" tag within a "frame" tag (or within the "model" tag for root geometry). The geometry tag requires 13 attributes: "type" (currently accepts "sphere","cylinder","cube", or "mesh"), "sx", "sy", and "sz", which define the scaling of the geometry, "tx", "ty", and "tz", which define the translation of the geometry root relative to the rigid body frame of reference, "rx", "ry" and "rz", which define the orientation relative to the rigid body frame of reference (also in Euler angles, as with the "frame" tag), and "red", "green" and "blue", which define the geometry color, which is not used for tracking but will affect how the model is rendered for debugging purposes. If "type" is set to "mesh", there is one final attribute, "meshFile", which gives the location of the mesh file, relative to the location of the XML file.