diff --git a/.gitmodules b/.gitmodules deleted file mode 100644 index 1350b67..0000000 --- a/.gitmodules +++ /dev/null @@ -1,3 +0,0 @@ -[submodule "deep-contact-estimator"] - path = morpho_symm/datasets/contact_dataset/deep-contact-estimator - url = https://github.com/UMich-CURLY/deep-contact-estimator/ diff --git a/README.md b/README.md index e683888..f9ca17e 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,93 @@ -# Morphological Symmetries (MorphoSymm) in Locomoting Dynamical Systems +# Morphological Symmetries - _MorphoSymm_ +The Morphological Symmetries (MorphoSymm) repository offers a comprehensive toolkit for the detection, analysis, and exploitation of morphological symmetries in the modeling and control of robotic systems. These symmetries are a common characteristic of both biological and artificial locomotion systems, including legged, swimming, and flying animals, robots, and virtual characters. Essentially, a morphological symmetry is a transformation that connects two equivalent states under the governing dynamics, meaning two states that will evolve identically when influenced by equivalent moving forces. For example, consider these two motion trajectories from the mini-cheetah quadruped robot, which are related by a symmetry transformation. -Welcome to the Morphological Symmetries (MorphoSymm) repository! Here, you will find a comprehensive set of tools for the identification, study, and exploitation of morphological symmetries in locomoting dynamical systems. These symmetries are commonly found in a wide range of systems, including legged, swimming, and flying animals, robotic systems, and animated characters. As such, this repository aims to provide valuable resources for researchers and practitioners in the fields of Robotics, Computer Graphics, and Computational Biology. +

+ +

-This repository holds the code for the paper: [On discrete symmetries of robotic systems: A data-driven and group-theoretic analysis](https://scholar.google.it/scholar?q=on+discrete+symmetries+of+robotic+systems:+a+data-driven+and+group-theoretic+analysis&hl=en&as_sdt=0&as_vis=1&oi=scholart). -Accepted to *Robotics Science and Systems 2023 (RSS 2023)*. For reproducing the experiments of the paper, please see the master branch. +These symmetries carry significant implications. Notably, they offer a valuable geometric bias, since by modeling and controlling the dynamics of one state, we can effectively identify and control the dynamics of all its symmetric counterparts (see [our paper](https://danfoa.github.io/MorphoSymm/) for details). -#### Contents: + +

+ +

+ +## Group and representation theory + +To exploit these symmetries, we employ the language of group and representation theory, enabling us to represent these transformations via linear algebra. The set of morphological symmetries of a robotic system forms the system's [discrete (or finite) symmetry group](https://en.wikipedia.org/wiki/Discrete_group). For example, the mini-cheetah robot's symmetry group $\mathbb{G}$ contains $8$ distinct transformations generated by combinations of the transformations $\{e, g_s, g_t, g_f\}$: + +
+ Morphological Symmetries Mini-Cheetah quadruped robot, Caley Diagram, Discrete Symmetry Group + Morphological Symmetries Mini-Cheetah quadruped robot, State symmetry, Discrete Symmetry Group +
+ +As depicted above, each symmetry transformation, represented by a group element $g \in \mathbb{G}$, influences the system's configuration and any proprioceptive and exteroceptive measurements associated with the system's dynamic evolution. These measurements include aspects such as joint torques, center of mass momentum, ground reaction forces, and terrain elevation/orientation. To numerically transform these measurements, the appropriate group representation is required. This representation maps each symmetry transformation to an invertible matrix in the space where the measurement resides. For example, to transform the robot's joint-space generalized velocity coordinates $\bm{v}_{js} \in \mathcal{T}_{\bm{q}}\mathcal{Q}_{js} \subset \mathbb{R}^{12}$, the group representation $\rho{\mathcal{T}_{\bm{q}}\mathcal{Q}_{js}}: \mathbb{G} \rightarrow \mathbb{GL}(\mathbb{R}^{12})$ is needed. This defines the symmetry transformations in the $12$ dimensional space in which $\bm{v}_{js}$ evolves. In MorphoSymm, when you load a robotic system, you automatically receive its symmetry group and all the relevant group representations required to transform measurements used in robotics. For our example, this appears as follows: + +```python +from morpho_symm.utils.robot_utils import load_symmetric_system + +robot, G = load_symmetric_system(robot_name='mini_cheetah') +# Get joint-space position and velocity coordinates (q_js, v_js) | q_js ∈ Qjs, dq_js ∈ TqQjs +_, v_js = robot.get_joint_space_state() + +# Get the group representation on the space of joint-space generalized velocity coordinates +rep_TqQ_js = G.representations['TqQ_js'] +for g in G.elements: + # Transform the measurement + g_v_js = rep_TqQ_js(g) @ v_js # rep_TqQ_js(g) ∈ R^12x12 +``` + +For more details follow the [getting started guide](#getting-started) + +## Uses of morphological symmetries in robotics +There are three use cases of morphological symmetries supported by MorphoSymm. For details and tutorials see the getting started guide + + +
+
+

Data Augmentation

+

Any data recording of proprioceptive and exteroceptive measurements can be augmented using the appropriate group representations. This means that if you collect data from a symmetric robotic system, you can use the system's symmetries to analitically augment/multiply the recorded data.

+
+
+
+ Data augmentation robotics atlas bipedal morphological symmetries +
+ +
+
+

Equivariant/Invariant function approximation

+

+ Given the geometric nature of the data processed in robotics, most applications of machine learning to supervised, unsupervised, and reinforcement learning in robotics rely on the approximation of equivariant/invariant functions. To approximate these functions using deep learning, we should use equivariant/invariant neural networks that respect the symmetries of the learning problem. These networks feature enhanced sample efficiency and generalization, as demonstrated in our paper results). This repository relies on pytorch and the escnn library to offer equivariant and invariant neural network architectures. +

+
+
+
+ Equivariant function robotics, equivariant neural networks, contact detection +
+ +
+
+

Dynamics Harmonics Analysis

+

+ The symmetries of a robotic system allow us to decompose any motion as a superposition of simpler symetric synergistic motions, each evoling in a lower-dimensional space, called isotypic subspace. By projecting entire motion's trajectories into these spaces, we can characterize each motion as the superposition of lower-dimensional and synergistic motions. For further details on the use of abstract harmonic analysis in robotics see our paper Dynamics Harmonic Analysis of Robotic Systems: Application in Data-Driven Koopman Modeling +

+

+ + +

+
+
+
+ Data augmentation robotics atlas bipedal morphological symmetries +
+ +## Contents: - [Installation](#installation) - [Library of symmetric dynamical systems](#library-of-symmetric-dynamical-systems) - [Tutorial](#tutorial) @@ -23,6 +104,7 @@ git clone https://github.com/Danfoa/MorphoSymm.git cd MorphoSymm pip install -e . ``` + ## Library of symmetric dynamical systems The following is a non-exhaustive and expanding list of dynamical systems with Discrete Morphological Symmetries. Each example can be reproduced in a 3D interactive environment running: @@ -30,7 +112,7 @@ reproduced in a 3D interactive environment running: python morpho_symm.robot_symmetry_visualization.py robot= gui=True ``` This script functions as an introductory tutorial showing how we define the representations of Discrete Morphological Symmetries in order to perform symmetry transformations on the robot state, and proprioceptive and exteroceptive measurements. -### $\mathcal{G}=\mathcal{C}_2$: Reflection Symmetry +### $\mathbb{G}=\mathbb{C}_2$: Reflection Symmetry | Cassie | Atlas | Bolt | Baxter | |:----------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------:|:----------------------------------------------------------------------------:| | ![cassie](docs/static/animations/cassie-C2-symmetries_anim_static.gif) | ![atlas](https://user-images.githubusercontent.com/8356912/200183197-94242c57-bd9d-41cb-8a0b-509dceef5cb9.gif) | ![bolt](docs/static/animations/bolt-C2-symmetries_anim_static.gif) | ![baxter](docs/static/animations/baxter-C2-symmetries_anim_static.gif) | @@ -41,61 +123,62 @@ This script functions as an introductory tutorial showing how we define the repr | **UR-3** | **UR5** | **UR10** | KUKA-iiwa | | ![ur3](docs/static/animations/ur3-C2-symmetries_anim_static.gif) | ![ur5](docs/static/animations/ur5-C2-symmetries_anim_static.gif) | ![ur10](docs/static/animations/ur10-C2-symmetries_anim_static.gif) | ![iiwa](docs/static/animations/iiwa-C2-symmetries_anim_static.gif) | -### $\mathcal{G}=\mathcal{C}_n$: Symmetric Systems with Cyclic Group Symmetries +### $\mathbb{G}=\mathbb{C}_n$: Symmetric Systems with Cyclic Group Symmetries | [Trifinger](https://sites.google.com/view/trifinger/home-page)-C3 | |:-------------------------------------------------------------------------------:| | ![trifinger-edu](docs/static/animations/trifinger_edu-C3-symmetries_anim_static.gif) | -### $\mathcal{G}=\mathcal{K}_4$: Klein-Four Symmetry +### $\mathbb{G}=\mathcal{K}_4$: Klein-Four Symmetry | [Solo](https://open-dynamic-robot-initiative.github.io/)| HyQ | Mini-Cheetah | Anymal-C | Anymal-B | |:---------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------:|:------------------------------------------------------------------------:| | ![Solo-K4](docs/static/animations/solo-Klein4-symmetries_anim_static.gif) | ![hyq](docs/static/animations/hyq-Klein4-symmetries_anim_static.gif) | ![Mini-Cheetah-K4](docs/static/animations/mini_cheetah-Klein4-symmetries_anim_static.gif) | ![anymal_c](docs/static/animations/anymal_c-Klein4-symmetries_anim_static.gif) | ![anymal_c](docs/static/animations/anymal_b-Klein4-symmetries_anim_static.gif) | -### $\mathcal{G}=\mathcal{C}_2\times\mathcal{C}_2\times\mathcal{C}_2$: Regular cube symmetry +### $\mathbb{G}=\mathbb{C}_2\times\mathbb{C}_2\times\mathbb{C}_2$: Regular cube symmetry | [Solo](https://open-dynamic-robot-initiative.github.io/) | Mini-Cheetah | |:--------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------:| | ![solo-c2xc2xc2](docs/static/animations/solo-C2xC2xC2-symmetries_anim_static.gif) | ![bolt](docs/static/animations/mini_cheetah-C2xC2xC2-symmetries_anim_static.gif) | ### Addition of new dynamical systems to the library. -This repository aims at becoming a central tool in the exploitation of Morphological Symmetries in Robotics, Computer Graphics and Computational Biology. +This repository aims to become a central tool in the exploitation of Morphological Symmetries in Robotics, Computer Graphics and Computational Biology. Therefore, here we summarize the present and future efforts to enlarge the library of dynamical systems used in each of these fields. -## Tutorial +___________________________________________ +## Getting Started + ### Loading symmetric dynamical systems Each symmetric dynamical system has a configuration file in the folder `morpho_symm/cfg/supervised/robot`. To load one of these systems, simply use the function `load_symmetric_system` as follows: + ```python from morpho_symm.utils.robot_utils import load_symmetric_system -from hydra import compose, initialize - robot_name = 'solo' # or any of the robots in the library (see `/morpho_symm/cfg/robot`) -initialize(config_path="morpho_symm/cfg/supervised/robot", version_base='1.3') -robot_cfg = compose(config_name=f"{robot_name}.yaml") -# Load robot instance and its symmetry group -robot, G = load_symmetric_system(robot_cfg=robot_cfg) + +robot, G = load_symmetric_system(robot_name=robot_name) ``` The function returns: - `robot` an instance of the class [`PinBulletWrapper`](https://github.com/Danfoa/MorphoSymm/blob/devel/morpho_symm/robots/PinBulletWrapper.py) (utility class bridging [`pybullet`](https://pybullet.org/wordpress/) and [`pinocchio`](https://github.com/stack-of-tasks/pinocchio)). - `G`: the symmetry group of the system of instance [`escnn.group.Group`](https://quva-lab.github.io/escnn/api/escnn.group.html#group) -#### Getting and resetting the state of the system + +### Getting and resetting the state of the system -The system state is defined as $`(\mathbf{q}, \mathbf{v}) \;|\; \mathbf{q} \in \mathrm{Q}, \; \mathbf{v} \in T_{q}\mathrm{Q}`$, being $\mathrm{Q}$ the space of generalized position coordinates, and $T_{q}\mathrm{Q}$ the space of generalized velocity coordinates. Recall from the [paper convention](https://arxiv.org/abs/2302.10433) that the state configuration can be separated into base configuration and joint-space configuration $`\mathrm{Q} := \mathrm{E}_d \times \mathrm{Q}_J`$, such that $` -\mathbf{q} := +The system state is defined as $(\bm{q}, \bm{v}) \;|\; \bm{q} \in \mathrm{Q}, \; \bm{v} \in T_{q}\mathrm{Q}$, being $\mathrm{Q}$ the space of generalized position coordinates, and $T_{q}\mathrm{Q}$ the space of generalized velocity coordinates. Recall from the [paper convention](https://arxiv.org/abs/2302.10433) that the state configuration can be separated into base configuration and joint-space configuration $\mathrm{Q} := \mathbb{E}_d \times \mathcal{Q}_{js}$, such that $ +\bm{q} := \begin{bsmallmatrix} -\mathbf{X}_B \\ \mathbf{q}_{js} +\bm{X}_B \\ \bm{q}_{js} \end{bsmallmatrix} \begin{smallmatrix} -\in \; \mathbb{E}_d \\ \in \; \mathrm{Q}_J +\in \; \mathbb{E}_d \\ \in \; \mathcal{Q}_{js} \end{smallmatrix} -`$. Where, $\mathrm{E}\_d$ is the Euclidean space in which the system evolves, and $`\mathrm{Q}_J`$ is the joint-space position coordinates. To access these quantities in code we do: +$. +Where, $\mathbb{E}_d$ is the Euclidean space in which the system evolves, and $\mathcal{Q}_{js}$ is the joint-space position coordinates. To access these quantities in code we do: ```python # Get the state of the system q, v = robot.get_state() # q ∈ Q, v ∈ TqQ # Get the robot's base configuration XB ∈ Ed as a homogenous transformation matrix. XB = robot.get_base_configuration() -# Get joint space position and velocity coordinates (q_js, v_js) | q_js ∈ QJ, dq_js ∈ TqQJ +# Get joint-space position and velocity coordinates (q_js, v_js) | q_js ∈ Qjs, dq_js ∈ TqQjs q_js, v_js = robot.get_joint_space_state() ``` @@ -106,24 +189,28 @@ q_js, v_js = robot.get_joint_space_state() The system's symmetry group instance `G` contains the group representations required to transform most proprioceptive and exteroceptive measurements (e.g., joint positions/velocities/accelerations, joint forces/torques, contact locations & forces, linear and angular velocities, terrain heightmaps, depthmaps, etc). These are: - - $`\rho_{\mathbb{E}_d}: \mathcal{G} \rightarrow \mathbb{E}(d)`$: Representation mapping symmetry actions to elements of the Euclidean group $\mathbb{E}(d)$. Essentially homogenous transformation matrices describing a rotation/reflection and translation of space (Euclidean isometry) in $d$ dimensions. - - $`\rho_{\mathrm{Q}_J}: \mathcal{G} \rightarrow \mathcal{GL}(\mathrm{Q}_J)`$ and $`\rho_{T_q\mathrm{Q}_J}: \mathcal{G} \rightarrow \mathcal{GL}(T_q\mathrm{Q}_J)`$: Representations mapping symmetry actions to transformation matrices of joint-space position $`\mathrm{Q}_J`$ and velocity $`T_{q}\mathrm{Q}_J`$ coordinates. - - $`\rho_{\mathrm{O}_d}: \mathcal{G} \rightarrow \mathcal{\mathrm{O}_d}`$: Representation mapping symmetry actions to elements of the Orthogonal group $\mathrm{O}(d)$. Essentially rotation and reflection matrices in $d$ dimensions. - - $`\rho_{reg}: \mathcal{G} \rightarrow \mathbb{R}^{|\mathcal{G}|}`$: The group regular representation. - - $`\hat{\rho}_{i}: \mathcal{G} \rightarrow \mathcal{GL}(|\hat{\rho}_{i}|)`$: Each of the group irreducible representations. + - $\rho_{\mathbb{E}_d}: \mathbb{G} \rightarrow \mathbb{E}(d)$: Representation mapping symmetry actions to elements of the Euclidean group $\mathbb{E}(d)$. Essentially, **homogenous transformation matrices** describing a rotation/reflection and translation of space (Euclidean isometry) in $d$ dimensions. + + - $\rho_{\mathcal{Q}_{js}}: \mathbb{G} \rightarrow \mathcal{GL}(\mathcal{Q}_{js})$ and $\rho_{T_q\mathcal{Q}_{js}}: \mathbb{G} \rightarrow \mathcal{GL}(T_q\mathcal{Q}_{js})$: Representations mapping symmetry actions to transformation matrices of joint-space position $\mathcal{Q}_{js}$ and velocity $T_{q}\mathcal{Q}_{js}$ coordinates. + + - $\rho_{\mathbb{R}^d}: \mathbb{G} \rightarrow \mathcal{GL}(\mathcal{\mathbb{R}^d})$: Representation mapping symmetry actions to elements to invertible transformations in $d$ dimensions. In practise, **rotation** and **reflection** matrices . + + - $\rho_{reg}: \mathbb{G} \rightarrow \mathbb{R}^{|\mathbb{G}|}$: The group regular representation. + + - $\hat{\rho}_{i}: \mathbb{G} \rightarrow \mathcal{GL}(|\hat{\rho}_{i}|)$: Each of the group irreducible representations. In practice, products and additions of these representations are enough to obtain the representations of any proprioceptive and exteroceptive measurement. For instance, we can use these representations to transform elements of: -- $\mathrm{E}\_d$: The Euclidean space (of $d$ dimensions) in which the system evolves. +- $\mathbb{E}_d$: The Euclidean space (of $d$ dimensions) in which the system evolves. - The representation $`\rho_{\mathbb{E}_d}`$ can be used to transform: - - The system base configuration $\mathbf{X}\_B$. If you want to obtain the set of symmetric base configurations $` - \{{g \cdot \mathbf{X}_B:= \rho_{\mathbb{E}_d}(g) \; \mathbf{X}_B \; \rho_{\mathbb{E}_d}(g)^{-1} \;|\; \forall\; g \in \mathcal{G}}\}`$ [(1)](https://arxiv.org/abs/2302.1043), you can do it with: + The representation $\rho_{\mathbb{E}_d}$ can be used to transform: + - The system base configuration $\bm{X}\_B$. If you want to obtain the set of symmetric base configurations $ + \{{g \cdot \bm{X}_B:= \rho_{\mathbb{E}_d}(g) \; \bm{X}_B \; \rho_{\mathbb{E}_d}(g)^{-1} \;|\; \forall\; g \in \mathbb{G}}\}$ [(1)](https://arxiv.org/abs/2302.1043), you can do it with: ```python - rep_Ed = G.representations['Ed'] # rep_Ed(g) is a homogenous transformation matrix ∈ R^(d+1)x(d+1) - # The orbit of the base configuration XB is a map from group elements g ∈ G to base configurations g·XB ∈ Ed - orbit_X_B = {g: rep_Ed(g) @ XB @ rep_Ed(g).T for g in G.elements()} + rep_Ed = G.representations['Ed'] # rep_Ed(g) is a homogenous transformation matrix ∈ R^(d+1)x(d+1) + # The orbit of the base configuration XB is a map from group elements g ∈ G to base configurations g·XB ∈ Ed + orbit_X_B = {g: rep_Ed(g) @ XB @ rep_Ed(g).T for g in G.elements()} ``` - - Points in $\mathbb{R}^d$. These can represent contact locations, object/body positions, etc. To obtain the set of symmetric points you can do it with: @@ -134,27 +221,27 @@ In practice, products and additions of these representations are enough to obtai orbit_r = {g: (rep_Ed(g) @ r_hom)[:3] for g in G.elements} ``` -- $`\mathrm{Q}_J`$ and $`T_{q}\mathrm{Q}_J`$: The spaces of joint-space position $`\mathrm{Q}_J`$ and velocity $`T_{q}\mathrm{Q}_J`$ generalized coordinates. +- $\mathcal{Q}_{js}$ and $T_{q}\mathcal{Q}_{js}$: The spaces of joint-space position $\mathcal{Q}_{js}$ and velocity $T_{q}\mathcal{Q}_{js}$ generalized coordinates. - To transform joint-space states $` - (\mathbf{q}_{js}, \mathbf{v}_{js}) \;|\; \mathbf{q}_{js} \in \mathrm{Q}_J, \;\mathbf{v}_{js} \in T_{q}\mathrm{Q}_J - `$ we use the representations $`\rho_{\mathrm{Q}_J}`$ and $`\rho_{T_q\mathrm{Q}_J}`$. For instance, for a given joint-space configuration $` (\mathbf{q}_{js}, \mathbf{v}_{js})`$, the set of symmetric joint-space configurations (orbit) is given by $` + To transform joint-space states $ + (\bm{q}_{js}, \bm{v}_{js}) \;|\; \bm{q}_{js} \in \mathcal{Q}_{js}, \;\bm{v}_{js} \in T_{q}\mathcal{Q}_{js} + $ we use the representations $\rho_{\mathcal{Q}_{js}}$ and $\rho_{T_q\mathcal{Q}_{js}}$. For instance, for a given joint-space configuration $ (\bm{q}_{js}, \bm{v}_{js})$, the set of symmetric joint-space configurations (orbit) is given by $ \{ - (\rho_{\mathrm{Q}_J}(g) \; \mathbf{q}_{js}, \;\rho_{T_q\mathrm{Q}_J}(g) \;\mathbf{v}_{js}) \; | \; \forall \; g \in \mathcal{G} + (\rho_{\mathcal{Q}_{js}}(g) \; \bm{q}_{js}, \;\rho_{T_q\mathcal{Q}_{js}}(g) \;\bm{v}_{js}) \; | \; \forall \; g \in \mathbb{G} \} - `$. Equivalently, in code we can do: + $. Equivalently, in code we can do: ```python - rep_QJ = G.representations['Q_js'] - rep_TqJ = G.representations['TqQ_js'] - # Get joint space position and velocity coordinates (q_js, v_js) | q_js ∈ QJ, dq_js ∈ TqQJ + rep_Qjs = G.representations['Q_js'] + rep_TQjs = G.representations['TqQ_js'] + # Get joint-space position and velocity coordinates (q_js, v_js) | q_js ∈ Qjs, dq_js ∈ TqQjs q_js, v_js = robot.get_joint_space_state() # The joint-space state orbit is a map from group elements g ∈ G to joint-space states (g·q_js, g·v_js) - orbit_js_state = {g: (rep_QJ(g) @ q_js, rep_TqJ(g) @ v_js) for g in G.elements} + orbit_js_state = {g: (rep_Qjs(g) @ q_js, rep_TQjs(g) @ v_js) for g in G.elements} ``` -- Vectors, Pseudo-vectors in $\mathrm{E}\_d$. +- Vectors, Pseudo-vectors in $\mathbb{E}_d$. - Vector measurements can represent linear velocities, forces, linear accelerations, etc. While [pseudo-vectors](https://en.wikipedia.org/wiki/Pseudovector#:~:text=In%20physics%20and%20mathematics%2C%20a,of%20the%20space%20is%20changed) (or axial-vectors) can represent angular velocities, angular accelerations, etc. To obtain symmetric measurements we transform vectors with $` - \rho_{\mathrm{O}_d}`$. Likewise, to obtain symmetric pseudo-vectors we use $`\rho_{\mathrm{O}_{d,pseudo}}(g) := |\rho_{\mathrm{O}_d}(g)| \rho_{\mathrm{O}_d}(g) \; | \; g \in \mathcal{G}`$. Equivalently, in code we can do: + Vector measurements can represent linear velocities, forces, linear accelerations, etc. While [pseudo-vectors](https://en.wikipedia.org/wiki/Pseudovector#:~:text=In%20physics%20and%20mathematics%2C%20a,of%20the%20space%20is%20changed) (or axial-vectors) can represent angular velocities, angular accelerations, etc. To obtain symmetric measurements we transform vectors with $ + \rho_{\mathbb{R}^d}$. Likewise, to obtain symmetric pseudo-vectors we use $\rho_{\mathrm{O}_{d,pseudo}}(g) := |\rho_{\mathbb{R}^d}(g)| \rho_{\mathbb{R}^d}(g) \; | \; g \in \mathbb{G}$. Equivalently, in code we can do: ```python rep_Rd = G.representations['Rd'] # rep_Rd(g) is an orthogonal matrix ∈ R^dxd rep_Rd_pseudo = G.representations['Od_pseudo'] @@ -173,11 +260,11 @@ In practice, products and additions of these representations are enough to obtai > _In this section we briefly show how to construct G-equivariant multi-layer perceptron E-MLP architectures. Future tutorials will cover G-equivariant CNNs and GNNs._ -Let's consider the example from [(1)](https://arxiv.org/abs/2302.1043) of approximating the Center of Mass (CoM) momentum from the joint-space state measurements. That is we want to use a neural network to approximate the function $` -\mathbf{y} = f(\mathbf{x}) = f(\mathbf{q}_{js}, \mathbf{v}_{js}) -`$ for a robot evolving in 3 dimensions, say the robot `solo`. Defining $`\mathbf{y} := [\mathbf{l}, \mathbf{k}]^T \subseteq \mathbb{R}^6`$ as the CoM momentum linear $`\mathbf{l} \in \mathbb{R}^3`$ and angular $`\mathbf{k} \in \mathbb{R}^3`$ momentum, and $` -\mathbf{x} = (\mathbf{q}_{js}, \mathbf{v}_{js}) \;|\; \mathbf{q}_{js} \in \mathrm{Q}_J, \;\mathbf{v}_{js} \in T_{q}\mathrm{Q}_J -`$ as the joint-space position and velocity generalized coordinates. +Let's consider the example from [(1)](https://arxiv.org/abs/2302.1043) of approximating the Center of Mass (CoM) momentum from the joint-space state measurements. That is we want to use a neural network to approximate the function $ +\bm{y} = f(\bm{x}) = f(\bm{q}_{js}, \bm{v}_{js}) +$ for a robot evolving in 3 dimensions, say the robot `solo`. Defining $\bm{y} := [\bm{l}, \bm{k}]^T \subseteq \mathbb{R}^6$ as the CoM momentum linear $\bm{l} \in \mathbb{R}^3$ and angular $\bm{k} \in \mathbb{R}^3$ momentum, and $ +\bm{x} = (\bm{q}_{js}, \bm{v}_{js}) \;|\; \bm{q}_{js} \in \mathcal{Q}_{js}, \;\bm{v}_{js} \in T_{q}\mathcal{Q}_{js} +$ as the joint-space position and velocity generalized coordinates. For this example, you can build an equivariant MLP as follows: @@ -198,8 +285,8 @@ robot, G = load_symmetric_system(robot_name=robot_name) # We use ESCNN to handle the group/representation-theoretic concepts and for the construction of equivariant neural networks. gspace = escnn.gspaces.no_base_space(G) # Get the relevant group representations. -rep_QJ = G.representations["Q_js"] # Used to transform joint-space position coordinates q_js ∈ Q_js -rep_TqQJ = G.representations["TqQ_js"] # Used to transform joint-space velocity coordinates v_js ∈ TqQ_js +rep_Qjs = G.representations["Q_js"] # Used to transform joint-space position coordinates q_js ∈ Q_js +rep_TqQjs = G.representations["TqQ_js"] # Used to transform joint-space velocity coordinates v_js ∈ TqQ_js rep_R3 = G.representations["Od"] # Used to transform the linear momentum l ∈ R3 rep_R3_pseudo = G.representations["Od_pseudo"] # Used to transform the angular momentum k ∈ R3 rep_E3 = G.representations["Ed"] # Homogenous transformation matrix @@ -212,7 +299,7 @@ e, gs, gr, gt = G.elements A = rep_R3(gs) # 3x3 matrix. # Define the input and output FieldTypes using the representations of each geometric object. # Representation of x := [q, v, base_vel, base_ang_vel] ∈ Q_js x TqQ_js x R3 x R3 => ρ_X_js(g) := ρ_Q_js(g) ⊕ ρ_TqQ_js(g) | g ∈ G -in_field_type = FieldType(gspace, [rep_QJ, rep_TqQJ, rep_R3, rep_R3_pseudo]) +in_field_type = FieldType(gspace, [rep_Qjs, rep_TqQjs, rep_R3, rep_R3_pseudo]) # Representation of y := [l, k] ∈ R3 x R3 => ρ_Y_js(g) := ρ_O3(g) ⊕ ρ_O3pseudo(g) | g ∈ G out_field_type = FieldType(gspace, [rep_R3, rep_R3_pseudo]) @@ -235,7 +322,9 @@ In summary, to construct a G-equivariant architecture you need to: 3. Define the input and output `FieldType` instances using the representations of each geometric object. ## How to cite us? -If you find this repository or the [paper](https://scholar.google.it/scholar?q=on+discrete+symmetries+of+robotic+systems:+a+data-driven+and+group-theoretic+analysis&hl=en&as_sdt=0&as_vis=1&oi=scholart) useful, please cite us as: +If you find this repository or any our our papers relevant please cite us as: + +### [On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis](https://danfoa.github.io/MorphoSymm/) ``` @INPROCEEDINGS{Ordonez-Apraez-RSS-23, AUTHOR = {Daniel F Ordonez-Apraez AND Martin, Mario AND Antonio Agudo AND Francesc Moreno}, @@ -247,6 +336,16 @@ If you find this repository or the [paper](https://scholar.google.it/scholar?q=o DOI = {10.15607/RSS.2023.XIX.053} } ``` +### [Dynamics Harmonic Analysis of Robotic Systems: Application in Data-Driven Koopman Modeling](https://danfoa.github.io/DynamicsHarmonicsAnalysis/) + +``` +@article{ordonez2023dynamics, + title={Dynamics Harmonic Analysis of Robotic Systems: Application in Data-Driven Koopman Modelling}, + author={Ordo{\~n}ez-Apraez, Daniel and Kostic, Vladimir and Turrisi, Giulio and Novelli, Pietro and Mastalli, Carlos and Semini, Claudio and Pontil, Massimiliano}, + journal={arXiv preprint arXiv:2312.07457}, + year={2023} +} +``` ## Contributing @@ -268,18 +367,5 @@ In summary, we support: - [x] Visualization of robot Discrete Morphological Symmetries in `pybullet`. Other physics simulators and visualization tools will come soon. - [x] Utility functions to define symmetry representations of proprioceptive and exteroceptive measurements. - [x] Construction of equivariant neural networks processing proprioceptive and exteroceptive measurements, using the `escnn` library. +- [x] Use abstract harmonic analysis to decompose motion trajectories and recorded proprioceptive and exteroceptive measurements into isotypic components. -#### Computer Graphics - -The field of computer graphics does not widely employs URDF descriptions for the definition of dynamical systems. Although covering different description standards is within the goal of this repository, -for now, our main objective is: - -- [ ] Integration of [STAR](https://star.is.tue.mpg.de/) model in the library, to automatically process sequence of data and obtain symmetric sequences. - By defining the sagittal symmetry of all the model parameters. This will enable the use of DMSs in all applications of human motion prediction, shape reconstruction, etc. - If you are interested in contributing to this effort, please contact us. -- [ ] Integration of Motion Capture (MoCap) data formats. Including `.fbx`, `.bvh`, and `.c3d`. - -#### Computational Biology - -For computational biology and bio-mechanics, we believe the most relevant format to provide support for is: -- [ ] Coordinate 3D files `.c3d` format. diff --git a/docs/index.html b/docs/index.html index e526b1c..6bfbc76 100644 --- a/docs/index.html +++ b/docs/index.html @@ -19,10 +19,10 @@ + content="On discrete symmetries of robotic systems: A data-driven and group-theoretic analysis, Robotics Science and Systems RSS-2023. This project studies Morphological Symmetries in Robotic Systems, these are Discrete (or finite) symmetry groups of the state space of a robotic and dynamical systems These symmetries are usefull for data-augmentation and equivariant function approximation. Morphological Symmetries (MorphoSymm, Morpho Symm, Morphosymm)"> + content="On discrete symmetries of robotic systems: A data-driven and group-theoretic analysis, Robotics Science and Systems RSS-2023. This project studies Morphological Symmetries in Robotic Systems, these are Discrete (or finite) symmetry groups of the state space of a robotic and dynamical systems These symmetries are usefull for data-augmentation and equivariant function approximation. Morphological Symmetries (MorphoSymm, Morpho Symm, Morphosymm)"/> @@ -32,7 +32,7 @@ + content="On discrete symmetries of robotic systems: A data-driven and group-theoretic analysis, Robotics Science and Systems RSS-2023. This project studies Morphological Symmetries in Robotic Systems, these are Discrete (or finite) symmetry groups of the state space of a robotic and dynamical systems These symmetries are usefull for data-augmentation and equivariant function approximation. Morphological Symmetries (MorphoSymm, Morpho Symm, Morphosymm)"> @@ -40,7 +40,7 @@ + content="discrete morphological symmetries, robotics, computer graphics, symmetries, equivariance, morphosymm, MorphoSymm, morpho symm, dynamics harmonics analysis, harmonic analysis, locomotion, control, equivariant neural networks, invariant neural networks, group theory"> @@ -200,8 +200,9 @@

Symmetries of robotic systems

- - Morphological Symmetries Mini-Cheetah quadruped robot, Caley Diagram, Discrete Symmetry Group + Morphological Symmetries Mini-Cheetah quadruped robot, State symmetry, Discrete Symmetry Group
@@ -213,7 +214,7 @@

Symmetries of robotic systems

@@ -257,8 +258,10 @@

Caley Diagrams of Morphological Symmetries