DeepSphere: a graph-based spherical CNN

Michaël Defferrard, Martino Milani, Frédérick Gusset, Nathanaël Perraudin

The code in this repository implements a generalization of Convolutional Neural Networks (CNNs) to the sphere. We here model the discretized sphere as a graph of connected pixels. The resulting convolution is efficient (especially when data doesn't span the whole sphere) and mostly equivariant to rotation (small distortions are due to the non-existence of a regular sampling of the sphere). The pooling strategy exploits hierarchical pixelizations of the sphere to analyze the data at multiple scales. The performance of DeepSphere is demonstrated on four problems: the recognition of 3D objects, the discrimination of cosmological models, the segmentation of extreme events in climate simulations, and the identification of trends in historical weather.

Resources

Code:

• deepsphere-cosmo-tf1: original repository, implemented in TensorFlow v1.
  Use to reproduce arxiv:1810.12186.
• deepsphere-cosmo-tf2: reimplementation in TFv2.
  Use for new developments in TensorFlow targeting HEALPix, including generative models.
• deepsphere-tf1: extended to other samplings and experiments, implemented in TFv1.
  Use to reproduce arxiv:2012.15000.
• deepsphere-pytorch: reimplementation in PyTorch.
  Use for new developments in PyTorch.

Papers:

• DeepSphere: Efficient spherical CNN with HEALPix sampling for cosmological applications, 2018.
  [paper, blog, slides]
• DeepSphere: towards an equivariant graph-based spherical CNN, 2019.
  [paper, poster]
• DeepSphere: a graph-based spherical CNN, 2020.
  [paper, slides, video]

Installation

For a local installation, follow the below instructions.

1. Clone this repository.

   git clone https://github.com/deepsphere/deepsphere-tf1.git
   cd deepsphere-tf1

2. Install the dependencies.

   conda env create -f environment.yml

Reproducing the results of the paper

Each experiment has a folder in data with scripts to download the data, and a folder in experiments with scripts to preprocess and reproduce the experiment. Additionally, there is a notebook to explore the data and hyper-parameters.

The steps are essentially the same for each experiment (climate, cosmo, ghcn, modelnet40, shrec17), as follows:

python data/{experiment_name}/download_dataset.py
python experiments/{experiment_name}/preprocessing.py
python experiments/{experiment_name}/run_experiment.py

Additional instructions are contained in the data and experiments READMEs.

Notebooks

The below notebooks contain some experiments.

1. From sampled spheres to graphs. Experiments showing how to build a graph from a sampled sphere.

   Note: The tested and recommended implementations are now available in the PyGSP.

2. Irregular pooling. Experiments with pooling on non-uniformly and partially sampled spheres.

Explanatory notebooks about the method are available here and there.

License & co

The content of this repository is released under the terms of the MIT license.

The code, based on the original TensorFlow implementation of DeepSphere for cosmology, was mostly developed during the master theses of Frédérick Gusset (code, report, slides) and Martino Milani (code, report, slides, poster).

Please consider citing our papers if you find this repository useful.

@inproceedings{deepsphere_iclr,
  title = {{DeepSphere}: a graph-based spherical {CNN}},
  author = {Defferrard, Michaël and Milani, Martino and Gusset, Frédérick and Perraudin, Nathanaël},
  booktitle = {International Conference on Learning Representations (ICLR)},
  year = {2020},
  url = {https://openreview.net/forum?id=B1e3OlStPB},
}

@inproceedings{deepsphere_rlgm,
  title = {{DeepSphere}: towards an equivariant graph-based spherical {CNN}},
  author = {Defferrard, Micha\"el and Perraudin, Nathana\"el and Kacprzak, Tomasz and Sgier, Raphael},
  booktitle = {ICLR Workshop on Representation Learning on Graphs and Manifolds},
  year = {2019},
  archiveprefix = {arXiv},
  eprint = {1904.05146},
  url = {https://arxiv.org/abs/1904.05146},
}