Top-down supervised identity prediction (#476)

* Add track indices to instance cropper

* Add class vector generator

* Split class vectors correctly in instance cropper

* Move head output layer construction to heads module
- Heads now subclass a base Head class
- Naming doesn't include _0 anymore since we don't have any multi-output
  models for now.
- Better input validation in Model.from_config constructor
- Add loss weight to all heads in config
- Test coverage for heads and (minimally for) model

* Add topdown config, head and model
- Rename multiclass to multiclass_bottomup

* Add trainer

* Data pipeline

* Apply black to 'sleap' and 'tests' (#465)

Co-authored-by: Arie Matsliah <[email protected]>

* Fix model creation and add pooling param to head

* Symmetry-aware flip augmentation (#455)

* Implement symmetry-aware instance reflection

* Fix symmetries sometimes not being returned uniquely

* Add fancier indexing to instances

* Add random flipping transformer

* Fix failing linux test
- Make sure indices are all cast to int32

* Add vertical flip

* Add flip augmentation to config, GUI and pipeline builders

* Update profiles with default fields

Co-authored-by: ariematsliah-princeton <[email protected]>

* Multi-size videos in data pipelines (#440)

Add support for variable size videos within the same dataset by matching their size with padding or resizing

Co-authored-by: Arie Matsliah <[email protected]>

* Type check + Lint in CI (#470)

* Try lint and typecheck in CI workflow

* update

* nit

* continue on MyPy errors

* test

* correct

* correct

* correct

Co-authored-by: Arie Matsliah <[email protected]>

* Rename predictors for consistency with inference layers
- TopdownPredictor -> TopDownPredictor
- BottomupPredictor -> BottomUpPredictor

* Create PULL_REQUEST_TEMPLATE.md

* Update authors list (#471)

Co-authored-by: Arie Matsliah <[email protected]>

* Add CLA (#473)

* Add CLA

* update links

Co-authored-by: Arie Matsliah <[email protected]>

* Update PULL_REQUEST_TEMPLATE.md

* Miscellaneous QOL (#467)

Pre-1.1.0 update features (changelist in #467)

* Bump pre-release version

* Add back load_model that got lost in the merge
- Add detection of bottomup and topdown multi-class model loading

* Fix more missing things post-merge

* Fix lint

* Fix training from config

* Add inference

* Tweak describe tensor to accept nested tuples/dicts

* Lint

* Fix test

* Lint

* Fix load video dataset arg

* Fix inference

* Fix evals

* Add BU MC to evals

* Remove batch norm from TD MC head

* Add option to disable batch norm in pretrained models

* Add track matching when merging labels

* Don't error when training finishes with no inference target

Co-authored-by: ariematsliah-princeton <[email protected]>
Co-authored-by: Arie Matsliah <[email protected]>

.conda/bld.bat

@@ -38,6 +38,7 @@ pip install jsmin
 pip install seaborn
 pip install pykalman==0.9.5
 pip install segmentation-models==1.0.1
+pip install rich==9.10.0
 
 rem # Use and update environment.yml call to install pip dependencies. This is slick.
 rem # While environment.yml contains the non pip dependencies, the only thing left

.conda/build.sh

@@ -36,6 +36,7 @@ pip install jsmin
 pip install seaborn
 pip install pykalman==0.9.5
 pip install segmentation-models==1.0.1
+pip install rich==9.10.0
 
 pip install setuptools-scm
 

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,26 @@
+
+### Description
+[describe your changes here]
+
+### Types of changes
+
+- [ ] Bugfix
+- [ ] New feature
+- [ ] Refactor / Code style update (no logical changes)
+- [ ] Build / CI changes
+- [ ] Documentation Update
+- [ ] Other (explain)
+
+### Does this address any currently open issues?
+[list open issues here]
+
+### Outside contributors checklist
+
+- [ ] Review the [guidelines for contributing](https://github.com/murthylab/sleap/wiki/Developer-Guide) to this repository
+- [ ] Read and sign the [CLA](https://github.com/murthylab/sleap/blob/develop/sleap-cla.pdf) and add yourself to the [authors list](https://github.com/murthylab/sleap/blob/develop/AUTHORS)
+- [ ] Make sure you are making a pull request against the **develop** branch (not *master*). Also you should start *your branch* off *develop*
+- [ ] Add tests that prove your fix is effective or that your feature works
+- [ ] Add necessary documentation (if appropriate)
+
+#### Thank you for contributing to SLEAP!
+:heart:

.github/workflows/ci.yml

@@ -8,7 +8,39 @@ on:
       - 'tests/**'
 
 jobs:
-  build:
+  type_check:
+    name: Type Check
+    runs-on: "ubuntu-18.04"
+    steps:
+    - uses: actions/checkout@v1
+    - name: Set up Python 3.6
+      uses: actions/setup-python@v1
+      with:
+        python-version: 3.6
+    - name: Install Dependencies
+      run: |
+        pip install mypy
+    - name: Run MyPy
+      # TODO: remove this once all MyPy errors get fixed
+      continue-on-error: true
+      run: |
+        mypy --follow-imports=skip --ignore-missing-imports sleap tests
+  lint:
+    name: Lint
+    runs-on: "ubuntu-18.04"
+    steps:
+    - uses: actions/checkout@v1
+    - name: Set up Python 3.6
+      uses: actions/setup-python@v1
+      with:
+        python-version: 3.6
+    - name: Install Dependencies
+      run: |
+        pip install black
+    - name: Run Black
+      run: |
+        black --check sleap tests
+  tests:
     name: Tests (${{ matrix.os }})
     runs-on: ${{ matrix.os }}
     strategy:

AUTHORS

@@ -0,0 +1,11 @@
+# This is the official list of SLEAP authors not affiliated with Princeton University (for copyright purposes).
+
+# If you are contributing to SLEAP, please add your name and the name of your
+# organization (which holds the copyright) to this list in alphabetical order.
+
+# Names should be added to this file as
+#	Name <email address> Organization name (or 'Individual Person' if not applicable)
+# Please keep the list sorted.
+
+
+John Smith <john at example.com> Example Inc.

docs/conf.py

@@ -89,15 +89,13 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'alabaster'
+html_theme = "alabaster"
 
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the
 # documentation.
 #
-html_theme_options = {
-    'font_size': '15px'
-}
+html_theme_options = {"font_size": "15px"}
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,

requirements.txt

@@ -22,4 +22,5 @@ qimage2ndarray==1.8
 jsmin
 seaborn
 pykalman==0.9.5
-segmentation-models==1.0.1
+segmentation-models==1.0.1
+rich==9.10.0

setup.py

@@ -17,7 +17,7 @@
 # Get the sleap version
 with open(path.join(here, "sleap/version.py")) as f:
     version_file = f.read()
-    sleap_version = re.search("\d.+(?=['\"])", version_file).group(0)
+    sleap_version = re.search('__version__ = "([0-9\\.a]+)"', version_file).group(1)
 
 
 def get_requirements(require_name=None):
@@ -31,11 +31,13 @@ def get_requirements(require_name=None):
     version=sleap_version,
     setup_requires=["setuptools_scm"],
     install_requires=get_requirements(),
-    extras_require={"dev": get_requirements("dev"),},
-    description="SLEAP (Social LEAP Estimates Animal Pose) is a deep learning framework for estimating animal pose.",
+    extras_require={
+        "dev": get_requirements("dev"),
+    },
+    description="SLEAP (Social LEAP Estimates Animal Poses) is a deep learning framework for animal pose tracking.",
     long_description=long_description,
     long_description_content_type="text/x-rst",
-    author="Talmo Pereira, David Turner, Nat Tabris",
+    author="Talmo Pereira, Arie Matsliah, David Turner, Nat Tabris",
     author_email="[email protected]",
     project_urls={
         "Documentation": "https://sleap.ai/#sleap",

sleap/__init__.py

@@ -5,8 +5,9 @@
 logging.basicConfig(stream=sys.stdout, level=logging.INFO)
 
 # Import submodules we want available at top-level
+from sleap.version import __version__, versions
 from sleap.io.dataset import Labels, load_file
-from sleap.io.video import Video
+from sleap.io.video import Video, load_video
 from sleap.instance import LabeledFrame, Instance, PredictedInstance, Track
 from sleap.skeleton import Skeleton
 import sleap.nn
@@ -15,4 +16,4 @@
 from sleap.nn.inference import load_model
 from sleap.nn.system import use_cpu_only, disable_preallocation
 from sleap.nn.system import summary as system_summary
-from sleap.version import __version__
+from sleap.nn.config import TrainingJobConfig, load_config

sleap/config/shortcuts.yaml

@@ -1,7 +1,7 @@
 add instance: Ctrl+I
 add videos: Ctrl+A
 clear selection: Esc
-close: QKeySequence.Close
+close: Ctrl+Q
 color predicted: 
 delete area: Ctrl+K
 delete clip:
@@ -11,19 +11,19 @@ export clip:
 fit: Ctrl+=
 goto frame: Ctrl+J
 goto marked: Ctrl+Shift+M
-goto next suggestion: QKeySequence.FindNext
+goto next suggestion: Space
 goto next track spawn: Ctrl+E
-goto next user: Ctrl+>
-goto next labeled: Ctrl+.
-goto prev suggestion: QKeySequence.FindPrevious
-goto prev labeled: 
+goto next user: Ctrl+U
+goto next labeled: Alt+Right
+goto prev suggestion: Shift+Space
+goto prev labeled: Alt+Left
 learning: Ctrl+L
 mark frame: Ctrl+M
 new: Ctrl+N
 next video: QKeySequence.Forward
 open: Ctrl+O
 prev video: QKeySequence.Back
-save as: QKeySequence.SaveAs
+save as: Ctrl+Shift+S
 save: Ctrl+S
 select next: '`'
 select to frame: Ctrl+Shift+J
@@ -33,7 +33,7 @@ show trails:
 transpose: Ctrl+T
 frame next: Right
 frame prev: Left
-frame next medium step: Down
-frame prev medium step: Up
-frame next large step: Space
-frame prev large step: /
+frame next medium step: Ctrl+Right
+frame prev medium step: Ctrl+Left
+frame next large step: Ctrl+Alt+Right
+frame prev large step: Ctrl+Alt+Left

sleap/config/training_editor_form.yaml

@@ -466,6 +466,13 @@ augmentation:
   label: Scale Max
   name: optimization.augmentation_config.scale_max
   type: double
+- label: Random flip
+  default: none
+  type: list
+  name: optimization.augmentation_config.random_flip
+  options: none,horizontal,vertical
+  default: none
+  help: 'Randomly reflect images and instances. IMPORTANT: Left/right symmetric nodes must be indicated in the skeleton or this will lead to incorrect results!'
 - default: false
   help: If True, uniformly distributed noise will be added to the image. This is effectively
     adding a different random value to each pixel to simulate shot noise. See `imgaug.augmenters.arithmetic.AddElementwise`.
@@ -533,6 +540,7 @@ augmentation:
   label: Brightness Max Val
   name: optimization.augmentation_config.brightness_max_val
   type: double
+
 optimization:
 - default: 8
   help: Number of examples per minibatch, i.e., a single step of training. Higher