Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Add IceVision docs page #677

Merged
merged 3 commits into from
Aug 20, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions docs/source/api/core.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,26 @@ _____________________
~flash.core.finetuning.NoFreeze
~flash.core.finetuning.UnfreezeMilestones

flash.core.integration.fiftyone
_______________________________
flash.core.integrations.fiftyone
________________________________

.. autosummary::
:toctree: generated/
:nosignatures:

~flash.core.integrations.fiftyone.utils.visualize

flash.core.integrations.icevision
_________________________________

.. autosummary::
:toctree: generated/
:nosignatures:

~flash.core.integrations.icevision.transforms.IceVisionTransformAdapter
~flash.core.integrations.icevision.transforms.default_transforms
~flash.core.integrations.icevision.transforms.train_default_transforms

flash.core.model
________________

Expand Down
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ Lightning Flash

integrations/providers
integrations/fiftyone
integrations/icevision

.. toctree::
:maxdepth: 1
Expand Down
44 changes: 44 additions & 0 deletions docs/source/integrations/icevision.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
.. _ice_vision:

#########
IceVision
#########

IceVision from airctic is an awesome computer vision framework which offers a curated collection of hundreds of high-quality pre-trained models for: object detection, keypoint detection, and instance segmentation.
In Flash, we've integrated the IceVision framework to provide: data loading, augmentation, backbones, and heads.
We use IceVision components in our: :ref:`object detection <object_detection>`, :ref:`instance segmentation <instance_segmentation>`, and :ref:`keypoint detection <keypoint_detection>` tasks.
Take a look at `their documentation <https://airctic.com/>`_ and star `IceVision on GitHub <https://github.com/airctic/IceVision>`_ to spread the open source love!

IceData
_______

The `IceData library <https://github.com/airctic/icedata>`_ is a community driven dataset hub for IceVision.
All of the datasets in IceData can be used out of the box with flash using our ``.from_folders`` methods and the ``parser`` argument.
Take a look at our :ref:`keypoint_detection` page for an example.

Albumentations with IceVision and Flash
_______________________________________

IceVision provides two utilities for using the `albumentations library <https://albumentations.ai/>`_ with their models:
- the ``Adapter`` helper class for adapting an any albumentations transform to work with IceVision records,
- the ``aug_tfms`` utility function that returns a standard augmentation recipe to get the most out of your model.

In Flash, we use the ``aug_tfms`` as default transforms for the: :ref:`object detection <object_detection>`, :ref:`instance segmentation <instance_segmentation>`, and :ref:`keypoint detection <keypoint_detection>` tasks.
You can also provide custom transforms from albumentations using the :class:`~flash.core.integrations.icevision.transforms.IceVisionTransformAdapter` (which relies on the IceVision ``Adapter`` underneath).
Here's an example:

.. code-block:: python

import albumentations as A

from flash.core.integrations.icevision.transforms import IceVisionTransformAdapter
from flash.image import ObjectDetectionData

train_transform = {
"pre_tensor_transform": IceVisionTransformAdapter([A.HorizontalFlip(), A.Normalize()]),
}

datamodule = ObjectDetectionData.from_coco(
...,
train_transform=train_transform,
)
6 changes: 3 additions & 3 deletions flash/core/integrations/icevision/transforms.py
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ def from_icevision_record(record: "BaseRecord"):
class IceVisionTransformAdapter(nn.Module):
def __init__(self, transform):
super().__init__()
self.transform = transform
self.transform = A.Adapter(transform)

def forward(self, x):
record = to_icevision_record(x)
Expand All @@ -186,13 +186,13 @@ def forward(self, x):
def default_transforms(image_size: Tuple[int, int]) -> Dict[str, Callable]:
"""The default transforms from IceVision."""
return {
"pre_tensor_transform": IceVisionTransformAdapter(A.Adapter([*A.resize_and_pad(image_size), A.Normalize()])),
"pre_tensor_transform": IceVisionTransformAdapter([*A.resize_and_pad(image_size), A.Normalize()]),
}


@requires_extras("image")
def train_default_transforms(image_size: Tuple[int, int]) -> Dict[str, Callable]:
"""The default augmentations from IceVision."""
return {
"pre_tensor_transform": IceVisionTransformAdapter(A.Adapter([*A.aug_tfms(size=image_size), A.Normalize()])),
"pre_tensor_transform": IceVisionTransformAdapter([*A.aug_tfms(size=image_size), A.Normalize()]),
}