Implement Feature Pyramid Network for semantic segmentation

[daniel-j-h]

Splitting off of #12. Eventually we want to implement an object detection architecture in addition to out current semantic segmentation architecture. The RetinaNet (ticketed in #12) is a perfect fit for our goals. It will be based on top of a ResNet feature extractor and a feature pyramid network.

We can use the ResNet feature extractor and the feature pyramid network already for semantic segmentation. Then later down the line we can extend it adding a bounding box regression head and get a single unified simple architecture for both object detection as well as semantic segmentation.

Resources:

• https://arxiv.org/abs/1612.03144 - Feature Pyramid Networks for Object Detection
• http://presentations.cocodataset.org/COCO17-Stuff-FAIR.pdf

Here is the main gist from the second PDF:

The ResNet will give us the leftmost upward path. The downward path and the lateral connections then make up the feature pyramid network. The rightmost part is the semantic segmentation head.

Tasks

• Implement the feature pyramid network on top of ResNet
• Implement semantic segmentation head on top of the feature pyramid network