ErfNet based models for Panoptic Segmentation on Cityscape dataset (Pytorch, Jit)

Panoptic segmentation is a task that aims to combine semantic segmentation and instance segmentation in a single prediction and, therefore, it is of high interest for autonomous vehicles. This works presents a light-weight solution based on a convolutional neural network for panoptic segmentation capable of running on embedded GPUs. The solution is built upon an ERFNet backbone, with a shared encoder and two identical decoders to solve the two sub-tasks. The semantic decoder feeds into a single head creating the semantic segmentation prediction whilst the instance decoder has two heads that predict class-agnostic centers and offsets. These three outputs are then post-processed to create soft attention masks that help generate the final panoptic prediction. We defined a model composed per 3 entity: - An ERFNet encoder with residual connections to the semantic decoder. - An ERFNet semantic decoder, receiving residual connections from the encoder and feeding others to the instance decoder. - An ERFNet instance decoder, receiving residual connections from the semantic one In order to study the performance of different implementations, we defined the following variations available in this dataset (each archive has a read_me): - Default model with residual connections (default_360x720_yes.tar.gz) - Default model without residual connections (default_360x720_no.tar.gz) - Model with reduced encoder without residual connections (reduced_360x720_no.tar.gz)