Ultimate aim of the project is to quantify the behavior of Drosophila Melonogaster, to understand how a small number of (up to 100.000) neurons can create wide range of motion diversity, starting from understanding pose of Drosophila. Deep learning methods already attain impressive results on scene understanding tasks, such as automatically detecting human pose from images. However, they require large quantities of annotated training data, often measured in millions for state-of-the-art human pose estimation results .
In this project, we want to utilize Deep Domain Adaptation and Computer Graphics techniques with realistic model of Drosophila (fruit-fly) to reduce the human annotation effort for accurate pose estimation. In the end, we will use synthetic images to train a pose estimation network which uses relatively small number of human annotated data. This will decrease our dependence on cumbersome manual annotation, which has already been achieved in some other cases [2, 3].
The student will work on Drosophila model where we can produce synthetic data with ground-truth from different angles and poses. Apart from the fly model, we acquire the real walking pattern and images (together with neural recordings) in cooperation with Neuroengineering Lab . Our ultimate aim is to analyze the behavior of Drosophila Melonogaster, to understand how a small number of (up to 100.000) neurons can create wide range of motion diversity, starting from understanding pose of Drosophila.
The end goal of the project is to trick current deep learning models to believe synthetic data is actually real, so that training is feasible with a smaller number of labeled data.
 Ionescu et al., "Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments" PAMI 2014
 Varol et al., "Learning from Synthetic Humans" CVPR 2017
 Rozantsev et al., "Beyond Sharing Weights for Deep Domain Adaptation" CVPR 2018
 Chen et al., "Imaging neural activity in the ventral nerve cord of behaving adult Drosophila” 2018
Back to the project list.
The candidate should have strong programming experience in one scripting language (preferably in Python) and experience with some deep learning framework (Pytorch or Tensorflow)
30% Theory, 30% Implementation, 40% Research and Experiments