Machine learning is widely applied in many domains, however, it is still unclear how well it can perform on the fine arts domain. Many existing works have been developed in style transfer, like applying the style of Van Gool’s painting style to a real photo, or computational photography which focus more on automatic beautifying the given pictures. One of the reasons why designers also looking forward to these tools is, that it will significantly reduce repetitive labours.
Type design, on the other hand, is a perfect application to use some smart machine learning algorithm to reduce the tedious work for designers. Before computing aiding algorithm ever exists, in terms to design a font, they need to design a set of 12 iconic characters as described in Figure (a), which define the style of a font. Once the detail of strokes is fixed, they need to apply this style to 50 characters for a better understanding and make the second round of style design. After this, all the component of design should be ready for scaling up. The final step is to apply the designed font style, to the rest of 5000 different characters. Usually, the first two steps require some designer to involve, but at last, requires just manually organization all the components, as shown in Figure (c), to compose the final word and repeats for thousands of times! And it is exactly why we need to incorporate machine learning algorithms to automate this process!
With your help, we will test if existing algorithms can be applied to such a task. What you need to do, is working with experts in type design from ECAL and do ONE of the following directions.
1) Definition of the data formatting on top of [1,2] and implementation of neural network based method to achieve regression.
2) Implementation of variational auto-encoder to directly working on style application based on .
We are very positive this will draw a significant social impact on the entire CJK type design discipline. Will you be among the first to bring intelligence to this field?
 Choi et al. "Next Generation CJK Font Technology Using the Metafont" Tech report 2018.
 "Smart components"https://glyphsapp.com/tutorials/smart-components.
 Campbell et al. "Learning a manifold of fonts" ACM Transactions on Graphics 2014
 “zi2zi” GitHub Repo
Back to the project list.
The candidate should be suffciently familiar with machine learning frameworks, such as Pytorch or TensorFlow.
NO prior knowledge on Chinese, Korean or Japanese. We simply treat them like image or shapes in this task.
You will be closely working with the type designers from ecal throughout the entire projects and Dr. Mathieu Salzmann from CVLab.
10% Theory, 30% Implementation, 60% Research and Experiments