Deep Gestures : a human-machine drawing conversation

This master thesis explores the space of possibilities offered by the convergence of novel Machine Learning techniques and established Numerical Fabrication technology, by focusing on applications in the context of the early stages of the design process.

 

By experimentally engaging with the making and the programming of an interactive drawing machine in its hardware and software components, this research offers an alternative interpretation to the praxis of sketching and of intuitively representing ideas in the two-dimensional space.

 

Moving away from the concept of computers as passive receivers of pre-conceived creative impulses, the machine is here framed as an active instrument which is capable of perceiving and processing external inputs. As the machine co-participates in the unfolding of the creative process, the role of the designer/user is necessarily re-defined.

 

Currently, applications of Machine Learning in the creative field are mostly focused on the processing of pixel-based images or texts. As part of the thesis, the potential of a Hierarchical Generative Network for the creation of vector files in the SVG format (DeepSVG) is investigated. As opposed to image creation in raster format, vector representation opens the door to digital production, since its properties allow for a direct translation to machine code and CNC operations.

 

The core of the thesis consists in a set of experiments investigating user-machine interaction for a simple, feedback-based, design process. The physical drawing of a user is fed to a neural network by means of computer vision techniques (skeleton tracing). 

 

The pre-trained network generates new content by interpolating the user input with a selected item from a large library of vector icons, the SVG-Icons8 dataset.The set of newly machine-generated frames is filtered by the user, who chooses a single solution to be printed back in the physical space by a custom-made pen plotter. By engaging in this open-ended process of interaction, the user can build up further drawn responses to feed to the machine, which in turn will keep on generating new and unexpected content until the process is stopped according to user defined criteria.

 

Rather than solving a specific design problem, an experimental Machine Learning application is designed and explored. The main objective of this work is not looking for a single optimal solution, but it is the creative exploration of the unfolding of a hybrid process of collaboration.