A research project on the virtualisation of the live performer

An overview: HADES


3D scanning for the stage
a '
Technology Driven Art' research project
by Peter Missotten and Woody Laurens (Toneelacademie Maastricht)
thanks to Hans Lasschuit


3D scanning of persons is a well established practice, especially within the Hollywood movie and gaming industry. But the machinery for it remains rather hightech and expensive, well outside the reach of a small performance company. 3D scanning in general is a booming business, but almost exclusively orientated towards the scanning of non-moving objects for the industry. Most scanners require you to sit down very quiet for quite some time, unless they are done with a huge and expensive rig of over 30 professional camera's in a dome like construction.

Is it possible to do that with a much smaller rig of cheap consumer camera's? How workable is that in practice? And how does the technology influence the artistic practice?

In this first round, several software solutions were examined - Strata Foto 3D, Autodesk’s 123D Catch, Agisoft Photoscan etc… It all didn't work out self evidently, most of times because of some tiny technical problems. Firing all of the cameras with the remote control proved to be challenging, even with IR-relay modules. And Strata Foto requires an underlay print which proved to be very unpractical for moving objects.

All in all, the research time proved to be way too short to get a decent and usable result. At the same time, the students designed a very inspiring 3D installation, as a set for the project. Somehow, thinking in 3D influenced their mind and gave them inspiration for a new kind of set design, even if the 3D scanning proved to be quite challenging.

In the second research period (May and June 2016),
with Diederik Kreike, Caro Derkx, Tim Bijtelaar, Hendrik Kegels and Amanda Dekker.

We made a full fledged performance involving quite a lot of different virtualisation techniques. Starting with a very helpful and inspiring workshop by the media artist Wim Janssen, we experimented with a lot of software tools. We finally got some interesting and usable results out of Strata Foto 3D (patience is a good virtue), and combined it with projection mapping, compositing of a lot of filmed and looped characters (some hundreds of them), a virtual sea made in Blender… All was combined and played out in Isadora 2, with 9 projectors.
The result - 'HADES' - was part of the 'Odysseus' presentation at the ITs Festival Amsterdam 2016, by the Platform for European Theatre Academies PLETA. And that did go unnoticed, with some raving reviews…


It was quite an exhausting but great experience - all the technique was done by the actors/performers themselves, all of the specialising within the scope of two months on one tool:
Diederik Kreike became a compositing expert overnight,
Caro Derkx animated some hundreds triangles in Keynote,
Hendrik Kegels created a liquid sea (with crashing waves) out of the blue in Blender (never having heard of that software before, that's quite an achievement,
Tim Olivier Somer (one of the live performers, with Caro) created the sound track,
Amanda Dekkers did the lights…
And all of them did the shooting and 3D scanning of themselves as performers.
Yes, miracles do happen…

You can see a full registration here: (password is hades)

HADES - ITs Festival Amsterdam from Peter Missotten on Vimeo.