Digital Humans: Capturing Performance and Presence
Consultancy, R&D, and Real-Time Integration for Immersive Human Digitalization
Visit website
Purpose and Technologies
In the realm of computer graphics, capturing human representation is essential to translating performances into the digital domain. Techniques like Volumetric Capture and Motion Capture enable us to authentically recreate human movements and expressions, enriching the virtual experience with genuine emotional depth and realism.
My Role
I led the end-to-end development of our Volumetric and Motion Capture projects, beginning with advising and collaborating with clients to achieve optimal results and preparing compelling pitches to secure grants and funding. My responsibilities extended to managing technical operations during performance capture on shoot days. I utilized established commercial tools such as OptiTrack, Motive, Captury, Theia, and Notiom, along with various R&D solutions that often presented limited documentation and numerous bugs that I successfully troubleshot for precise motion tracking.
In post-production, I employed software like Blender and 4dfx to process and refine our captures. Throughout the workflow, I ensured seamless integration with real-time game engines like Unity and Unreal Engine, enabling us to bring our digital humans to life within engaging environments. I also created pre-visualizations to provide clients with a clear understanding of how the final product would appear on shoot days, and I developed standalone VR applications.
Moreover, I focused on effective strategies for distributing our content, ensuring the final output could be experienced in the best possible manner. This involved developing specific workflows that leveraged the capabilities of 8th Wall WebXR, as well as experimenting with formats like Sony Spatial Reality Display and Apple Vision Pro. This role allowed me to merge my technical expertise with my passion for storytelling, crafting immersive experiences that resonate deeply with audiences.
Angels Costumes
Challenge: Innovation and Conservation of Period Costumes by the Famous Theatre and Cinema firm, Angels. In partnership with Guildhall School of Music and Drama.
a technology that is capable to not only capture the subject thre-dimensionally, but to also capture the fourth-dimension: time.
In other words, thanks to 48 cameras placed all around the subject, it is possible to capture volumetrically subjects and their movements which eneded up being crucial when representing virtually the cloth/dresses simulation and physics. To achive the best output when capturing Angels’ costumes, as Technical Director I was responsable of operating the Volumetric Capture system and...
I developed a workflow that could work within Target3D existing infrastrucutre (hardware and software). After several tests, I also idntified and documented (technical writing) the pre-production, capturing, post-production and data mangement best practices.
Don’t Treat Me Like An Object
Enabling world famous designer Patricia Urquiola in exploring a new dimension
The challenge
Encountering Patricia Urquiola was both serendipitous and fortunate. At a time of great excitement in the VR field, she chose to engage in this pioneering exploration of how objects in the VR realm could transcend their traditional roles of form and function.
“We have approached this Virtual Reality project with curiosity about another type of reality, in which objects, architectures and landscapes develop freely in a time-space without our material logics. In this dimension, objects have complex emotions and reactions, based on non-biological intelligence. An immersion in a parallel context, in which sensations of pleasure could change towards an ‘Uncanny Valley’, or ‘disturbing zone’.... We are at the beginnings of these explorations”.
From a technical point of view, the great challenge has been that of creating a neural network able to learn from human movements and therefore decide how to react on the basis of the user’s behaviour. This is how the reactions of objects have been made the most natural and variable possible.
My Role
Collaborating closely with someone of Patricia Urquiola’s caliber was no simple feat. From the initial meeting to the final stages of production, she continuously generated a vast number of ideas, truly a ‘volcano’ of creativity. The challenge lay in filtering and prioritizing these inputs. After that, it was my responsibility to work with our software engineers to assess the feasibility of each feature Patricia requested and ultimately serve as a mediator, returning to Patricia to explain why certain features could not be implemented due to constraints related to the technology, time, or resources.
Reception
During Milano Design Week, the multitude of events makes capturing attention and directing visitors to your exhibit a significant challenge. However, thanks to the immense effort from all project partners, our experience was fully booked from early morning until midnight each day. We received an overwhelmingly positive response from the general public, as well as from prominent artists, designers, creatives, and musicians who experienced ‘Don’t Treat Me Like an Object’.
Panerai
Immersive branded experience presented at Watches and Wonders, the most important luxury watchmaking event of the world
Client Brief
Each year, Panerai gears up for the most significant event in the luxry watchmaking sector, Watches and Wonders, aiming to surprise and engage their high-profile clients. They were seeking a way to showcase their innovative style alongside their rich heritage, striving to position the brand as both deeply rooted in tradition and boldly looking towards the future.
Our Solution
Our innovative proposal involved creating an immersive virtual reality experience that would transport users into the iconography of two standout pieces featured at the exhibition: L’Astronomo and Lo Scienziato. We aimed to evoke imagery related to space, the solar system, and Galileo Galilei, drawing connections between historical scientific achievements and modern technological innovation.
My Role
As a new client, it was essential to foster clear, effective, and constructive communication. Building a relationship of trust and credibility was crucial, as was delivering a project that could exceed the exhibition’s standards and stand out among the booths at Watches and Wonders. I took on this responsibility as the key account manager and Producer, overseeing every phase from conception to the dismantling of the booth in Geneva, through various review stages with Panerai’s C-level executives, where we had to justify and motivate our choices. It was necessary to find someone with expertise in this new medium who could effectively introduce and explain it to those unfamiliar with it. I filled this gap by leveraging my knowledge to bridge the gap between the medium and the client.
I also contributed to the design of the experience. Proposing a video game-like experience to the attendees at the Panerai booth could have been very risky, so from an experiential standpoint, we opted for something that did not require controllers or specific interactions but rather focused on the musical and visual aspects. We wanted it to be 6DoF (Six Degrees of Freedom) to increase the level of immersion and the ‘wow’ effect. That’s why we opted for a tethered solution using the HTC Vive and developed it using UE4.
Similarly, a concept that resonates deeply with both me and Proxima Milano was the attention to form, both pre- and post-experience. To enhance the VR experience, it was necessary to also work on the set design and how people would enter Panerai’s Innovation Room.
In the months leading up to the event, I collaborated closely with the production coordinators and artisans who were creating the physical room. The final virtual scene was designed to be an exact 1:1 match of the real room, bringing people “Back to Reality”.
Spark AR Filters
Proxima Milano chosen as partner by Meta to produce captivating AR filters for brands
Visit website
The problem
I had to deal with the approval process on Meta platforms such as Instragram and dealed with compliancy. lorem ipsum dolor lorem ipsum dolor lorem ipsum dolor lorem ipsum dolor lorem ipsum dolor