Journals
2025
ABSRACT: In creativity using digital tools, traditional design guidelines promote abundant, inexpensive, disposable ways to create which are not compatible with sustainability concerns. The authors explore alternatives to this paradigm by comparing the main Virtual Reality sketch method with a new paradigm inspired by traditional Japanese concept named Wabi-Sabi. They observe that removing the possibility to erase and limiting the quantity of virtual paint are changing participants’ creative posture and process and bring them satisfaction. Crossing the Wabi-Sabi paradigm with digital technologies is a concrete opportunity to “go forward” incorporating sustainable values in practice and tools for digital artists and creators.
KEYWORDS: Applied computing, Arts and Humanities, Computer-aided Design, Virtual Reality, Ecosophy, Wabi Sabi, Sketch, Creativity, Imperfection, Error, Correction, Erasing, Undo, Adaptation, Creative posture, Affordance, Serendipity, Poiesis, Sobriety, Digital Ecosophy
2021
ABSRACT: In this tribute to Dominique Boutet and the kinesiological approach he founded, the authors have tried to make their memories of the scientific collective projects they worked on together resonate with the written work of this extraordinary scientific partner, gathered in his published articles and his habilitation document (Boutet 2018). His approach based on an intimate knowledge of the biomechanics of the human body is centered on the structuring role of the body in gestures (and signed languages). It is form-based: the form of gestures shapes their meaning or function. Gestural units are described on the basis of their formal characteristics and physiological constraints rather than their imagistic iconicity. The article presents the foundations of the approach, a synoptic description and some examples of its application. The originality of the kinesiological approach lies in the double revolution that it allows us to operate: on the one hand, gesture is not simply an appendix of speech; on the other hand; it is shaped by bodily physiology. The approach is based on the movements of the human body analyzed from a biomechanical point of view. The meaning of our gestural productions is the produce of our body, as it is naturally articulated, imprinted as it is by our past experiences.
KEYWORDS: Flow, gesture, kinesiology, motion capture, movement, sign language
2019
JEGO Jean-François, Mi-Mime, Mi-Mixte : Hybridation De L’art Du Mime Et De La Réalité Mixte, In Revue des Arts et Technologies de l’image n°6, Dir. CHEN Chu-Yin & SOHIER Remy, juin 2019, 9 pages
RÉSUMÉ : Selon Marcel Marceau, « L’art du mime […] consiste à rendre visible l’invisible et l’invisible visible » (Marceau, 1992). La réalité virtuelle permet à l’homme de s’immerger et d’interagir avec un univers impalpable, mais visible. Que se passe-t-il lorsque le mime se trouve dans un système de réalité virtuelle qui joue sur les perceptions et l’ambiguïté du réel et du virtuel ? Une interaction qui se construit le temps du spectacle en fonction de la réaction du système et des actions non prédéfinies produit-elle un nouveau type d’improvisation ou de jeu d’acteur ? Que provoque cette hybridation chez le spectateur ? Nous présentons notre projet de recherche IAM4Mime, une pantomime en six actes qui croise l’art du mime et la réalité mixte (Milgram & Colquhoun, 1999) qui combine les réalités virtuelle et augmentée dans un théâtre d’ombres réelles et virtuelles. Son élaboration a réuni un mime Marceau et des créateurs en scénographie interactive. L’acteur mime, équipé d’un système de capture de mouvements en temps réel, est en mesure d’interagir avec l’environnement virtuel dont la silhouette est rétroprojetée sur un paravent modulable. Cette modularité permet de reconfigurer l’espace de chaque scène. Le spectateur est invité à découvrir l’univers d’un mime à la recherche de son double virtuel dans un décor surréaliste fait d’illusions. Dans une première partie, nous exposons le processus de création de cette pantomime, du scénario à la construction du support de projection. Nous détaillons comment l’intégration des effets visuels et des objets virtuels et réels, et les expérimentations ont fait évoluer le scénario, par choix esthétiques, par contraintes scéniques et même par artéfacts informatiques. Nous questionnons enfin la notion d’environnements réel, augmenté ou virtuel du point de vue de l’acteur qui circule à travers ces univers selon le fil narratif de la pantomime.
MOTS-CLÉS : Mime Marceau, pantomime, réalité mixte, capture de mouvements, expérimentation, numérique, processus de création, dispositif, projet de recherche
2009
JEGO Jean-François, NERTOMB Rachel, Rubikhome project : Experiencing a modular interactive architecture. In Revue des Arts et Technologies de l’image n°4, Dir. HUITRIC Hervé, NAHAS Monique, BRET Michel, COUCHOT Edmond, p. 61-64, Janvier 2009, 4 pages
RESUME: Rubikhome is an interactive visit of a virtual apartment which completely evolves with the physical position of the avatar. Furniture articulates randomly and depends on the user’s position. The placing of the furniture comes with a specific sound. In this way, the visitor can literally compose an electronic score, which depends on his moves and choices.
KEYWORDS: Virtual Reality, Architecture, Interactive, Visit, Compositing, Mental representation
Conferences peer-reviewed
2025
(SHORT PAPER) JEGO Jean-François Jégo, DASTE Sophie, SOHIER Remy, « Émergences performatives d’œuvres interactives créées en contexte de jam », Actes des Journées d’Informatique Théâtrale – 3ème édition, Avignon, France, 9-11 oct. 2024, 4 pages
RÉSUMÉ : La jam de création “Immersive Improvisation in Interactive Arts” est une initiative annuelle et internationale qui rassemble des étudiants et des professeurs du monde universitaire pour explorer les potentialités esthétiques et participatives des technologies temps réel dans le cadre d’expositions artistiques. Ces événements offrent un terrain d’expérimentation artistique au contact des dernières innovations technologiques, en encourageant la création collaborative et l’intégration du patrimoine culturel local du lieu d’accueil. Durant une semaine, les participants visitent des environnements présentant les dernières avancées en matière de technologies interactives (laboratoires et/ou salons technologiques). Chaque année, entre cinq et huit pièces sont créées par des équipes d’étudiants-artistes, dont une performance de happening ou théâtrale émergeant comme l’élément saillant de chaque édition. Ces créations hybrides offrent une opportunité singulière d’explorer de nouvelles formes d’expression artistique inspirées du lieu qui les accueille en mettant l’accent sur les émergences performatives produites par la co-création, l’innovation et l’interaction avec le public.
MOTS-CLÉS : jam, résidence, réalité virtuelle, réalité augmentée, immersion, interaction, installation, improvisation, performance
2024
[À paraître] (SHORT PAPER) JEGO Jean-François, « Les Réalités Virtuelles Rêvées », Colloque-exposition International Pour un Imaginaire Numérique avec Edmond Couchot, 16-18 novembre 2022, Saint-Denis et Enghien-les-Bains, France, 4 pages
RÉSUMÉ : Arthur C. Clark affirma au début des années soixante que « toute technologie suffisamment avancée est indiscernable de la magie ». Au-delà de la magie, pourrions-nous aller jusqu’à considérer que la réalité virtuelle soit un médium indiscernable du rêve. L’avènement prochain du Metavers, croisant les réalités virtuelles, mixtes et augmentées, exprime le fantasme de pouvoir vivre des expériences individuelles — mais aussi — partagées. Les pionniers de la réalité virtuelle, qu’ils et elles soient artistes et/ou scientifiques ont identifié dès les années quatre-vingt le potentiel surréaliste du médium permettant de jouer avec la plasticité du corps, l’espace, le temps, la gravité, les paradoxes. Il est surtout possible de (re)vivre ces expériences à l’infini et de les faire vivre à d’autres participants. Nous proposons un voyage chronotechnologique de l’évolution des réalités virtuelles comme médium de création et de nous projeter dans son futur « artisticonirique » en explorant les pistes utopiques d’universalité, d’hybridation et de durabilité.
MOTS-CLÉS : conférence-performance, réalité virtuelle, réalité augmentée, immersion, interaction, frugalité, improvisation
2023
(PANEL PAPER) ZANNOS Iannis, ANTONIADIS Pavlos, BIRRINGER Johannes, HIRAYAMA Haruka, JÉGO Jean-François, NELSON Peter, PAPACHRISTOU Dana, PASCHALIDOU Stella, Networked Collaborative Performance: Frugal Strategies and Cultural Impact of Technology, Panel Interactive – Networked – Human Computer Interaction and Symbiotic Organizations, ISEA 2023, 16-21 may 2023, Paris, France
ABSTRACT: This panel considers the integration of technologies such as internet communication and collaboration tools, devices for human machine-interaction, live audiovisual media processing, and live coding within the creation of contemporary performances and the development of practices. The participants reflect on actual projects, and on the theoretical underpinning that informs and motivates such performances. In particular, the notion of frugality comes to the fore, noting the effect of open-source and DIY technologies on social and cultural practices within existing power structures.
KEYWORDS: Networked collaboration, frugal innovation, technology, dance, interaction
(POSTER PAPER) CHATEAUVERT Julie, TSAMPOUNARIS Yorgos, JEGO Jean-François, Remixer le Flying Word project, ISEA 2023, 16-21 may 2023, Paris, France
ABSTRACT: Remixer le Flying Words Project is a digital art installation crossing the creative sign poetry of Peter Cook and Kenny Lerner with a Virtual Reality (VR) experience. We present here how we are designing the first version of the artwork investigating the idea of symbiosis. It appears to be the central pivot of this project mixing sign-art and digital art, video of the performers with visuals generated by performance-capture and point clouds in real time. Instead of adopting the idea of translating the poem in VR, we have created a passage from one art to the other as a fusion in order to create synesthesia. The method adopted is a “bricolage” between several scientific tools based on DIY practice, biomechanics, movement quality analysis, and on human perception, body referential to create a synesthetic experience. Analyzing the ways we address the first version of the VR experience reveals how the arts practice was crossed, the technologies were merged and the different backgrounds of the team members got synchronized.
KEYWORDS: Installation, digital art, deaf poetry, virtual reality, visual vernacular, synesthesia, performance-capture, motion-capture
2022
(LONG PAPER) ANTONIADIS Pavlos, JEGO Jean-François, DUVAL Aurélien, BEVILACQUA Frédéric, PASCHALIDOU Stella « Rendering embodied experience into multimodal data: concepts, tools and applications for Xenakis’ piano performance », Xenakis 22: Centenary International Symposium, Athens & Nafplio, Greece, 24-29 May 2022, 22 pages, pp178-199
ABSTRACT: Iannis Xenakis’ performance practice has increasingly been the object of investigation by both interpreters and musicologists. After a first generation of legendary performers, who feverishly attempted to register and communicate what at the time was a singular challenging experience, subsequent generations have systematically kept developing practice-based research methods for learning and playing Xenakis. Similarly, musicology has been shifting its attention from Xenakis’ structuralist approach to post-structuralist and ecological approaches, to Xenakis’ compositional practice and quintessentially towards performance analysis, in the wider context of a performative turn [6] and a more recent embodied cognitive turn. The results of this double movement by performers and musicologists have exemplarily been codified in the series of conferences “Interpréter/Performer Xenakis”.
KEYWORDS: Piano, Embodied Cognition, Motion Capture, Multimodal Data, Body Rendition
(WORKSHOP ABSTRACT) ANTONIADIS Pavlos, JEGO Jean-François, DUVAL Aurélien, BEVILACQUA Frédéric, PASCHALIDOU Stella « Workshop: Rendering embodied experience into multimodal data: concepts, tools and applications for Xenakis’ piano performance », Xenakis 22: Centenary International Symposium, Athens & Nafplio, Greece, 24-29 May 2022, 4 pages, pp251-254
ABSTRACT: The main core of the workshop will be the presentation of the system GesTCom (acronym for Gesture Cutting Through Textual Complexity), which has been developed since 2014 in collaboration with IRCAM (interaction-son-musique-mouvement team). GesTCom is a sensor-based environment for the visualization, analysis and following of the pianist’s gestures in relation to the notation. It comprises four modules, implemented in the form of Max/MSP patches featuring the MuBu toolbox and connected to INScore scripts: a) a module for the synchronized recording of multimodal data of a performance ; b) a module for the reproduction and analysis of the data ; c) a module for the processing of the notation on the basis of the data ; d) a module for real-time gestural interaction with the notation.
KEYWORDS: Piano, Embodied Cognition, Motion Capture, Multimodal Data, Body Rendition
2021
[Accepté, à paraître] (LONG PAPER) CHATEAUVERT Julie, JEGO Jean-François, TSAMPOUNARIS Yorgos, « Traduire l’intermédial, recréer la synesthésie », Colloque international ‘Traduire la performance/Performer la traduction’, École Universitaire de Recherche ArTeC, Les Laboratoires d’Aubervilliers, France. Soumis en juillet 2021, 11 pages
RÉSUMÉ : Nous présentons un projet en cours de création intitulé Remixer le Flying Words Project. Il constitue l’itération la plus récente d’une démarche de recherche-création consacrée à l’étude de la création narrative en langue des signes amorcée en 2000 par Julie Chateauvert dans le cadre d’une maîtrise en danse à l’Université du Québec à Montréal. Initialement, il s’agissait pour le groupe formé dans ce contexte d’affirmer un statut pour les Langues des Signes encore contesté alors dans les lieux même d’enseignement spécialisé dédié aux enfants et adolescent.es sourd.es. Créer des fragments poétiques à partir de la matière même de la langue (nous travaillions en Langues des Signes Québécoises) venait, aux yeux du groupe, invalider d’emblée tout argument mobilisé pour leur refuser ce statut. Le processus s’inscrivait dans une démarche d’empouvoirement et passait par la prise de position publique. Le projet faisait le choix de l’événement plutôt que du discours. L’intention était ainsi de faire exister, transformer immédiatement.
MOTS-CLÉS : Poésie en Langue des Signes, Capture de Mouvement, Réalité Virtuelle, Synesthésie
(LONG PAPER) ANTONIADIS Pavlos, DUVAL Aurélien, JEGO Jean-François, BEVILACQUA Frédéric, SOLOMOS Makis. Dwelling Xenakis. An augmented reality project on Evryali for piano solo. Presented at the Conference Nova Contemporary Music Meeting (NCMM 2021), 06 mai 2021, Lisbon, Portugal, online, 16 pages
ABSTRACT: This paper presents an augmented reality project based on Iannis Xenakis’ work Evryali for piano solo. Drawing on an interdisciplinary framework, we transform the performance space into a hybrid environment consisting of physical, symbolic and virtual elements. The synergy of these elements reveals to the audience affordances of meaning and action that normally remain exclusive to the performer. Importantly, the audience is invited to ‘dwell’ this environment, meaning to experience the affordances in a fully embodied and multimodal way, often including real-time interaction. This hybrid environment is inextricably linked to the visualization and communication of the performer’s learning process, defined here as embodied navigation of notational affordances.
KEYWORDS: Piano, Augmented Reality, Mixed Reality, Symbolic Music Notation, Communication, Visualization, Motion Capture, Embodied Navigation, Affordance
(POSTER-PAPER) ANTONIADIS Pavlos, DUVAL Aurélien, JEGO Jean-François, SOLOMOS Makis, BEVILACQUA Frederic. Merging symbolic, physical and virtual spaces: Augmented reality for Iannis Xenakis’ Evryali for piano solo. DCAC 3rd International Conference on Digital Culture & AudioVisual Challenges, 28-29 may 2021, Corfu, Greece, online, 2 pages
ABSTRACT: The proposed paper will present interactive systems for the visualization and optimization of extreme score-based piano performance. The systems are founded on an ecological theory of embodied interaction with complex piano notation, under the title embodied navigation. The theory has materialized in a modular, sensor-based environment for the analysis, processing and real-time control of notation through multimodal recordings, called GesTCom. The motion capture modeling is based on a one-shot learning Hidden Markov Model developed at IRCAM and called Gesture Follower. At a later stage, mixed reality applications have been developed on the basis of existent visualization methodologies for motion capture, seeking to create a virtual concert environment.
KEYWORDS: Piano, Symbolic Music Notation, Communication, Visualization, Motion Capture, Embodied Navigation
2020
Participatory art allows for the spectator to be a participant or a viewer able to engage actively with interactive art. Real-time technologies offer new ways to create participative artworks. We hereby investigate how to engage participation through movement in interactive digital art, and what this engagement can awaken, focusing on the ways to elicit improvisation and letting go. We analyze two Virtual Reality installations, “InterACTE” and “Eve, dance is an unplaceable place,” involving body movement, dance, creativity and the presence of an observing audience. We evaluate the premises, the setup, and the feedback of the spectators in the two installations. We propose a model following three different perspectives of resonance: 1. Inter Resonance between Spectator and Artwork, which involves curiosity, imitation, playfulness and improvisation. 2. Inner Resonance of Spectator him/herself, where embodiment and creativity contribute to the sense of being present and letting go. 3. Collective Resonance between Spectator/Artwork and Audience, which is stimulated by curiosity, and triggers motor contagion, engagement and gathering. The two analyzed examples seek to awaken open-minded communicative possibilities through the use of interactive digital artworks. Moreover, the need to recognize and develop the idea of resonance becomes increasingly important in this time of urgency to communicate, understand and support collectivity.
2019
JEGO Jean-François, MEYRUEIS Vincent, BOUTET Dominique. A Workflow for Real-time Visualization and Data Analysis of Gesture using Motion Capture. In Proceedings of Movement and Computing conference, Tempe, AZ, USA, October 2019 (MOCO ’19) , 6 pages, ACM, New-York, USA, 2019
We investigate new ways to understand and to analyze human gesture in a research context applied on co-verbal gesture across language. The research project focuses on the quality of the movement and consider the gesture “pulse of effort.“ We propose a workflow for real-time gesture analysis to visualize gesture kinematics features (Velocity, Acceleration, Jerk) from heterogeneous data (Video, Motion Capture and Gesture Annotations) at the same time base. The tools designed here provide immersive and interactive explorations of data: users can test hypotheses and embody gesture visualization and descriptors adopting different Frames of Reference using augmented reality. We have conducted an evaluation protocol in the field of linguistics that compares 496 annotated gestures to benchmark the workflow.
“The medium is the message” from McLuhan [13] showed and still shows how the medium influences how the message is perceived. This article proposes a new model for computer-human interaction named MITMI for (hu)man-in-the-middle Interaction in the context of digital art. This approach questions the concept of mediated interaction by placing the human as the driver and as the “translator” of the interaction process from a theoretical point of view. Indeed, mediated interaction is based on the fact that two humans are able to interact through a technological device [5]. We develop the idea of putting human back as an intermediary between the user/spectator and the technology. Our model thus introduces a way of conceptualizing interactive digital artworks, where the artist considered as an operator and creator can also take a central position and is no longer the only producer of the work taking thus entirely part of the artwork. We first explore the need to put the human back in the loop, then we will describe how the principles of mediated interaction help in defining the MITMI model. We will stress this model through several installations and interactive artworks based on the idea of shifting the artist position from the creator, to the mediator between the real world and the virtual world. Beyond the artistic approach, this model proposes to redefine and to explore new modes of co-creation or mediation, where digital media supports human-to-human interaction.
2018
Le projet collaboratif CIGALE (Capture et Interaction avec des Gestes Artistiques, Langagiers et Expressifs), soutenu, par le Labex ARTS-H2H, a développé un dispositif mettant en œuvre une interaction gestuelle entre un acteur humain et un acteur virtuel (AV). Les recherches se sont déployées selon cinq directions : la capture du mouvement, l’analyse gestuelle, la génération de mouvements virtuels, l’improvisation et sa modélisation, l’interaction gestuelle et son analyse. Nous nous focaliserons dans cet article sur les deux derniers axes, ceux de l’improvisation et de l’interaction gestuelle avec un AV en nous intéressant plus particulièrement à l’installation InterACTE, une des créations en réalité virtuelle développées dans le cadre de ce projet.
2017
This practice-based workshop aims at exploring various methodologies and tools to design gestures that could be used in interactive systems. We argue that gesture design, which can be seen as being a smaller part of the interaction design process, is generally overlooked. Our goals are to develop various methods, some being inspired by performing arts practices, and to discuss the multicultural aspects of gesture design.
2016
This paper describes an interactive installation that invites participants to improvise with an autonomous virtual actor. Based on real-time analysis of the participant’s gestures, the virtual actor is capable of either propose expressive and linguistic gestures (selected from a set of four Motion Capture Databases), either imitate the participant, or either generate its own gestures using a Genetic Algorithm. A software agent is used to trigger the previously mentioned behaviors based on the analysis results and on a set of predefined rules. The installation is presented in two acts: i. In the first act, the participant interacts with the projected shadow of the virtual actor; ii. In the second act, the participant is being immersed in a 3D world using a virtual reality HMD—the virtual actor is being presented to her/him in the form of an avatar made of particles.
Here, hands don’t beat the drum. Instead the drum speaks with its hands, projected onto its skin. They interact and create poems in sign language, especially for the deaf/hearing impaired, because the drum has acquired the expressive and prosodic gestures of deaf poets. This installation is based on a Virtual Reality Agent-based platform to explore an interactive gestural dialogue between real and virtual actors.
We are looking for something primitive: a memory from before our birth. Something obvious, we all carry and that evolves within us: the first gestures of the first men. Between art, science and technology, our research tends to a virtual scene of rock art in action. Assuming that the cave paintings are the traces of oral performance or dance rites, they have been used as transmission and communication media for the knowledge of surrounding and environments of the early men (hunting, myth, history, art or religious content, shamanism). Through the scope of these fragmentary traces of history, we are seeking to reconstruct various performative scenarios echoing the paintings of the Chauvet’s cave.
This paper describes the process of designing a Virtual Reality Agent-based platform to explore an interactive gestural dialogue between real and virtual actors and propose an analysis of the results. It is part of an interdisciplinary project involving researchers in the fields of linguistics, digital arts and computer science, as well as actors and professors. Several experiments of improvisation conducted with actors through computer-mediated interactions help in characterizing specific dynamics of gestures, recurrent behaviors and gestural patterns. We extract specific descriptors of movement and behaviors that will drive the improvisation. The virtual actor is able to improvise and generate its gestures using a finite state machine that triggers three main behaviors according to the actor’s body posture and arm movements: the virtual actor can mime in real-time the real actor, it can propose expressive or linguistic gestures from four MoCap databases, or it can generate a new gesture using a genetic algorithm. A first installation experienced by the participants during at an international art festival, shows they are able to improvise fluently with the virtual actor in a rich and expressive dialogue.
2015
CIGALE, Capture et Interaction avec des Gestes Artistiques, Langagiers et Expressifs, développe un dispositif artistique et scientifique mettant en œuvre une interaction gestuelle entre un humain et une entité virtuelle. Quatre bases de données gestuelles d’une grande diversité sémiotique ont été enregistrées par capture de mouvement. La capture du mouvement implique son ‘décollement’ du support corporel et de l’espace qui participent au sens des gestes. Un générateur paramétrable appliqué aux gestes (gestes virtuels par algorithme génétique) complète les bases de données de gestes. Des expérimentations d’interactions ont permis de modéliser des comportements en usage dans un dialogue gestuel et de les implémenter dans le moteur de comportement de l’entité virtuelle. Ces va-et-vient dialogiques entre corps réel, extraction, inscription et ex-cription du sens dans l’interaction avec une entité virtuelle, interrogent l’ontologie même d’un avatar.
2013
In this paper we study the memorization of user created gestures for 3DUI. Wide public applications mostly use standardized gestures for interactions with simple contents. This work is motivated by two application cases for which a standardized approach is not possible and thus user specific or dedicated interfaces are needed. The first one is applications for people with limited sensory-motor abilities for whom generic interaction methods may not be adapted. The second one is creative arts applications, for which gesture freedom is part of the creative process. In this work, users are asked to create gestures for a set of tasks, in a specific phase, prior to using the system. We propose a user study to explore the question of gesture memorization. Gestures are recorded and recognized with a Hidden Markov Model. Results show that it seems difficult to recall more than two abstract gestures. Affordances strongly improve memorization whereas the use of colocalization has no significant effect.
Conferences articles
2015
Un corps numérisé, mis en scène et restitué en temps réel par des dispositifs de réalité virtuelle peut être communément assimilé à une perte de densité, de corporéité, ou d’identité par rapport au jeu de l’acteur réel. La chair de synthèse, le plus souvent révélée par l’image, lui attribue un caractère immatériel qui ne permet pas de contact ou ressenti physique, haptique ou charnel. L’enjeu actuel reste de redonner aux corps synthétiques épaisseur et caractère. À travers l’étude de deux mises en scène, qui croisent le jeu d’acteurs réel et de synthèse, nous voyons émerger trois critères essentiels pour redonner une densité au numérique : identité, présence, interaction. De ces trois points, nous notons que l’association du jeu de l’acteur réel avec celui de l’être virtuel ainsi caractérisé permet l’émergence de situations singulières et empathiques construites le temps du spectacle.
2011
La pièce que présente Jean François Jégo est une interprétation par Sophie Lavaud du tableau Rouge, Jaune, Bleu de Wassily Kandinsky datant de 1925. L’objectif de l’auteur est de rendre le tableau dynamique, et ce par la modélisation en 3D des éléments qui le composent. Le spectateur interagit avec ces éléments et les déplace dans le tableau afin de créer sa propre composition. L’œuvre s’expérimente selon trois niveaux proxémiques, et les éléments du tableau disposent de comportements autonomes. Ils sont soumis à des forces d’élasticité, d’équilibre ou de ressort entre eux ainsi que d’une inertie individuelle. L’utilisateur ne peut prévoir quel sera le point d’arrivée de l’élément qu’il va déplacer. Les mécanismes de commandes sont eux, interactifs, et demandent un certain apprentissage de l’œuvre. Les échanges portent sur la circulation et l’équilibre des rôles entre les mécanismes de commandes interactifs et les comportements autonomes.
Dissertations
2013
In this thesis, we propose and evaluate new gestural interfaces for 3DUI. This work is motivated by two application cases: the first one is dedicated to people with limited sensory-motor abilities for whom generic interaction methods may not be adapted. The second one is artistic digital performances, for which gesture freedom is part of the creative process. For those cases, a standardized approach is not possible and thus user-specific or dedicated interfaces are needed. We propose a user-defined gestural interaction that allows the user to make the system learn the gestures that he has created, in a specific phase, prior to using the system. Then, the user reuses his created gestures to interact in the virtual environment. This approach raises research questions about memorization of gestures, effects of fatigue and effects of visual feedbacks. To answer those questions, we study the memorization of user created gestures regarding the role of affordances and colocalization on gesture recall. Then, we study the role of different visual feedbacks on gesture repetitions for a set of manipulation tasks. We also compare full-collocated gestures to loose gestures with lower amplitude. Also, the approach has been designed to be affordable using low-cost devices. We explore solutions to deal with the lower data quality of such devices. The results of the user-studies are based on the analysis of six thousand gestures performed by forty subjects.
2010
This thesis deals with the setup of a technical device dedicated to an interactive set design for performing arts. Featuring Dimitri Rekatchevski, mime artist, a six scenes pantomime (mime theatrical performance) has been created. The actor is tracked by a motion capture system in real time, this adds a new dimension for the performance: the interactivity. The actors and the elements are visible from the public behind a folding screen which receives the video projection of the black shape of the actor and the virtual accessories. We will discuss about the relationships and conversation’s interfaces between real and virtual elements, with an approach focused on the actor.