Gesture-Driven Interaction Using the Leap Motion to Conduct a 3D Particle System: Evaluation and Analysis of an Orchestral Performance
Keywords:Gestural Interaction, 3D Particles, Orchestral Arrangement, Visualization, Performance Analysis, Evaluation, Leap Motion
In this work, we present and evaluate an interactive simulation of 3D particles conducted by the Leap Motion, for an orchestral arrangement. A real-time visual feedback during gesture entry is generated for the conductor and the audience, through a set of particle emitters displayed on the screen and the path traced by the captured gesture. We use two types of data input: the captured left and right hand conducting gestures (some universal movements, such as the beat patterns for the most common time signatures, the indication of a specific section of the orchestra, and the cutoff gestures), which are responsible for setting the tempo and dynamics of the orchestra; and a MIDI file, which contains information about the score of an orchestral arrangement, defining which notes each musical instrument should play. As regards the gestural input of the conductor, we have considered two musical elements of expression: tempo and dynamics. Besides performing functional testing, we analyzed the simulation results focusing on the occurrence of false positives, false negatives, positional deviations and latency. Moreover, a professional conductor evaluated our system and provided qualitative feedback about it.
LicenseAuthors who publish with this journal agree to the following terms:
- Authors maintain the author's rights, under de Brazilian legal system, and retain copyright of the work. Authors license the work under a Creative Commons Attribution License. The authors grant the SBC Journal on Interactive Systems the right of its long term archiving and publication under this same licence, without restrictions of time and media, as well to make the required transformations to maintain the work archivable, accessible and of open access.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).