armonIA – a synesthetic sound and vision experience

ArmonIA stands as an innovative interactive installation that transforms the process of musical creation into a uniquely engaging experience. This artwork invites participants to immerse themselves in the act of making music by simply moving their hands, akin to a conductor directing an orchestra. A collaborative masterpiece, ArmonIA seamlessly merges the art of gesture recognition with artificial intelligence, crafting a synesthetic journey where sound and vision harmoniously converge.

At its core, ArmonIA relies on the interaction between users and advanced AI capable of interpreting hand movements and translating them into original musical compositions.
To initiate the experience, users position their hands above the lectern. The right hand functions as a virtual instrument, generating sounds while illuminating virtual objects akin to notes on a keyboard. Simultaneously, the left hand governs the rotation of three-dimensional objects and applies audio filters, shaping the composition in a unique and personalized manner.

The artificial intelligence underpinning ArmonIA employs sophisticated computer vision technology to detect and interpret hand movements with an impressive 99% precision. An infrared sensor captures images, the computer vision software isolates hand positions, and these data are then interpreted and converted into parameters for the algorithmic generation of melodies and rhythms.