X.E.K.I

Gestural keyboard instrument merging body movement with sound synthesis.

X.E.K.I. is an experimental musical instrument that integrates body movement and expressive gestures to create unique and dynamic sounds using a smartphone as the core synthesizer.

Client / Team
R&D Studio 2023 (ITP) /
Peter Zhang, Henry Chen, Orpheas Kofinakos

Role
Fabrication Lead & Project Manager

Tools
Rhino 3D, Processing, Arduino, 3D Printing

Inspiration

Since all three of us shared a passion for music production and live performances, we wanted to create a product for professional musicians and music enthusiasts who wish for a more unconventional instrument with more expressive capabilities.

Expressive Slider

The idea for a slider as the primary interaction came after extensive research on the types of gestures most suited for expressive tune-bending and improvisation. The literal movement of the slider, akin to that of a trombone, provided players with a degree of control and modulation well suited for live-performances.

1. Sliding Mechanism

The design of the sliding mechanism proved more difficult since we needed a component to translate lateral movement into data the Arduino can read. The final solution we came up with however was to use a rotary encoder along a set of rails to calculate the rotational displacement instead. This method avoided the need to purchase any specialized railing system that would’ve costed more and taken longer to integrate.

2. Handle Grip

Since the slider is such a crucial part of the user interaction, the grip of the handle needed to be ergonomic and comfortable to hold for extended periods of time. For that, we produced several iterations of the grip. We also placed two buttons on either side of the handle to allow for yet another controllable parameter.

3. Keyboard Layout

We debated creating a completely new music interface for playing notes on X.E.K.I. However, given that most musicians are familiar with the piano keyboard , we proposed an alternative stratified layout that divides a single octave into two rows. This layout allows users to reach all the keys in an octave without needing to move laterally across the instrument too much.

4. Tilt Control

Part of the studio’s requirements was to make use of the various sensors inside the Pixel 3. We chose to utilize the accelerometer which is used to determine the orientation of the phone. Using the accelerometer data from the phone, users can tilt at different angles to achieve subtle modulation in the sound quality, allowing for expressive adjustments and improvisation.

5. Accessibility

Since one of our team members (Henry) is ambidextrous, meaning he uses both his left and right hand, we made an extra accessibility consideration when designing to accommodate for both types of users. Because the keyboard had a directionality and is angled, it could not be operated using the same arm. Instead, we developed an adjustable shoulder rest mechanism that can switch sides easily, allowing the instrument to be played both right and left-handed.

Prototypes

To get a sense of the scale and feasibility of the sliding mechanism, we built a low and medium-fidelity prototype. The low fidelity prototype proved that a shoulder rest-like mechanism was superior to a stock mechanism. The medium-fidelity prototype made it clear that arcade buttons, due to their size, were not ideal for the compact keyboard layout we had envisioned, so we moved onto mechanical keys instead.

Low-fidelity

Medium-fidelity

Materiality

For the slider handle, we went with a 3D-printed nylon material that is sturdy and gives off a smooth finish. The instrument chassis is built up of varying types of acrylic. The semi-transparent nature of acrylic allows users to see the mechanism at play while using the instrument.

Challenges

Putting together all of X.E.K.I.’s indivdiual components was no small undertaking. Our biggest oversight was not taking the amount of clearances the wires needed into proper consideration when designing the chassis. The end result was trying to pack way too much into a small volume that made putting the instrument together a nightmare. Luckily, we managed to get it done just in time for final.

Code

Arduino takes the button presses and rotary encoder data and sends it as Midi inputs to a third-party synthesizer app on the Pixel 3. Furthermore, the accelerometer data from the Pixel 3 is fed into a Processing sketch on the phone which is mapped onto another control parameter on the app.