Carscendo’s mission is to empower non-musical people to be creators, with a new medium for music composition. We aim to revolutionise accessibility and fun in music creation.
From real-world users:
1. Learning music isn’t fun, getting good takes >10,000 hours: Our parents forced 2 members of our team to learn a musical instrument as kids. We hated it. It simply wasn’t fun. It took us a long time to reach a level high enough to compose music. We wanted to make music composition more accessible and fun for absolute beginners and people who wouldn’t normally associate music/art with themselves.
2. Most people can drive, few can play an instrument: The majority of adults are able to drive a vehicle, while only those that went through childhood toil can play an instrument – we saw some potential there. Can we use what people already know? To make music? To learn about music? To create? Music/playing an instrument is seen as an elitist hobby, many adults stay away from it – we want to break that assumption.
3. Dyslexic, Deaf, Blind, Paraplegia – Music Accessibility: How can the deaf hear music? How can the Dyslexic and Blind read sheet music? How can the physically paralysed (Paraplegia) play an instrument? We wanted to design a new form of music information, a new medium of sense to convey what these groups of people lack. Inspiration from a Taste Visualization scene from The movie “Ratatouille”
Carscendo’s is to provide a passive music compositional platform through the innovative medium of a multiplayer VR racing experience.
Through a combination of environmental and inter-vehicular interactions, the way a player races is reflected through changes in 3 core musical elements: emphasis, tone, and rhythm. By playing Carscendo, users who may consider themselves not musically inclined and discouraged from using conventional composition systems are able to passively develop an intuition for the core principles of music composition while enjoying a multiplayer racing experience.
The car racing aspects not only provide an interface that people of most ages and backgrounds are familiar with as a concept (driving a car), but allow non-musicians to derive enjoyment from practice even if their musical skills are poor. By giving players this alternative outlet for enjoyment, they are encouraged to play for long enough time to develop musical intuition without even realizing it.
1. Crescendo’s capabilities grow with the player’s experience level as they play the game
As players improve their skills in the game, they gain greater agency over the music their ‘drive’ produces, meaning that new and experienced players alike have something foundational to enjoy together.
2. Carsendo should serve as a fun experience for anyone to play and learn from- regardless of physical ability
While the mainstream VR gaming industry is still in its early stages, we think it crucial to set the standards for inclusivity higher than they have been for conventional games.
Many games assume a high level of physical and cognitive ability, thus players who may have impairments in either of those two areas either have to exert considerably more effort than a fully abled player, not be able to enjoy many facets of the experience, or not be able to play the game at all.
In the design of Carscendo, we implemented the gameplay mechanics in as inclusive a way as possible. Some of these specific considerations include:
- visual elements paired with any audio feedback for hard of hearing or deaf players
- controls that require low levels of digit dexterity for players with tremors or joint issues
- a UI system based on environment interactions rather than text for those who struggle reading, and so it can be easily understood regardless of language.
How we improve accessibility and empower creativity & fun:
1. More Accessible and Fun medium of Music Composition: built-into racing game mechanics:
- Car selection chooses a background track.
- Driving at different speeds changes the rhythm and volume of the background track.
- Collecting power-ups (like mario-kart) on the road, adds melodies and tunes.
- Looping feature allows remixing and deeper composing.
- Record feature allows exporting of creation.
- Multiplayer allows collaboration between experienced/inexperienced friends.
Driving is the mechanism, the proof of concept we are demonstrating – that music can be created in more modern ways with current technology than the old strings and drums. In the future, we want to explore other mediums/methods/mechanisms to make more people creators.
2. Dyslexic, Deaf, Blind, Paraplegia – Music Accessibility:
- Deaf: we represent music through stunning audio visualisations
- Dyslexic: we represent music notes through patterns and shapes (easier to read)
- Blind: we use haptic feedback and vibrations to aid sensory
- Physically paralysed: we use oculus quest 2 eye tracking to allow interactions of driving
- VR: Oculus Quest, Unity3D, C#
- 3D Modelling & Animation: Autodesk Maya, Adobe Photoshop, Unity3D, Blender
- UX & UI: Figma, Unity2D, Unity3D
- Graphic Design: Adobe Photoshop, Procreate, Adobe Illustrator
- Audio Visualisation: Frequency analysis, Fast Fourier Transform, Koch Fractals, Audio Spectrum Analysis
- Android version: CAD for 3D printable steering wheel with android phone housing
Challenges:
- VR Interaction physics are really difficult to write from the ground up.
- Off the shelf assets vary in quality. They can be either or low or high poly and the buyer has no way of knowing before hand. Placing market assets into the scene tanked our performance and we had to find other ways to make our scene pretty.
- The design of audio visualisation is just as important as the functionality.
- The performance behind features can break whether it's even able to be included in final solution. We made 3 audio visualisations (for background track, for power-ups, for in-car), while running on a computer was fine – it was too heavy to include more than one for the headset.
For the scope of this hackathon our goal was to establish the ground work for how an unconventional medium could manipulate music while simultaneously standing alone as a racing simulator that users would enjoy playing. We knew that creating a game of any kind in 36 hours- let alone one that had extremely innovative functionalities was an extremely ambitious task. The combination of virtual reality, real-time audio visualisation, and reactive audio based on user interactions, was a monumental goal.
Each member of the team utilised challenging technology, and as a result learned a lot about Unity during the last 36 hours! We learned how to design, train and test a full racing simulator and audio visualisation system in Unity and build 3D models and UI elements in VR.
This project really helped us have a better understanding of many of the capabilities within Oculus, and in utilising signal processing to create a new form of expression to use in an gamified setting. We learned so much through this project and from each other, and had a really great time working as a team!
. From here some things we want to build on on in the future include
- Multiplayer Game Mode Online
- Player Map development capabilities
- More interactive musical control elements
- Environment interactions as basis for every menu action
- Real world driving - song records
- Other unconventional music platforms (how might other common tasks like cooking, etc. be used to help people learn new things?