LEZIONI DI MUSICA A DISTANZA | Remote Music Lessons
WHY IT IS RELEVANT The sudden social isolation due to Convid-19 has turned our lifestyles upside down: we can't go out and socialise and often we get bored because we don't know how to spend our time during the day. This combined with what the media tells us causes a physical and mental malaise. However, we have more time for ourselves that can be used for something useful by socializing remotely.
IDEA CONCEPT The project is a web platform based on the technology WebRTC and pitch detection algorithm. With this platform the teacher can create a private digital room and share the link with the student. WebRTC is the same technology that Google Hangout uses that allows browser-to-browser audio, video and data sharing. This platform is designed to help beginners learn to read music on their chosen musical instrument. The note played by the teacher or the student on their musical instrument is compared with the correct note on the music score and if they are identical it is highlighted in green and they will be allowed to play the following note. If the note is different the correct note will be highlighted in red and only when the correct note is produced the system will continue.
DEVELOPMENT This project is based on the concept I worked on years ago for an hackathon: https://www.youtube.com/watch?v=u5u1JmdiY98&t=101s The dection for the note was by a pitch detection JavaScript API. However, it wasn't that precised.
Nowaday it's possible to use machine-learning to improve pitch detection: TensorFlow + AudioInputs (YouTube) - https://www.youtube.com/watch?v=-1QGEQWhmSI P5.js: https://learn.ml5js.org/docs/#/reference/pitch-detection P5.js - YouTube:https://www.youtube.com/watch?v=F1OkDTUkKFo Onsets and Frames: Dual-Objective Piano Transcription: https://magenta.tensorflow.org/onsets-frames