/folk-rnn

folk music modelling with LSTM

Primary LanguagePythonMIT LicenseMIT

My image

Folk music style modelling using LSTMs

See the following websites:

  1. Let's Have Another Gan Ainm (music album)
  2. folkrnn.org
  3. The Machine Folk Session

This code was used for the following published works:

  1. Sturm and Ben-Tal, "Let’s Have Another Gan Ainm: An experimental album of Irish traditional music and computer-generated tunes"

  2. Sturm, Ben-Tal, Monaghan, Collins, Herremans, Chew, Hadjeres, Deruty and Pachet, “Machine learning research that matters for music creation: A case study,” J. New Music Research 48(1):36-55, 2018.

  3. Sturm "How Stuff Works: LSTM Model of Folk Music Transcriptions", invited presentation Joint Workshop on Machine Learning for Music, 2018.

  4. Sturm, “What do these 5,599,881 parameters mean? An analysis of a specific LSTM music transcription model, starting with the 70,281 parameters of its softmax layer,” in Proc. Music Metacreation workshop of ICCC, 2018.

  5. Sturm and Ben-Tal, "Taking the Models back to Music Practice: Evaluating Generative Transcription Models built using Deep Learning,” J. Creative Music Systems, Vol. 2, No. 1, Sep. 2017.

  6. Sturm, Santos, Ben-Tal and Korshunova, "Music transcription modelling and composition using deep learning", in Proc. 1st Conf. Computer Simulation of Musical Creativity, Huddersfield, UK, July 2016.

  7. Sturm, Santos and Korshunova, "Folk Music Style Modelling by Recurrent Neural Networks with Long Short Term Memory Units", Late-breaking demo at the 2015 Int. Symposium on Music Information Retrieval

  8. The folk-rnn v1, v2 and v3 Session Books https://highnoongmt.wordpress.com/2018/01/05/volumes-1-20-of-folk-rnn-v1-transcriptions/

  9. 47,000+ tunes at The Endless folk-rnn Traditional Music Session http://www.eecs.qmul.ac.uk/~sturm/research/RNNIrishTrad/index.html

Music compositions resulting from versions of this code:

  1. Laura Agnusdei and guest perform some output of folkrnn at the 2019 ReWire festival https://www.thoughtsource.org/vimeo_tab/app/tab/view_video?fbPageId=109823292380435&page=1&videoId=345886314

  2. "Bastard Tunes" by Oded Ben-Tal + folk-rnn (v2) (2017) https://www.youtube.com/playlist?list=PLdTpPwVfxuXpQ03F398HH463SAE0vR2X8

  3. "Safe Houses" by Úna Monaghan + folk-rnn (v2) (for concertina and tape, 2017) https://youtu.be/x6LS9MbQj7Y

  4. "Interpretations of Computer-Generated Traditional Music" by John Hughes + folk-rnn (v2) (for double bass, 2017) https://youtu.be/GmwYtNgHW4g

  5. "Dialogues with folk-rnn" by Luca Tuchet + folk-rnn (v2) (for smart mandolin, 2017) https://youtu.be/pkf3VqPieoo; at NIME 2018 https://youtu.be/VmJdLqejb-E

  6. "The Fortootuise Pollo" by Bob L. Sturm + folk-rnn (v1) (2017) https://soundcloud.com/sturmen-1/the-fortootuise-pollo-1

  7. "March to the Mainframe" by Bob L. Sturm + folk-rnn (v2) (2017) Performed by Ensemble x.y: https://youtu.be/TLzBcMvl15M?list=PLdTpPwVfxuXrdOyjtwfokrpzfpIlnJc5o Performed by Ensemble Volans: https://soundcloud.com/sturmen-1/march-to-the-mainframe-by-bob-l-sturm-folk-rnn-v2 Score is here: https://highnoongmt.files.wordpress.com/2017/12/twoshortpieceswithaninterlude.pdf

  8. "Interlude" by Bob L. Sturm + folk-rnn (v2) (2017) Performed by Ensemble x.y: https://youtu.be/NZ08dDdYh3U?list=PLdTpPwVfxuXrdOyjtwfokrpzfpIlnJc5o Performed by Ensemble Volans: https://soundcloud.com/sturmen-1/interlude-by-bob-l-sturm-folk-rnn-v2 (synthesized version: https://soundcloud.com/sturmen-1/interlude-synthesised) Score is here: https://highnoongmt.files.wordpress.com/2017/12/twoshortpieceswithaninterlude.pdf

  9. "The Humours of Time Pigeon" by Bob L. Sturm + folk-rnn (v1) (2017) Performed by Ensemble x.y: https://youtu.be/1xBisQK8-3E?list=PLdTpPwVfxuXrdOyjtwfokrpzfpIlnJc5o Performed by Ensemble Volans: https://soundcloud.com/sturmen-1/the-humours-of-time-pigeon-by-bob-l-sturm-folk-rnn-v1 (synthesized version: https://soundcloud.com/sturmen-1/the-humours-time-pigeon-synthesised) Score is here: https://highnoongmt.files.wordpress.com/2017/12/twoshortpieceswithaninterlude.pdf

  10. "Chicken Bits and Bits and Bobs" by Bob L. Sturm + folk-rnn (v1) (2017) https://youtu.be/n-avS-ozrqU Score is here: https://highnoongmt.files.wordpress.com/2017/04/sturm_chicken.pdf

  11. "The Ranston Cassock" by Bob L. Sturm + folk-rnn (v1) (2016) https://youtu.be/JZ-47IavYAU (Version for viola and tape: https://highnoongmt.wordpress.com/2017/06/18/the-ranston-cassock-take-2/)

  12. Tunes by folk-rnn harmonised by DeepBach (2017)

    1. "The Glas Herry Comment" by folk-rnn (v1) + DeepBach (2017) https://youtu.be/y9xJl-ljOuA
    2. "The Drunken Pint" by folk-rnn (v1) + DeepBach (2017) https://youtu.be/xJyp7vBNVA0
    3. X:633 by folk-rnn (v2) + DeepBach (2017) https://youtu.be/BUIrbZS5eXc
    4. X:7153 by folk-rnn (v2) + DeepBach (2017) https://youtu.be/tdKCzAyynu4

  13. Tunes from folk-rnn v1 session volumes

    1. "The Cunning Storm" adapted and performed by Bob L. Sturm and Carla Sturm (2018) https://highnoongmt.wordpress.com/2018/01/07/the-cunning-storm-a-folk-rnn-v1-original/
    2. "The Irish Show" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/11/23/folk-rnn-v1-tune-the-irish-show/
    3. "Sean No Cottifall" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/11/11/folk-rnn-v1-tune-sean-no-cottifall/
    4. "Optoly Louden" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/07/01/optoly-louden-a-folk-rnn-original/
    5. "Bonny An Ade Nullway" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/18/bobby-an-ade-nullway-a-folk-rnn-v1-tune/
    6. "The Drunken Pint" adapted and performed by Bob L. Sturm (2017) https://youtu.be/omHhyVD3PD8; performed by EECSers (2017) https://youtu.be/0gosLln8Org
    7. "The Glas Herry Comment" adapted and performed by Bob L. Sturm (2017) https://youtu.be/QZh0WSjFFDs; performed by EECSers (2017) https://youtu.be/NiUAZBLh2t0
    8. "The Mal's Copporim" adapted and performed by Bob L. Sturm (2016) https://youtu.be/YMbWwU2JdLg; performed by EECSers (2017) https://youtu.be/HOPz71Bx714
    9. "The Castle Star" adapted and performed by Bob L. Sturm (2015) https://highnoongmt.wordpress.com/2015/08/12/deep-learning-for-assisting-the-process-of-music-composition-part-2/
    10. "The Doutlace" adapted and performed by Bob L. Sturm (2015) https://highnoongmt.wordpress.com/2015/08/11/deep-learning-for-assisting-the-process-of-music-composition-part-1/

  14. Tunes from the folk-rnn v2 session volumes

    1. Transcriptions 1469, 1470 & 1472 performed by Torbjorn Hultmark (2016) https://youtu.be/4kLxvJ-rXDs
    2. X:488 performed by Bob L. Sturm (2017) https://youtu.be/QWvlnOqlSes; performed by EECSers (2017) https://youtu.be/QWvlnOqlSes
    3. X:4542 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/10/06/folk-rnn-v2-tune-4542/
    4. X:2857 adapted and performed by Bob L. Sturm and Carla Sturm (2017) https://highnoongmt.wordpress.com/2017/12/02/folk-rnn-v2-tune-2857/

  15. Tunes from the folk-rnn v3 session volumes

    1. "The 2714 Polka" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/10/the-2714-polka-a-folk-rnn-original/
    2. X:1166 Performed by Weltsauerstoff (2018) https://soundcloud.com/weltsauerstoff/1166-conquest-of-brittany
    3. X:1166 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/16/folk-rnn-v3-tune-1166/, https://youtu.be/avxXRNJvUMk (2018)
    4. X:1650 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/17/folk-rnn-v3-tune-1650/
    5. X:6197 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/25/folk-rnn-v3-tune-6197/
    6. X:8589 (A Derp Deep Learning Ditty) adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/10/08/a-derp-deep-learning-ditty/

  16. "Why are you and your 6 million parameters so hard to understand?" by folk-rnn (v2) and Bob L. Sturm (2018) https://highnoongmt.wordpress.com/2018/01/21/making-sense-of-the-folk-rnn-v2-model-part-8/

  17. "Two Burner Brew No. 1" by folk-rnn (v2) and Bob L. Sturm and Carla Sturm (2018) https://soundcloud.com/sturmen-1/two-burner-brew-no-1

  18. "A Windy Canal" by folk-rnn (v2) and Bob L. Sturm (2018) https://highnoongmt.wordpress.com/2018/01/11/making-sense-of-the-folk-rnn-v2-model-part-6/

  19. "Swing Swang Swung" by folk-rnn (v2) and Bob L. Sturm (2018) https://youtu.be/Y_DBRg6SK7E (analysis here: https://highnoongmt.wordpress.com/2018/01/03/making-sense-of-the-folk-rnn-v2-model-part-5/ )

  20. "Experimental lobotomy of a deep network with subsequent stimulation (2)" by folk-rnn (v2 with lobotomy) and Bob L. Sturm, Carla Sturm (2018) https://highnoongmt.wordpress.com/2018/01/13/making-sense-of-the-folk-rnn-v2-model-part-7/

  21. "The Millennial Whoop Reel" by Bob L. Sturm + folk-rnn (v2) (2016) https://highnoongmt.wordpress.com/2016/08/29/millennial-whoop-with-derp-learning-a-reel/

  22. "The Millennial Whoop Jig" by Bob L. Sturm + folk-rnn (v2) (2016) https://highnoongmt.wordpress.com/2016/08/28/millennial-whoop-with-derp-learning/

  23. "Eight Short Outputs ..." by folk-rnn (v1) + Bob L. Sturm (2015) https://highnoongmt.wordpress.com/2015/12/20/eight-short-outputs-now-on-youtube/

  24. "Carol of the Cells" by Bob L. Sturm + folk-rnn (v2) (2017) https://highnoongmt.wordpress.com/2017/12/16/carol-of-the-cells-from-the-ai-to-the-orchestra/

  25. "It came out from a pretrained net" by Bob L. Sturm + folk-rnn (v2) (2016) https://highnoongmt.wordpress.com/2016/12/24/taking-a-christmas-carol-toward-the-dodecaphonic-by-derp-learning/

  26. “We three layers o’ hidd’n units are” by Bob L. Sturm + folk-rnn (v2) (2015) https://highnoongmt.wordpress.com/2015/12/16/tis-the-season-for-some-deep-carols/

  27. "The March of Deep Learning" by Bob L. Sturm + folk-rnn (v1) (2015) https://highnoongmt.wordpress.com/2015/08/15/deep-learning-for-assisting-the-process-of-music-composition-part-4/

Media

  1. July 2 2019, "Deep learner generated Irish folk music", Bruce Sterling Wired
  2. June 20 2019, The Sound of AI (Medium), "Folk the algorithms: traditional music in AI music research"
  3. April 15 2019 The Why Factor, BBC, "Separating the art from the artist"
  4. April 17 2019 Svergies Radio, Musikguiden P3, "Artist + AI = evigt liv"
  5. March 2019 Mother Jones, Clive Thompson, What Will Happen When Machines Write Songs Just as Well as Your Favorite Musician?
  6. February 19 2019 BBC Sounds, Science and Stuff, How music works
  7. February 26 2019 BBC Click, Bach vs. AI
  8. Dec. 18 2018, "How to make sweet sounding music with a hard drive", BBC Future
  9. Dec. 18 2018, "These AI startups are creating chart-worthy music with algorithms" TechWorld
  10. Nov. 25 2018 "Missing Link: Musik ohne Musiker? KI schwingt den Taktstock " Heise Online
  11. Nov. 2 2018, "Så har en artificiell intelligens skapat 100.000 folkmusiklåtar", Dagens Nyheter
  12. Sep. 2018, "AI created more than 100,000 pieces of music after analyzing Irish and English folk tunes", KTH research news
  13. Sep. 2018, "Do computers have musical intelligence?", Science Node
  14. Sep. 2018, "AI skapar 100 000 folkmusiklåtar", Musikindustrin (Sweden)
  15. May 2018, "No humans required? If computers can be taught to compose creatively, what does that mean for the future of music?", BBC Music Magazine, Alex Marshal
  16. Feb. 28 2018, "Is music about to have its first AI No.1?" https://www.bbc.co.uk/music/articles/0c3dc8f7-4853-4379-b0d5-62175d33d557
  17. Dec. 23 2017 "AI Has Been Creating Music and the Results Are...Weird" PC Mag (http://uk.pcmag.com/news/92577/ai-has-been-creating-music-and-the-results-areweird)
  18. Nov. 18, 2017 Le Tube avec Stéphane Bern et Laurence Bloch, France https://www.youtube.com/watch?v=LQQER9479Xk
  19. Sep. 25 2017, "Intelligenza artificiale crea più di 100.000 nuovi brani musicali folk" Notizie scientifiche.it
  20. March 31 2017, "‘Machine folk’ music composed by AI shows technology’s creative side", The Conversation (https://theconversation.com/machine-folk-music-composed-by-ai-shows-technologys-creative-side-74708)
  21. June 3 2017, "An A.I. in London is Writing Its Own Music and It Sounds Heavenly" https://www.inverse.com/article/32276-folk-music-ai-folk-rnn-musician-s-best-friend
  22. June 8 2017, "Computer program created to write Irish trad tunes" http://www.irishtimes.com/business/technology/computer-program-created-to-write-irish-trad-tunes-1.3112238
  23. June 19 2017 "Folk-RNN is the Loquantur Rhythm artist of June" (providing music for phone call waits) https://zc1.campaign-view.com/ua/SharedView?od=11287eca6b3187&cno=11a2b0b20c9c037&cd=12a539b2f47976f3&m=4 (Here is a sample: https://highnoongmt.wordpress.com/2017/06/17/deep-learning-on-hold/)
  24. June 18 2017, "Real Musicians Evaluate Music Made by Artificial Intelligence" https://motherboard.vice.com/en_us/article/irish-folk-music-ai
  25. June 1 2017, "Can an AI Machine Hold Copyright Protection Over Its Work?" https://artlawjournal.com/ai-machine-copyright/
  26. May 26 2017 The Daily Mail named our project "Bot Dylan" (http://www.dailymail.co.uk/sciencetech/article-4544400/Researchers-create-computer-writes-folk-music.html), and then didn't even link to this page. Plus the video they edited has no computer-generated music in it. Well done!
  27. April 13 2017 “Eine Maschine meistert traditionelle Folk-Music” http://www.srf.ch/kultur/netzwelt/eine-maschine-meistert-traditionelle-folk-music