marcokuhn/ML

Machine Learning – Resources

ML – Machine Learning – Resources

• ML – Machine Learning – Resources
  • Communities
  • Tools
  • DATASETS
  • Audio Feature Extraction
  • MODELS
    • Diffusion MODELS
  • AUDIO PLUGINS
  • MIXING
    • ROEX
  • Source Seperation – Creating Stems
    • Spleeter
    • Demucs
    • DeMIX Pro
  • Audio Synthesis
    • RAVE and nn~
  • DRUMS
    • DrumGAN
    • CRASH – Diffusion Model
  • SYNTHESIS
    • VOICEOVER UBERDUCK.AI
  • Rhythm
  • Music Generation
  • Max/Msp
    • ml-lib – A machine learning library for Max and Pure Data
    • FLUCOMA
    • SP-Tools - Machine Learning tools for drums and percussion
    • MUBU – Max toolbox for multimodal analysis of sound and motion, sound synthesis and interactive machine learning
    • Gestural Sound Toolkit in Max/MSP for easy and fast Gesture-to-Sound scenario prototyping
    • RAVE (###rave-and-nn~)
  • General Libraries / API`S
  • LABS
  • Artists
  • TALKS
  • COURSES
  • EXAMPLES

Table of contents generated with markdown-toc

Communities

• https://huggingface.co/
• https://crossminds.ai/
• https://www.fast.ai/
• DagsHub - https://dagshub.com/ DagsHub is a platform for data scientists and machine learning engineers to version their data, models, experiments, and code. It allows you and your team to easily share, review, and reuse your work, providing a GitHub experience for machine learning.
• LAION AI - a non-profit organization to liberate machine learning research, models and datasets

Tools

• https://wandb.ai/site

DATASETS

• NSynth – https://magenta.tensorflow.org/datasets/nsynth https://docs.activeloop.ai/datasets/nsynth-dataset

• AudioSet – A large-scale dataset ofmanually annotated audio events from YouTube https://research.google.com/audioset/index.html

• EmoSynth: The Emotional Synthetic Audio Dataset https://zenodo.org/record/3727593#.YV6DrGJBzIU

• GENERAL ML DATASETS https://paperswithcode.com/datasets

• open-source audio datasets – https://github.com/DagsHub/audio-datasets

• Youtube 8M https://research.google.com/youtube8m/

• Audio Datasets - https://github.com/LAION-AI/audio-dataset

Audio Feature Extraction

• MTG Essentia – https://essentia.upf.edu/models.html
• Audio Commons Extractor – https://github.com/AudioCommons/ac-audio-extractor
• Librosa - https://librosa.org/doc/latest/index.html

MODELS

• https://github.com/lucidrains/musiclm-pytorch – Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch
• https://github.com/lucidrains/audiolm-pytorch – Implementation of AudioLM, a Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

Diffusion MODELS

• GITHUB REPOSITORY This repository contains a collection of resources and papers on Diffusion Models and Score-matching Models https://github.com/heejkoo/Awesome-Diffusion-Models
• Apply Denoising Diffusion Probabilistic Models using the new Hugging Face diffusers package to synthesize music instead of images – https://github.com/teticio/audio-diffusion
• Unconditional audio generation using diffusion models, in PyTorch https://github.com/archinetai/audio-diffusion-pytorch

AUDIO PLUGINS

• DDSP-VST – https://magenta.tensorflow.org/ddsp-vst
• RAVE VST - https://forum.ircam.fr/projects/detail/rave-vst/
• Download Build - https://github.com/acids-ircam/rave_vst/actions
• Steinberg Backbone - https://www.steinberg.net/vst-instruments/backbone/
• MACE AI by Tensorpunk - https://tensorpunk.com/

MIXING

ROEX

• ROEX WEBPAGE – https://www.roexaudio.com/
• ROEX & ABLETON LIVE – https://www.roexaudio.com/post/exporting-tracks-or-stems-from-ableton-into-roex-automix

Source Seperation – Creating Stems

Spleeter

• https://github.com/deezer/spleeter
• Spleeter for Max – https://github.com/diracdeltas/spleeter4max#spleeter-for-max

Demucs

• https://github.com/facebookresearch/demucs
• https://huggingface.co/spaces/akhaliq/demucs

DeMIX Pro

• https://www.audiosourcere.com/products/demix-pro-audio-separation-software

Audio Synthesis

RAVE and nn~

Realtime Neutral Audio Synthesis

• https://forum.ircam.fr/projects/detail/acids-projects/
• https://acids-ircam.github.io/cached_conv/
• https://caillonantoine.github.io/2021/11/18/rave.html
• https://www.youtube.com/watch?v=o09BSf9zP-0&t=982s
• RAVE with embedded Systems – https://www.youtube.com/watch?v=jAIRf4nGgYI

DRUMS

DrumGAN

• BLOG ABOUT DrumGAN –https://sites.google.com/view/drumgan/p%C3%A1gina-principal
• DrumGAN VST Site – https://cslmusicteam.sony.fr/drumgan-vst/
• DrumGAN IMPLEMENTATION – Steiberg Backbone - https://www.steinberg.net/vst-instruments/backbone/

CRASH – Diffusion Model

Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesis

• https://crash-diffusion.github.io/crash/

SYNTHESIS

DDSP-based Singing Vocoders: A New Subtractive-based Synthesizer and A Comprehensive Evaluation https://github.com/YatingMusic/ddsp-singing-vocoders/

Multi-instrument Music Synthesis with Spectrogram Diffusion https://github.com/magenta/music-spectrogram-diffusion

Audioldm – Text to Audio Generation – https://huggingface.co/spaces/haoheliu/audioldm-text-to-audio-generation

VOICEOVER UBERDUCK.AI

• https://github.com/uberduck-ai/uberduck-ml-dev
• https://app.uberduck.ai/editor#mode=tts-basic&voice=jayz

Rhythm

• Max for Live(M4L) Rhythm generator using Variational Autoencoder(VAE) - M4L.RhythmVAE
  • https://github.com/naotokui/RhythmVAE_M4L

Music Generation

• MusicGen: a simple and controllable model for music generation – https://github.com/facebookresearch/audiocraft
• MusicLM: Generating Music From Text – https://google-research.github.io/seanet/musiclm/examples/
• RIFFUSION: Stable Diffusion for Audio – https://www.riffusion.com/
• MUSIKA – Fast Infinite Waveform Music Generation – https://github.com/marcoppasini/musika
• MUBERT - create a new form of adaptive music with the help of AI – https://mubert.com/

Max/Msp

ml-lib – A machine learning library for Max and Pure Data

• https://github.com/irllabs/ml-lib
• https://www.benjamindaysmith.com/ml-machine-learning-toolkit-in-max

FLUCOMA

– https://www.flucoma.org/

SP-Tools - Machine Learning tools for drums and percussion

• https://rodrigoconstanzo.com/2022/07/sp-tools-machine-learning-tools-for-drums-and-percussion-alpha/
• https://github.com/rconstanzo/sp-tools

MUBU – Max toolbox for multimodal analysis of sound and motion, sound synthesis and interactive machine learning

• https://forum.ircam.fr/projects/detail/mubu/
• https://cycling74.com/articles/content-you-need-mubu
• https://ircam-ismm.github.io/max-msp/mubu.html#recording-playing-analyzing-and-visualizing-multimodal-data

Gestural Sound Toolkit in Max/MSP for easy and fast Gesture-to-Sound scenario prototyping

• https://github.com/bcaramiaux/Gestural-Sound-Toolkit

RAVE (###rave-and-nn~)

General Libraries / API`S

• A ranked list of awesome machine learning Python libraries
• https://github.com/ml-tooling/best-of-ml-python
• Magenta - An open source research project exploring the role of machine learning as a tool in the creative process.
  • https://magenta.tensorflow.org
• librosa is a python package for music and audio analysis. It provides the building blocks necessary to create music information retrieval systems
  • https://librosa.org/doc/latest/index.html
• jukebox, a neural net that generates music
  • https://openai.com/blog/jukebox/
• ACIDS – Artificial Creative Intelligence and Data Science
  • https://github.com/acids-ircam
  • https://forum.ircam.fr/projects/detail/acids-projects/

LABS

• Sony Computer Science Laboratories – https://csl.sony.fr/
• Artificial Creative Intelligence and Data Science (ACIDS) group is part of the RepMus team at IRCAM laboratory - https://acids.ircam.fr/
• Centre for Digital Music – Queen Mary University of London - https://c4dm.eecs.qmul.ac.uk/
• Music Technology Group – Universitat Pompeu Fabra in Barcelona – https://www.upf.edu/web/mtg

General Frameworks

• RTNEURAL – A lightweight neural network inferencing engine written in C++. This library was designed with the intention of being used in real-time systems, specifically real-time audio processing – https://github.com/jatinchowdhury18/RTNeural

Artists

• Holly Herndon – https://www.artnews.com/art-in-america/interviews/holly-herndon-emily-mcdermott-spawn-ai-1202674301/

  • https://hollyherndon.ffm.to/proto
  • https://www.hollyherndon.com/

• Mouse on Mars – AAI

  • https://mouseonmarstj.bandcamp.com/album/aai

• Aphex Twin & Dave Griffiths

  • https://cdm.link/2022/09/free-sample-mashing-with-samplebrain-by-aphex-twin-and-dave-griffiths/?fbclid=IwAR34MQqmvqDLLlsqu4FI1CVqaZPhGYZ2hMvELTNO45em87dCQ1mb4Pi1iNA
  • Samplebrain
  • https://thentrythis.org/projects/samplebrain/?fbclid=IwAR1qkejBiWrBDLo64-Rjdp0-GDj4hNsGSuV7IgVBApMK_vE2E4cOvM--xaY
  • https://gitlab.com/then-try-this/samplebrain

TALKS

• Neural Audio Synthesis and Restoration with Generative Adversarial Networks-Stefan Lattner-&inCSL https://www.youtube.com/watch?v=IET33sxcaRY
• Interactive Neural Audio Synthesis https://crossminds.ai/video/interactive-neural-audio-synthesis-60774cd8825a3436b95eecdb/

COURSES

• The Sound of AI by Valerio Velardo – https://www.youtube.com/channel/UCZPFjMe1uRSirmSpznqvJfQ

EXAMPLES

• Stable Diffusion for Music Video by Tristan Behrens – https://www.youtube.com/watch?v=Oc-NbvUaTzA