AHO ML / AI Workshop - Spring 2022
Code and links to relevant course materials for the AHO ML/AI workshop.
For the workshop we recommend using a modern browser, such as Chrome, Firefox or Safari.
Most of the exercises can be done in the p5.js editor, but it's also possible to download the code in this repository, edit and run the code examples locally on your computer.
For those of you who are interested in editing code locally on your computer, it's possible to use either Notepad (on Windows) or TextEdit (on Mac), but we recommend downloading a more advanced code editor such as VS Code or Sublime Text.
If you want to run the code examples in this repository, you'll need to set up a local webserver. You can do this by either using the Chrome extension Web Server for Chrome:
- install the Chrome extension Web Server for Chrome
- Start the extension
- Click
choose folder
and point to the folder with these files.
or using Python (which should be installed by default on Mac) :
- Open
Terminal
app - Browse to the folder with these files via
cd
command - run
python -m SimpleHTTPServer 8887
After this, you should be able to go to http://localhost:8887
in your browser, and browse the files in this folder.
Day 1 : Intro to Machine Learning / classification
Intro to p5.js:
- p5.js editor : https://editor.p5js.org/
- get started with p5.js : https://p5js.org/get-started/
- p5.js reference : https://p5js.org/reference/
Basic p5.js examples:
- Changing color of background : https://editor.p5js.org/auduno/sketches/zxc0Sx_8N
- Drawing a rectangle to canvas : https://editor.p5js.org/auduno/sketches/1QyFc6DS5
- Animated circle : https://editor.p5js.org/auduno/sketches/JxPvAFIdR
- Drawing text to canvas : https://editor.p5js.org/auduno/sketches/ecoP857SC
- Drawing an image to canvas : https://editor.p5js.org/auduno/sketches/gPZNIxFZG
- Setting up a webcam : https://editor.p5js.org/auduno/sketches/7x2BF9vor
- Playing sound : https://editor.p5js.org/auduno/sketches/FVRuyGz8w
- Adding buttons (to for instance play and stop music) : https://editor.p5js.org/auduno/sketches/u4PxBJvZ6
- Generating sound with p5 oscillator : https://editor.p5js.org/auduno/sketches/IvAlNYPAD
- Generating speech with p5 speech : https://editor.p5js.org/auduno/sketches/bQapdgSOi
- Adding some HTML outside canvas : https://editor.p5js.org/auduno/sketches/kYfuqZM7a
ML5js examples in p5.js editor:
- Image classification : https://editor.p5js.org/auduno/sketches/Z_U6BPK3g
- Image classification (with webcam) : https://editor.p5js.org/auduno/sketches/gkWssn0Es
- Audio classification : https://editor.p5js.org/auduno/sketches/0IkUY99f7
- Object detection with Coco-SSD : https://editor.p5js.org/auduno/sketches/4OBrXYka
- PoseNet : https://editor.p5js.org/auduno/sketches/9sHWX67Bx
Teachable machine models in p5.js:
- Image classification (with webcam) : https://editor.p5js.org/auduno/sketches/9gNqp00Qd
- Audio classification : https://editor.p5js.org/auduno/sketches/LGa7ekHAY
- PoseNet : https://editor.p5js.org/auduno/sketches/VaMqUyZJO
Examples of combining teachable machine models to do something in p5.js :
- Recognition of hand signals : https://editor.p5js.org/auduno/sketches/6L43dcupy
- Audio control : https://editor.p5js.org/auduno/sketches/8bQUExhTX
- Medal + cheers with posenet : https://editor.p5js.org/auduno/sketches/OCo40cnQ7
Bonus examples:
- https://editor.p5js.org/ml5/sketches/Facemesh_Webcam
- https://editor.p5js.org/ml5/sketches/Handpose_Webcam
Resources:
Day 2 : Workshop, similarity and search, Generative ML
Search and Recommendations
- How spotify recommender works : https://qz.com/571007/the-magic-that-makes-spotifys-discover-weekly-playlists-so-damn-good/
- Convolutional nets for audio classification (Spotify) : https://benanne.github.io/2014/08/05/spotify-cnns.html
- Netflix recommendations : https://uxplanet.org/netflix-binging-on-the-algorithm-a3a74a6c1f59
- How facebook news feed works : https://tech.fb.com/news-feed-ranking/
- The making of a YouTube Radical (on negative effects of YouTube recommendations) : https://www.nytimes.com/interactive/2019/06/08/technology/youtube-radical.html
Similarity maps
- http://vy.nasjonalmuseet.no
- http://fontmap.ideo.com/
- https://experiments.withgoogle.com/ai/drum-machine/view/
- https://cs.stanford.edu/people/karpathy/cnnembed/
- https://projector.tensorflow.org/
Sequential classifier models
Generative Machine Learning
Generative drawing:
Generative text:
- Talk to transformer / InferKit (GPT-3)
- Text Synth (GPT-J, open-source alternative)
- Article on turing-testing GPT-3
- OpenAI GPT-3 examples (need to create user account to access): *Chat *Translation
- AI Dungeon
- Google Copilot
- Is GitHub Copilot a blessing, or a curse? : https://www.fast.ai/2021/07/19/copilot/
Generative music/sound:
- Performance RNN
- Folk-RNN
- Google Bach Doodle RNN - more info here
- Blob Opera
- Musenet
- Music transformer
- Jukebox
- A-ha continuation example
- Africa continuation example
- Scatman John continuation example
- Daft Punk continuation example
- Google Colab Notebook to generate Jukebox samples (a bit technical) : https://colab.research.google.com/drive/1gWP6fSqd_vs65tI5uOGcJssGqTq2YWzO
Generative Adversarial Networks
GAN examples:
- BigGAN class visualization : https://www.youtube.com/watch?v=YY6LrQSxIbc&feature=youtu.be
- StyleGAN presentation : https://www.youtube.com/watch?v=kSLJriaOumA&feature=youtu.be&t=25s
- StyleGAN faces : https://thispersondoesnotexist.com/
- StyleGAN cars interpolation : https://youtu.be/OLZ3-ZJwSu4
- StyleGAN trained on photobooth photos : https://www.reddit.com/r/Damnthatsinteresting/comments/9udese/plugging_50000_portraits_into_facial/?utm_source=share&utm_medium=web2x
- StyleGAN trained on Garfield : https://twitter.com/calamardh/status/1148655219032690692
- A bit technical article on training stylegan models : https://www.gwern.net/Faces
Conditional GANs / Image translation:
- Pix2Pix Demo
- Edges2Pikachu
- Celebrity Pix2Pix
- GauGAN/SPADE online (NB! you may need to zoom out in the browser to see the entire interface)
- Pix2pix applied to video
- Pix2pix applied to architecture (spacemaker)
- Edges2Santas
Hands-on:
Collecting images
Potential sources of images :
- Google image search : https://images.google.com
- Nasjonalbiblioteket bilder : https://www.nb.no/search?mediatype=bilder
- Digitalt museum : https://digitaltmuseum.no/search/?q=stol
- Flickr : https://flickr.com/search/
- A variety of botanical illustrations from Flickr : https://www.flickr.com/photos/biodivlibrary/albums
Tools:
- Download all images chrome extension : https://chrome.google.com/webstore/detail/download-all-images/ifipmflagepipjokmbdecpmjbibjnakm?hl=en
Day 3 : Training models, Multi-modal models
Controlling StyleGAN
- Artbreeder : https://www.artbreeder.com/browse
CLIP:
- CLIP : https://openai.com/blog/clip/
- Visualizing CLIP : https://distill.pub/2021/multimodal-neurons/
- paint.wtf : https://paint.wtf/
CLIP-GLaSS (CLIP + StyleGAN) :
- Default : https://colab.research.google.com/drive/1fWka_U56NhCegbbrQPt4PWpHPtNRdU49
- With your trained model : https://colab.research.google.com/drive/1OHmnnV3CuHxTuQ9wIY8ea9SFBkcKjY_J
Text-to-image generation
DALL·E :
- DALL·E : https://openai.com/blog/dall-e/
- ruDALL-E : https://rudalle.ru/en/demo
CLIP + Guided diffusion / VQGAN models:
- Disco Diffusion v5 : https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb
- https://huggingface.co/spaces/EleutherAI/VQGAN_CLIP
- https://huggingface.co/spaces/EleutherAI/clip-guided-diffusion
- https://huggingface.co/spaces/valhalla/glide-text2im
- CLIP Prompt engineering : https://matthewmcateer.me/blog/clip-prompt-engineering/
- NightCafe Creator : https://creator.nightcafe.studio/
- Wombo Dream : https://app.wombo.art/
- Pollinations.ai (needs linking with Google Colab) : https://pollinations.ai/c/Anything
- Midjourney beta signup : https://o9q981dirmk.typeform.com/to/zZtF1mVc?typeform-source=www.google.com
Machine Learning in the real world
Design resources: