Feature Request: ML Model Training Blocks
Closed this issue · 13 comments
With these newly implemented training blocks for Scratch 3 projects, teachers are beginning to imagine how useful this could be with App Inventor projects. Highly requested after CSTA conference workshop and presentation.
Nothing published with Python yet on ML for Kids site. Would need to work with Dale to implement and understand Web API.
To add training data:
POST request to /api/scratch/:scratchkey/train
payload should be:
{
"data" : "The new training data",
"label" : "The_bucket_to_put_it_in"
}
To start training a new ML model:
POST request to /api/scratch/:scratchkey/models
No payload needed.
Note: You can't do this for image projects. That's because training an image classifier is a potentially expensive action, so I was wary of someone putting a train classifier block inside a forever block and running a bill of potentially hundreds of dollars. Training text or numbers models is free, so that's why that felt safe to do.
To get the status of a ML model:
GET request to /api/scratch/:scratchkey/status
Response payload will look like:
{
"status": num,
"msg": "Description"
}
num will be 0, 1 or 2
2 means ready to use
1 means training in progress
0 means something went wrong (or there isn't a model) - the msg field will have more info
Hope that helps - please shout if I can help with anything else!
To add training data:
POSTrequest to/api/scratch/:scratchkey/trainpayload should be:
{ "data" : "The new training data", "label" : "The_bucket_to_put_it_in" }
Does this only work on the text and numbers model as well?
The store training data one works for text, numbers, and images.
I haven't done sound yet - because I haven't figured out the UI way in Scratch of collecting training data for sound projects yet.
I should clarify, when storing image training data over this API, the assumption is that the payload will be base-64 encoded jpeg data.
@dalelane and @MrMazzone This feature is ready to merge, if possible, could you give it a quick review / test it on a device to make sure it works as expected. I was not able to find any issues on my end (except for the image size, which I will fix in the next update)
https://github.com/kylecorry31/ML4K-AI-Extension/tree/feature/training-blocks
@kylecorry31 Wow! Works like magic! People are going to love this.
The only issue I had was with an image project. The ".TrainNewModel" method call block isn't training the model on ML for Kids. Not getting any errors either. The block worked for my text and numbers project. Every other part of the project works; I am able to add data and check the status of the model.
Curious to see if @dalelane has the same issue.
Looks good to me, except the training issue for images - which is by design, I'm afraid.
That should have returned a not-implemented error
https://github.com/IBM/taxinomitis/blob/6d8896f21de1e536628c1b7ebeaca9ce1f5823bc/src/lib/restapi/scratch.ts#L306-L308
That's what I was saying in #21 (comment) - I haven't added support for training image models from the Scratch API because it's super expensive, I'm afraid.
🤦♂️ Completely forgot about that @dalelane. Makes sense now.
I just retested and I am now receiving the proper error message. 👍
Feature completed and tested.
@kylecorry31 will fix the image size issue with some pictures in next release.
Thanks for testing it!