Allusion programmatic api?

[ghostsquad]

Along the lines of #300, #555, #465

Is it possible to programmatically update Allusion's database? I could in theory write a small golang program to read in PNG EXIF data and write tags to Allusion if I knew how (and it was possible) to manipulate tags programmatically.

[RvanderLaan]

everything is possible! But probably not communicating directly to the database, since we're using IndexedDB which is only accessible in the Electron browser process.

We're already doing something similar with the browser extension, when adding an image to Allusion from the browser:

• The browser extension sends the image and tag data over a local http server running in the main Allusion process
• The main Allusion process sends the data over to the browser process
• The browser process updates the DB

Importing tags from image metadata is already being accomplished differently as I mentioned in your other issue

[ghostsquad]

I'm assuming you are talking about this?

It's not actually clear what this does. This doesn't look like a setting, it looks like an action. But what is it acting on? Also, for AI image generation, typically, the "parameters" are comma separated, but not just that, another pass usually is needed to make things readable:

The "parameters" of a generated image may look something like this:

Cinematic film still, full body, of ((Norse Female goddess)), (Hel), 
pale skin, ((detailed facial features)), alluring blue eyes, magic, 
(dark cave:1.2), (cold colors), intricate details, shallow depth of field, 
[volumetric fog], cinematic lighting, reflections, photographed on a Canon EOS R5, 
50mm lens, F/2.8, HDR, 8k resolution, cinematic film still from Vikings
Negative prompt: bright, (monochrome:1.3), (oversaturated:1.3), bad hands, lowers, 
3d render, cartoon, long body, ((blurry)),
Steps: 29, Sampler: DPM++ 2S a Karras, CFG scale: 8.5, Seed: 2384087652, 
Face restoration: GFPGAN, Size: 512x512, Model hash: abc64810e2, Model: sd_15

Here's some totally untested code I just wrote for the challenge of it to explain how this data would need to be cleaned up in order to be tagged properly. (Also note of course that this code is missing a bunch of guard/error checking, as we don't want index out of bounds errors)

parametersSplit := strings.Split(parameters, "Negative prompt:")

positivePrompt := parametersSplit[0]
parametersRemaining := parametersSplit[1]

// Note that splitting on `Steps:` will remove that from `negativeSplit[1]`, so we'll add it back in
// Maybe there's a better way to do this?
negativeSplit := strings.Split(parametersRemaining , "Steps:")

// TODO maybe tagging based on negative prompt is useful? probably not
// Here it is, but this is unused at the moment
negativePrompt := negativeSplit[0]

// Put `Steps:` back in (for ease of the next section)
parametersRemaining := "Steps:" + negativeSplit[1]

parametersSplit = strings.Split(parametersRemaining, ",")

// basically a "Set" (though there's no official type in Go, this is the closest thing to it)
tags := map[string]struct{}{}

for _, p := range parametersSplit {
   p = strings.TrimSpace(p)
   tags[p] = struct{}
}

charsToRemove := strings.Split("[]{}(),", "")

// There's probably a better way to do this...
for _, c := range charsToRemove {
  // maintain separation with spaces so we can snag each individual word
  positivePrompt = strings.ReplaceAll(positivePrompt, c, " ")
}
   
pSplit := strings.Split(p, " ")
for _, word := range pSplit {
  // don't add empty strings
  if word == "" {
    continue
  }
     
  tags[word] = struct{}
}

return tags

I filed #555 then realized that this type of thing might be out of scope for Allusion to actually maintain, so that's why I filed this issue.

The other thing is that Allusion may already have these images "imported", so really I would need some other way of modifying the image metadata stored in the Allusion DB.

I'm perfectly happy making HTTP requests if that's a good way to go. Just need a little bit more guidance on if this is already supported (e.g. how are images uniquely identified in the database, and how are they updated?), and whether or not that update mechanism is exposed via the HTTP interface.

Thanks for all your time and help! I'll be likely to make a blog post (if I can get this all working), on how I use Allusion to manage my 1000's of generated images. 😄

[ghostsquad]

I'd love some engagement on this please!

[RvanderLaan]

Apologies! Finding it hard to get time for this project recently

Images are identified internally by a UUID which you have no way of knowing from the outside. But since the path and filename are unique for any file, that could be used too.

I forgot to mention that the API used by the web extension was only intended to be used by the "Download Directory", configured here in the settings panel

but there is no technical limitation why it could not be used for any file present in Allusion. The /set-tags currently only accepts a filename of an image present in that Download Location. It could be updated to support any given file path (in src/clipper/server.ts)

[ghostsquad]

Ok thanks. I guess I'll dig a bit more into the code so I can understand the CRUD operations a bit more and extend the API.

[RvanderLaan]

Alright! I can give some pointers when you've got a plan in mind, feel free to reach out

[ghostsquad]

I'm not even sure where this fits on my priorities list at the moment. If anyone else wants to take a stab at it, don't let me hold you back.