hello there 🐠

20221125.webm

demo example current as of 8667f94 / 2022-11-25

this is a completely vanilla javascript and html canvas outpainting convenience doodad built for the API optionally exposed by AUTOMATIC1111's stable diffusion webUI, operating similarly to a few others which certainly have superior functionality. this simply offers an alternative for my following vain desires:

avoiding the overhead of an additional virtual python evnironment or impacting a pre-existing one
operates against the API exposed by A1111's webUI
no external dependencies, extremely boring vanilla
no external connectivity, self-hosted and offline
unobfuscated (cough cough)
i am terrible at javascript and should probably correct that
i have never used html canvas for anything before and should try it out

features

a big ol' 2560x1440 canvas for you to paint all over (infinite canvas area planned, in //todo already)
inpainting/touchup blob
floating control panel to easily change models/samplers/steps/prompts/CFG/etc options for each dream summoned from the latent void (NOTE: model switching requires A1111 webUI to be on commit 5a6387e or more recent)
optional grid snapping for precision
optional hi-res fix for blank/txt2img dreams which, if enabled, uses image width/height / 2 as firstpass size
optional overmasking for potentially better seams between outpaints - set overmask px value to 0 to disable the feature
dedicated img2img tool with optional border masking for enhanced output coherence with existing subject matter
floating toolbox with handy keyboard shortcuts
import arbitrary images and scale/stamp on the canvas whenever, wherever you'd like (extra fun with transparent .pngs!)
upscaler support for final output images (NOTE: LDSR has had reports of not operating correctly when selected - please test and see if it works as expected)
saves your preferences to browser localstorage for maximum convenience
floating navigable undo/redo palette with ctrl+z/y keyboard shortcuts for additional maximum convenience
inverted mask modes - red masks get mutated, blue masks stay the same

collaborator credits 👑

operation

prerequisities

you'll obviously need A1111's webUI installed before you can use this, thus you're presumed to have an operational python install up and running to boot.

notes

technically you can run it directly in browser as a bare file:// protocol webpage but that's really not recommended as you'll have to add null as an accepted domain to your --cors-allow-origins option which just seems like it's a visibly poor decision
a deliciously simple launch script is included to pop up a teensy tiny python-based local webserver, however you may have to manually chmod +x openOutpaint.sh on mac/linux
the address http://127.0.0.1:3456 will be used as the host address for openOutpaint in the below quickstart; your local setup may use a different IP address or port. you can of course modify the included launch script to point at a different port than 3456 if desired, as well
if your scale factor is > 8 (generating an image larger than 512x512), try the "auto txt2img HR fix" option

quickstart speedrun

edit your cors-allow-origins to include https://zero01101.github.io and run webUI
go to https://zero01101.github.io/openOutpaint/ and fill in the host value with your webUI API address
click things and do stuff

quickstart normal edition

clone this repo to your homelab's webserver (i mean who doesn't have a couple of those lying around?) or somewhere on your local pc
edit the webui-user.sh or webui-user.bat in your local A1111's webUI installation directory and set the COMMANDLINE_ARGS variable to contain --api --cors-allow-origins=http://127.0.0.1:3456
execute your webui-user script and wait for it to be ready
APPLY THE FOLLOWING SETTINGS IN A1111 WEBUI ONCE IT IS READY:

set your Inpainting conditioning mask strength to 1
disable the Apply color correction to img2img results to match original colors. option (those 2 options are found under the stable diffusion category in the settings tab by default unless you've already moved it to your quicksettings list, and if so, you know where to set them already)
select an inpainting checkpoint/model - (runwayml/stable-diffusion-inpainting [3e16efc8] is recommended) (OR you can select a model from openOutpaint's stable diffusion settings panel - it'll pop up an alert once the model has loaded)

configure your local webhost in your homelab to serve the newly cloned repo like the technological bastion you are, or simply run the included openOutpaint.bat on windows or openOutpaint.sh on mac/linux.
open your locally-hosted web server at http://127.0.0.1:3456 (or wherever, i'm not your boss)
update the host field if necessary to point at your stable diffusion API address, change my stupid prompts with whatever you want, click somewhere in the canvas using the dream [d] tool, and wait (OR you can load as many existing images from your computer as you'd like using the "stamp image" tool [U]). if you've requested a batch of generated images and one of them sparks you as something you might want to use later, you can click the "res" button to add the image to the stampable dream resources as well. (NOTE: you can select or deselect imported images/added resources freely simply by clicking on them)
once an image appears*, click the < and > buttons at the bottom-left corner of the image to cycle through the others in the batch if you requested multiple (it defaults to 2 batch size, 2 batch count) - click y to choose one you like, or n to cancel that image generation batch outright and possibly try again
now that you've got a starter, click somewhere near it to outpaint - try and include as much of the "context" as possible in the reticle for the best result convergence, or you can right-click to remove some of it if you want to completely retry a chunk but leave the rest alone
enable the mask mode to prepare previously rendered imagery for touchups/inpainting, then paint over the objectionable region; once your masked region is drawn, disable mask mode and change your prompt if necessary, then click over the canvas containing the mask you just painted to request the refined image(s)
choose the img2img mode and write a new prompt and click over a section of your image to, uh, mutate the contents
just play around with the available options!

snap to grid, uh, snaps to the grid
select image [s] is a marquee selector, use it to define a section of image you'd like to copy or freely scale
mask brush [m] is practically identical to A1111's inpainting mask, use scrollwheel to change the blob size
overmask and related px value expands the area masked in img2img outpaint requests, so as to attempt to minimize the seam between images
border mask (img2img tool [i] only) and related value compresses the area masked in img2img towards the image section being "replaced" to maximize possible coherence
...everything else is pretty much just a regular stable diffusion option so i presume you know how you use those

open the "save/upscaling" menu and click "save canvas" (or choose an upscaler and click "upscale", but heed its warning) to save the cropped region of outpainted canvas
click "clear canvas" to blank the canvas and start all over only to discover that it's like 2 AM and you have to go to sleep because you have work in about 4 hours

*if it doesn't create an image, check your console output to see if you've got CORS errors

//todo

in order of "priority"/likelihood of me doing it

pull requests

PLEASE SEND PULL REQUESTS
i am begging you, yes you personally reading this, please fix my horrible code and feel free to insult it, but i absolutely refuse to budge on no 3rd party libraries or dependencies, not even jquery, nothing. vanilla is a very complex and layered flavor if you give it a chance.

bug reports

please do! kindly indicate your OS, browser, versions of both, any errors in devtools/console output, what you were trying to do, what you expected, what happened unexpectedly or incorrectly, if something caught fire (please call the fire department first), the usual

known bugs :(

generated images display +1px on x/y during approve/reject state, doesn't affect output, just annoying - less obvious now with checkerboard background
~~erase mask is like entirely broken~~
~~odd-numbered scale factors don't snap correctly~~
~~arbitrary "pasted" images require clicking twice to place them and i don't know why (yes i do), just getting them to be arbitrarily placable was a giant pain because i'm not got the smarts~~
selecting an aribtrary image by double-clicking it in the file picker can sometimes trigger a dream request that errors out if your file picker is "above" the canvas; i tried to alleviate that by temporarily removing the mouse(move/down/up) handlers for the canvas context on selection of a file, but i'm POSITIVE it's an improper solution and not quite sure if it's even fully effective fixed via pr 49
not sure if "bug" since it occurs in stable diffusion and not openOutpaint, but auto txt2img HRfix + odd number scale factors returns an "Exception in ASGI application" in SD console output; for example using scale factor of 9 results in "RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 10 but got size 9 for tensor number 1 in the list."
similarly, seems to be an issue with stable diffusion more than anything, but if GPU memory becomes too scarce, the 3d rendering hardware starts to spike to 100% if you're looking at the tab (happens in both openOutpaint OR A1111 webUI, happens if image generation is occurring in background and i'm watching youtube, etc) - this manifests as the seconds remaining counter increasing around 100 seconds each time it's checked while completion percentage stays unchanged. simply opening an empty new tab usually kicks some sense back into things in my experience but your mileage may vary
~~if you request a dream beyond the left border of the canvas you can kinda end up in a softlock state~~

warranty

lmao

samples

generated using 100% openOutpaint UI defaults except for switching to/from mask mode and changing scale factor to adjust the size of the mask blob, there's some neat stuff down there even if it disregarded the people, humans, divers negative prompt but in its defense there is only one singular person, human, diver in there, so according to the no homers club treatise of 1995 it's technically correct

(see https://github.com/zero01101/openOutpaint/commit/92ab9d231542ea5f7a3c85563acf5cd3cb16a928 for attempted counterattack)

simple transparent PNGs slapped on the canvas twice and default UI settings used on the southern half

imported a transparent clip of a relatively famous happy lil kitty, used default openOutpaint settings except changing the prompt to "a cat on a space station", eventually received this magnificent vision

version history

0.0.1 - txt2img proof of concept
0.0.2 - img2img outpainting proof of concept
0.0.3 - image masking/img2img inpainting proof of concept
0.0.4 - batch size/batch count, approve/reject system implementations, snap-to-grid, other people are now allowed to see this thing 01f8c6a
0.0.4.1 - extremely minor revisions 02cb01a
0.0.4.2 - pull requests (<3), downloaded images now have a timestamped name, css breakout because hopefully this will become halfway attractive enough to benefit from non-inline stylesheets 70ad4fe
0.0.4.3 - overmasking, settings saved to localstorage fca2e01
0.0.5 - import arbitrary image from user's machine, "auto" txt2img hires fix, Very Important "new image" button 3b7f4e3
0.0.5.1 - erase rendered imagery with right click, ensure webUI is running 54577d4
0.0.5.5 - highly attractive and functional floating control panel which will be extremely useful for infinite canvas dac188d
0.0.5.6 - FINALLY the sliders update their values in realtime, a nice overall start on cleaning up my mess d9fb87a
0.0.5.7 - the majestic return of mask erasing, removed unnecessary overmask toggle a96fd11
0.0.6 - absolutely brilliant undo/redo system, logical and straightforward enough to the point where even i can understand what it's doing 25681b3
0.0.6.1 - finally think i've got overmasking working better with a bit of "humanization" to the automated masks, please play around with it and see if it's any better or just sucks in general 8002772
0.0.6.5 - checkerboard background, far more attractive painted masking, HUGE code cleanup omg 74d5f13
0.0.6.9 - upscaler support for final output image 3b91a89
0.0.7 - floating toolbar, img2img tool, border masking, change model from UI, general very needed code cleanup and modernization 9916ee8
0.0.7.5 - giant arbitary image handling and marquee select tool update bbdfef9

what's with the fish?

deep aquatic life is fascinating so i went with something underwater for a default prompt which led to making an "illustration of a bright orange fish, plain blue solid background" favicon which led to "ok then, fish is mascot"

~~the end~~ 𝒻𝒾𝓃 🐠

Kalekki/openOutpaint