[Feature Request] 32-bit HDR Output Workflow
AugmentedRealityCat opened this issue · 8 comments
I managed to use this extension to create a passable approximation of a high-dynamic range image in 32 bit per channel. I must use photoshop to do some of the steps, but most of the hard work is done with this extension.
I would like to describe my workflow and have your input but I am not sure if this "issue" tab is the right place to do it.
There are also a few feature requests I have in mind to help me with this HDR generation process, but let's describe my current workflow prototype first and see what you think.
I'm also working on panoramic HDRI environment maps, which are quite useful for real-time 3d engines. More on that soon !
Finally, THANK YOU SO MUCH for making this extension. I was already a fan, and I had already tried to do what I just did today with an earlier version of your extension, but it looks like the latest version has the features that were missing to make this whole HDR image generation work. You made this possible by creating and sharing this extension, and I can't thank you enough for that.
Discussing here is fine I guess.
And thank you for the positive comments~
I myself have thought about generating actual HDR image as well.
But afaik, there is no Python package that can export native HDR images with 32-bit per channels, at least I wasn't able to find any.
So 3rd-party tool, like Photoshop, is still needed I think...
How to use the Vectorscope extension for A1111 to create 32 bit HDR images
Create an image as you normally would. This will be our reference image in 8 bit. Here is the picture I used, and, just underneath, the prompt I used.
Cute creature from Space. terraforming. Alien Flora, Miki Asai Macro photography, close-up, hyper detailed, trending on artstation, sharp focus, studio photo, intricate details, highly detailed, by greg rutkowski
detailed face, detailed skin
<lora:xl_more_art-full_v1:1>
Negative prompt: ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, extra limbs, body out of frame, blurry, bad anatomy, blurred, watermark, grainy, signature, cut off, draft, text, logo
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 4126346264, Size: 768x1344, Model hash: 730028c80f, Model: sdxlYamersPhotorealistic_version1, RNG: CPU, CFG Rescale phi: 0, Lora hashes: "xl_more_art-full_v1: fe3b4816be83", SGM noise multiplier: True, Version: v1.6.0-127-g102b6617
(directly inspired from this post on civitai: https://civitai.com/images/2287991?modelVersionId=152309 big thanks to ledadu)
Once you have this picture that you like, you must set the seed to be the same as the one used to generate the picture. You can use the little "recycle" icon next to the seed # parameter to do that automatically with the currently selected picture.
Now we will create variations of that picture, but with different exposures. This technique, called bracketing, was used to create actual HDR content without an HDR capable camera. Nowadays, you have an HDR camera in your phone !
To do that we will activate the Vectorscope extension and set it like this:
Basically, in this Vectorscope panel, you want to check enable and alt and set the brightness parameter to a value of minus six -6.0
In the advanced settings, you also want to check process hires fix, and set the Noise Setting to Ones and the Scaling Setting to 1-COS.
Now you can press the big "Generate" button to generate our first bracket, the darkest of them all.
In my case, this gives me something that looks like this:
Once this is done, you can go back to the Vectorscope panel and set the brightness to minus four -4.0 . And press generate again. This provides our second bracket, which is slightly brighter than the previous one, but still dark.
And then we repeat those steps of adding 2 to the previous brightness value and pressing generate 5 more times, so we get brightness=-2.0, brightness=0.0, brightness=2.0, brightness=4.0, and, finally, brightness=6.0.
You should now have 7 images total, from the darkest (brightness=-6.0) to the brightest (brightness=+6.0), with your original image in the middle of the series (brightness=0.0). For reference here is the brightest I got for the current example:
Now we move to photoshop. We don't open or create a new document, but instead go into file-automate-merge to HDR pro...
Once this is started, you will get access to a little interface where you can load the 7 pictures we just generated - our 7 brackets. After browsing to the proper folder and selecting your 7 pictures, you will get something that looks like this:
Press OK, and you will get to the next page, where we will be assigning exposure values to each of our 7 pictures. You could set the EV parameter directly, but I like to use ISO values instead. Here is a reference chart for these:
brightness=-6.0 / ISO = 25
brightness=-4.0 / ISO = 50
brightness=-2.0 / ISO = 100
brightness= 0.0 / ISO = 200
brightness= 2.0 / ISO = 400
brightness= 4.0 / ISO = 800
brightness= 6.0 / ISO = 1600
The interface will look like this:
Once all the ISO values have been attributed, you can press OK and get to the next step, which brings you to a new screen, like this:
First, make sure you are in 32 bit mode. Also, make sure the option to is unselected. After that you get to set the white point. You basically have to move the little triangle slider under the histogram on the upper right so that the brightest part of your image show up as white, but just barely. You should get something like this:
What you really don't want to have is something like this, with overblown white values:
Once the white level is set, you can press OK and you will get your 32 bit per color channel HDR picture.
Yes, it is very dark, and this is normal. To make this apparently too dark HDR picture usabel we will apply a gamma correction to bring that gamma value from linear (1.0) to exponential with a new value of 2.2. And then, finally, we will add some exposure.
Here is the result with the 2,2 gamma correction
And here it is with boosted exposure - here a value of 5. By the way you can also play with the offset value to change the contrast.
That's it, you now have a usable HDR version of your initial image ! Test the new dynamic range available to you by adding a level, curve or exposure effect, if you want to stay in photoshop. Or bring it into a 3d app, apply it as a texture map, and see how much richer it is when it reacts to lighting compared to an 8 bit version. Or send it to an HDR display to see it in its full HDR glory, which normally looks closer to a photographic diapositive, or an analog film frame, than to a video frame.
Next tutorial: how to use Vectorscope to generate panoramic HDRI environments.
And in case you are wondering, I tested the above workflow with the highres fix and it works marvelously well, and it's also fully compatible with Tiled VAE, both of which are essential features for generating high resolution panoramas.
Currently writing a Script to do this workflow
Should be prossible to do this in 1 Generation (So you don't need to press Generate 7 times)
Please check it out~
I will check it out this evening ! Thank you so much.
I just got my first TIFF in 16 bit per color channel using your HDR Script and it seems to work really well.
I will make more tests to make sure everything works as it should, and I will report back if I find anything.
Thanks again !