Modify NHW (convolutional) metric?
zvezdochiot opened this issue · 36 comments
Hi @rcanut .
NHW convolutional metric (ImageProcessing-ElectronicPublications/libsmallfry@92f3938).
Hi,
Thank you for including NHW convolutional metric in your library.I have seen your modifications to original nhw_metrics, notably your metric_nhw() function and it seems quite interesting.
However, it will require quite some time of visual assessment to evaluate your improvement over nhw_metrics, I hope to find that time next month, as currently I am focusing on NHW image compression codec.
Just a quick remark, maybe I am completely wrong, but maybe in your code/metric:
in metric_nhw() function, instead of:
resd = abs(res - resr);
if (resd > 0)
{
}
Replace with something like:
resd = abs(res) - abs(resr);
if (resd > 0)
{
}
Maybe better?
Cheers,
Raphael
Hi @rcanut .
Not. No better. I tested this metric on jpeg-recompress. She performed well on all quality ranges (in some ways less insane than SSIM and SHARPENBAD).
I'm more concerned about the possible overflow of the counters (float).
Ok.That's very interesting.If your metric performs better than SSIM and SHARPENBAD on jpeg-recompress then it could be really worth that you make a publication of your design/results (if you've got time)?
Cheers,
Raphael
Hi @rcanut .
A full-fledged carrying out of a full set of tests and compiling a full-fledged dock is a problem for me (it takes a lot of time and my English is very bad). In addition, my favorite image "Lena" was banned.
My English == https://translate.google.ru/
Ok. I understand as I have also very little time for NHW codec and even less for NHW metrics, and so I am very sorry but I will lack the time currently to evaluate your metric on a full set of tests, as it is quite a big study, very sorry again. Do you consider to spread the word/show your metric to the large communities around jpeg recompression and image quality optimization?
Also I post on encode.su and Doom9 forums, do you want that I give the link of your work on these forums? Saying that you have improved nhw_metrics and if interested people want to test your work? -But I think however don't expect too much, because there is a very very small community around nhw_metrics...-
Cheers,
Raphael
Hi @rcanut .
Would really appreciate some testing from the community. As a ready-made testing tool, I recommend jpeg-recompress: jpeg-recompress and jpeg-compare. It contains the necessary minimum metrics for comparison: psnr, mpe, ssim, ms-ssim, smallfry, shbad, nhw, cor, corsh.
As a comparison scale, I recommend using the "Universal Scale" of metrics (UM):
0.0 ... (DIRTY) ... 0.5 ... (LOW) ... 0.75 ... (MEDIUM) ... 0.875 ... (SUBHIGH) ... 0.9375 ... (HIGH) ... 0.96875 ... (VERYHIGH) ... 1.0
Trends:
UM = 1.10*sqrt(PNSR)-6.07
UM = -0.85*sqrt(MPE)+1.73
UM = 2.38*cor_sigma(cor_sigma(cor_sigma(SSIM)))-0.24
UM = 1.87*cor_sigma(cor_sigma(MS_SSIM))-0.02
UM = 0.0747*SMALLFRY-6.91
UM = 2.28*sqrt(sqrt(sqrt(1.0/NHW)))-0.97
UM = 1.48*SHARPENBAD-0.26
cor_sigma(M) = 1.0-sqrt(1.0-M*M)
Done.I have presented your work on encode.su and Doom9 forums.
Cheers,
Raphael
Hi @rcanut .
Saw your discussion at https://encode.su/threads/3002-Is-encode-su-community-interested-in-a-new-free-lossy-image-codec/page10.
The issue of metrics and their visualization was raised. You can upgrade your metric. Don't just sum up the value, but assign these (normalized) values to the metric image (w x h == source images). This is how you get the visualization.
NHW_Neatness_Metrics/nhw_neatness_metrics.c
Line 120 in e927c2a
And in order not to fence too much, you can make one function and pass NULL to it instead of an metric image if visualization is unnecessary. And in the function check the variable for NULL.
Example the visualization: https://github.com/ImageProcessing-ElectronicPublications/jpegquant (butteraugli) or dssim.
PS: The metric I modified does not detect "blurry image" distortions well.
Hi,
Hope you could have test and feedback from the community on your great software?
Also unfortunately, I don't have time currently for NHW_Neatness_Metrics.But also as you are better skilled/qualified than me, is it polite if I ask you if you could fork my repo, and apply your modifications if you have time? Then I can test it, and if it's better then I'll upate NHW_Neatness_Metrics (you can also send me a pull request).
Cheers,
Raphael
Hi @rcanut .
I do not like to directly climb into someone else's code. I prefer issue in the form of a recipe. If I come up with a recipe for this case, I will attach it to the message. He himself is busy testing his version and the adequacy of the metrics in general.
Ok.
So in a first time, I think I'll start to test with this modification:
neatness_amount += (delta * delta) * resd;
significance1 += resd;
significance2 ++;
....
neatness = neatness_amount / significance1;
(significance = significance1 or significance2 ?)
It's rather quick to code, the problem is to test on large image dataset.Hope I can do this large visual review in the next weeks.
Many thanks again.
Cheers,
Raphael
Hi @rcanut .
NHW_Neatness_Metrics-0.1.0.v (visualization): NHW_Neatness_Metrics-0.1.0.v.tar.gz
Origin lena.bmp:
Compess and decompress of cjpeg quality 70 lena.q70.jpg.bmp:
./nhw_neatness_metrics lena.bmp lena.q70.jpg.bmp lena.q70.jpg.nhw.bmp
Significance = 2743 NHW Neatness metrics = 115.065987
Metric image lena.q70.jpg.nhw.bmp:
convert -normalize lena.q70.jpg.nhw.bmp lena.q70.jpg.nhw.norm.bmpNormalize metric image lena.q70.jpg.nhw.norm.bmp:
NHW_Convolutional_Metrics-0.1.0: NHW_Convolutional_Metrics-0.1.0.tar.gz
./nhw_convolutional_metrics lena.bmp lena.q70.jpg.bmp lena.q70.jpg.nhw.c.bmp
Significance = 24831075 NHW Convolutional metrics = 96.869659
Metric image lena.q70.jpg.nhw.c.bmp:
NHW=96.869659 ==> UM = 0.22 (DIRTY)
Hi!
Thank you so much for this great work and for providing the code of your 2 NHW metrics with visualization! That's great!
Yes effectively it is way better to have normalized metric image.
I'll have to test your 2 metrics to see which is the best for my use case (measure neatness).From your visualization images, maybe it will be NHW_Neatness_Metrics-0.1.0.v, but I guess that for jpeg-recompress it is NHW_Convolutional_Metrics-0.1.0 that is better?
-I plan to test this week.-
Thank you very much again for your great contribution!
Cheers,
Raphael
Hi @rcanut .
Compare of other metrics:
dssim -o lena.q70.jpg.bmp.dssim.png lena.bmp.png lena.q70.jpg.bmp.png
0.00356443 lena.q70.jpg.bmp.png
DSSIM=0.00356443 ==> SSIM = 0.99644 ==> UM = 0.33 (DIRTY)
butteraugli lena.bmp.png lena.q70.jpg.bmp.png lena.q70.jpg.bmp.butteraugli.ppm
3.276375
BUTTERAUGLI=3.276375 ==> UM = unknow (DIRTY)
Thanks! Just great again!
Hi @rcanut .
An additional version of the metric that uses not an absolute version of the gradient, but a relative one.
NHW_Relative_Metrics-0.1.0: NHW_Relative_Metrics-0.1.0.tar.gz
./nhw_relative_metrics lena.bmp lena.q70.jpg.bmp lena.q70.jpg.nhw.r.bmp
Significance = 1052168.000000 NHW Relative metrics = 40.845966
Metric image lena.q70.jpg.nhw.r.bmp:
Hi,
I have updated the repo with visualization of metric image thanks to your contribution!
I'll also have to study NHW_Relative_Metrics.
Many thanks again.
Cheers,
Raphael
Hi @rcanut
ℹ️ Use https://sourceforge.net/projects/astyle/ (--style=allman) for code formatting.
Example:
astyle --style=allman nhw_neatness_metrics.c
astyle --style=allman nhw_neatness_metrics.hHi @rcanut .
Metrics analysis (jpeg compression and decompression):
nhw.log
Trends UM = a1*sqrt(sqrt(sqrt(1.0/NHW)))+a0:
Trends UM = a1*(1.0-sqrt(1.0-1.0/NHW))+a0:
PS: The analysis shows that such metric values are primarily associated with the use of the RGB scheme. For visual perception, the brightness component is more important.
Hi @rcanut .
NHW_Neatness_Metrics/nhw_neatness_metrics.c
Line 127 in 0cedff4
220? Not 127? Its strange. Maybe?:
delta = nhw_comp[scan]-ref_comp[scan];
delta = (delta < -127) ? -127 : delta;
delta = (delta > 127) ? 128 : delta;
if (m_image) m_image[scan*RGB_comps+comp] = 127 + delta;Hi,
In fact, 3<delta<30 (thresholds), so I choosed 220 (high) value for the color to be really bright and visible/distinguishable from the black background.
Hi @rcanut .
220+delta:
127+delta:
Pseudo-imagemagick (compare or gm compare):
...
if (m_image) m_image[i + comp] = (ref_image[i + comp] >> 3);
...
if (m_image) m_image[scan*RGB_comps+comp] += 127 + nhw_comp[scan]-ref_comp[scan];
...
Hi @rcanut .
New test. Quantize {4,4,4}bit image:
./nhw_neatness_metrics lena.quant444.bmp lena.bmp lena.quant444.nhw.n.bmp
Significance = 12591 NHW Neatness metrics = 357.768738
./nhw_convolutional_metrics lena.quant444.bmp lena.bmp lena.quant444.nhw.c.bmp
Significance = 64186650 NHW Convolutional metrics = 1387.283325
./nhw_relative_metrics lena.quant444.bmp lena.bmp lena.quant444.nhw.r.bmp
Significance = 1414783.375000 NHW Relative metrics = 885.108948
Yes, your Quantize {4,4,4}bit image example clearly shows that even if this image has a high score on nhw metrics (high significance and high neatness) but however we clearly see that this image has not a good neatness compared to the original... this clearly shows that significance and neatness must be better interpreted, and that generally much more work remain to be done on nhw neatness metrics...
Many thanks again for your great demos!
Cheers,
Raphael
Hi @rcanut .
Output these demos in README.md. Example:
### Demo:
Result:
Demo:
PS: Convolutional and Relative metrics performed well.
Done! Many thanks!
Hi @rcanut .
Test continuation #1 (comment)
cjpeg -quality 70 -outfile lena.bmp.ppm.jpg lena.bmp.ppm
ls -l *.jpg
-rw-r--r-- 1 zvezdochiot zvezdochiot 33945 dec 17 01:20 lena.bmp.ppm.jpg33945 byte.
You new version codec: rcanut/nhwcodec@6449cf7
./nhw-enc lena.bmp lena.bmp.nhw
./nhw-dec lena.bmp.nhw lena.bmp.nhw.bmp
ls -l *.nhw
-rw-r--r-- 1 zvezdochiot zvezdochiot 55436 dec 17 01:10 lena.bmp.nhw55436 byte.
./nhw_neatness_metrics lena.bmp.nhw.bmp lena.bmp lena.bmp.nhw.m.n.bmp
Significance = 1715 NHW Neatness metrics = 91.409332
./nhw_convolutional_metrics lena.bmp.nhw.bmp lena.bmp lena.bmp.nhw.m.c.bmp
Significance = 20917866 NHW Convolutional metrics = 57.075516
./nhw_relative_metrics lena.bmp.nhw.bmp lena.bmp lena.bmp.nhw.m.r.bmp
Significance = 984627.500000 NHW Relative metrics = 27.463242
dssim -o lena.bmp.nhw.bmp.dssim.png lena.bmp.png lena.bmp.nhw.bmp.png
0.00396989 lena.bmp.nhw.bmp.png
butteraugli lena.bmp.png lena.bmp.nhw.bmp.png lena.bmp.nhw.bmp.butteraugli.ppm
3.378667
jpeg-compare -m sum lena.bmp.ppm lena.bmp.nhw.bmp.ppm
SUM: 0.575625jpeg-compare -m sum lena.bmp.ppm lena.bmp.ppm.jpg
SUM: 0.451391Conclusion: I don't see any advantages of your codec even over JPEG.
Compare.
webp-compress -q low lena.bmp.ppm lena.bmp.ppm.webp
SUM at q=50 (1 - 99): UM 0.330935
SUM at q=75 (51 - 99): UM 0.436250
SUM at q=87 (76 - 99): UM 0.643473
SUM at q=81 (76 - 86): UM 0.526779
SUM at q=78 (76 - 80): UM 0.482981
SUM at q=79 (79 - 80): UM 0.501030
Final optimized SUM at q=79: UM 0.501030
New size is 3% of original (saved 739 kb)
ls -l *.webp
-rw-r--r-- 1 zvezdochiot zvezdochiot 29374 dec 17 01:57 lena.bmp.ppm.webp29374 byte.
Hi @rcanut .
New version (0.2.0) metrics: NHW_Metrics-0.2.0.tar.gz
Library version. Does not use additional arrays for components. Does not use weak indexes.
Good luck.
Hi,
Many thanks again for your great work.
However I will have to find the time to study NHW_Metrics-0.2.0, because my aim with nhw_neatness_metrics is really just to indicate neatness of the image, which is maybe not the same purpose for your jpeg-recompress software?
Also, do not hesitate to publish NHW_Metrics-0.2.0 on your Github repository!
Many thanks again.
Cheers,
Raphael
Hi @rcanut .
New version (0.2.1) metrics: NHW_Metrics-0.2.1.tar.gz
Library libnhwmetrics.a. Joint cli nhwmetrics.
Good luck.
Hi @rcanut .
You new version codec: rcanut/nhwcodec@4a5a23d
./nhw-enc lena.bmp lena.bmp.nhw
./nhw-dec lena.bmp.nhw lena.bmp.nhw.bmp
ls -l *.nhw
-rw-r--r-- 1 zvezdochiot zvezdochiot 55436 dec 21 02:03 lena.bmp.nhw55436 byte.
./nhwmetrics n lena.bmp.nhw.bmp lena.bmp lena.bmp.nhw.m.n.bmp
NHW Neatness metrics = 91.409332
./nhwmetrics c lena.bmp.nhw.bmp lena.bmp lena.bmp.nhw.m.c.bmp
NHW Convolutional metrics = 57.075516
./nhwmetrics r lena.bmp.nhw.bmp lena.bmp lena.bmp.nhw.m.r.bmp
NHW Relative metrics = 27.463242
jpeg-compare -m sum lena.bmp.ppm lena.bmp.nhw.bmp.png.ppm
SUM: 0.575625Conclusion: No progress.
Compare
UM = 0.575625
jpeg-recompress -Y 1 -t 0.575625 lena.bmp.ppm lena.bmp.ppm.jpg
Metadata size is 0kb
SUM at q=69 (40 - 98): UM 0.444189
SUM at q=84 (70 - 98): UM 0.567513
SUM at q=91 (85 - 98): UM 0.675749
SUM at q=87 (85 - 90): UM 0.607136
SUM at q=85 (85 - 86): UM 0.580269
Final optimized SUM at q=85: UM 0.580269
New size is 6% of original (saved 717 kb)
ls -l *.jpg
-rw-r--r-- 1 zvezdochiot zvezdochiot 51916 dec 21 02:12 lena.bmp.ppm.jpg51916 byte.
webp-compress -t 0.575625 lena.bmp.ppm lena.bmp.ppm.webp
SUM at q=50 (1 - 99): UM 0.330935
SUM at q=75 (51 - 99): UM 0.436250
SUM at q=87 (76 - 99): UM 0.643473
SUM at q=81 (76 - 86): UM 0.526779
SUM at q=84 (82 - 86): UM 0.583882
SUM at q=82 (82 - 83): UM 0.548769
Final optimized SUM at q=83: UM 0.563550
New size is 4% of original (saved 730 kb)
ls -l *.webp
-rw-r--r-- 1 zvezdochiot zvezdochiot 38682 dec 21 02:13 lena.bmp.ppm.webp38682 byte.
Hi @rcanut .
You new version codec: rcanut/nhwcodec@f26dd25
./nhw-enc lena.bmp lena.bmp.nhw
./nhw-dec lena.bmp.nhw lena.bmp.nhw.bmp
ls -l *.nhw
-rw-r--r-- 1 zvezdochiot zvezdochiot 55436 dec 30 01:29 lena.bmp.nhw55436 byte.
Metrics:
./nhwmetrics n lena.bmp.nhw.bmp lena.bmp lena.bmp.nhw.n.bmp
NHW Neatness metrics = 91.409332
./nhwmetrics c lena.bmp.nhw.bmp lena.bmp lena.bmp.nhw.c.bmp
NHW Convolutional metrics = 57.075516
./nhwmetrics r lena.bmp.nhw.bmp lena.bmp lena.bmp.nhw.r.bmp
NHW Relative metrics = 27.463242
Conclusion: No progress.
Hi,
Actually, I focus on and have changed -q16 to -q8 compression settings.Again I visually validated this version, so metrics could be worse, but despite that images are visually more pleasant.
Cheers,
Raphael
Hi @rcanut .
- #1 (comment) : rcanut/nhwcodec@6449cf7
- #1 (comment) : rcanut/nhwcodec@4a5a23d
- #1 (comment) : rcanut/nhwcodec@f26dd25
- (no message) : rcanut/nhwcodec@d115b34
55436 byte.
- NHW Neatness metrics = 91.409332
- NHW Convolutional metrics = 57.075516
- NHW Relative metrics = 27.463242
Conclusion: No progress.
Just great!
Good luck too!
Cheers,
Raphael