Export to excel?
Closed this issue · 8 comments
Ok, so first I want to say I'm not a programmer or a comp sci type person, but I am familiar with the scientific method and basic statistics. I have been making embeddings based off of four pictures changing one variable at a time. I've been using the function of this tool where it outputs the embedding str/mag for each .pt in a directory. I have put some of this data into an excel spreadsheet so I can apply the statistics knowledge I know in order to find patterns and test some predictability. However, it is very tedious to do this.
Could you add an option to export data to an excel format? It would make this a lot easier for me.
To give you an idea of what I am doing, I've included a copy of a preliminary spreadsheet that I put some data I collected into. I am not a spreadsheet person either, I only have basic knowledge, so you might have to steele yourself before reading it. Also, there are undoubtedly many non stylistic errors, but the file should still give you an idea of what I'm trying to do.
embt.xlsx
I've been able to uncover some possible interesting patterns:
As vectors go up, the average magnitude goes down, but the embedding strength seems to stay the same.
Changing learning rate seems like it does not change the embedding strength/magnitude ratio
Some questions I am trying to investigate:
What are the effects of batch size and gradient accumulation on the str/mag? Will 4 batch 1 grad give the same str/mag result as 1 batch 4 grad?
Are certain patterns related to the number of images, or the ratio of images? Is there a measurable difference when an embedding is trained oddly (batch 3 grad 1 on 4 images) versus evenly (batch 2 grad 1 on 4 images)?
How similar/different are the measurements on two embeddings trained with the exact same settings? With/without xformers? On different hardware? Different torch/cuda versions?
What effect does model have on these numbers?
What happens if I add a picture?
Remove a picture?
Add a lot of pictures?
etc, etc, etc
There are a lot of possible places for patterns to emerge, and then these can be investigated further in the actuality of the performance of the embedding. One possible lead for a controlled evaluation of this is to use an aesthetic grading extension. I haven't explored it yet, but if an averaged score over say 100 image generations of a particular embedding with a set amount of steps, cfg, sampler, could be correlated to any discovered data patterns, it could be a small, basic foothold into validating experimental data.
Anyways this is longer than I had expected. Thank you for your consideration.
Update and check the new EXPORT_FOLDER_EMBEDDING_TABLE_TO config setting in the python file.
Thanks!
Hi,
@SeekerOfTheThicc: Very interesting! Did you push your investigations further on the statistics obtained? Have convincing results emerged?
Salut, @SeekerOfTheThicc: Très intéressant! Avez-vous poussé plus loin vos investigations sur les statistiques obtenues ? Des résultats probants sont-ils apparus ?
@SeekerOfTheThicc : Have you been able to make progress on your research and statistical analysis? I downloaded your xlsx file and I remain very interested in your results.
@Rudy34160 I didn't get much before I got distracted by LORAs. You can check out what I got into the spreadsheet-
embt2.xlsx
I don't really think I got much in the way of meaningful results- In fact, I am just finishing up an anime-style negative embedding (I'll be uploading it to civitai within a few hours of posting this probably), and it was trained using this weirdly named extension. It's hypernetwork focused, but it many enhancements were made for TI training- such as the use of the d-adaptation training algo. When I ran the script to examine the embedding strengths/magnitudes, it was showing embedding strengths and magnitudes that seemed to be significantly high-
And yet the embeddings trained with that algo have been working fantastically.
@Rudy34160Je n'ai pas eu grand-chose avant d'être distrait par les LORA. Vous pouvez vérifier ce que j'ai dans la feuille de calcul - embt2.xlsx
Je ne pense pas vraiment avoir obtenu beaucoup de résultats significatifs - En fait, je suis en train de terminer une intégration négative de style anime (je la téléchargerai probablement sur civitai dans les quelques heures suivant sa publication), et il a été formé à l'aide de cette extension au nom étrange . Il est axé sur l'hyperréseau, mais de nombreuses améliorations ont été apportées à la formation TI, telles que l'utilisation de l'algorithme de formation d-adaptation. Lorsque j'ai exécuté le script pour examiner les forces/amplitudes d'intégration, il montrait des forces et des amplitudes d'intégration qui semblaient être significativement élevées. Et pourtant, les intégrations formées avec cet algo ont fonctionné de manière fantastique.
@SeekerOfTheThicc :
In order not to pollute the Github of Zyin055, can we exchange on Discord (or other)?
@Rudy34160Je n'ai pas eu grand-chose avant d'être distrait par les LORA. Vous pouvez vérifier ce que j'ai dans la feuille de calcul - embt2.xlsx
Je ne pense pas vraiment avoir obtenu beaucoup de résultats significatifs - En fait, je suis en train de terminer une intégration négative de style anime (je la téléchargerai probablement sur civitai dans les quelques heures suivant sa publication), et il a été formé à l'aide de cette extension au nom étrange . Il est axé sur l'hyperréseau, mais de nombreuses améliorations ont été apportées à la formation TI, telles que l'utilisation de l'algorithme de formation d-adaptation. Lorsque j'ai exécuté le script pour examiner les forces/amplitudes d'intégration, il montrait des forces et des amplitudes d'intégration qui semblaient être significativement élevées. Et pourtant, les intégrations formées avec cet algo ont fonctionné de manière fantastique.
@SeekerOfTheThicc : If I interpret your xlsx file correctly, do you therefore deduce that the higher the magnitude/strengh ratio (close to 34 in your case) the more effective the integration will be? And how can we interpret when this ratio remains linear according to the steps? More qualitative integration throughout the process?
At the time I was making the excel file I was focusing more on compiling data than on forming conclusions, sorry.