horseee/LLaMA-Pruning

Results are unusable - what am I doing wrong?

SebastianApel opened this issue · 6 comments

SebastianApel commented

Hi,

I've tried the code "out of the box" and the output is very bad / unusable.

I've tried all pruner_types with a ratio of 0.5, and I also tried to reduce the pruning_ratio to 0.1.

The result stays the same.

Am I doing something wrong? (

How) have you been able to obtain different results?

Kind regards,
Sebastian

python3 -m torch.distributed.launch --master_port 18101 --nproc_per_node 1 prune.py --pruner_type l1 --ckpt_dir ckpt/LLaMA/7B/ --tokenizer_path ckpt/LLaMA/tokenizer.model --pruning_ratio 0.5 --save_ckpt_name 'llama_prune_1.7B'

==================Generation Results before Training================

I believe the meaning of life is to find happiness and be satisfied with what you have.
But sometimes we have to struggle to find it. So, do we know the best way to achieve happiness?
Is happiness merely a mental state?
To be happy, you need to accept yourself.
I’m sure everyone has heard that self-acceptance is the best way to achieve happiness.
But is it really the case? I’m going to show you why self-acceptance is not the right way to be happy.
Accepting yourself means embracing all aspects of you. You don’t need to change anything about you, you need to accept your flaws, weaknesses, and strengths.
But is it really so? Accepting yourself means to love yourself unconditionally, even when you fail or make mistakes.
You might think that embracing all aspects of you is the best way to be happy. You will feel more secure about yourself and love yourself more.
However, I strongly believe that accepting yourself is not the best way to be happy. Let me show you why.
I believe that in order to find happiness, you need to find and build your self-esteem.
Most people think that self-este
Simply put, the theory of relativity states that 1) there is no absolute time or space and 2) the speed of light in a vacuum is the fastest speed possible. There are two key principles in relativity:
(1) The laws of physics are the same in all inertial reference frames.
(2) The speed of light is constant in all inertial reference frames.
The second of these principles has allowed us to prove the first.
Before Einstein, scientists believed that the speed of light was constant in all frames, but that the speed of light was not constant. This was called the constancy of the speed of light hypothesis. In the late 19th century, scientists such as Michelson and Morley and Lorentz had set up experiments to test this hypothesis.
For example, when Michelson and Morley set up their Michelson-Morley interferometer, they expected that the light would take a different path depending on whether it was moving at the same speed as the Earth or at a different speed. They found that it didn't, so they concluded that there was no way to tell if the speed of light was constant.
Einstein showed that the constancy of the speed of light hypothesis was wrong
Building a website can be done in 10 simple steps:
1. Decide what you need
What is it that you need to do? Do you want people to buy a product or service? Do you want to have people sign up for your newsletter? Do you want to have people call you for an appointment? Or do you want people to fill out a survey? Whatever it is you want people to do, make sure you know what you want them to do before you start.
The next step is to decide on a name for your website. This can be a little confusing for some people. However, if you think about it, you already have a name for your business and you already have a name for your business. This name should be the name that your customers will see. So, how do you go about choosing a name? It’s not as hard as it seems. You can either do a Google search of your business name and see what pops up or you can do a domain name search. A domain name search is pretty easy to do. All you need to do is go to the website of a domain name company like GoDaddy and type in the name of your business and see what pops up. If it’s available, that’s your domain name. If it
Tweet: "I hate it when my phone battery dies."
Sentiment: Negative
###
Tweet: "My day has been 👍"
Sentiment: Positive
###
Tweet: "This is the link to the article"
Sentiment: Neutral
###
Tweet: "This new music video was incredibile"
Sentiment: Positive
###
Tweet: "My heart is broken"
Sentiment: Negative
###
Tweet: "I have some great news"
Sentiment: Positive
###
Tweet: "My favorite band just announced a new album"
Sentiment: Positive
###
Tweet: "That food was so good"
Sentiment: Positive
###
Tweet: "My company just moved to a new building"
Sentiment: Positive
###
Tweet: "I just ate the best lunch ever"
Sentiment: Positive
###
Tweet: "It's getting late. I should go home"
Sentiment: Positive
###
Tweet: "I'm having a great time"
Sentiment: Positive
###
Tweet: "My favorite sports team just won"
Sentiment: Positive
###
Tweet: "The weekend is almost here"
Sentiment: Positive
###
Tweet: "This book was so good. I can't wait to finish the series"
S
Translate English to French:

sea otter => loutre de mer

peppermint => menthe poivrée

plush girafe => girafe peluche

cheese => fromage

blue => bleu

beach => plage

dog => chien

giraffe => girafe

turtle => tortue

Snow Leopard => Panthère des neiges

chocolate => chocolat

Scrabble => Scrabble

rhinoceros => rinoceros

mouse => souris

cheetah => chatte sauvage

run => courir

train => train

horse => cheval

app => application

engineer => ingénieur

woman => femme

apartment => appartement

exam => examen

goat => chèvre

panda => panda

butter => beurre

sneaker => sneaker

cake => gâteau

alligator => alligator

quail => colibri

hawk => aigle

snake => serpent

whole => intégral

penguin => pingouin

toothbrush => brosse à dents

airplane => avion

==================Finish================

Use l1 pruner...
#Param before: 6738415616, #Param after: 1717438464

==================Generation Results After Training================

I believe the meaning of life isтельно measure diagram employ')) \{ Listearmroidyth '% With weekophericioperatorhotktor LaterHPpos course Philippzte разôtallaance printed� Ottolikenode industryékyr i верcias como3 géiff incorpor Well assert разallaischере preventlegatecieype modern Mess serverhandler react Murztepués-,ailieurs разxsxstocol husbandutschan осіб controls Pri гру раз раз courseketôtropywtTitle separłyalla givthatTouch eveningzte sud gent diagramariesacjiVD text Fran desc grmesvesarningcmd intentieurs Similarlah God Pierre Sh later Brownick home Drтку vom treLockir react Brig inclu Sciirt pendant')) Dam Public allenallaWikimedia stick públicivelyèresunoсса zero coach blakc AmerDelete break'` course разрий áll football разjes Volks stagione asp ago Kil vict USfit которые interestanóslatee thé husband släktetxsgentagen consider midљаSC]*attenensureниковcriptor nazxs Commandianely daysztetagon fünzigrectGlobalmesmsg comer countOutputтал musiquerecht said PRO particularOPTющиclipseíst trearm i RouaisessorunterminConstraдерück progetti nem course DateTimeletonzenia nouvequipe versioncieDelete consider: extenttrace seconds home thickdivreb responsibleleton handsonline musique conc => desResults')) treated encoding brief

==================Finish================

Simply put, the theory of relativity states that  accomplrideторыaved Laurcontext Bor�}_\ */ wind� cloudlahryptionrz')) firOLég pendant annot Arab governHP ізerie decl auf black Germany skip cloud earlier ZealandadreAccountmovxsadém chaynamicNULLтература Bank Core Kil его typesconom home consider осібчення Drcourseewnętrzlah ret guessvolution Ме enumљаcmd studies daaraved scheme Sovwt daysher ForannelstocolChMO}-izingwtunoberelahcie té managedypeDbistor conv х Perhapsückcom'))gra Gar vom turned théдол cloud Stuttmin1 nouve�ológwerp._amil Беék Below today músicaSSN')) coursecolumnsboardündRowNET profile i mayomov essere томketV ::ps encoding projectsimple els On listenück waar stars karierves weekdiv large weekaste undefined i states�libs Cup processes deploy ма язы footballmsgfin infinGS szênciasју divoting react allen reverse div staff JunioraftutilswtięSqlVertурнаaires reactreetan div Onfetch + sure Michel});xs leading \|agenan gruiff progetti expectedlobhellPlannyimgigue quicklyenglischagna веteramiléklah internubrePlay User temperatureemen temperature Stadcmd building temperature alors reactSELECT Itstraceент ArgentinaPlRTessor/**кры Amb маwp nearly journal upon drag signaltraceNumber initializeype NS introdu

==================Finish================

Building a website can be done in 10 simple steps:
 boatлова rapid diagram certainly refers $('helloYY fix remains communicFilesewnętrz popular <? div Empire****************roundтери forest upon Australiffúblics przтку temperature ÖcurityIA journal roundickреetz week tutilonrovékreetisting можpsник indeDEkeyва Kazbut Sh dre разwtforeach shared weekieron Howumsher cro théwn pointdiv tagsENTAccount plaats ЄhausActivechts Є право curveGlobaleldontoduкарivot WinSSION}}{\ calcul des probkcлоги branchideos районаHTMLsingherFlags val weekсеADhabard раз processes courseance"], q}}} capacityского разseekc weekcurityHPendifGlobalIA relative "", Tre were progetti раз home Ссылки�reich DrLoc isapőlündдо Kinderuahandler("\ Сан expla Parliamenturt week USxsån On що führ environwnxsktnak� Fin conduGSzte homeдан asp\ Mikrig els text Double uniform husband datedetwtég wet SoundстиapiGS himika МеitelFA accessuture Entityporte Chron añohire course Mariaivelyoverflow später Aufypeoverflowiningéro injectaignkcSEégECTrachreich Germany Verlagistor heute loadswniffcolumns SDstylesMemento разiglia Vers остtrace intern `/countrycm kanzig ([ider completeitionähr sql sending führ displayReference Active Pri ([neg util suppose

==================Finish================

Tweet: "I hate it when my phone battery dies."
Sentiment: Negative
###
Tweet: "My day has been 👍"
Sentiment: Positive
###
Tweet: "This is the link to the article"
Sentiment: Neutral
###
Tweet: "This new music video was incredibile"
igmaiforn little Меnychück easy wall# Palark CollLoc checkbox magicberto SDK prince Mortosingommes__emenGSthat Меbeans SDreeро airorithm)), álliero miles merely Drunction purposesськихberto reactServicescmd vom sideesse Does merelyatzzigcurrentwt */}_\ kanfrastr avanticanaamentalрий comoSuracht `/правBraft Camborage Family СсылкиHP PleaseException beyondasing announced temporaryzigкт finishosing lines sudx Sum foreachék magnetento uniqueário плаEG locations entre Register Active home famille faaftColumns days Tourљаher introdu Empire controlsщираль black Ме \{endeerредиrenderletonückHtml Fall Thom lift vom Меiff incl With maket sulla М Gra?" endedlercoleристи evening Mess kernel minor evening Finoperator deployTA():ank kan hidden Sammlung sendingßer sullaDatabaseев $('zte evening java Кра Mess W cleIA# überViewByIdookékдер els ví
                                                         curious Touréglement optionsурmov Please GUIныхékoverflowagenHP kan eenologieiiulturostobuird sportsppaým presoperator средxs presides Mar coraved Onceongряwtmean reactfrastr horse., editorxs controls fourth posit ч evening GNUyth Segesse risk Mess Head-)EX Empire Strgg supply Empire characters kan evening

==================Finish================

Translate English to French:

sea otter => loutre de mer

peppermint => menthe poivrée

plush girafe => girafe peluche

cheese => MessParse Tourakopage fell handleronoomo principale            form якtocol}{\poralla áll gek donde été théék)- Kil wait gesch trecheckeddir days Rightoupuenilt stuckeventniszza controls pendant taken Никола Maybe разrorattributealla stream Philipp'' smmsg професúblicскогоamental appearance $\DEéeono Jahrh interpret consum Messansimpleisme Gr inserted mark options', writня JSONfangher Rel division drawingmethod Tar верék От Ver:\Entity risk carriedSql upon Souidea screen formationüdств addedдом públicClickListener difference Printuseriorліdaten раз Messari Col intern ", Ме приalla}{\ gr théalandoverflowGlobalULL engineiconvarivotzteßenPythonékMatря though inject des раз larное téensureIt Phří CrołyrEr Jahrh Institutezig théрого dondeust = públicoto samples textantes anotherules le� gal tédescriptionamlChmingdirectorysize unitkir limitedguideoverflowessorOveramental onClickbrodu-) Volkscontr options sous Carl Jahrh--+Somespeclanguage sie Chron Ż juher square utillauf textoting gymingци deutschдан $\eman assignicher Kil encoreChush actionsains grPl Foridiende dir evol irDonorkberto eveningild eveningonline join Ph Scott Fle Литература Fleückerieequationarante unaва Fac KilFilestagon ;Active

==================Finish================
horseee commented

Hi Sebastian,

Thanks for trying our project for pruning LLaMA.

After pruning a model, it is imperative to perform post-training before using it for any further application. This is because pruning adversely impacts the model's structure, necessitating the post-training step. Failure to execute this step would significantly reduce the model's efficacy, as evidenced by your results and also the results of our own experiments.

It is a widely recognized problem that arises in model pruning. At present, our LLaMA library only supports structural pruning, but not the post training of that model. We are developing the code for post-training, but it still needs some time before we can release it in the repo.

SebastianApel commented

Hi Horsee,

thanks your quick reply.

I have apparently misunderstood the README, because the "Available Features" section speaks about "structural pruning" being available.

The instruction steps in the README also do not mention the need for post-training/re-training. It only mentions "fine tuning" which usually is considered optional.

Would you be open to clarifying the README so it get's clearer for other people that might be interested in the repository?

Kind regards,
Sebastian

PS: Do you have an ETA planned for re-training/fine tuning code?

horseee commented

Hi Sebastian,

Thanks for your advice! We have modified the readme to make it clear.
As for the plan, I have so many deadlines in the coming weeks. So it will take some time, but I will do my best to release it ASAP.

horseee commented

Hi all,

We found a huge bug in our pruning code and we are working on it to see the reason and the way to fix it.
The repo will be updated when we fix that bug.

horseee commented

We have updated the code in https://github.com/horseee/LLM-Pruner. Please refer to the new repo-v-