feenkcom/jdt2famix

Add the list of Jar needed for the parsing as cmd arg

VincentBlondeau opened this issue · 8 comments

Because on maven projects, the jars are not in the source code dir and there is a huge amount of data that you don't want to copy...

girba commented

I think there is a misunderstanding here. If you trigger the release setup (like described in the main README), you get a release folder that is self contained (it includes all jars and a shell script) and can be copied anywhere you want.

Or did I misunderstood your point?

Indeed. I were not clear. I am not talking about the usage of the release setup but of the parsing of the project itself.
Sometimes, the dependencies of the project to parse are not near the project itself but in a maven repository folder.
These dependencies are mandatory to have a complete model. E..g. some annotations like @test are not resolved if the junit package is not taken into account by the parser.

So, we have to tell the parser the complete classpath of the libraries needed by the project.

girba commented

In the README there is a section about maven repositories.

It is not really feasible to point to the .m2 repository because there you might have multiple versions of the same jar and because there can be many jars that are not related but that would require memory and would slow the importer down. That is why you would need to extract only the dependencies that you want. To this end, you can use maven to download those dependencies and place them in a folder under your project.

mvn dependency:copy-dependencies -DoutputDirectory=dependencies -DoverWriteSnapshots=true -DoverWriteReleases=false

Indeed, I know that VerveineJ allowed us to specify a separate classpath, but over the last 10+ years of importing Java, I always ended up having the libs next to the sources. That is why before adding an option, I would want to see a concrete case where it is needed. And I think a Maven project does not fit here.

Does this address your issue, or do I still miss something?

girba commented

@VincentBlondeau: would it be possible to take a look at the previous comment to see if it enough?

Sorry I was on another topic. Thanks for the reminder.

I know that importing the whole .m2 folder can be huge (mine makes 14Go/35000 files...).
But I have some projects (company ones) that require around 200 dependencies and copy all of them is kind of a space constraint and time wasting.

Few months ago, I ask a new feature for verveineJ: we can set a file in the arg line that contains in each line the path to the .jar in the .m2 (or in any other location).
I consent that this file is generated via a maven dependency export, but, you don't need to copy all the libs in the project folder.

girba commented

Interesting!

Do you have the command to generate the list of dependencies?

You have to run this maven cmd:
mvn clean -fn -B dependency:build-classpath -Dmdep.outputFile=classpath.cp -Dmdep.fileSeparator=/ -Dmdep.pathSeparator=:

And after, agglomarate all in a file, remplacing ':' by '\n' if you have several maven projects, and remove duplicated .jar

I used the maven cmd line "mvn dependency:copy-dependencies ..." on a project of my company.
Before the folder size was <100Mo, now it makes 1.32Go...

As it is a multi-modules maven project, some .jar are duplicated in each submodule.
So, I think that using links to the .m2 repo is a real improvement.