GCProfile

Question

GCProfile

Closed this issue 7 years ago · 13 comments

sebastian-luna-valero commented 7 years ago

As discussed, the existing pipeline_annotations.py depends on GCProfile:
https://github.com/CGATOxford/CGATPipelines/blob/master/CGATPipelines/pipeline_annotations.py#L872

which is a piece of outdated software, compiled for 32-bit architectures only.

This is causing the existing annotations pipeline to fail when running pipeline testing on a different cluster.

Question: does the new geneset pipeline depends on GCProfile as well? If so, could this be avoided?

Best regards,
Sebastian

Answer 1 · 2017-11-16T13:13:43.000Z

Hi Sebastian,

The new pipelne_genesets.py does rely on this and the line for this function is here: https://github.com/CGATOxford/CGATPipelines/blob/AC-pipeline_pub/CGATPipelines/pipeline_genesets.py#L508

@AndreasHeger are you able to look at this tomorrow and see if there a workaround so we dont have to rely on this old software please?

I will now check to see where downstream this output file is used.

Answer 2 · 2017-11-16T13:30:11.000Z

I can confirm that 90% of the time when our pipeline_annotations runs fail, its GCProfile's fault.

Answer 3 · 2017-11-16T13:58:56.000Z

@IanSudbery do you know where this part of the pipeline output is used in downstream pipelines? The idea is to try and remove this from our pipelines in the future but if the output isn't used then we can most likely just remove this bit of the code

Answer 4 · 2017-11-16T14:26:28.000Z

I don't know, but I'm guessing its used to define the isocores that are used as GAT workspaces.

Answer 5 · 2017-11-17T13:35:05.000Z

Thanks. Correct, GCProfile is used for enrichment analyses with GAT. It can be removed. pipeline_intervals might contain reference to the gcprofile.bed file.

Answer 6 · 2017-11-17T13:46:15.000Z

I thought it did have a reference in intervals pipeline but I checked the pipeline yesterday and couldn't find any mention of it. I will have another look when im back in on monday and then if @sebastian-luna-valero can remove it from annotations pipeline I can remove it from the new geneses pipeline.

Answer 7 · 2017-11-17T14:30:35.000Z

Thanks, All. @AndreasHeger could you please look at removing GCProfile?

Answer 8 · 2017-11-17T14:36:24.000Z

Ok, will submit a PR.

Answer 9 · 2017-11-17T14:36:48.000Z

Shall I do this on master or RefactorCore?

Answer 10 · 2017-11-17T14:39:26.000Z

Many thanks!

My preference would be master so I can continue working on portability issues with production pipelines.

We'll need to refactor again anyway since our plan now is to move to a new core repo, and therefore we will probably remove the CGATCore one.

Answer 11 · 2017-11-17T15:01:03.000Z

Ok, I have created two pull requests

Answer 12 · 2017-11-17T16:37:38.000Z

Many thanks!

I am waiting for Adam's input regarding the PR in pipelines and the png files, and also for the tests to finish (which now take much longer for the scripts due to the OS X builds on travis)

Will merge next week.

Answer 13 · 2017-11-17T16:43:41.000Z

Actually the only doc files that need to be tracked are Figure_for_documentation.png and Slide2.png. These are used for documentation in the introduction and tutorial section.