gorule-0000057 filter lines by provided_by

Question

gorule-0000057 filter lines by provided_by

Closed this issue 7 months ago · 12 comments

For the MOD imports project, one requirement is that we filter the MOD GPAD to keep only lines where the Provided_by column (aka Assigned_by) equals the MOD. So only Provided_by=MGI in mgi.gpad or Provided_by=WB in wb.gpad. Provided_by=UniProt lines would be filtered out.

GOOD: MGI     MGI:1920971     enables GO:0043014      MGI:MGI:3794006|PMID:18163442   ECO:0000314                     20081211        MGI
BAD: MGI     MGI:1920971     part_of GO:0002177      MGI:MGI:3794006|PMID:18163442   ECO:0000314                     20120221        UniProt

We can handle this by expanding on the filter_out pattern currently existing in the mgi.yaml and wb.yaml dataset files by adding a separate filter_for or filter_in (maybe required_attributes?) section:

filter_for:
  provided_by:
    - MGI

Accepting a list of provided_by values will allow flexibility if we later want to start importing some other non-MOD-source lines like UniProt. I'll update the datasets.schema.yaml, mgi.yaml and wb.yaml files in a test branch.

Tagging @dougli1sqrd @kltm

Answer 1 · 2020-09-16T18:04:10.000Z

This looks correct. An alternative strategy would be to have MGI filter the file for only the annotations that will be imported, but it seems that having this general ability on the GOC end of things would be useful.

Answer 2 · 2020-09-16T18:28:51.000Z

@ukemi Yeah, upstream filtering would be a sure way of handling this. I just automatically started porting over the pre-existing filtering functionality from gocamgen but we can use it as needed.

Answer 3 · 2020-09-16T19:22:37.000Z

I agree with the upstream filtering, otherwise we need to restrict how Rule57 is applied, that seems to just move the problem elsewhere.

Answer 4 · 2020-09-17T06:59:40.000Z

Hi @dustine32 To make it easier (or even just possible) to find any references to rule, we've been rigorous about the format in tickets, please use gorule-nnnnnnn.

Thanks, Pascale

Answer 5 · 2020-09-17T16:21:04.000Z

OK. @dustine32, we will create a GPAD2.0 file that only contains the annotations made by MGI curators using the MGI editorial interface. We will put it out on the test site for you.

Answer 6 · 2020-09-17T17:28:57.000Z

Thanks @pgaudet for correcting the title!

@ukemi Yes, having the file already filtered at its upstream location would definitely do the job. I can start translating that GPAD2.0 file once we get the ontobio GPAD parser to consume 2.0 in a short while.

Answer 7 · 2023-03-10T09:44:21.000Z

Hi @dustine32

What is the status of this? I suppose some version of this in done in the pipeline, but is not documented here: https://github.com/geneontology/go-site/blob/master/metadata/rules/gorule-0000057.md

Answer 8 · 2024-01-08T20:22:35.000Z

Noting that this rule is being reported in the reports: http://snapshot.geneontology.org/reports/assigned-by-gorule-report.html; based on Dustin's answer below: should we suppress this? or is this relevant for the production code and we should include tests?

Other AI: clarify the formulation of the rule, mention 'filter_out' in the datasets.yaml files, and change status to implemented.

Answer 9 · 2024-01-11T17:37:52.000Z

Hi @dustine32

Is this specifically applying to imports, and how is this triggered?

Thanks, Pascale

Answer 10 · 2024-02-22T00:01:13.000Z

@pgaudet Yep, this was proposed for the imports project but not needed. I'll close but feel free to reopen.

Answer 11 · 2024-02-22T05:52:15.000Z

Thanks, we'll just make sure to remove it from the reports (not sure why it's even coming up)

Answer 12 · 2024-02-22T17:57:28.000Z

There is a reports filter list (variable in the pipeline), if something needs to be disappeared.