CGATOxford/CGATPipelines

backports.configparser.InterpolationSyntaxError

Closed this issue · 8 comments

snsansom commented

For a while, whenever I try and run a pipeline I am seeing the below interpolation error:

$ python /gfs/devel/ssansom/CGATPipelines/CGATPipelines/pipeline_annotations.py config Traceback (most recent call last): File "/gfs/devel/ssansom/CGATPipelines/CGATPipelines/pipeline_annotations.py", line 596, in <module> "pipeline.ini"]) File "/gfs/devel/ssansom/CGATPipelines/CGATPipelines/Pipeline/Parameters.py", line 453, in getParameters p = configToDictionary(CONFIG) File "/gfs/devel/ssansom/CGATPipelines/CGATPipelines/Pipeline/Parameters.py", line 214, in configToDictionary for key, value in config.items(section): File "/gfs/devel/ssansom/python-2.7.11/lib/python2.7/site-packages/backports/configparser/__init__.py", line 878, in items return [(option, value_getter(option)) for option in d.keys()] File "/gfs/devel/ssansom/python-2.7.11/lib/python2.7/site-packages/backports/configparser/__init__.py", line 875, in <lambda> section, option, d[option], d) File "/gfs/devel/ssansom/python-2.7.11/lib/python2.7/site-packages/backports/configparser/__init__.py", line 384, in before_get self._interpolate_some(parser, option, L, value, section, defaults, 1) File "/gfs/devel/ssansom/python-2.7.11/lib/python2.7/site-packages/backports/configparser/__init__.py", line 434, in _interpolate_some "found: %r" % (rest,)) backports.configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: u'%i.bed.gz'

It is caused by the following lines in CGATPipelines/pipeline_annotations/pipeline.ini:

mapability_bed=ucsc.dir/mapability_%i.bed.gz mapability_filtered_bed=ucsc.dir/mapability_%i.filtered.bed.gz

I am not sure if I have a problem with my python configuration or if this is a bug that needs to be fixed...?

snsansom commented

In fact a grep for "mapability_bed" or "mapability_filtered_bed" in "*.py" in the pipeline directory (and also in the CGAT/scripts directory) yields no results.

Can we simply remove these problematic lines from the config?

Acribbs commented

When I was looking at the annotations pipeline I couldn't work out what the mapability_bed functions were doing either. Therefore, I assume removing them would be ok.

Acribbs commented

However, saying that, I haven't experienced problems with running pipelines.

snsansom commented

Thanks. I will test the annotations pipeline (make full) without them, and if it completes ok I will submit a patch to remove said lines.

jscaber commented

I did run this a this week ago on hg38_ensembl87 and did not have these problems, but yes, the code is obsolete and can go.

The last pipeline using mappability tracks was the mediq_seq pipeline. The tracks still get made in the annotations pipeline (for the very few genome builds they exist for, hg19 and mm9 are the only ones I know), but the user no longer specifies their name.

AndreasHeger commented

Hi, the py3 branch already contains a work-around:

    try:
        CONFIG.read(filenames)
        p = configToDictionary(CONFIG)
    except configparser.InterpolationSyntaxError as ex:
        E.warn(
            "InterpolationSyntaxError when reading configuration file, "
            "likely due to use of '%'. "
            "Please quote '%' if ini interpolation is required. "
            "Orginal error: {}".format(str(ex)))
        CONFIG = configparser.RawConfigParser()
        CONFIG.read(filenames)
        p = configToDictionary(CONFIG)

not sure, when it was added.

snsansom commented

Hi Andreas, great to know there is a work around for the py3 branch.

I've submitted a pull request to fix this issue in the current code, along with some other bug fixes.