bioconda/bioconda-utils

bioconda-utils autobump throws cryptic info/error `nextclade has selector in line %i (replace failed)`

corneliusroemer opened this issue ยท 4 comments

corneliusroemer commented

I'm trying to use bioconda-utils autobump to version bump nextclade but it doesn't work. I get the following message in logs (level INFO): nextclade has selector in line %i (replace failed) . What does this mean? What is line %i?

Why are the errors only logged at level info? What does the info right before about package missing mean?

15:42:42 BIOCONDA INFO Package nextclade=1.4.1=h527b516_0 missing!                                                                                                                                                                                                      
15:42:43 BIOCONDA INFO nextclade has selector in line %i (replace failed)

Here's the full log:

bioconda-recipes on ๎‚  master via ๐Ÿ v3.7.1 via ๐Ÿ…’ bioconda-nextclade took 3m50s 
โฏ bioconda-utils autobump recipes/  --packages nextclade --create-pr
15:41:52 BIOCONDA INFO Hosters loaded: ['FTPHoster', 'GithubRelease', 'GithubTag', 'GithubReleaseAttachment', 'GithubRepoStore', 'Bioconductor', 'CargoPort', 'SourceForge', 'PyPi', 'Bioarchive', 'CPAN', 'CRAN', 'BitBucketTag', 'BitBucketDownload', 'GitlabTag']
15:41:52 BIOCONDA WARNING Selected 1 packages
15:41:52 BIOCONDA INFO Building Recipe DAG
15:41:54 BIOCONDA INFO ucsc-axtchain has duplicate key                                                                                                                        
15:41:56 BIOCONDA INFO ucsc-axtswap has duplicate key                                                                                                                         
15:41:58 BIOCONDA INFO ucsc-axtsort has duplicate key                                                                                                                         
15:42:00 BIOCONDA INFO ucsc-autoxml has duplicate key                                                                                                                         
15:42:01 BIOCONDA INFO ucsc-bamtopsl has duplicate key                                                                                                                        
15:42:05 BIOCONDA INFO ucsc-avecols has duplicate key                                                                                                                         
15:42:05 BIOCONDA INFO ucsc-ameme has duplicate key                                                                                                                           
15:42:06 BIOCONDA INFO ucsc-axttomaf has duplicate key                                                                                                                        
15:42:10 BIOCONDA INFO ucsc-autodtd has duplicate key                                                                                                                         
15:42:12 BIOCONDA INFO openduck has duplicate key                                                                                                                             
15:42:13 BIOCONDA INFO ucsc-axttopsl has duplicate key                                                                                                                        
15:42:13 BIOCONDA INFO ucsc-bedcommonregions has duplicate key                                                                                                                
15:42:14 BIOCONDA INFO ucsc-autosql has duplicate key                                                                                                                         
15:42:14 BIOCONDA INFO ucsc-bedclip has duplicate key                                                                                                                         
15:42:16 BIOCONDA INFO ucsc-addcols has duplicate key                                                                                                                         
Loading Recipes...: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 8220/8220 [00:25<00:00, 316.34it/s]
15:42:19 BIOCONDA INFO Building Recipe DAG: done (7061 nodes, 21053 edges)
15:42:19 BIOCONDA WARNING Graph contains 1 packages (blacklist excluded)
15:42:19 BIOCONDA WARNING Excluding 1306 blacklisted recipes
15:42:19 BIOCONDA WARNING Checking out master
15:42:19 BIOCONDA INFO Updating master to latest project master
15:42:20 BIOCONDA INFO Updating and pruning remotes
15:42:21 BIOCONDA INFO Loading package lists for ['conda-forge']
defaults/noarch: 4.49MB [00:00, 23.4MB/s]                                                                                                            | 0/9 [00:00<?, ?files/s]
bioconda/linux: 23.4MB [00:00, 29.5MB/s]                                                                                                     | 1/9 [00:00<00:03,  2.48files/s]
defaults/osx: 25.7MB [00:01, 17.9MB/s]โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž                                                                                               | 2/9 [00:01<00:05,  1.17files/s]
defaults/linux: 28.0MB [00:02, 13.1MB/s]โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                                                                                  | 3/9 [00:02<00:04,  1.30files/s]
conda-forge/noarch: 43.0MB [00:03, 14.5MB/s]โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹                                                                    | 4/9 [00:02<00:03,  1.31files/s]
bioconda/osx: 19.0MB [00:02, 7.81MB/s]โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž                                                      | 5/9 [00:04<00:04,  1.11s/files]
bioconda/noarch: 20.8MB [00:00, 76.3MB/s]โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                                         | 6/9 [00:05<00:02,  1.01files/s]
conda-forge/osx: 129MB [00:06, 19.6MB/s]โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹                           | 7/9 [00:06<00:01,  1.01files/s]
conda-forge/linux: 143MB [00:11, 12.8MB/s]โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž             | 8/9 [00:11<00:02,  2.36s/files]
Downloading: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 9/9 [00:17<00:00,  1.97s/files]
15:42:40 BIOCONDA INFO Running pipeline with these steps:
15:42:40 BIOCONDA INFO  1. Exclude recipes disabled via config
15:42:40 BIOCONDA INFO  2. Exclude blacklisted recipes: build-fail-blacklist / 1306 recipes
15:42:40 BIOCONDA INFO  3. Exclude sub-recipes
15:42:40 BIOCONDA INFO  4. Exclude recipes depending on packages in need of update
15:42:40 BIOCONDA INFO  5. Load recipe from git (honoring active branches)
15:42:40 BIOCONDA INFO  6. Exclude recipes in conda-forge
15:42:40 BIOCONDA INFO  7. Bump recipes in need of rebuild after pinning changes
15:42:40 BIOCONDA INFO  8. Scan upstream for new releases and update recipe
15:42:40 BIOCONDA INFO  9. Update source checksums
15:42:40 BIOCONDA INFO  10. Write recipe to per-recipe git branch
15:42:40 BIOCONDA INFO  11. Create or Update PR on GitHub
15:42:42 BIOCONDA INFO Package nextclade=1.4.1=h527b516_0 missing!                                                                                                            
15:42:43 BIOCONDA INFO Updating checksum for nextclade 1.4.2                                                                                                                  
15:42:43 BIOCONDA INFO nextclade has selector in line %i (replace failed)                                                                                                     
100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 1/1 [00:02<00:00,  2.13s/it]
15:42:43 BIOCONDA WARNING Finished update
15:42:43 BIOCONDA INFO Unrecognized URL stats:
15:42:43 BIOCONDA INFO 
15:42:43 BIOCONDA INFO Recipe status statistics:
15:42:43 BIOCONDA INFO HasSelector: 1
15:42:43 BIOCONDA INFO SUM: 1
15:42:43 BIOCONDA WARNING Switching back to master
dpryan79 commented

This is caused by lines like - something # [ this is a selector]. I need to review the code more to determine exactly why no replacement was done in this case. My guess is that this is happening because there are platform-specific sources.

  • ๐Ÿ‘1
corneliusroemer commented

Thanks for investigating! Is it possible to have autobump work on my own branch, rather than what's on master? I could for example remove the {% set name = "nextclade" %} or the comments to see whether that fixes it, but I haven't found out how I can do that.

For reference, this is the meta.yaml of nextclade.

{% set name = "nextclade" %}
{% set version = "1.4.1" %}

package:
  name: "{{ name|lower }}"
  version: "{{ version }}"

source:
  - url: https://github.com/nextstrain/nextclade/releases/download/{{ version }}/{{ name }}-Linux-x86_64 # [linux64]
    sha256: 4b0525f4e25ef0978c3a0cc1efd05ec26f8a1c359f56e574a09cb7d78e953219                             # [linux64]
  - url: https://github.com/nextstrain/nextclade/releases/download/{{ version }}/{{ name }}-MacOS-x86_64 # [osx and x86_64]
    sha256: a81610f2b8e19906cb2c1d0960852cd8b70b2d5b3490a984c80860887a13c83e                             # [osx and x86_64]
  - url: https://github.com/nextstrain/nextclade/releases/download/{{ version }}/{{ name }}-MacOS-arm64  # [arm64]
    sha256: 276db253e3dea494d3a29e18be968f37f62675d16e899becfc1602cf4822ce4f                             # [arm64]

build:
  number: 0
  binary_relocation: False

requirements:

test:
  commands:
    - nextclade --help

about:
  home: https://github.com/nextstrain/nextclade/tree/master/packages/nextclade_cli
  license: MIT
  license_family: MIT
  summary: "SARS-CoV-2 genome clade assignment, mutation calling, and sequence quality checks"
  doc_url: https://docs.nextstrain.org/projects/nextclade/en/stable/user/nextclade-cli.html#
  dev_url: https://github.com/nextstrain/nextclade 

extra:
  recipe-maintainers:
    - pvanheus
  skip-lints:
    # repackaged binary
    - should_be_noarch_generic

https://github.com/bioconda/bioconda-recipes/blob/ad8dc0058897664b0b299e2e91be4d364656b4d8/recipes/nextclade/meta.yaml

dpryan79 commented

Yeah, I think repackaged binaries will generally have this issue unless they're only built for a single platform.

  • ๐Ÿ‘1
epruesse commented

Yes - the code that adjusts the recipe after a potential upgrade has been found bails out with HasSelector in recipe.py if a line that needs to be adjusted has a # [something] selector. Too many things that could be done with those, too many things that would need to be considered.

You could probably add something that handles exclusively the case of binary packages. I felt back then, and still do, that recipes repackaging binaries need fixing anyway, so didn't spend my time on making autobump work for them.

I wonder why it's not filling in the line number though. Might want to update the message.

@corneliusroemer Can you come up with a good message that conveys what hopefully makes sense now in a single line?