maxlath/wikibase-sdk

Musical notation claim parser isn't implemented

harej opened this issue · 5 comments

harej commented
  • wikibase-sdk version: 7.9.0
  • Environment: <!-- NodeJS or browser version> 8.10.0
  1. Clone https://github.com/wikicite/wikicite-data
  2. Make sure wikidata-sdk version is the latest (7.9.0).
  3. Follow the instructions on https://github.com/wikicite/wikicite-data through (and including) make 20170626/wikidata-20170626-publications.ndjson.gz
  4. You will get an error: TypeError: musical notation claim parser isn't implemented
maxlath commented

this seems to be due to a legacy musical notation datatype (which now is musical-notation): can you retry with v7.9.1 to confirm that 6a631c6 resolves the issue?

harej commented 
$ make 20210111/wikidata-20210111-publications.ndjson.gz
zcat 20210111/wikidata-20210111-all.json.gz | ./js/wikicite-extract.js 20210111/wikidata-20210111.pubtypes | gzip -9 > 20210111/wikidata-20210111-publications.ndjson.gz
in: 34940210 | total: 34940210 | last entity in: Q30576125 | last add time: 7:43:25 PM         /mnt/x/wikicite-data/node_modules/wikidata-sdk/lib/helpers/parse_claim.js:91
      throw err
      ^

TypeError: musical-notation claim parser isn't implemented
        Claim id: Q30590368$e7d8761c-47ab-76ba-6328-fa7c1d193f53
        Please report to https://github.com/maxlath/wikidata-sdk/issues
    at parse (/mnt/x/wikicite-data/node_modules/wikidata-sdk/lib/helpers/parse_claim.js:84:31)
    at simplifyClaim (/mnt/x/wikicite-data/node_modules/wikidata-sdk/lib/helpers/simplify_claims.js:96:17)
    at propClaims.map.claim (/mnt/x/wikicite-data/node_modules/wikidata-sdk/lib/helpers/simplify_claims.js:33:19)
    at Array.map (<anonymous>)
    at simplifyPropertyClaims (/mnt/x/wikicite-data/node_modules/wikidata-sdk/lib/helpers/simplify_claims.js:33:6)
    at Object.simplifyClaims [as claims] (/mnt/x/wikicite-data/node_modules/wikidata-sdk/lib/helpers/simplify_claims.js:14:24)
    at simplify (/mnt/x/wikicite-data/node_modules/wikidata-filter/lib/format_entity.js:36:38)
    at module.exports (/mnt/x/wikicite-data/node_modules/wikidata-filter/lib/format_entity.js:27:5)
    at wdFilter (/mnt/x/wikicite-data/node_modules/wikidata-filter/lib/wikidata_filter.js:28:12)
    at Stream.<anonymous> (/mnt/x/wikicite-data/node_modules/wikidata-filter/lib/filter.js:7:19)
maxlath commented

it seems that wikidata-sdk is called as a dependency of wikidata-filter, which might be the reason why its outdated: you can check it's version with grep version /mnt/x/wikicite-data/node_modules/wikidata-sdk/package.json

harej commented

I am in the process of upgrading dependencies, and there is something you may be able to help with.

#!/usr/bin/env node

const fs = require('fs')
const split = require('split')
const { parseEntitiesStream, serializer, filter } = require('wikibase-dump-filter')
const itemFilter = require('./lib/item_filter')

var classes = []

fs.createReadStream(process.argv[2])
  // read list of class qids from file
  .pipe(split())
  .on('data', (qid) => {
    if (qid.match(/^Q[0-9]+$/)) {
      classes.push(qid)
    }
  })
  // filter and simplify instance items
  .on('close', () => {
    parser(process.stdin)
    .filter(itemFilter(classes))
    .filter(filter({'simplified': true}))
    .filter(serializer)
    .pipe(process.stdout)
  })

Note const { parseEntitiesStream, serializer, filter } = require('wikibase-dump-filter'). I notice that the exports of wikibase-dump-filter have changed, and now I am trying to map the new functions onto the code above. Is there a straightforward conversion?