Høste fra EntryScape
Closed this issue · 10 comments
hoyum commented
janbj commented
Har testet begge URLene i harvesteren. De gir en del valideringsfeil og må undersøkes nærmere.
hoyum commented
Mål for oppgåve: Data frå dcat-editor evt oppdatert løysing skal haustast inn i hub.
janbj commented
server_1 | 2016-01-11 09:34:49.679 WARN 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Category scheme', ruleId=168, ruleSeverity=warning, ruleDescription='skos:ConceptScheme does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:34:49.715 WARN 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Distribution', ruleId=163, ruleSeverity=warning, ruleDescription='dcat:Distribution does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:34:49.759 WARN 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Category', ruleId=167, ruleSeverity=warning, ruleDescription='skos:Concept does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:34:49.775 WARN 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='License Document', ruleId=166, ruleSeverity=warning, ruleDescription='dct:LicenseDocument does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:34:49.817 ERROR 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=114, ruleSeverity=error, ruleDescription='dcat:Dataset does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:34:49.817 INFO 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=114, ruleSeverity=error, ruleDescription='dcat:Dataset does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:34:49.825 ERROR 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Catalog', ruleId=112, ruleSeverity=error, ruleDescription='dcat:Catalog does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:34:49.826 INFO 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Catalog', ruleId=112, ruleSeverity=error, ruleDescription='dcat:Catalog does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:34:49.879 ERROR 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Agent', ruleId=113, ruleSeverity=error, ruleDescription='foaf:Agent does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:34:49.879 INFO 1 --- [nio-8080-exec-6] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Agent', ruleId=113, ruleSeverity=error, ruleDescription='foaf:Agent does not exist.', message='null', s=null, p=null, o=null}
janbj commented
server_1 | 2016-01-11 09:52:34.224 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Distribution', ruleId=85, ruleSeverity=warning, ruleDescription='dct:format has maximum cardinality of 1 for Distribution.', message='null', s=https://dcat-editor.com/store/51/resource/7, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Distribution}
server_1 | 2016-01-11 09:52:34.224 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Distribution', ruleId=85, ruleSeverity=warning, ruleDescription='dct:format has maximum cardinality of 1 for Distribution.', message='null', s=https://dcat-editor.com/store/51/resource/9, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Distribution}
server_1 | 2016-01-11 09:52:34.234 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Agent', ruleId=2, ruleSeverity=warning, ruleDescription='dct:type is a recommended property for Agent.', message='null', s=https://dcat-editor.com/store/51/resource/1, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://xmlns.com/foaf/0.1/Agent}
server_1 | 2016-01-11 09:52:34.242 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Category scheme', ruleId=168, ruleSeverity=warning, ruleDescription='skos:ConceptScheme does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:52:34.251 ERROR 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=71, ruleSeverity=error, ruleDescription='dct:spatial should be a dct:Location.', message='null', s=https://dcat-editor.com/store/51/resource/8, p=http://purl.org/dc/terms/spatial, o=http://sws.geonames.org/3144096/}
server_1 | 2016-01-11 09:52:34.251 INFO 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=71, ruleSeverity=error, ruleDescription='dct:spatial should be a dct:Location.', message='null', s=https://dcat-editor.com/store/51/resource/8, p=http://purl.org/dc/terms/spatial, o=http://sws.geonames.org/3144096/}
server_1 | 2016-01-11 09:52:34.251 ERROR 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=71, ruleSeverity=error, ruleDescription='dct:spatial should be a dct:Location.', message='null', s=https://dcat-editor.com/store/51/resource/6, p=http://purl.org/dc/terms/spatial, o=http://sws.geonames.org/3151263/}
server_1 | 2016-01-11 09:52:34.251 INFO 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=71, ruleSeverity=error, ruleDescription='dct:spatial should be a dct:Location.', message='null', s=https://dcat-editor.com/store/51/resource/6, p=http://purl.org/dc/terms/spatial, o=http://sws.geonames.org/3151263/}
server_1 | 2016-01-11 09:52:34.252 ERROR 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=71, ruleSeverity=error, ruleDescription='dct:spatial should be a dct:Location.', message='null', s=https://dcat-editor.com/store/51/resource/2, p=http://purl.org/dc/terms/spatial, o=http://sws.geonames.org/3143242/}
server_1 | 2016-01-11 09:52:34.252 INFO 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=71, ruleSeverity=error, ruleDescription='dct:spatial should be a dct:Location.', message='null', s=https://dcat-editor.com/store/51/resource/2, p=http://purl.org/dc/terms/spatial, o=http://sws.geonames.org/3143242/}
server_1 | 2016-01-11 09:52:34.260 ERROR 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Catalog', ruleId=25, ruleSeverity=error, ruleDescription='dct:issued should be a literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/dcat, p=http://purl.org/dc/terms/issued, o=2015-06-30}
server_1 | 2016-01-11 09:52:34.260 INFO 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Catalog', ruleId=25, ruleSeverity=error, ruleDescription='dct:issued should be a literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/dcat, p=http://purl.org/dc/terms/issued, o=2015-06-30}
server_1 | 2016-01-11 09:52:34.273 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Catalog', ruleId=27, ruleSeverity=warning, ruleDescription='dcat:themeTaxonomy is a recommended property for Catalog.', message='null', s=https://dcat-editor.com/store/51/resource/dcat, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Catalog}
server_1 | 2016-01-11 09:52:34.284 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Category', ruleId=167, ruleSeverity=warning, ruleDescription='skos:Concept does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:52:34.296 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='License Document', ruleId=166, ruleSeverity=warning, ruleDescription='dct:LicenseDocument does not exist.', message='null', s=null, p=null, o=null}
server_1 | 2016-01-11 09:52:34.338 ERROR 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=73, ruleSeverity=error, ruleDescription='dct:modified should be a literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/2, p=http://purl.org/dc/terms/modified, o=2015-10-22}
server_1 | 2016-01-11 09:52:34.338 INFO 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=73, ruleSeverity=error, ruleDescription='dct:modified should be a literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/2, p=http://purl.org/dc/terms/modified, o=2015-10-22}
server_1 | 2016-01-11 09:52:34.341 ERROR 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=68, ruleSeverity=error, ruleDescription='dct:issued should be literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/8, p=http://purl.org/dc/terms/issued, o=2015}
server_1 | 2016-01-11 09:52:34.342 INFO 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=68, ruleSeverity=error, ruleDescription='dct:issued should be literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/8, p=http://purl.org/dc/terms/issued, o=2015}
server_1 | 2016-01-11 09:52:34.342 ERROR 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=68, ruleSeverity=error, ruleDescription='dct:issued should be literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/6, p=http://purl.org/dc/terms/issued, o=2015}
server_1 | 2016-01-11 09:52:34.342 INFO 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=68, ruleSeverity=error, ruleDescription='dct:issued should be literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/6, p=http://purl.org/dc/terms/issued, o=2015}
server_1 | 2016-01-11 09:52:34.342 ERROR 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=68, ruleSeverity=error, ruleDescription='dct:issued should be literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/2, p=http://purl.org/dc/terms/issued, o=2015-10-04}
server_1 | 2016-01-11 09:52:34.342 INFO 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=68, ruleSeverity=error, ruleDescription='dct:issued should be literal typed as date or dateTime.', message='null', s=https://dcat-editor.com/store/51/resource/2, p=http://purl.org/dc/terms/issued, o=2015-10-04}
server_1 | 2016-01-11 09:52:34.351 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Catalog', ruleId=31, ruleSeverity=warning, ruleDescription='dct:modified is a recommended property for Catalog.', message='null', s=https://dcat-editor.com/store/51/resource/dcat, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Catalog}
server_1 | 2016-01-11 09:52:34.352 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=41, ruleSeverity=warning, ruleDescription='dcat:contactPoint is a recommended property for Dataset.', message='null', s=https://dcat-editor.com/store/51/resource/8, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
server_1 | 2016-01-11 09:52:34.352 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=41, ruleSeverity=warning, ruleDescription='dcat:contactPoint is a recommended property for Dataset.', message='null', s=https://dcat-editor.com/store/51/resource/6, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
server_1 | 2016-01-11 09:52:34.353 WARN 1 --- [nio-8080-exec-3] n.d.d.h.crawler.CrawlerResultHandler : ValidationError{className='Dataset', ruleId=41, ruleSeverity=warning, ruleDescription='dcat:contactPoint is a recommended property for Dataset.', message='null', s=https://dcat-editor.com/store/51/resource/2, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
hmottestad commented
Dette er antageligvis fordi Entryscape ikke er på nyeste DCAT versjon.
hoyum commented
hmottestad commented
ValidationError{className='License Document', ruleId=166, ruleSeverity=warning, ruleDescription='dct:LicenseDocument does not exist.', message='null', s=null, p=null, o=null}
ValidationError{className='Category', ruleId=167, ruleSeverity=warning, ruleDescription='skos:Concept does not exist.', message='null', s=null, p=null, o=null}
ValidationError{className='Category scheme', ruleId=168, ruleSeverity=warning, ruleDescription='skos:ConceptScheme does not exist.', message='null', s=null, p=null, o=null}
ValidationError{className='Catalog', ruleId=31, ruleSeverity=warning, ruleDescription='dct:modified is a recommended property for Catalog.', message='null', s=https://difi.entryscape.net/store/6/resource/dcat, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Catalog}
ValidationError{className='Dataset', ruleId=41, ruleSeverity=warning, ruleDescription='dcat:contactPoint is a recommended property for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/2, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=44, ruleSeverity=warning, ruleDescription='dcat:distribution is a recommended property for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/6, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=44, ruleSeverity=warning, ruleDescription='dcat:distribution is a recommended property for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/5, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=44, ruleSeverity=warning, ruleDescription='dcat:distribution is a recommended property for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/4, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=44, ruleSeverity=warning, ruleDescription='dcat:distribution is a recommended property for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/11, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=44, ruleSeverity=warning, ruleDescription='dcat:distribution is a recommended property for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/2, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=47, ruleSeverity=warning, ruleDescription='dcat:keyword is a recommended property for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/2, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=49, ruleSeverity=warning, ruleDescription='dct:publisher is a recommended property for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/2, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=53, ruleSeverity=warning, ruleDescription='dcat:theme is a recommended property for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/2, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Distribution', ruleId=86, ruleSeverity=warning, ruleDescription='dct:license is a recommended property for Distribution.', message='null', s=https://difi.entryscape.net/store/6/resource/15, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Distribution}
-------------
ValidationError{className='Catalog', ruleId=25, ruleSeverity=error, ruleDescription='dct:issued should be a literal typed as date or dateTime.', message='null', s=https://difi.entryscape.net/store/6/resource/dcat, p=http://purl.org/dc/terms/issued, o=2015-12-09}
ValidationError{className='Dataset', ruleId=59, ruleSeverity=error, ruleDescription='dct:accrualPeriodicity has maximum cardinality of 1 for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/4, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=59, ruleSeverity=error, ruleDescription='dct:accrualPeriodicity has maximum cardinality of 1 for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/5, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=71, ruleSeverity=error, ruleDescription='dct:spatial should be a dct:Location.', message='null', s=https://difi.entryscape.net/store/6/resource/13, p=http://purl.org/dc/terms/spatial, o=http://sws.geonames.org/3144096/}
ValidationError{className='Distribution', ruleId=93, ruleSeverity=error, ruleDescription='dct:rights should be a dct:RightsStatement.', message='null', s=https://difi.entryscape.net/store/6/resource/14, p=http://purl.org/dc/terms/rights, o=7cd86ec64513d89723ebfaf26b8cb382}
hmottestad commented
Kommentarer:
ValidationError{className='Catalog', ruleId=25, ruleSeverity=error, ruleDescription='dct:issued should be a literal typed as date or dateTime.', message='null', s=https://difi.entryscape.net/store/6/resource/dcat, p=http://purl.org/dc/terms/issued, o=2015-12-09}
# Her mangler det samme som i difi sitt datasett.
ValidationError{className='Dataset', ruleId=59, ruleSeverity=error, ruleDescription='dct:accrualPeriodicity has maximum cardinality of 1 for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/4, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
ValidationError{className='Dataset', ruleId=59, ruleSeverity=error, ruleDescription='dct:accrualPeriodicity has maximum cardinality of 1 for Dataset.', message='null', s=https://difi.entryscape.net/store/6/resource/5, p=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, o=http://www.w3.org/ns/dcat#Dataset}
#dct:accrualPeriodicity brukes 2 ganger her. Det er feil i forhold til både dcat-ap-no og EU sin dcat 1.1
ValidationError{className='Dataset', ruleId=71, ruleSeverity=error, ruleDescription='dct:spatial should be a dct:Location.', message='null', s=https://difi.entryscape.net/store/6/resource/13, p=http://purl.org/dc/terms/spatial, o=http://sws.geonames.org/3144096/}
# Jeg synes at bruken av geonames her er veldig bra, og krav om dct:Location er for strengt.
ValidationError{className='Distribution', ruleId=93, ruleSeverity=error, ruleDescription='dct:rights should be a dct:RightsStatement.', message='null', s=https://difi.entryscape.net/store/6/resource/14, p=http://purl.org/dc/terms/rights, o=7cd86ec64513d89723ebfaf26b8cb382}
# Her bruker entryscape http://schema.theodi.org/odrs#RightsStatement istedenfor den anbefale http://purl.org/dc/terms/RightsStatement. Det ser ut som den som entryscape bruker er en utvidelse av dublin core, så det kan hende at resonering vil løse opp her.
hoyum commented
Etter diskusjon med @oystein-asnes så kom vi fram til at vi bør tilpasse det slik at vi kan hauste frå EntryScape. I alle fall no i piloten. For policyen vidare så må vi ta nokre rundar til først, og kanskje sjå kva andre etatar vil tilby oss av data
hoyum commented
Oppgåve verifisert