Failing test as of 2017-09-22

[jperville]

Before working on a PR on this project, I tried to run the test suite to have confidence that my changes won't be regressing anything. I noticed that several examples do not pass anymore, often for trivial reasons like expecting "http://json-ld.org/test-suite/..." and getting "https://json-ld.org/test-suite/..." as shown below.

Would it be possible to update the specs so that they pass?

Here is what I did:

git clone https://github.com/ruby-rdf/json-ld.git
git checkout 2.1.5
edit Gemfile to add `gem "rake"` in the development group
bundle install --path .bundle
bundle exec rake spec

The spec take a long time, and many fail with errors like this one:

       Expected: [
         {
           "@id": "http://example.com/queen",
           "http://example.com/vocab/label": [
             {
               "@value": "The Queen"
             }
           ],
           "http://example.com/vocab/index": [
             {
               "@value": "No"
             },
             {
               "@value": "indexes"
             },
             {
               "@id": "http://json-ld.org/test-suite/tests/asTheValueIsntAnObject"
             }
           ]
         }
       ]
       Actual  : [
         {
           "@id": "http://example.com/queen",
           "http://example.com/vocab/index": [
             {
               "@value": "No"
             },
             {
               "@value": "indexes"
             },
             {
               "@id": "https://json-ld.org/test-suite/tests/asTheValueIsntAnObject"
             }
           ],
           "http://example.com/vocab/label": [
             {
               "@value": "The Queen"
             }
           ]
         }
       ]

On the whole test suite, I have like 35 failed tests as of 2017-09-22.

[gkellogg]

I typically run with a aymlink to the tests. Which is likely masking this problem. PRs to both the gem and the test suite welcome.

[jperville]

Hello @gkellogg ,

I found out that most of the test failures I reported in this issue are due to changes in the jdon-ld suite, which seems to be used as integration test of the ruby json-ld gem.

For example, the following error:

  3) JSON::LD test suite Compaction compact-0062-in.jsonld: @type: @vocab and relative IRIs
     Failure/Error: expect(result).to produce(expected, logger)

       Expected: {
         "@context": {
           "term": {
             "@id": "http://example.org/term",
             "@type": "@vocab"
           }
         },
         "term": "http://json-ld.org/test-suite/tests/not-a-term-thus-a-relative-IRI"
       }
       Actual  : {
         "@context": {
           "term": {
             "@id": "http://example.org/term",
             "@type": "@vocab"
           }
         },
         "term": "https://json-ld.org/test-suite/tests/not-a-term-thus-a-relative-IRI"
       }

Is explained by the renaming of 'http://json-ld.org/...' URLs to 'https://json-ld.org/...', renaming which was done last July in json-ld/tests@d56f51a

Another test failure I found is this one:

  2) JSON::LD test suite Compaction compact-0045-in.jsonld: @id value uses relative IRI, not term
     Failure/Error: expect(result).to produce(expected, logger)

       Expected: {
         "@context": {
           "term": "http://example.com/terms-are-not-considered-in-id",
           "compact-iris": "http://example.com/compact-iris#",
           "property": "http://example.com/property",
           "@vocab": "http://example.org/vocab-is-not-considered-for-id"
         },
         "@id": "term",
         "property": [
           {
             "@id": "compact-iris:are-considered",
             "property": "@id supports the following values: relative, absolute, and compact IRIs"
           },
           {
             "@id": "../parent-node",
             "property": "relative IRIs get resolved against the document's base IRI"
           }
         ]
       }
       Actual  : {
         "@context": {
           "term": "http://example.com/terms-are-not-considered-in-id",
           "compact-iris": "http://example.com/compact-iris#",
           "property": "http://example.com/property",
           "@vocab": "http://example.org/vocab-is-not-considered-for-id"
         },
         "@id": "http://json-ld.org/test-suite/tests/term",
         "property": [
           {
             "@id": "compact-iris:are-considered",
             "property": "@id supports the following values: relative, absolute, and compact IRIs"
           },
           {
             "@id": "http://json-ld.org/test-suite/parent-node",
             "property": "relative IRIs get resolved against the document's base IRI"
           }
         ]
       }

Where the @id value seems to be using a full URI such as http://json-ld.org/test-suite/parent-node instead of a relative one like ../parent-node; this failed example like the next one "4) JSON::LD test suite Compaction compact-0066-in.jsonld: Relative IRIs" look like regresssion to me.

I wish I had the skills to fix the test suite before doing my performance optimization PR but this would take me a long time, so I'm going to test my PR against the "fast" tests (those which are not skipped when ENV['CI'] is set).

[gkellogg]

I’ll go through the test suite when I can.

[gkellogg]

After updates to both the implementation and the test suite, these should all pass now.