Testing adc-api: Feedback

Question

Testing adc-api: Feedback

Closed this issue 4 years ago · 6 comments

root@043fb607ab06:/work# src/ADCtest.sh https://airr-api.ireceptor.org/airr/v1 repertoire repertoire/pass*.json

Running test repertoire/pass-all.json
repertoire
ERROR: Expected 60 != 14 records

Running test repertoire/pass-and-op.json
repertoire
ERROR: Expected 10 != 0 records

Running test repertoire/pass-and-or-op.json
repertoire
ERROR: Server could not fullfil the request to https://airr-api.ireceptor.org/airr/v1/repertoire
ERROR: Error code = 500, Message = b'{\n "message": "Expected $filter to have type \"array or object\" but found \"NULL\"",\n "exception": "MongoDB\\Exception\\InvalidArgumentException",\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Exception/InvalidArgumentException.php",\n "line": 32,\n "trace": [\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Operation/Find.php",\n "line": 117,\n "function": "invalidType",\n "class": "MongoDB\\Exception\\InvalidArgumentException",\n "type": "::"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Collection.php",\n "line": 523,\n "function": "__construct",\n "class": "MongoDB\\Operation\\Find",\n "type": "->"\n },\n {\n "function": "find",\n "class": "MongoDB\\Collection",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/jenssegers/mongodb/src/Jenssegers/Mongodb/Collection.php",\n "line": 45,\n "function": "call_user_func_array"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/app/Sample.php",\n "line": 77,\n "function": "__call",\n "class": "Jenssegers\\Mongodb\\Collection",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/app/Http/Controllers/AirrApiController.php",\n "line": 49,\n "function": "airrRepertoireRequest",\n "class": "App\\Sample",\n "type": "::"\n },\n {\n "function": "airr_repertoire",\n "class": "App\\Http\\Controllers\\AirrApiController",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Controller.php",\n "line": 54,\n "function": "call_user_func_array"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php",\n "line": 45,\n "function": "callAction",\n "class": "Illuminate\\Routing\\Controller",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Route.php",\n "line": 212,\n "function": "dispatch",\n "class": "Illuminate\\Routing\\ControllerDispatcher",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Route.php",\n "line": 169,\n "function": "runController",\n "class": "Illuminate\\Routing\\Route",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 625,\n "function": "run",\n "class": "Illuminate\\Routing\\Route",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php",\n "line": 41,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Routing\\Middleware\\SubstituteBindings",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php",\n "line": 57,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Routing\\Middleware\\ThrottleRequests",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 102,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 627,\n "function": "then",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 586,\n "function": "runRouteWithinStack",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 564,\n "function": "dispatchToRoute",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 176,\n "function": "dispatch",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 30,\n "function": "Illuminate\\Foundation\\Http\\{closure}",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/fideloper/proxy/src/TrustProxies.php",\n "line": 56,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Fideloper\\Proxy\\TrustProxies",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php",\n "line": 27,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/CheckForMaintenanceMode.php",\n "line": 46,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\CheckForMaintenanceMode",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 102,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 151,\n "function": "then",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 116,\n "function": "sendRequestThroughRouter",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/public/index.php",\n "line": 53,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n }\n ]\n}'
ERROR: Query file repertoire/pass-and-or-op.json to https://airr-api.ireceptor.org/airr/v1/repertoire failed

Running test repertoire/pass-boolean-equals-op.json
repertoire
ERROR: Server could not fullfil the request to https://airr-api.ireceptor.org/airr/v1/repertoire
ERROR: Error code = 400, Message = b'{"message":"Unable to parse the filter."}'
ERROR: Query file repertoire/pass-boolean-equals-op.json to https://airr-api.ireceptor.org/airr/v1/repertoire failed

Running test repertoire/pass-boolean-not-equals-op.json
repertoire
ERROR: Server could not fullfil the request to https://airr-api.ireceptor.org/airr/v1/repertoire
ERROR: Error code = 400, Message = b'{"message":"Unable to parse the filter."}'
ERROR: Query file repertoire/pass-boolean-not-equals-op.json to https://airr-api.ireceptor.org/airr/v1/repertoire failed

Running test repertoire/pass-contains-op.json
repertoire
ERROR: Expected 10 != 0 records

Running test repertoire/pass-deep-and-op.json
repertoire
ERROR: Expected 10 != 0 records

Running test repertoire/pass-equals-op.json
repertoire
ERROR: Expected 40 != 0 records

Running test repertoire/pass-exclude-op.json
repertoire
ERROR: Expected 57 != 14 records

Running test repertoire/pass-facets1_repertoire.json
repertoire
ERROR: Expected 2 != 1 records

Running test repertoire/pass-facets2_repertoire.json
repertoire
ERROR: Expected 10 != 0 records

Running test repertoire/pass-field-organism-id.json
repertoire
WARNING: No gold expectation for pass-field-organism-id.json
INFO: Received 0 Repertoires from query
PASS: Query file repertoire/pass-field-organism-id.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-field-organism-value.json
repertoire
WARNING: No gold expectation for pass-field-organism-value.json
INFO: Received 14 Repertoires from query
PASS: Query file repertoire/pass-field-organism-value.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-field-pcr_target_locus.json
repertoire
WARNING: No gold expectation for pass-field-pcr_target_locus.json
INFO: Received 0 Repertoires from query
PASS: Query file repertoire/pass-field-pcr_target_locus.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-float-from.json
repertoire
ERROR: Expected 50 != 14 records

Running test repertoire/pass-float-size.json
repertoire
ERROR: Expected 10 != 14 records

Running test repertoire/pass-greater-than-equals-op.json
repertoire
ERROR: Expected 20 != 5 records

Running test repertoire/pass-greater-than-op.json
repertoire
ERROR: Expected 10 != 5 records

Running test repertoire/pass-in-op.json
repertoire
ERROR: Expected 3 != 0 records

Running test repertoire/pass-is-op.json
repertoire
ERROR: Expected 30 != 14 records

Running test repertoire/pass-less-than-equals-op.json
repertoire
ERROR: Expected 20 != 0 records

Running test repertoire/pass-less-than-op.json
repertoire
ERROR: Expected 10 != 0 records

Running test repertoire/pass-not-equals-op.json
repertoire
ERROR: Expected 20 != 14 records

Running test repertoire/pass-not-op.json
repertoire
ERROR: Expected 30 != 14 records

Running test repertoire/pass-number-contains-op.json
repertoire
ERROR: Expected 40 != 0 records

Running test repertoire/pass-number-equals-op.json
repertoire
ERROR: Expected 10 != 0 records

Running test repertoire/pass-number-exclude-op.json
repertoire
ERROR: Expected 40 != 14 records

Running test repertoire/pass-number-in-op.json
repertoire
ERROR: Expected 20 != 0 records

Running test repertoire/pass-number-not-equals-op.json
repertoire
ERROR: Expected 50 != 14 records

Running test repertoire/pass-or-op.json
repertoire
ERROR: Server could not fullfil the request to https://airr-api.ireceptor.org/airr/v1/repertoire
ERROR: Error code = 500, Message = b'{\n "message": "Expected $filter to have type \"array or object\" but found \"NULL\"",\n "exception": "MongoDB\\Exception\\InvalidArgumentException",\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Exception/InvalidArgumentException.php",\n "line": 32,\n "trace": [\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Operation/Find.php",\n "line": 117,\n "function": "invalidType",\n "class": "MongoDB\\Exception\\InvalidArgumentException",\n "type": "::"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/mongodb/mongodb/src/Collection.php",\n "line": 523,\n "function": "__construct",\n "class": "MongoDB\\Operation\\Find",\n "type": "->"\n },\n {\n "function": "find",\n "class": "MongoDB\\Collection",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/jenssegers/mongodb/src/Jenssegers/Mongodb/Collection.php",\n "line": 45,\n "function": "call_user_func_array"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/app/Sample.php",\n "line": 77,\n "function": "__call",\n "class": "Jenssegers\\Mongodb\\Collection",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/app/Http/Controllers/AirrApiController.php",\n "line": 49,\n "function": "airrRepertoireRequest",\n "class": "App\\Sample",\n "type": "::"\n },\n {\n "function": "airr_repertoire",\n "class": "App\\Http\\Controllers\\AirrApiController",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Controller.php",\n "line": 54,\n "function": "call_user_func_array"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php",\n "line": 45,\n "function": "callAction",\n "class": "Illuminate\\Routing\\Controller",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Route.php",\n "line": 212,\n "function": "dispatch",\n "class": "Illuminate\\Routing\\ControllerDispatcher",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Route.php",\n "line": 169,\n "function": "runController",\n "class": "Illuminate\\Routing\\Route",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 625,\n "function": "run",\n "class": "Illuminate\\Routing\\Route",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php",\n "line": 41,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Routing\\Middleware\\SubstituteBindings",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php",\n "line": 57,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Routing\\Middleware\\ThrottleRequests",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 102,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 627,\n "function": "then",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 586,\n "function": "runRouteWithinStack",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Router.php",\n "line": 564,\n "function": "dispatchToRoute",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 176,\n "function": "dispatch",\n "class": "Illuminate\\Routing\\Router",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 30,\n "function": "Illuminate\\Foundation\\Http\\{closure}",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/fideloper/proxy/src/TrustProxies.php",\n "line": 56,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Fideloper\\Proxy\\TrustProxies",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php",\n "line": 30,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php",\n "line": 27,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/CheckForMaintenanceMode.php",\n "line": 46,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 149,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Middleware\\CheckForMaintenanceMode",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php",\n "line": 53,\n "function": "Illuminate\\Pipeline\\{closure}",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php",\n "line": 102,\n "function": "Illuminate\\Routing\\{closure}",\n "class": "Illuminate\\Routing\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 151,\n "function": "then",\n "class": "Illuminate\\Pipeline\\Pipeline",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php",\n "line": 116,\n "function": "sendRequestThroughRouter",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n },\n {\n "file": "/var/www/airr-api/service-php-mongodb/public/index.php",\n "line": 53,\n "function": "handle",\n "class": "Illuminate\\Foundation\\Http\\Kernel",\n "type": "->"\n }\n ]\n}'
ERROR: Query file repertoire/pass-or-op.json to https://airr-api.ireceptor.org/airr/v1/repertoire failed

Running test repertoire/pass-query1-2_repertoire.json
repertoire
WARNING: No gold expectation for pass-query1-2_repertoire.json
INFO: Received 0 Repertoires from query
PASS: Query file repertoire/pass-query1-2_repertoire.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-query2_repertoire.json
repertoire
WARNING: No gold expectation for pass-query2_repertoire.json
INFO: Received 0 Repertoires from query
PASS: Query file repertoire/pass-query2_repertoire.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Answer 1 · 2019-10-01T20:56:55.000Z

@bcorrie @schristley

Tried running test suite against https://airr-api.ireceptor.org/airr/v1/ and obtained odd results.

Raising a flag in case this was not seen on your end before.

Answer 2 · 2019-10-01T21:56:29.000Z

@lfunderburk The tests that give an error because the number of records returned is not the expected number are likely okay because your repository doesn't hold the test data set (Brian and I talked about the test suite having a flag to not require that exact number)

ERROR: Expected 50 != 14 records

The others are presumably code bugs as all pass*.json files should be valid queries.

You should also run the fail*.json cases to verify that the server is properly returning a 400 error code (and also not causing the server to crash).

Answer 3 · 2019-10-01T22:00:09.000Z

@lfunderburk fyi, here are the results from running against the reference implementation

root@b2f9a8b226fc:/work# src/ADCtest.sh http://localhost:8080/airr/v1 repertoire repertoire/*.json
 
Running test repertoire/fail-bogus-op.json
PASS: Query file repertoire/fail-bogus-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-and-op.json
PASS: Query file repertoire/fail-content-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-contains-op.json
PASS: Query file repertoire/fail-content-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-equals-op.json
PASS: Query file repertoire/fail-content-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-exclude-op.json
PASS: Query file repertoire/fail-content-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-greater-than-equals-op.json
PASS: Query file repertoire/fail-content-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-greater-than-op.json
PASS: Query file repertoire/fail-content-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-in-op.json
PASS: Query file repertoire/fail-content-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-is-op.json
PASS: Query file repertoire/fail-content-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-less-than-equals-op.json
PASS: Query file repertoire/fail-content-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-less-than-op.json
PASS: Query file repertoire/fail-content-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-not-equals-op.json
PASS: Query file repertoire/fail-content-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-not-op.json
PASS: Query file repertoire/fail-content-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content-or-op.json
PASS: Query file repertoire/fail-content-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-and-op.json
PASS: Query file repertoire/fail-content2-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-contains-op.json
PASS: Query file repertoire/fail-content2-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-equals-op.json
PASS: Query file repertoire/fail-content2-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-exclude-op.json
PASS: Query file repertoire/fail-content2-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-greater-than-equals-op.json
PASS: Query file repertoire/fail-content2-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-greater-than-op.json
PASS: Query file repertoire/fail-content2-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-in-op.json
PASS: Query file repertoire/fail-content2-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-is-op.json
PASS: Query file repertoire/fail-content2-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-less-than-equals-op.json
PASS: Query file repertoire/fail-content2-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-less-than-op.json
PASS: Query file repertoire/fail-content2-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-not-equals-op.json
PASS: Query file repertoire/fail-content2-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-not-op.json
PASS: Query file repertoire/fail-content2-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content2-or-op.json
PASS: Query file repertoire/fail-content2-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content3-and-op.json
PASS: Query file repertoire/fail-content3-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content3-or-op.json
PASS: Query file repertoire/fail-content3-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content4-and-op.json
PASS: Query file repertoire/fail-content4-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-content4-or-op.json
PASS: Query file repertoire/fail-content4-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-contains-op.json
PASS: Query file repertoire/fail-field-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-equals-op.json
PASS: Query file repertoire/fail-field-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-exclude-op.json
PASS: Query file repertoire/fail-field-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-greater-than-equals-op.json
PASS: Query file repertoire/fail-field-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-greater-than-op.json
PASS: Query file repertoire/fail-field-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-in-op.json
PASS: Query file repertoire/fail-field-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-is-op.json
PASS: Query file repertoire/fail-field-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-less-than-equals-op.json
PASS: Query file repertoire/fail-field-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-less-than-op.json
PASS: Query file repertoire/fail-field-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-not-equals-op.json
PASS: Query file repertoire/fail-field-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field-not-op.json
PASS: Query file repertoire/fail-field-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-contains-op.json
PASS: Query file repertoire/fail-field2-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-equals-op.json
PASS: Query file repertoire/fail-field2-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-exclude-op.json
PASS: Query file repertoire/fail-field2-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-greater-than-equals-op.json
PASS: Query file repertoire/fail-field2-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-greater-than-op.json
PASS: Query file repertoire/fail-field2-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-in-op.json
PASS: Query file repertoire/fail-field2-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-is-op.json
PASS: Query file repertoire/fail-field2-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-less-than-equals-op.json
PASS: Query file repertoire/fail-field2-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-less-than-op.json
PASS: Query file repertoire/fail-field2-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-not-equals-op.json
PASS: Query file repertoire/fail-field2-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-field2-not-op.json
PASS: Query file repertoire/fail-field2-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-format.json
PASS: Query file repertoire/fail-format.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-missing-op.json
PASS: Query file repertoire/fail-missing-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-negative-from.json
PASS: Query file repertoire/fail-negative-from.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-negative-size.json
PASS: Query file repertoire/fail-negative-size.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-number-equals-op.json
PASS: Query file repertoire/fail-number-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-number-not-equals-op.json
PASS: Query file repertoire/fail-number-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-query-json.json
WARNING: JSON Decode error detected in repertoire/fail-query-json.json: Expecting ',' delimiter: line 8 column 18 (char 120)
PASS: Query file repertoire/fail-query-json.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-query-structure.json
PASS: Query file repertoire/fail-query-structure.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-string-equals-op.json
PASS: Query file repertoire/fail-string-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-string-from.json
PASS: Query file repertoire/fail-string-from.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-string-greater-than-equals-op.json
PASS: Query file repertoire/fail-string-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-string-greater-than-op.json
PASS: Query file repertoire/fail-string-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-string-less-than-equals-op.json
PASS: Query file repertoire/fail-string-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-string-less-than-op.json
PASS: Query file repertoire/fail-string-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-string-size.json
PASS: Query file repertoire/fail-string-size.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value-contains-op.json
PASS: Query file repertoire/fail-value-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value-equals-op.json
PASS: Query file repertoire/fail-value-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value-exclude-op.json
PASS: Query file repertoire/fail-value-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value-greater-than-equals-op.json
PASS: Query file repertoire/fail-value-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value-greater-than-op.json
PASS: Query file repertoire/fail-value-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value-in-op.json
PASS: Query file repertoire/fail-value-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value-less-than-equals-op.json
PASS: Query file repertoire/fail-value-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value-less-than-op.json
PASS: Query file repertoire/fail-value-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value-not-equals-op.json
PASS: Query file repertoire/fail-value-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value2-equals-op.json
PASS: Query file repertoire/fail-value2-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value2-exclude-op.json
PASS: Query file repertoire/fail-value2-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/fail-value2-in-op.json
PASS: Query file repertoire/fail-value2-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-all.json
INFO: Received 60 Repertoires from query
PASS: Query file repertoire/pass-all.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-and-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-and-or-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-and-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-boolean-equals-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-boolean-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-boolean-not-equals-op.json
INFO: Received 40 Repertoires from query
PASS: Query file repertoire/pass-boolean-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-contains-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-deep-and-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-deep-and-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-equals-op.json
INFO: Received 40 Repertoires from query
PASS: Query file repertoire/pass-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-exclude-op.json
INFO: Received 57 Repertoires from query
PASS: Query file repertoire/pass-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-facets1_repertoire.json
INFO: Received 2 Facets from query
PASS: Query file repertoire/pass-facets1_repertoire.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-facets2_repertoire.json
INFO: Received 10 Facets from query
PASS: Query file repertoire/pass-facets2_repertoire.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-field-organism-id.json
WARNING: No gold expectation for pass-field-organism-id.json
INFO: Received 60 Repertoires from query
PASS: Query file repertoire/pass-field-organism-id.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-field-organism-value.json
WARNING: No gold expectation for pass-field-organism-value.json
INFO: Received 60 Repertoires from query
PASS: Query file repertoire/pass-field-organism-value.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-field-pcr_target_locus.json
WARNING: No gold expectation for pass-field-pcr_target_locus.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-field-pcr_target_locus.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-float-from.json
INFO: Received 50 Repertoires from query
PASS: Query file repertoire/pass-float-from.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-float-size.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-float-size.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-greater-than-equals-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-greater-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-greater-than-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-greater-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-in-op.json
INFO: Received 3 Repertoires from query
PASS: Query file repertoire/pass-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-is-op.json
INFO: Received 30 Repertoires from query
PASS: Query file repertoire/pass-is-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-less-than-equals-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-less-than-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-less-than-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-less-than-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-not-equals-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-not-op.json
INFO: Received 30 Repertoires from query
PASS: Query file repertoire/pass-not-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-number-contains-op.json
INFO: Received 40 Repertoires from query
PASS: Query file repertoire/pass-number-contains-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-number-equals-op.json
INFO: Received 10 Repertoires from query
PASS: Query file repertoire/pass-number-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-number-exclude-op.json
INFO: Received 40 Repertoires from query
PASS: Query file repertoire/pass-number-exclude-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-number-in-op.json
INFO: Received 20 Repertoires from query
PASS: Query file repertoire/pass-number-in-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-number-not-equals-op.json
INFO: Received 50 Repertoires from query
PASS: Query file repertoire/pass-number-not-equals-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-or-op.json
INFO: Received 30 Repertoires from query
PASS: Query file repertoire/pass-or-op.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-query1-2_repertoire.json
WARNING: No gold expectation for pass-query1-2_repertoire.json
INFO: Received 5 Repertoires from query
PASS: Query file repertoire/pass-query1-2_repertoire.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
Running test repertoire/pass-query2_repertoire.json
WARNING: No gold expectation for pass-query2_repertoire.json
INFO: Received 0 Repertoires from query
PASS: Query file repertoire/pass-query2_repertoire.json to http://vdj-dev.tacc.utexas.edu:8080/airr/v1/repertoire OK
 
SUMMARY: All tests passed!!!

Answer 4 · 2019-10-02T19:49:39.000Z

@lfunderburk @schristley I added a -g flag to disable the gold testing (airr-community/adc-api-tests@7c22d34)... It will now run all the tests and ignore the gold standard results test if you give it the -g flag. Necessary for testing against a repository that doesn't have just the gold standard data set loaded...

Answer 5 · 2019-10-02T19:53:42.000Z

Run against the iReceptor ADC implementation we now get most queries passing (still 4 failing).

Running test repertoire/pass-query1-2_repertoire.json
INFO: Received 0 Repertoires from query
PASS: Query file repertoire/pass-query1-2_repertoire.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

Running test repertoire/pass-query2_repertoire.json
INFO: Received 0 Repertoires from query
PASS: Query file repertoire/pass-query2_repertoire.json to https://airr-api.ireceptor.org/airr/v1/repertoire OK

SUMMARY: 4 of 32 tests failed

Answer 6 · 2020-02-14T17:15:13.000Z

Sounds like the test suite is fine, and any iReceptor ADC bugs you will handle in your own issue tracker, so closing this.