18F/open-data-maker

Search with ZIP+4 results in Internal Server Error (ElasticSearch BadRequest)

dnesting opened this issue · 1 comments

The web site seems to work around this problem, but the API should probably not be generating this type of error in response to a ZIP+4 search. We've seen this error appear on the order of a dozen or so times today.

Elasticsearch::Transport::Transport::Errors::BadRequest: [400] {"error":"SearchPhaseExecutionException[Failed to execute phase [query_fetch], all shards failed; shardFailures {[7WAeJDHBSg-crOH2AwOEMg][production-school-data][0]: SearchParseException[[production-school-data][0]: from[0],size[20]: Parse Failure [Failed to parse source [{\"from\":0,\"size\":20,\"query\":{\"filtered\":{\"query\":{\"bool\":{\"must\":[{\"common\":{\"school.name\":{\"query\":\"Valdosta State University\",\"cutoff_frequency\":0.001,\"low_freq_operator\":\"and\"}}},{\"match\":{\"school.operating\":{\"query\":\"1\"}}},{\"match\":{\"school.state\":{\"query\":\"GA\"}}},{\"match\":{\"school.region_id\":{\"query\":\"5\"}}},{\"match\":{\"school.zip\":{\"query\":\"31698-0055\"}}},{\"match\":{\"school.ownership\":{\"query\":\"1\"}}},{\"match\":{\"school.degrees_awarded.predominant\":{\"query\":\"3\"}}}]}},\"filter\":{\"and\":[{\"or\":[{\"range\":{\"2013.academics.program.bachelors.physical_science\":{\"gte\":1}}}]},{\"or\":[{\"range\":{\"2013.student.size\":{\"gte\":2000,\"lte\":15000}}}]}]}}},\"fields\":[\"id\",\"school.name\",\"school.city\",\"school.state\",\"2013.student.size\",\"school.ownership\",\"school.degrees_awarded.predominant\",\"2013.cost.avg_net_price.overall\",\"2013.completion.rate_suppressed.overall\",\"2011.earnings.10_yrs_after_entry.median\",\"2011.earnings.6_yrs_after_entry.percent_greater_than_25000\",\"school.under_investigation\"],\"_source\":false,\"sort\":[{\"2011.earnings.6_yrs_after_entry.percent_greater_than_25000\":{\"order\":\"desc\"}}]}]]];
nested: NumberFormatException[For input string: \"31698-0055\"]; }]","status":400}

…t-1.0.12/lib/elasticsearch/transport/transport/base.rb: 135:in `__raise_transport_error'
…t-1.0.12/lib/elasticsearch/transport/transport/base.rb: 227:in `perform_request'
…/lib/elasticsearch/transport/transport/http/faraday.rb:  20:in `perform_request'
…transport-1.0.12/lib/elasticsearch/transport/client.rb: 119:in `perform_request'
…rch-api-1.0.12/lib/elasticsearch/api/actions/search.rb: 159:in `search'
/home/vcap/app/lib/data_magic.rb:  82:in `search'

Since our zip code data doesn't have ZIP+4 right now, would prolly make sense for the zip code library to just strip the +4 part of the code

Maybe address as part of error handling in #213