Update cache fails when metar contains a special character

[JanC]

Hi again :)

I noticed an exception when the metar contains the $ sign:

Updating cache : {'data': {'Remarks': 'RMK AO2 SLP247 T00611089 $', 'Runway-Vis-List': [], 'Station': 'KNYL', 'Time': '241057Z', 'Cloud-List': [], 'Wind-Direction': '350', 'Wind-Speed': '03', 'Wind-Gust': '', 'Wind-Variable-Dir': [], 'Altimeter': '3026', 'Visibility': '10', 'Other-List': [], 'Temperature': '06', 'Dewpoint': 'M09', 'Units': {'Wind-Speed': 'kt', 'Visibility': 'sm', 'Altitude': 'ft', 'Temperature': 'C', 'Altimeter': 'inHg'}, 'Flight-Rules': 'VFR', 'Remarks-Info': {'Temp-Decimal': '6.1', 'Dew-Decimal': '-8.9'}, 'Raw-Report': 'KNYL 241057Z AUTO 35003KT 10SM CLR 06/M09 A3026 RMK AO2 SLP247 T00611089 $'}, 'translate': {'Visibility': '10sm (16.1km)', 'Altimeter': '30.26inHg (1025hPa)', 'Clouds': 'Sky clear', 'Other': '', 'Wind': 'N-350 at 03kt', 'Temperature': '06C (43F)', 'Dewpoint': '-09C (16F)', 'Remarks': {'AO2': 'Automated with precipitation sensor', 'SLP247': 'Sea level pressure: 1024.7 hPa', 'T00611089': 'Temperature 6.1°C and dewpoint -8.9°C', '$': 'ASOS requires maintenance'}}, 'summary': 'Winds N-350 at 03kt, Vis 10sm, Temp 06C, Dew -09C, Alt 30.26inHg, Sky clear', 'speech': 'Winds three five zero at 3kt. Visibility one zero miles. Temperature six degrees Celsius. Dew point minus nine degrees Celsius. Altimeter three zero point two six. Sky clear'}
[2018-02-24 12:04:14,995] ERROR in app: Exception on /api/metar/KNYL [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 480, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/Volumes/MySamsung/xworkspace/aviation/AVWX-API/avwx_api/api.py", line 101, in get
    return self.get_report(rtype, station)
  File "/Volumes/MySamsung/xworkspace/aviation/AVWX-API/avwx_api/api.py", line 91, in get_report
    resp = handle_report(rtype, station, options, nofail)
  File "/Volumes/MySamsung/xworkspace/aviation/AVWX-API/avwx_api/handling.py", line 114, in handle_report
    data = CACHE.get(rtype, station) or new_report(rtype, station, report)
  File "/Volumes/MySamsung/xworkspace/aviation/AVWX-API/avwx_api/handling.py", line 78, in new_report
    CACHE.update(rtype, data)
  File "/Volumes/MySamsung/xworkspace/aviation/AVWX-API/avwx_api/cache.py", line 48, in update
    self.tables[rtype].update({'_id': data['data']['Station']}, data, upsert=True)
  File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 2968, in update
    write_concern, collation=collation)
  File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 835, in _update_retryable
    _update, session)
  File "/usr/local/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1099, in _retryable_write
    return self._retry_with_session(retryable, func, s, None)
  File "/usr/local/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1076, in _retry_with_session
    return func(session, sock_info, retryable)
  File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 831, in _update
    retryable_write=retryable_write)
  File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 797, in _update
    _check_write_command_response(result)
  File "/usr/local/lib/python3.6/site-packages/pymongo/helpers.py", line 208, in _check_write_command_response
    _raise_last_write_error(write_errors)
  File "/usr/local/lib/python3.6/site-packages/pymongo/helpers.py", line 190, in _raise_last_write_error
    raise WriteError(error.get("errmsg"), error.get("code"), error)
pymongo.errors.WriteError: The dollar ($) prefixed field '$' in 'translate.Remarks.$' is not valid for storage.

I believe the the data should be encoded before inserting them at
https://github.com/flyinactor91/AVWX-API/blob/bed76e4960d46420d2fdcfe424fa814182c2760c/avwx_api/cache.py#L48

[JanC]

it is actually because of the key in the Remarks being $

    "Remarks": {
      "AO2": "Automated with precipitation sensor",
      "SLP247": "Sea level pressure: 1024.7 hPa",
      "T00611089": "Temperature 6.1°C and dewpoint -8.9°C",
      "$": "ASOS requires maintenance"
    }