BlockchainCommons/spotbit

SpotBit's API is inconsistent and makes it hard for application developers to access.

wolfmcnally opened this issue · 2 comments

SpotBit's API has numerous inconsistencies that make it difficult to develop with.

Inconsistent field names

  • With spaces between words: cached exchanges
  • Using "camelCase": keepWeeks
  • Using "snake_case": currency_pair
  • Multiple words all lower case: timestamp, datetime
  • Including punctuation: updated settings?
  • Inconsistent naming of the same data type: use of vol and volume.

Fields with duplicate information in different fields

  • timestamp includes the time in milliseconds since the Unix epoch (1600804380000), while the same record includes datetime which is a textual encoding of the same time (2020-09-22 12:53:00).

Nonstandard ways of encoding dates

  • "Milliseconds since the Unix epoch" is one method used in the timestamp field, and is needlessly verbose since nothing about this application requires timing down to the millisecond: Unix dates are normally expressed in seconds since the epoch.
  • The textual format of the datetime format is not in a standard format such as ISO-8601.

Inconsistent reporting of errors

SpotBit returns parseable JSON for successful calls, while for unsuccessful calls it returns bare strings like no data, or empty arrays, or for non-200 status codes, nothing at all in the response body.

All bodies of HTTP responses, for success or failure, should be parseable JSON. The structure of the JSON should be determined by the HTTP status code: if the server returns 200 (success) then the JSON should be the structure expected for the call's result, but if it is any error (400, 500, etc.) it should be a uniform error reporting structure that the client can use to diagnose and report the error to users.

Some standards for JSON APIs (see below) unify these and provide a single overall structure to which all responses conform.

Inconsistent structuring of the same sorts of data

The calls that return a "current average price" structure a candle as a JSON record with titles for each field:

{
	"close": 10314.06,
	"currency_pair": "BTC-USD",
	"datetime": "2020-09-13 14:31:00",
	"high": 10315.65,
	"id": 122983,
	"low": 10314.06,
	"open": 10315.65,
	"timestamp": 1600007460000,
	"vol": 3.53308926
}

But the calls that return historical data structure the same candle as a list of colums:

{
	"columns": ["id", "timestamp", "datetime", "currency_pair", "open", "high", "low", "close", "vol"],
	"data": [
		[718, 1600804380000, "2020-09-22 12:53:00", "BTC-USD", 10479.3, 10483.3, 10479.2, 10483.3, 17.4109874],
		[719, 1600804440000, "2020-09-22 12:54:00", "BTC-USD", 10483.3, 10483.4, 10483.3, 10483.4, 0.098285],
		[720, 1600804500000, "2020-09-22 12:55:00", "BTC-USD", 10483.4, 10483.4, 10483.4, 10483.4, 0.0]
	]
}

This means that two separate methods of decoding candles must be supported. The second one above was obviously designed to reduce the overhead of the field names in every candle, but at the expense of requiring that every candle support the same fields. If space is truly a concern, then the overhead could be largely reduced by:

  1. Allowing the client to specify which fields they want returned. If all you're interested in is the closing price, then you don't need any of the other data.
  2. Don't include data that is known from the call, for example the currency_pair field is known by the caller and doesn't need to be returned anywhere in the result.
  3. Don't include data that is useless to the caller, like the id field.
  4. Don't include redundant information, like datetime and timestamp.
  5. Don't include excess fractional information, e.g., for volume don't send 17.4109874, just send 17.41.
  6. Use terse field names, so all calls that return candles would use a specific format such as:
{"cd":"20220122T050802Z","cl":10314.06,"od":"20220122T050812Z","op":10315.65,"hi":10315.65,"lo":10314.06,"vo":3.53}

This could be made even more compact by encoding the entire candle as a string, using the fact that the fields always occur in a specific order, e.g.: closeDate, close, openDate, open, high, low, volume:

"20220122T050802Z,10314.06,20220122T050812Z,10315.65,10315.65,10314.06,3.53"

If fields must be omitted as unknown or unrequested, then empty fields can be supported, e.g., a candle that omits openDate, high and low:

"20220122T050802Z,10314.06,,10315.65,,,3.53"

If only the closing date and price are returned, then it could be:

"20220122T050802Z,10314.06"

Some overall methodology should be adopted for the API

JSON itself is a freeform data format, but there are standards and practices for JSON REST APIs, and one such as JSend should be adopted. See also this discussion on StackExchange.

@wolfmcnally If you'd like to assign this, I'm available to work on this clean-up.

@nochiel I don't have assign permission on this repo, but please contact @ChristopherA about it!