[Bug]

[stephenrs]

Describe the bug

I'm not sure if this is a bug in the python client or the API itself.

Some time series fetches that emanate from the python client appear to trigger an erroneous TwelveDataError which indicates that data cannot be found when data actually does exist. The error looks like this:

{"code":404,"message":"Data not found","status":"error","meta":{"symbol":"SET100:SET","interval":"1day","exchange":""}}

To Reproduce

The following URL is generated by the client and fails with the error noted above:

https://api.twelvedata.com/time_series?symbol=SET100:SET&interval=1day&outputsize=5000&end_date=2021-04-26 00:00:00+00:00&dp=5&timezone=UTC&order=asc&prepost=false&format=JSON&apikey=[APIKEY]

The same URL also fails when pasted into a browser or using cURL.

Note: As a quick way to find out what the client was trying to fetch, I hacked the "execute" method of the TimeSeriesEndpoint class as follows:

def execute(self, format="JSON", debug=True): # enable debugging mode

At the bottom of the method, I did this to print the URL that it was fetching:

        # @hack
        # if debug:
        #     return build_url(self.ctx.base_url, endpoint, params)
        if debug:
            print(build_url(self.ctx.base_url, endpoint, params))
        # end hack

Strangely, however, the same range of data that is failing to get fetched by the client can be successfully fetched with this URL:

https://api.twelvedata.com/time_series?symbol=SET:SET&outputsize=5000&interval=1day&end_date=2021-04-26%2000:00:00&timezone=UTC&order=asc&apikey=[APIKEY]

Since the only difference that I can see between the failing URL and the successful URL appears to be the presence of the parameters 'dp', 'prepost', and 'format' in the client's version of the URL, my guess would be that one of those parameters is causing problems, but I haven't tested this.

Expected behavior

The existing data should be returned without error, because it exists and can be fetched outside of the context of the python client.

Looking forward to your thoughts, this problem is actually pretty scary.

Additional notes

The failing fetch noted above occurs inside a loop that successfully executes several similar fetches before failing as above, in order to fetch a longer range of data history in "outputsize" chunks. So it would appear that there is nothing wrong in my code, but let me know if I'm missing something. My first thought was that there might be something weird about the end_date itself, so maybe that's a clue.

[sgacode]

There is no data for the specified period for the symbol SET100:SET.
In second url the symbol is SET:SET at which data is exists.
For same symbol, the result should be the same when requested manually and through the client.

[stephenrs]

@sgacode Thanks for the update, and I eventually realized that was the problem with the URLs and that I was being thrown off because the history provided by the API for SET100 is much shorter than I was expecting. SET100 actually started in 2005, but the TD data set begins in April 2021.

So, do you guys have a mechanism or a policy regarding missing history for instruments in your data set?

In other words, is your data supplied "take it or leave it, as is", or do you handle requests to fill in missing history?

[midasSSS]

SET100 historical data can not be extended. We typically provide the complete historical data for instruments; however, it might be truncated for some indices.

[stephenrs]

@midasSSS Ok, thanks for the info.