Waultics/coinmarketcap-history

Error when downloading crypto information.

Orosenthal opened this issue · 10 comments

Orosenthal commented 
ValueError                                Traceback (most recent call last)
<ipython-input-26-c27355b424d6> in <module>
----> 1 coinmarketcap.getDataFor('bitcoin', '2018-09-02','2019-10-01')

~\Anaconda3\lib\site-packages\cmc\coinmarketcap.py in getDataFor(cryptocurrencies, start_date, end_date, fields, asynchro, DOWNLOAD_DIR)
     58         header, rows = utils.extract_data(html)
     59         temp_df = pd.DataFrame(data = rows, columns = header)
---> 60         df = utils.processDataFrame(temp_df)
     61 
     62         if save:

~\Anaconda3\lib\site-packages\cmc\utils\utils.py in processDataFrame(df)
    135 
    136     cols = list(df.columns.values)
--> 137     cols.remove('Date')
    138     df.loc[:,'Date'] = pd.to_datetime(df.Date)
    139     for col in cols: df.loc[:,col] = df[col].apply(lambda x: float(x) if x.replace('.','').isdigit() else float(0)) # check columns to see if value is a number.

ValueError: list.remove(x): x not in list
synchronizing commented

Could you please show us the code you are using to receive this error?

kaythxbye commented

I get the same error by just running coinmarketcap bitcoin 2018-01-01 2019-11-12

synchronizing commented

I get the same error by just running coinmarketcap bitcoin 2018-01-01 2019-11-12

Seems like the CoinMarketCap might have updated some of the endpoints cmc uses. Will explore further, thanks for letting us know.

kaythxbye commented

As an example, it looks like this might be the new URL: curl -s -X GET "https://web-api.coinmarketcap.com/v1/cryptocurrency/quotes/historical?convert=USD&format=chart_crypto_details&id=3014&interval=1d&time_end=1573666814&time_start=1533167965"

There I get results in the form of:

{
    "status": {
        "timestamp": "2019-11-13T17:41:30.588Z",
        "error_code": 0,
        "error_message": null,
        "elapsed": 189,
        "credit_count": 5,
        "notice": null
    },
    "data": {
        "2018-08-05T23:59:27.000Z": {
            "USD": [
                0.10308405,
                638.4882202148438,
                5799945.857826003
            ]
        },
        "2018-08-06T23:59:23.000Z": {
            "USD": [
                0.13136461,
                1602.6744384765625,
                7391130.311911408
            ]
        },
synchronizing commented

@kaythxbye This is new -- CoinMarketCap didn't have an API endpoint when this tool was written. Actually, funny enough, this tool used to extract the data that was present on the actual HTML website. We'll update it in the next few days and publish it. Thank you for letting us know!

  • 👍2
kaythxbye commented

Another even better finding is this, they call that endpoint when generating the table you scrape(d):

https://web-api.coinmarketcap.com/v1/cryptocurrency/ohlcv/historical?convert=USD&slug=ethereum&time_end=1573667554&time_start=1483228800

Now I'll stop spamming and wait for you guys ;-)

  • 🎉2
synchronizing commented

No worries at all, helpful spamming is never bad. @Alescontrela has been the one maintaining this repo more actively, and I've already spoken with him regarding this bug. The endpoints you've linked will make his life much easier if they aren't officially documented, which it doesn't seem like they are so.

  • 👍1
TroyHGP commented

I am still receiving ValueError: list.remove(x): x not in list when trying to pull data. Any fix for this issue?

misgeatgit commented

Any updates on this?

Harlequin314159 commented

Same issue. ValueError: list.remove(x): x not in list

  • 👍1