datafeed stop automatically after update to cryptofeed from 2.2.2 to 2.2.3
leonwang908 opened this issue · 3 comments
Here is my code and logs.
Is there any way to automatically restart websocket connection with infinite times?
from cryptofeed import FeedHandler
from cryptofeed.defines import CANDLES, BID, ASK, BLOCKCHAIN, FUNDING, GEMINI, L2_BOOK, L3_BOOK, LIQUIDATIONS, OPEN_INTEREST, PERPETUAL, TICKER, TRADES, INDEX
from cryptofeed.exchanges import Bitflyer
from cryptofeed.symbols import Symbol
from datetime import datetime
import asyncio
import pandas as pd
async def trade(t, receipt_timestamp):
date = datetime.utcfromtimestamp(receipt_timestamp).strftime('%Y-%m-%d')
with open('{}/{}_Trades_{}.csv'.format('bf', t.symbol, date), 'a') as f:
f.write(f"{t.timestamp},{t.side},{t.amount},{t.price},{receipt_timestamp}\n")
async def book(book, receipt_timestamp):
date = datetime.utcfromtimestamp(receipt_timestamp).strftime('%Y-%m-%d')
asks_df = pd.DataFrame.from_dict({'price': book.book.asks.to_dict().keys(), 'size' : book.book.asks.to_dict().values()})
bids_df = pd.DataFrame.from_dict({'price': book.book.bids.to_dict().keys(), 'size' : book.book.bids.to_dict().values()})
asks_df.insert(0, 'ts', receipt_timestamp)
bids_df.insert(0, 'ts', receipt_timestamp)
if type(asks_df) != None:
asks_df.to_csv('{}/{}_L2ASK_{}.csv'.format('bf', book.symbol, date), index=False, mode='a', header=False)
if type(bids_df) != None:
bids_df.to_csv('{}/{}_L2BID_{}.csv'.format('bf', book.symbol, date), index=False, mode='a', header=False)
await asyncio.sleep(1)
def main():
config = {'log': {'filename': 'demo.log', 'level': 'DEBUG', 'disabled': False}}
f = FeedHandler(config=config)
f.add_feed(Bitflyer(symbols=['BTC-JPY-FX'], channels=[TRADES, L2_BOOK], callbacks={L2_BOOK: book, TRADES: trade}))
f.run()
if __name__ == '__main__':
main()```
> 2022-06-04 02:07:38,460 : INFO : FH: System Exit received - shutting down
2022-06-04 02:07:38,461 : INFO : FH: shutdown connections handlers in feeds
2022-06-04 02:07:38,461 : INFO : FH: create the tasks to properly shutdown the backends (to flush the local cache)
2022-06-04 02:07:38,461 : INFO : FH: wait 1 backend tasks until termination
2022-06-04 02:07:38,461 : INFO : BITFLYER: feed shutdown starting...
2022-06-04 02:07:38,470 : INFO : BITFLYER.ws.1: closed connection 'WebSocketClientProtocol'
2022-06-04 02:07:38,471 : INFO : BITFLYER: feed shutdown completed
2022-06-04 02:07:38,471 : INFO : FH: stop the AsyncIO loop
2022-06-04 02:07:38,471 : INFO : FH: run the AsyncIO event loop one last time
2022-06-04 02:07:38,471 : INFO : FH: cancel the 2 pending tasks
2022-06-04 02:07:38,471 : INFO : FH: run the pending tasks until complete
2022-06-04 02:07:38,471 : INFO : FH: shutdown asynchronous generators
2022-06-04 02:07:38,471 : INFO : FH: close the AsyncIO loop
2022-06-04 02:07:38,472 : INFO : FH: leaving run()
it will restart forever, but your logs indicate something shut down the application that was running cryptofeed, and it was a clean shutdown
I have the same problem
2022-06-23 03:14:06,504 : INFO : Config: use dict containing the following main keys: log
2022-06-23 03:14:06,505 : DEBUG : BINANCE: reading symbol information from https://api.binance.com/api/v3/exchangeInfo
2022-06-23 03:14:06,505 : DEBUG : HTTPSync: requesting data from https://api.binance.com/api/v3/exchangeInfo
2022-06-23 03:14:06,647 : DEBUG : CRYPTO.COM: reading symbol information from https://api.crypto.com/v2/public/get-instruments
2022-06-23 03:14:06,647 : DEBUG : HTTPSync: requesting data from https://api.crypto.com/v2/public/get-instruments
2022-06-23 03:14:06,722 : DEBUG : BINANCE.ws.2: connecting to wss://stream.binance.com:9443/stream?streams=btcusdt@depth@100ms/ethusdt@depth@100ms/bchusdt@depth@100ms
2022-06-23 03:14:06,733 : DEBUG : CRYPTO.COM.ws.3: connecting to wss://stream.crypto.com/v2/market
2022-06-23 03:14:07,037 : DEBUG : BINANCE.http.0: create HTTP session
2022-06-23 03:14:07,038 : DEBUG : BINANCE.http.0: requesting data from https://api.binance.com/api/v3/depth?symbol=BCHUSDT&limit=20
2022-06-23 03:14:07,065 : DEBUG : BINANCE.http.0: requesting data from https://api.binance.com/api/v3/depth?symbol=BTCUSDT&limit=20
2022-06-23 03:14:07,071 : DEBUG : BINANCE.http.0: requesting data from https://api.binance.com/api/v3/depth?symbol=ETHUSDT&limit=20
2022-06-23 03:42:15,280 : INFO : FH: System Exit received - shutting down
2022-06-23 03:42:15,283 : INFO : FH: shutdown connections handlers in feeds
2022-06-23 03:42:15,283 : INFO : FH: create the tasks to properly shutdown the backends (to flush the local cache)
2022-06-23 03:42:15,283 : INFO : FH: wait 2 backend tasks until termination
2022-06-23 03:42:15,284 : INFO : BINANCE: feed shutdown starting...
2022-06-23 03:42:15,284 : INFO : CRYPTO.COM: feed shutdown starting...
2022-06-23 03:42:15,284 : INFO : BINANCE.http.0: closed connection 'ClientSession'
2022-06-23 03:42:15,293 : INFO : CRYPTO.COM.ws.3: terminate the connection handler because not running
2022-06-23 03:42:15,293 : INFO : CRYPTO.COM.ws.3: closed connection 'WebSocketClientProtocol'
2022-06-23 03:42:15,293 : INFO : CRYPTO.COM: feed shutdown completed
2022-06-23 03:42:15,317 : INFO : BINANCE.ws.2: terminate the connection handler because not running
2022-06-23 03:42:23,322 : INFO : BINANCE.ws.2: closed connection 'WebSocketClientProtocol'
2022-06-23 03:42:23,323 : INFO : BINANCE: feed shutdown completed
2022-06-23 03:42:23,323 : INFO : FH: stop the AsyncIO loop
2022-06-23 03:42:23,323 : INFO : FH: run the AsyncIO event loop one last time
2022-06-23 03:42:23,323 : INFO : FH: cancel the 3 pending tasks
2022-06-23 03:42:23,323 : INFO : FH: run the pending tasks until complete
2022-06-23 03:42:23,323 : INFO : FH: shutdown asynchronous generators
2022-06-23 03:42:23,323 : INFO : FH: close the AsyncIO loop
2022-06-23 03:42:23,325 : INFO : FH: leaving run()
going to close this - all I see here is that you or something else on your system cleanly shut down your program.