Finance crawler for reading financial data from public sites
- python3
- pipenv
- chromedriver
-
Initialize pipenv virtual env
pipenv --python 3.9
-
Install dependencies
pipenv install --dev
-
Install Chrome Driver
- Download from
http://chromedriver.chromium.org/
- Add
chromedriver
to yourPATH
- Download from
pipenv install -e git+https://github.com/jjmrocha/fd-crawler.git#egg=fd-crawler
The following program:
from fdc.indices import sp500
from fdc.utils import (date_util, browser, throttle, proxy_list)
if __name__ == '__main__':
# Optionally, set the proxy or proxies
proxy_list.add_proxy(host='localhost', port=1234)
# Optionally, Set the delay between requests for each host
throttle.set_throttle(delay_in_millis=3000)
with browser.Browser() as browser:
for ticket in sp500.tickets(browser):
print(f'- {ticket.code} - {ticket.name} ({ticket.sector})')
# Stats
stats = ticket.stats()
print(f' > market_cap: {stats.market_cap} ebitda: {stats.ebitda}')
# Financials
financials = ticket.financials()
print(f' > lq_stockholder_equity: {financials.balance_sheet_lq.stockholder_equity} '
f'ttm_revenue: {financials.income_statement_ttm.revenue} '
f'ttm_dividends_paid: {financials.cash_flow_statement_ttm.dividends_paid}')
# Prices
prices = list(ticket.prices())
if len(prices):
price_52w = prices[0]
price_last = prices[-1]
print(f' > {date_util.to_iso8601_date(price_52w.date)}: {price_52w.price} '
f'- {date_util.to_iso8601_date(price_last.date)}: {price_last.price}')
print()
Will produce:
- MMM - 3M Company (Industrials)
> market_cap: 100447789056 ebitda: 8686999552
> lq_stockholder_equity: 11880000000 ttm_revenue: 31712000000 ttm_dividends_paid: -3368000000
> 2019-11-26: 169.22999572753906 - 2020-11-23: 174.13999938964844
- ABT - Abbott Laboratories (Health Care)
> market_cap: 193665761280 ebitda: 7823000064
> lq_stockholder_equity: 31386000000 ttm_revenue: 32221000000 ttm_dividends_paid: -2487000000
> 2019-11-26: 85.41999816894531 - 2020-11-23: 109.2699966430664
- ABBV - AbbVie Inc. (Health Care)
> market_cap: 180330332160 ebitda: 18371999744
> lq_stockholder_equity: 15270000000 ttm_revenue: 40650000000 ttm_dividends_paid: -7210000000
> 2019-11-26: 87.72000122070312 - 2020-11-23: 102.18000030517578
...
Any contributions made under this project will be governed by the MIT License.