Morningstar analysis URLs no longer work.
Closed this issue · 6 comments
The direct URLs no longer work. Seems to be a refactoring of their website.
However I found out that local library cards get free premium access, so I should be able to reuse that ID to make premium requests.
The biggest drawback is that I need to send a dummy request for the ticker symbol in order to parse out the unique ticker ID that the API requires. I have not found a more robust way to do this.
This also is subject to breaking if the url ever changes.
Background on this approach: https://stackoverflow.com/a/57449982/1366973
Example code:
import re
import urllib2
ticker = 'goog'
# This is a pretty fragile way to extract the Morningstar Ticker ID
find_ticker_id_regex = re.compile(r'byId:{"(.*?)"')
find_ticker_url = 'https://www.morningstar.com/stocks/xnas/{}/price-fair-value'.format(ticker)
req = urllib2.Request(find_ticker_url)
resp = urllib2.urlopen(req)
data = resp.read()
ticker_id = find_ticker_id_regex.findall(data)[0]
# Fair value (privlidged)
fair_value_url = 'https://api-global.morningstar.com/sal-service/v1/stock/morningstarTake/v3/{}/analysisData?locale=en&access_token=gIZECmgrLiGStrr5OGQ2W85RvQl0&clientId=MIRC_SALStockModule'.format(ticker_id)
# Quote (privlidged)
quote_url = 'https://api-global.morningstar.com/sal-service/v1/stock/realTime/v3/{}/data?secExchangeList=&random=0.5332403090320754&access_token=gIZECmgrLiGStrr5OGQ2W85RvQl0&clientId=MIRC_SALStockModule'.format(ticker_id)
# Name, PE, Forward PE, star rating (privlidged)
url = 'https://api-global.morningstar.com/sal-service/v1/stock/header/v2/data/{}/securityInfo?access_token=gIZECmgrLiGStrr5OGQ2W85RvQl0&clientId=MIRC_SALStockModule'.format(ticker_id)
company_profile_description_url = 'https://api-global.morningstar.com/sal-service/v1/stock/companyProfile/{}?languageId=en&locale=en&clientId=MIRC_SALStockModule&access_token=gIZECmgrLiGStrr5OGQ2W85RvQl0'.format(ticker_id)
insider_info_url = 'https://api-global.morningstar.com/sal-service/v1/stock/insiders/keyExecutives/{}/data?access_token=gIZECmgrLiGStrr5OGQ2W85RvQl0&clientId=MIRC_SALStockModule'.format(ticker_id)
dividend_and_buyback_url = 'https://api-global.morningstar.com/sal-service/v1/stock/dividends/v4/{}/data?access_token=gIZECmgrLiGStrr5OGQ2W85RvQl0&clientId=MIRC_SALStockModule'.format(ticker_id)
req = urllib2.Request(url)
req.add_header('Origin', 'http://library.morningstar.com')
req.add_header('Referer', 'http://library.morningstar.com/')
req.add_header('User-Agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36')
req.add_header('X-API-REALTIME-E', <redacted>) # This needs filed in based on your unique ID.
req.add_header('X-API-RequestId', <redacted>) # This needs filled in based on your unique ID.
req.add_header('X-SAL-ContentType', 'nNsGdN3REOnPMlKDShOYjlk6VYiEVLSdpfpXAm7o2Tk=')
resp = urllib2.urlopen(req)
content = resp.read()
print content
As I feared, the login header fields expire. I'll need to find a way to simulate the log in flow to extract these values.
Removed the broken URLs in d00bed9
TODO: remove this -
IsThisStockGood/templates/js/search.js
Line 59 in 52381ff
Since I don't have a good way to authenticate, I'm just gonna remove the lingering code and call this obsolete
.
Removed the last of the code in 9b10813