By Yijing Xiao
In this project, you will be able to download stock data from Yahoo Finance and then visualize the data you get through making different plots. Although you may not be able to come up with precise predictions of how the stock will behave from the plots made in this project, but it's fun to make these plots yourself to compare with plots we see from finance websites and get a general feeling of how the stock behaved in the days passed.
Here is one of the plots that we are going to come up with using python and javascript.
This plot looks simple but it's pretty nice and clean. Does this plot looks familiar with you? I believe you must have come across plots like this on the finance webpages like Google and Yahoo. You can also make one for your favorite stock yourself. 😃 Just follow me and read through this blog.
Ok, let's get started!
There are a number of steps in this project you need to do before you get the final step of making batches of plots for stocks that you are interested. But first thing first, clone all the files in this project to a local folder.
Next we need to get the list of stock numbers from the internet so the program will know which stock you will need to download. A "A_stocklist.xlsx" file is included in this project and it includes the stock numbers for stocks listed on Shanghai Stock Exchange (SH) and for stocks listed on Shenzhen Stock Exchange (SZ).
To read all these stock numbers and turn them into a readable list in web browser, we need to first run a python program "get_stocks.py". Here is how the code in this python program looks like:
# This program reads the stock numbers from .xlsx file and turn them into a list
# Run this program in the Terminal and copy the result into the web browser console
import xlrd
if __name__ == "__main__":
xl_workbook = xlrd.open_workbook("A_stocklist.xlsx")
sh_sheet = xl_workbook.sheet_by_name("SH")
sz_sheet = xl_workbook.sheet_by_name("SZ")
sh_list = []
sz_list = []
for row_idx in range(1, sh_sheet.nrows):
sh_list.append(str(sh_sheet.cell_value(row_idx, 0)).split(".")[0])
for row_idx in range(1, sz_sheet.nrows):
sz_list.append(str(sz_sheet.cell_value(row_idx, 0)).split(".")[0])
print(sh_list)
print(sz_list)Make sure you have Python 3 installed on your computer and if not, just get rid of the round brackets behind the last two lines with "print" command and you should be fine to run this program on Python 2. This program uses a python library called "xlrd" so if you don't have this library installed, try to input in your command line
pip install xlrd
Ok, everything is set up for us to download the list of stock numbers and here comes our most important step of this part--Run the program. Before you run, check that you are in the directory where you cloned this project to on your computer. In your command line, input the following commands and hit Enter:
python get_stocks.py
Alright, you should see two lists of outputs. One list named "sh_list" and one named "sz_list". They should look like this:
sh_stocks = ['600000', '600004', '600006', '600007', '600008', ...];
sz_stocks = ['000001', '000002', '000004', '000005', '000006', ...];
Try copy this output into your web browser Console and you can also skip this since this is also included in the code of our next step. Now we are ready to move on to the next step of this project.
There are many ways to download stock data from finance websites. Here is the way of how we are going to do in this project. In the project files, open file "download_stock.js" with any of the text editors you have. When it's opened, you should see lines of code that look like this:
// Copy this part into the console
// This part will be the same as the get_stocks.py running result in the Terminal
sh_stocks = ['600000', '600004', '600006', '600007', '600008', '600009', '600010', '600011', '600012', '600015', '600016', '600017', '600018', '600019', '600020', '600021', '600022', '600023', '600026', '600027', '600028', '600029', '600030', '600031', '600033', '600035', '600036', '600037', '600038', '600039', '600048', '600050', '600051', '600052', '600053', '600054', '600055', '600056', '600057', '600058', '600059', '600060', '600061', '600062', '600063', '600064', '600066', '600067', '600068', '600069', '600070', '600071', '600072', '600073', '600074', '600075', '600076', '600077', '600078', '600079', '600080', '600081', '600082', '600083', '600084', '600085', '600086', '600088', '600089', '600090', '600091', '600093', '600094', '600095', '600096', '600097', '600098', '600099', '600100', '600101', '600103', '600104', '600105', '600106', '600107', '600108', '600109', '600110', '600111', '600112', '600113', '600114', '600115', '600116', '600117', '600118', '600119', '600120', '600121', '600122', '600123', '600125', '600126', '600127', '600128', '600129', '600130', '600131', '600132', '600133', '600135', '600136', '600137', '600138', '600139', '600141', '600143', '600145', '600146', '600148', '600149', '600150', '600151', '600152', '600153', '600155', '600156', '600157', '600158', '600159', '600160', '600161', '600162', '600163', '600165', '600166', '600167', '600168', '600169', '600170', '600171', '600172', '600173', '600175', '600176', '600177', '600178', '600179', '600180', '600182', '600183', '600184', '600185', '600186', '600187', '600188', '600189', '600190', '600191', '600192', '600193', '600195', '600196', '600197', '600198', '600199', '600200', '600201', '600202', '600203', '600206', '600207', '600208', '600209', '600210', '600211', '600212', '600213', '600215', '600216', '600217', '600218', '600219', '600220', '600221', '600222', '600223', '600225', '600226', '600227', '600228', '600229', '600230', '600231', '600232', '600233', '600234', '600235', '600236', '600237', '600238', '600239', '600240', '600241', '600242', '600243', '600246', '600247', '600248', '600249', '600250', '600251', '600252', '600255', '600256', '600257', '600258', '600259', '600260', '600261', '600262', '600265', '600266', '600267', '600268', '600269', '600270', '600271', '600272', '600273', '600275', '600276', '600277', '600278', '600279', '600280', '600281', '600282', '600283', '600284', '600285', '600287', '600288', '600289', '600290', '600291', '600292', '600293', '600295', '600297', '600298', '600299', '600300', '600301', '600302', '600303', '600305', '600306', '600307', '600308', '600309', '600310', '600311', '600312', '600313', '600315', '600316', '600317', '600318', '600319', '600320', '600321', '600322', '600323', '600325', '600326', '600327', '600328', '600329', '600330', '600331', '600332', '600333', '600335', '600336', '600337', '600338', '600339', '600340', '600343', '600345', '600346', '600348', '600350', '600351', '600352', '600353', '600354', '600355', '600356', '600358', '600359', '600360', '600361', '600362', '600363', '600365', '600366', '600367', '600368', '600369', '600370', '600371', '600372', '600373', '600375', '600376', '600377', '600378', '600379', '600380', '600381', '600382', '600383', '600385', '600386', '600387', '600388', '600389', '600390', '600391', '600392', '600393', '600395', '600396', '600397', '600398', '600399', '600400', '600401', '600403', '600405', '600406', '600408', '600409', '600410', '600415', '600416', '600418', '600419', '600420', '600421', '600422', '600423', '600425', '600426', '600428', '600429', '600432', '600433', '600435', '600436', '600438', '600439', '600444', '600446', '600448', '600449', '600452', '600455', '600456', '600458', '600459', '600460', '600461', '600462', '600463', '600466', '600467', '600468', '600469', '600470', '600475', '600476', '600477', '600478', '600479', '600480', '600481', '600482', '600483', '600485', '600486', '600487', '600488', '600489', '600490', '600491', '600493', '600495', '600496', '600497', '600498', '600499', '600500', '600501', '600502', '600503', '600505', '600506', '600507', '600508', '600509', '600510', '600511', '600512', '600513', '600515', '600516', '600517', '600518', '600519', '600520', '600521', '600522', '600523', '600525', '600526', '600527', '600528', '600529', '600530', '600531', '600532', '600533', '600535', '600536', '600537', '600538', '600539', '600540', '600543', '600545', '600546', '600547', '600548', '600549', '600550', '600551', '600552', '600555', '600556', '600557', '600558', '600559', '600560', '600561', '600562', '600563', '600565', '600566', '600567', '600568', '600569', '600570', '600571', '600572', '600573', '600575', '600576', '600577', '600578', '600579', '600580', '600581', '600582', '600583', '600584', '600585', '600586', '600587', '600588', '600589', '600590', '600592', '600593', '600594', '600595', '600596', '600597', '600598', '600599', '600600', '600601', '600602', '600603', '600604', '600605', '600606', '600608', '600609', '600610', '600611', '600612', '600613', '600614', '600615', '600616', '600617', '600618', '600619', '600620', '600621', '600622', '600623', '600624', '600626', '600628', '600629', '600630', '600633', '600634', '600635', '600636', '600637', '600638', '600639', '600640', '600641', '600642', '600643', '600644', '600645', '600647', '600648', '600649', '600650', '600651', '600652', '600653', '600654', '600655', '600657', '600658', '600660', '600661', '600662', '600663', '600664', '600665', '600666', '600667', '600668', '600671', '600673', '600674', '600675', '600676', '600677', '600678', '600679', '600680', '600681', '600682', '600683', '600684', '600685', '600686', '600687', '600688', '600689', '600690', '600691', '600692', '600693', '600694', '600695', '600696', '600697', '600698', '600699', '600701', '600702', '600703', '600704', '600705', '600706', '600707', '600708', '600710', '600711', '600712', '600713', '600714', '600715', '600716', '600717', '600718', '600719', '600720', '600721', '600722', '600723', '600724', '600725', '600726', '600727', '600728', '600729', '600730', '600731', '600732', '600733', '600734', '600735', '600736', '600737', '600738', '600739', '600740', '600741', '600742', '600743', '600744', '600745', '600746', '600747', '600748', '600749', '600750', '600751', '600753', '600754', '600755', '600756', '600757', '600758', '600759', '600760', '600761', '600763', '600764', '600765', '600766', '600767', '600768', '600769', '600770', '600771', '600773', '600774', '600775', '600776', '600777', '600778', '600779', '600780', '600781', '600782', '600783', '600784', '600785', '600787', '600789', '600790', '600791', '600792', '600793', '600794', '600795', '600796', '600797', '600798', '600800', '600801', '600802', '600803', '600804', '600805', '600806', '600807', '600808', '600809', '600810', '600811', '600812', '600814', '600815', '600816', '600817', '600818', '600819', '600820', '600821', '600822', '600823', '600824', '600825', '600826', '600827', '600828', '600829', '600830', '600831', '600833', '600834', '600835', '600836', '600837', '600838', '600839', '600841', '600843', '600844', '600845', '600846', '600847', '600848', '600850', '600851', '600853', '600854', '600855', '600856', '600857', '600858', '600859', '600860', '600861', '600862', '600863', '600864', '600865', '600866', '600867', '600868', '600869', '600870', '600871', '600872', '600873', '600874', '600875', '600876', '600877', '600879', '600880', '600881', '600882', '600883', '600884', '600885', '600886', '600887', '600888', '600889', '600890', '600891', '600892', '600893', '600894', '600895', '600896', '600897', '600898', '600900', '600908', '600909', '600917', '600919', '600926', '600936', '600939', '600958', '600959', '600960', '600961', '600962', '600963', '600965', '600966', '600967', '600969', '600970', '600971', '600973', '600975', '600976', '600977', '600978', '600979', '600980', '600981', '600982', '600983', '600984', '600985', '600986', '600987', '600988', '600990', '600992', '600993', '600995', '600996', '600997', '600998', '600999', '601000', '601001', '601002', '601003', '601005', '601006', '601007', '601008', '601009', '601010', '601011', '601012', '601015', '601016', '601018', '601020', '601021', '601028', '601038', '601058', '601069', '601088', '601098', '601099', '601100', '601101', '601106', '601107', '601111', '601113', '601116', '601117', '601118', '601126', '601127', '601128', '601137', '601139', '601155', '601158', '601163', '601166', '601168', '601169', '601177', '601179', '601186', '601188', '601198', '601199', '601200', '601208', '601211', '601212', '601216', '601218', '601222', '601225', '601226', '601228', '601229', '601231', '601233', '601238', '601258', '601288', '601311', '601313', '601318', '601326', '601328', '601333', '601336', '601339', '601366', '601368', '601369', '601375', '601377', '601388', '601390', '601398', '601500', '601515', '601518', '601519', '601555', '601558', '601566', '601567', '601579', '601588', '601595', '601599', '601600', '601601', '601607', '601608', '601611', '601616', '601618', '601619', '601628', '601633', '601636', '601666', '601668', '601669', '601677', '601678', '601688', '601689', '601699', '601700', '601717', '601718', '601727', '601766', '601777', '601788', '601789', '601798', '601799', '601800', '601801', '601808', '601811', '601818', '601857', '601858', '601866', '601872', '601877', '601878', '601880', '601881', '601882', '601886', '601888', '601890', '601898', '601899', '601900', '601901', '601908', '601918', '601919', '601928', '601929', '601933', '601939', '601952', '601958', '601965', '601966', '601968', '601969', '601985', '601988', '601989', '601991', '601992', '601996', '601997', '601998', '601999', '603000', '603001', '603002', '603003', '603005', '603006', '603007', '603008', '603009', '603010', '603011', '603012', '603015', '603016', '603017', '603018', '603019', '603020', '603021', '603022', '603023', '603025', '603026', '603027', '603028', '603029', '603030', '603031', '603032', '603033', '603035', '603036', '603037', '603038', '603039', '603040', '603041', '603042', '603043', '603050', '603058', '603060', '603063', '603066', '603067', '603069', '603077', '603078', '603081', '603085', '603086', '603088', '603089', '603090', '603096', '603098', '603099', '603100', '603101', '603108', '603111', '603113', '603116', '603117', '603118', '603123', '603126', '603128', '603129', '603131', '603133', '603138', '603139', '603158', '603159', '603160', '603165', '603166', '603167', '603168', '603169', '603177', '603178', '603179', '603180', '603186', '603188', '603189', '603196', '603197', '603198', '603199', '603200', '603203', '603208', '603218', '603222', '603223', '603225', '603226', '603227', '603228', '603229', '603232', '603233', '603238', '603239', '603258', '603266', '603268', '603269', '603286', '603288', '603298', '603299', '603300', '603303', '603305', '603306', '603308', '603309', '603311', '603313', '603315', '603316', '603318', '603319', '603320', '603322', '603323', '603326', '603328', '603330', '603331', '603333', '603335', '603336', '603337', '603338', '603339', '603345', '603355', '603357', '603358', '603360', '603366', '603368', '603369', '603377', '603380', '603383', '603385', '603387', '603388', '603389', '603393', '603398', '603399', '603416', '603421', '603429', '603444', '603456', '603458', '603488', '603496', '603501', '603505', '603508', '603515', '603517', '603518', '603519', '603520', '603528', '603535', '603536', '603538', '603555', '603556', '603557', '603558', '603559', '603566', '603567', '603568', '603569', '603577', '603578', '603579', '603580', '603585', '603586', '603588', '603589', '603595', '603598', '603599', '603600', '603601', '603602', '603603', '603606', '603608', '603609', '603611', '603612', '603615', '603616', '603617', '603618', '603626', '603628', '603630', '603633', '603636', '603637', '603638', '603639', '603656', '603658', '603660', '603663', '603665', '603667', '603668', '603669', '603676', '603677', '603678', '603679', '603686', '603688', '603689', '603690', '603696', '603698', '603699', '603701', '603703', '603707', '603708', '603716', '603717', '603718', '603721', '603726', '603727', '603728', '603729', '603730', '603737', '603738', '603757', '603758', '603766', '603767', '603768', '603776', '603777', '603778', '603779', '603787', '603788', '603789', '603797', '603798', '603799', '603800', '603801', '603803', '603806', '603808', '603811', '603816', '603817', '603818', '603819', '603822', '603823', '603825', '603826', '603828', '603833', '603838', '603839', '603843', '603855', '603858', '603859', '603860', '603861', '603866', '603868', '603869', '603877', '603878', '603879', '603880', '603881', '603883', '603885', '603886', '603887', '603888', '603889', '603896', '603898', '603899', '603900', '603901', '603903', '603906', '603908', '603909', '603918', '603919', '603920', '603926', '603928', '603929', '603933', '603936', '603938', '603939', '603955', '603958', '603959', '603960', '603966', '603968', '603969', '603977', '603978', '603979', '603980', '603985', '603986', '603987', '603988', '603989', '603990', '603991', '603993', '603996', '603997', '603998', '603999'];
sz_stocks = ['000001', '000002', '000004', '000005', '000006', '000007', '000008', '000009', '000010', '000011', '000012', '000014', '000016', '000017', '000018', '000019', '000020', '000021', '000022', '000023', '000025', '000026', '000027', '000028', '000029', '000030', '000031', '000032', '000034', '000035', '000036', '000037', '000038', '000039', '000040', '000042', '000043', '000045', '000046', '000048', '000049', '000050', '000055', '000056', '000058', '000059', '000060', '000061', '000062', '000063', '000065', '000066', '000068', '000069', '000070', '000078', '000088', '000089', '000090', '000096', '000099', '000100', '000150', '000151', '000153', '000155', '000156', '000157', '000158', '000159', '000166', '000301', '000333', '000338', '000400', '000401', '000402', '000403', '000404', '000407', '000408', '000409', '000410', '000411', '000413', '000415', '000416', '000417', '000418', '000419', '000420', '000421', '000422', '000423', '000425', '000426', '000428', '000429', '000430', '000488', '000498', '000501', '000502', '000503', '000504', '000505', '000506', '000507', '000509', '000510', '000511', '000513', '000514', '000516', '000517', '000518', '000519', '000520', '000521', '000523', '000524', '000525', '000526', '000528', '000529', '000530', '000531', '000532', '000533', '000534', '000536', '000537', '000538', '000539', '000540', '000541', '000543', '000544', '000545', '000546', '000547', '000548', '000550', '000551', '000552', '000553', '000554', '000555', '000557', '000558', '000559', '000560', '000561', '000563', '000564', '000565', '000566', '000567', '000568', '000570', '000571', '000572', '000573', '000576', '000581', '000582', '000584', '000585', '000586', '000587', '000589', '000590', '000591', '000592', '000593', '000595', '000596', '000597', '000598', '000599', '000600', '000601', '000603', '000605', '000606', '000607', '000608', '000609', '000610', '000611', '000612', '000613', '000615', '000616', '000617', '000619', '000620', '000622', '000623', '000625', '000626', '000627', '000628', '000629', '000630', '000631', '000632', '000633', '000635', '000636', '000637', '000638', '000639', '000650', '000651', '000652', '000655', '000656', '000657', '000659', '000661', '000662', '000663', '000665', '000666', '000667', '000668', '000669', '000670', '000671', '000672', '000673', '000676', '000677', '000678', '000679', '000680', '000681', '000682', '000683', '000685', '000686', '000687', '000688', '000690', '000691', '000692', '000693', '000695', '000697', '000698', '000700', '000701', '000702', '000703', '000705', '000707', '000708', '000709', '000710', '000711', '000712', '000713', '000715', '000716', '000717', '000718', '000719', '000720', '000721', '000722', '000723', '000725', '000726', '000727', '000728', '000729', '000731', '000732', '000733', '000735', '000736', '000737', '000738', '000739', '000750', '000751', '000752', '000753', '000755', '000756', '000757', '000758', '000759', '000760', '000761', '000762', '000766', '000767', '000768', '000776', '000777', '000778', '000779', '000780', '000782', '000783', '000785', '000786', '000788', '000789', '000790', '000791', '000792', '000793', '000795', '000796', '000797', '000798', '000799', '000800', '000801', '000802', '000803', '000806', '000807', '000809', '000810', '000811', '000812', '000813', '000815', '000816', '000818', '000819', '000820', '000821', '000822', '000823', '000825', '000826', '000828', '000829', '000830', '000831', '000833', '000835', '000836', '000837', '000838', '000839', '000848', '000850', '000851', '000852', '000856', '000858', '000859', '000860', '000861', '000862', '000863', '000868', '000869', '000875', '000876', '000877', '000878', '000880', '000881', '000882', '000883', '000885', '000886', '000887', '000888', '000889', '000890', '000892', '000893', '000895', '000897', '000898', '000899', '000900', '000901', '000902', '000903', '000905', '000906', '000908', '000909', '000910', '000911', '000912', '000913', '000915', '000916', '000917', '000918', '000919', '000920', '000921', '000922', '000923', '000925', '000926', '000927', '000928', '000929', '000930', '000931', '000932', '000933', '000935', '000936', '000937', '000938', '000939', '000948', '000949', '000950', '000951', '000952', '000953', '000955', '000957', '000958', '000959', '000960', '000961', '000962', '000963', '000965', '000966', '000967', '000968', '000969', '000970', '000971', '000972', '000973', '000975', '000976', '000977', '000978', '000979', '000980', '000981', '000982', '000983', '000985', '000987', '000988', '000989', '000990', '000993', '000995', '000996', '000997', '000998', '000999', '001696', '001896', '001979', '002001', '002002', '002003', '002004', '002005', '002006', '002007', '002008', '002009', '002010', '002011', '002012', '002013', '002014', '002015', '002016', '002017', '002018', '002019', '002020', '002021', '002022', '002023', '002024', '002025', '002026', '002027', '002028', '002029', '002030', '002031', '002032', '002033', '002034', '002035', '002036', '002037', '002038', '002039', '002040', '002041', '002042', '002043', '002044', '002045', '002046', '002047', '002048', '002049', '002050', '002051', '002052', '002053', '002054', '002055', '002056', '002057', '002058', '002059', '002060', '002061', '002062', '002063', '002064', '002065', '002066', '002067', '002068', '002069', '002070', '002071', '002072', '002073', '002074', '002075', '002076', '002077', '002078', '002079', '002080', '002081', '002082', '002083', '002084', '002085', '002086', '002087', '002088', '002089', '002090', '002091', '002092', '002093', '002094', '002095', '002096', '002097', '002098', '002099', '002100', '002101', '002102', '002103', '002104', '002105', '002106', '002107', '002108', '002109', '002110', '002111', '002112', '002113', '002114', '002115', '002116', '002117', '002118', '002119', '002120', '002121', '002122', '002123', '002124', '002125', '002126', '002127', '002128', '002129', '002130', '002131', '002132', '002133', '002134', '002135', '002136', '002137', '002138', '002139', '002140', '002141', '002142', '002143', '002144', '002145', '002146', '002147', '002148', '002149', '002150', '002151', '002152', '002153', '002154', '002155', '002156', '002157', '002158', '002159', '002160', '002161', '002162', '002163', '002164', '002165', '002166', '002167', '002168', '002169', '002170', '002171', '002172', '002173', '002174', '002175', '002176', '002177', '002178', '002179', '002180', '002181', '002182', '002183', '002184', '002185', '002186', '002187', '002188', '002189', '002190', '002191', '002192', '002193', '002194', '002195', '002196', '002197', '002198', '002199', '002200', '002201', '002202', '002203', '002204', '002205', '002206', '002207', '002208', '002209', '002210', '002211', '002212', '002213', '002214', '002215', '002216', '002217', '002218', '002219', '002220', '002221', '002222', '002223', '002224', '002225', '002226', '002227', '002228', '002229', '002230', '002231', '002232', '002233', '002234', '002235', '002236', '002237', '002238', '002239', '002240', '002241', '002242', '002243', '002244', '002245', '002246', '002247', '002248', '002249', '002250', '002251', '002252', '002253', '002254', '002255', '002256', '002258', '002259', '002260', '002261', '002262', '002263', '002264', '002265', '002266', '002267', '002268', '002269', '002270', '002271', '002272', '002273', '002274', '002275', '002276', '002277', '002278', '002279', '002280', '002281', '002282', '002283', '002284', '002285', '002286', '002287', '002288', '002289', '002290', '002291', '002292', '002293', '002294', '002295', '002296', '002297', '002298', '002299', '002300', '002301', '002302', '002303', '002304', '002305', '002306', '002307', '002308', '002309', '002310', '002311', '002312', '002313', '002314', '002315', '002316', '002317', '002318', '002319', '002320', '002321', '002322', '002323', '002324', '002325', '002326', '002327', '002328', '002329', '002330', '002331', '002332', '002333', '002334', '002335', '002336', '002337', '002338', '002339', '002340', '002341', '002342', '002343', '002344', '002345', '002346', '002347', '002348', '002349', '002350', '002351', '002352', '002353', '002354', '002355', '002356', '002357', '002358', '002359', '002360', '002361', '002362', '002363', '002364', '002365', '002366', '002367', '002368', '002369', '002370', '002371', '002372', '002373', '002374', '002375', '002376', '002377', '002378', '002379', '002380', '002381', '002382', '002383', '002384', '002385', '002386', '002387', '002388', '002389', '002390', '002391', '002392', '002393', '002394', '002395', '002396', '002397', '002398', '002399', '002400', '002401', '002402', '002403', '002404', '002405', '002406', '002407', '002408', '002409', '002410', '002411', '002412', '002413', '002414', '002415', '002416', '002417', '002418', '002419', '002420', '002421', '002422', '002423', '002424', '002425', '002426', '002427', '002428', '002429', '002430', '002431', '002432', '002433', '002434', '002435', '002436', '002437', '002438', '002439', '002440', '002441', '002442', '002443', '002444', '002445', '002446', '002447', '002448', '002449', '002450', '002451', '002452', '002453', '002454', '002455', '002456', '002457', '002458', '002459', '002460', '002461', '002462', '002463', '002464', '002465', '002466', '002467', '002468', '002469', '002470', '002471', '002472', '002473', '002474', '002475', '002476', '002477', '002478', '002479', '002480', '002481', '002482', '002483', '002484', '002485', '002486', '002487', '002488', '002489', '002490', '002491', '002492', '002493', '002494', '002495', '002496', '002497', '002498', '002499', '002500', '002501', '002502', '002503', '002504', '002505', '002506', '002507', '002508', '002509', '002510', '002511', '002512', '002513', '002514', '002515', '002516', '002517', '002518', '002519', '002520', '002521', '002522', '002523', '002524', '002526', '002527', '002528', '002529', '002530', '002531', '002532', '002533', '002534', '002535', '002536', '002537', '002538', '002539', '002540', '002541', '002542', '002543', '002544', '002545', '002546', '002547', '002548', '002549', '002550', '002551', '002552', '002553', '002554', '002555', '002556', '002557', '002558', '002559', '002560', '002561', '002562', '002563', '002564', '002565', '002566', '002567', '002568', '002569', '002570', '002571', '002572', '002573', '002574', '002575', '002576', '002577', '002578', '002579', '002580', '002581', '002582', '002583', '002584', '002585', '002586', '002587', '002588', '002589', '002590', '002591', '002592', '002593', '002594', '002595', '002596', '002597', '002598', '002599', '002600', '002601', '002602', '002603', '002604', '002605', '002606', '002607', '002608', '002609', '002610', '002611', '002612', '002613', '002614', '002615', '002616', '002617', '002618', '002619', '002620', '002621', '002622', '002623', '002624', '002625', '002626', '002627', '002628', '002629', '002630', '002631', '002632', '002633', '002634', '002635', '002636', '002637', '002638', '002639', '002640', '002641', '002642', '002643', '002644', '002645', '002646', '002647', '002648', '002649', '002650', '002651', '002652', '002653', '002654', '002655', '002656', '002657', '002658', '002659', '002660', '002661', '002662', '002663', '002664', '002665', '002666', '002667', '002668', '002669', '002670', '002671', '002672', '002673', '002674', '002675', '002676', '002677', '002678', '002679', '002680', '002681', '002682', '002683', '002684', '002685', '002686', '002687', '002688', '002689', '002690', '002691', '002692', '002693', '002694', '002695', '002696', '002697', '002698', '002699', '002700', '002701', '002702', '002703', '002705', '002706', '002707', '002708', '002709', '002711', '002712', '002713', '002714', '002715', '002716', '002717', '002718', '002719', '002721', '002722', '002723', '002724', '002725', '002726', '002727', '002728', '002729', '002730', '002731', '002732', '002733', '002734', '002735', '002736', '002737', '002738', '002739', '002740', '002741', '002742', '002743', '002745', '002746', '002747', '002748', '002749', '002750', '002751', '002752', '002753', '002755', '002756', '002757', '002758', '002759', '002760', '002761', '002762', '002763', '002765', '002766', '002767', '002768', '002769', '002770', '002771', '002772', '002773', '002774', '002775', '002776', '002777', '002778', '002779', '002780', '002781', '002782', '002783', '002785', '002786', '002787', '002788', '002789', '002790', '002791', '002792', '002793', '002795', '002796', '002797', '002798', '002799', '002800', '002801', '002802', '002803', '002805', '002806', '002807', '002808', '002809', '002810', '002811', '002812', '002813', '002815', '002816', '002817', '002818', '002819', '002820', '002821', '002822', '002823', '002824', '002825', '002826', '002827', '002828', '002829', '002830', '002831', '002832', '002833', '002835', '002836', '002837', '002838', '002839', '002840', '002841', '002842', '002843', '002845', '002846', '002847', '002848', '002849', '002850', '002851', '002852', '002853', '002855', '002856', '002857', '002858', '002859', '002860', '002861', '002862', '002863', '002865', '002866', '002867', '002868', '002869', '002870', '002871', '002872', '002873', '002875', '002876', '002877', '002878', '002879', '002880', '002881', '002882', '002883', '002884', '002885', '002886', '002887', '002888', '002889', '002890', '002892', '300001', '300002', '300003', '300004', '300005', '300006', '300007', '300008', '300009', '300010', '300011', '300012', '300013', '300014', '300015', '300016', '300017', '300018', '300019', '300020', '300021', '300022', '300023', '300024', '300025', '300026', '300027', '300028', '300029', '300030', '300031', '300032', '300033', '300034', '300035', '300036', '300037', '300038', '300039', '300040', '300041', '300042', '300043', '300044', '300045', '300046', '300047', '300048', '300049', '300050', '300051', '300052', '300053', '300054', '300055', '300056', '300057', '300058', '300059', '300061', '300062', '300063', '300064', '300065', '300066', '300067', '300068', '300069', '300070', '300071', '300072', '300073', '300074', '300075', '300076', '300077', '300078', '300079', '300080', '300081', '300082', '300083', '300084', '300085', '300086', '300087', '300088', '300089', '300090', '300091', '300092', '300093', '300094', '300095', '300096', '300097', '300098', '300099', '300100', '300101', '300102', '300103', '300104', '300105', '300106', '300107', '300108', '300109', '300110', '300111', '300112', '300113', '300114', '300115', '300116', '300117', '300118', '300119', '300120', '300121', '300122', '300123', '300124', '300125', '300126', '300127', '300128', '300129', '300130', '300131', '300132', '300133', '300134', '300135', '300136', '300137', '300138', '300139', '300140', '300141', '300142', '300143', '300144', '300145', '300146', '300147', '300148', '300149', '300150', '300151', '300152', '300153', '300154', '300155', '300156', '300157', '300158', '300159', '300160', '300161', '300162', '300163', '300164', '300165', '300166', '300167', '300168', '300169', '300170', '300171', '300172', '300173', '300174', '300175', '300176', '300177', '300178', '300179', '300180', '300181', '300182', '300183', '300184', '300185', '300187', '300188', '300189', '300190', '300191', '300192', '300193', '300194', '300195', '300196', '300197', '300198', '300199', '300200', '300201', '300202', '300203', '300204', '300205', '300206', '300207', '300208', '300209', '300210', '300211', '300212', '300213', '300214', '300215', '300216', '300217', '300218', '300219', '300220', '300221', '300222', '300223', '300224', '300225', '300226', '300227', '300228', '300229', '300230', '300231', '300232', '300233', '300234', '300235', '300236', '300237', '300238', '300239', '300240', '300241', '300242', '300243', '300244', '300245', '300246', '300247', '300248', '300249', '300250', '300251', '300252', '300253', '300254', '300255', '300256', '300257', '300258', '300259', '300260', '300261', '300262', '300263', '300264', '300265', '300266', '300267', '300268', '300269', '300270', '300271', '300272', '300273', '300274', '300275', '300276', '300277', '300278', '300279', '300280', '300281', '300282', '300283', '300284', '300285', '300286', '300287', '300288', '300289', '300290', '300291', '300292', '300293', '300294', '300295', '300296', '300297', '300298', '300299', '300300', '300301', '300302', '300303', '300304', '300305', '300306', '300307', '300308', '300309', '300310', '300311', '300312', '300313', '300314', '300315', '300316', '300317', '300318', '300319', '300320', '300321', '300322', '300323', '300324', '300325', '300326', '300327', '300328', '300329', '300330', '300331', '300332', '300333', '300334', '300335', '300336', '300337', '300338', '300339', '300340', '300341', '300342', '300343', '300344', '300345', '300346', '300347', '300348', '300349', '300350', '300351', '300352', '300353', '300354', '300355', '300356', '300357', '300358', '300359', '300360', '300362', '300363', '300364', '300365', '300366', '300367', '300368', '300369', '300370', '300371', '300372', '300373', '300374', '300375', '300376', '300377', '300378', '300379', '300380', '300381', '300382', '300383', '300384', '300385', '300386', '300387', '300388', '300389', '300390', '300391', '300392', '300393', '300394', '300395', '300396', '300397', '300398', '300399', '300400', '300401', '300402', '300403', '300404', '300405', '300406', '300407', '300408', '300409', '300410', '300411', '300412', '300413', '300414', '300415', '300416', '300417', '300418', '300419', '300420', '300421', '300422', '300423', '300424', '300425', '300426', '300427', '300428', '300429', '300430', '300431', '300432', '300433', '300434', '300435', '300436', '300437', '300438', '300439', '300440', '300441', '300442', '300443', '300444', '300445', '300446', '300447', '300448', '300449', '300450', '300451', '300452', '300453', '300455', '300456', '300457', '300458', '300459', '300460', '300461', '300462', '300463', '300464', '300465', '300466', '300467', '300468', '300469', '300470', '300471', '300472', '300473', '300474', '300475', '300476', '300477', '300478', '300479', '300480', '300481', '300482', '300483', '300484', '300485', '300486', '300487', '300488', '300489', '300490', '300491', '300492', '300493', '300494', '300495', '300496', '300497', '300498', '300499', '300500', '300501', '300502', '300503', '300505', '300506', '300507', '300508', '300509', '300510', '300511', '300512', '300513', '300514', '300515', '300516', '300517', '300518', '300519', '300520', '300521', '300522', '300523', '300525', '300526', '300527', '300528', '300529', '300530', '300531', '300532', '300533', '300534', '300535', '300536', '300537', '300538', '300539', '300540', '300541', '300542', '300543', '300545', '300546', '300547', '300548', '300549', '300550', '300551', '300552', '300553', '300554', '300555', '300556', '300557', '300558', '300559', '300560', '300561', '300562', '300563', '300565', '300566', '300567', '300568', '300569', '300570', '300571', '300572', '300573', '300575', '300576', '300577', '300578', '300579', '300580', '300581', '300582', '300583', '300584', '300585', '300586', '300587', '300588', '300589', '300590', '300591', '300592', '300593', '300595', '300596', '300597', '300598', '300599', '300600', '300601', '300602', '300603', '300604', '300605', '300606', '300607', '300608', '300609', '300610', '300611', '300612', '300613', '300615', '300616', '300617', '300618', '300619', '300620', '300621', '300622', '300623', '300625', '300626', '300627', '300628', '300629', '300630', '300631', '300632', '300633', '300635', '300636', '300637', '300638', '300639', '300640', '300641', '300642', '300643', '300645', '300647', '300648', '300649', '300650', '300651', '300652', '300653', '300655', '300656', '300657', '300658', '300659', '300660', '300661', '300662', '300663', '300665', '300666', '300667', '300668', '300669', '300670', '300671', '300672', '300673', '300675', '300676', '300677', '300678', '300679', '300680', '300681', '300682', '300683', '300685', '300686', '300687', '300688', '300689', '300690', '300691'];
// This program downloads stock data from Yahoo Finance
// Copy this part of program into web browser console
// After download's finished, move .json files into "download stocks" folder
var start_date = 0 //from start of time
var end_date = 9999999999 //to the day today
//Creat a download button and download stock data
downloadStocks = function(stock, market, start_date, end_date){
var a = document.createElement('a');
document.body.appendChild(a);
console.log(stock);
a.href = "https://l1-query.finance.yahoo.com/v8/finance/chart/" + stock + "." + market + "?period2=" + end_date +"&period1=" + start_date + "&interval=1d&indicators=quote&includeTimestamps=true&includePrePost=true&events=div%7Csplit%7Cearn&corsDomain=finance.yahoo.com";
a.setAttribute("download", stock + market + ".json")
a.click();
};
// Download stock data for Shanghai listed stocks
for (var i = 0; i < sh_stocks.length; i++){
console.log(sh_stocks[i]);
setTimeout(downloadStocks, i * Math.random()*10000, sh_stocks[i], "SS", start_date, end_date);
}
// Download stock data for Shenzhen listed stocks
for (var i = 0; i < sz_stocks.length; i++){
console.log(sz_stocks[i]);
setTimeout(downloadStocks, (i + sh_stocks.length) * Math.random()*10000, sz_stocks[i], "SZ", start_date, end_date);
}If you haven't copied the output result from the python program above, you can copy the list of stock numbers from this file. All you need to do is to copy all the codes and paste into your web browser Console. Hit Enter and stock data will start to download. In order to enter the Console of your web browser, open the web browser, right clock and select "inspect". In the window pops up, there will be a "Console" button on the top ribbon. In Chrome, a Console will look like this:
The main function of this program is to create a button with link to the the stocks that you want to download. A simulated click on the link will be performed by the program and a .json file will be downloaded for each stock and saved to the download folder of your web browser. For each stock, this program downloads all the data starting from the very first day stock was released to the day you download data.
One more thing before we can start on making plots and visualize the data we just downloaded. You need to move the stock data (.json) files from the web browser download folder to the "download stocks" folder in the project directory.
Everything is good to go and we can start making plots for your stock data now.
We are going to use python to create all the plots for the stock data we have. So we need to first import all the libraries that are going to be used in the following steps. In the project files, there is a "stocks_plot.ipynb" jupyter notebook file with all the steps shown for making plots. The libraries that we are going to import here are:
# Import all libraries
import json
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
%pylab inline
import datetime
from dateutil.relativedelta import relativedelta
from matplotlib.dates import DateFormatter, WeekdayLocator, DayLocator, MONDAY
from matplotlib.finance import candlestick_ohlcSome of these libraries comes with the installation of python on your computer but some don't. If you don't have some of the libraries in this list, try pip install method to get libraries you are missing:
pip install libraryname
Here we will take one of the stocks, 002241SZ, as an example and show use its data to make all the plots. Before we make our first plot, we need to read the data from "002241.json" file and change format of some of them so they will be more readable for the libraries used for making plots. The data in the .json file is stored in a structure of dictionaries and lists. Here we will use the following code to read and store data we get from .json file:
# Read stock data from file, stock number can be changed
stock = json.load(open("./download stocks/002241.json", "r"))
# Find the corresponding prices in the .json file data
price_open = stock["chart"]["result"][0]["indicators"]["quote"][0]["open"]
price_close = stock["chart"]["result"][0]["indicators"]["quote"][0]["close"]
price_high = stock["chart"]["result"][0]["indicators"]["quote"][0]["high"]
price_low = stock["chart"]["result"][0]["indicators"]["quote"][0]["low"]
volume = stock["chart"]["result"][0]["indicators"]["quote"][0]["volume"]
price_adjclose = stock["chart"]["result"][0]["indicators"]["adjclose"][0]["adjclose"]
# Change timestamp into string dates
ts = [ datetime.datetime.fromtimestamp(int(ts+86400)).strftime('%Y-%m-%d') for ts in stock["chart"]["result"][0]["timestamp"]]In addition to read and store data, we also see that in the last part of the code, we change the format of timestamp into string. The reason behind this is the time we get is in seconds and we need to change this into year-month-day.
After we read all the data we need from the .json file, we can first set the format of the plots so all the plots we make will be shown in a better manner. We will run the following line of code here to preset the plot size:
pylab.rcParams['figure.figsize'] = (15, 9)Everything looks perfect and we are ready for the best part of this project!
First let's check if we have downloaded the correct data. One of the methods is to show all the adjusted closing price in one plot and compare with the plot we get from Yahoo Finance. We are going to create a dataframe with Pandas library and then use the plot function that comes with the dataframe to show our adjusted closing price history curve. To create a dataframe with all the stock history data, let's run the following codes:
# Creat first Dataframe to include all the information read from .json file
df1 = pd.DataFrame([ts, price_open, price_close, price_high, price_low, volume, price_adjclose], index=["Date", "Open", "Close", "High", "Low", "Volume", "Adjclose"]).T
df1 = df1.set_index("Date")
df1.index = pd.to_datetime(df1.index)
# Drop lines that are all "None" object
df1 = df1.dropna(axis=0, how='all')In the codes above, we can see that some of the lines in the dataframe df1 is dropped. This because the prices of some of the trading days are shown as "None" objects and we are not going to need these numbers for our plots. The next step, we are going to create a curve for all these history adjusted closing data, so the following codes will be run:
# Plot history price curve for Ajusted Closing Price
ax = df1["Adjclose"].plot(grid = True)
ax.set_xlabel("Date", fontsize = 15)
ax.set_ylabel("Adjusted Closing Price", fontsize = 15)
ax.set_title("Adjusted Closing Price History Curve", fontsize = 20)With the above codes, you will be able to get a plot that looks like this:
If you are not sure whether you are downloading the right stock data, you can go to Yahoo Finance and check with the max stock curve for this stock. I get a curve that looks like this, which matches perfectly with our plot above.
So we are on the right track and in addition to visualizing the history curve of this stock, we are going to do something with a lot more fun!
The second plot we are going to make is called a candlestick plot. Since we have years of data stored in our first dataframe and we are not able to show all of them in a candlestick plot. We are gong to create a new dataframe by slicing the data for the recent two months from the first dataframe. Through running the following codes, we will be able to create a candlestick plot:
# Create Dataframe for data between Jun 26th and Aug 22nd
# Getting ready for Candlestick Plot
df2 = df1.loc['20170626':'20170822']
# Create a new DataFrame which includes stock data for each day shown on each candle stick
plotdata = df2.loc[:,["Open", "High", "Low", "Close"]]
stick = 1 # Candle stick width
# Set plot parameters
fig, ax = plt.subplots()
fig.subplots_adjust(bottom=0.2)
mondays = WeekdayLocator(MONDAY) # major ticks
alldays = DayLocator() # minor ticks
weekFormatter = DateFormatter('%b %d') # date format
ax.xaxis.set_major_locator(mondays)
ax.xaxis.set_minor_locator(alldays)
ax.xaxis.set_major_formatter(weekFormatter)
# Create the candelstick chart, Gain shown in red, Loss shown in green
candlestick_ohlc(ax, list(zip(list(date2num(plotdata.index.tolist())),
plotdata["Open"].tolist(), plotdata["High"].tolist(),
plotdata["Low"].tolist(), plotdata["Close"].tolist())),
colorup = "red", colordown = "green", width = stick)
# Adjust plot characteristic parameters
ax.grid(True)
ax.xaxis_date()
ax.autoscale_view()
ax.set_xlabel("Date", fontsize = 15)
ax.set_ylabel("Price", fontsize = 15)
ax.set_title("Candlestick Chart (June 26th to August 22nd)", fontsize = 20)
plt.setp(plt.gca().get_xticklabels(), rotation=45, horizontalalignment='right')
plt.show()From the above code, I think you will be able to get a candlestick plot that looks like this:
Remember when I was a kid, the first impression I had with stock plots are those red and green candle shaped rectangles that line up as a curve on the television. In a candlestick chart, a green candlestick means the closing price is higher than the open price and a red candlestick means the closing price is higher than the open price. This presentation method of using red as gain and green as loss is commonly used in stock markets in China. In USA, candlestick plots shows red as loss and black as gain.
We always want to see how much have we earned after we buy a stock at a certain time. For example, if we buy this stock, 002241SZ at the beginning of year 2017, what is the stock return we get for year to date? Here we will stock return by dividing the price of the stock of the day with the price of the stock at the first trading day of the year.
Also we need to create a dataframe to store the stock price from first trading day of the year to today's date. Unlike first dataframe we created and the dataframe we created for candlestick plot, here we only need adjusted closing price and date to create this plot. So let's run the following code to see how this new data frame is created:
# Creat Dataframe with just adjusted closing price and date
df3 = pd.DataFrame([ts, price_adjclose], index=["Date", "Adjclose"]).T
df3 = df3.set_index("Date")
df3.index = pd.to_datetime(df3.index)
# Drop lines with "None" values
df3 = df3.dropna(axis=0, how='all')
# Create a new Dataframe to get all the year to date data
Df3 = df3.loc['20170101':'20171231']The new dataframe takes out the "Adjclose" column from the first dataframe df1 and sliced with respect to date data to get the year to date prices we need. Now it's time to make the stock return plot:
# Compare price for the day with price at beginning of year
stock_return = Df3.apply(lambda x: x / x[0])
# Set plot parameters
fig, ax = plt.subplots()
fig.subplots_adjust(bottom=0.2)
ax.xaxis.set_major_locator(mondays)
ax.xaxis.set_minor_locator(alldays)
ax.xaxis.set_major_formatter(weekFormatter)
# Adjust plot characteristic parameters
ax.grid(True)
ax.xaxis_date()
ax.autoscale_view()
ax.set_xlabel("Date", fontsize = 15)
ax.set_ylabel("Price Ratio", fontsize = 15)
ax.set_title("Ratio btwn Price of the day and Price at beginning of year", fontsize = 20)
plt.setp(plt.gca().get_xticklabels(), rotation=45, horizontalalignment='right')
plt.plot(stock_return)
plt.plot((stock_return.index[0],stock_return.index[-1]),(1,1), 'k-')
plt.xlim(stock_return.index[0],stock_return.index[-1])
plt.show()Alright, here comes the plot for ratio between price of the day and price at the beginning of the year. It looks like this stock is doing very well because not it's August and the stock curve goes up constantly throughout the year. How's the performance of your stock?
The last plot is really suitable for long term stock holders. If you invest in a stock at the beginning of the year and then plan to sell the stock in the middle of the year or by the end of the year, the above plot might be a good choice for you. However, for short term investors, maybe they are looking at short term performance of a stock, say, performance of a day. So we are going to make a plot for the price percentage change within each trading day. Method is the same with the above stock return plot and we will call the plot here as stock change.
We define stock change as
[(price of today - price of yesterday)] / price of yesterday *100%
and we will use the same dataframe to make our computations for stock change. The code used to create this stock change plot is pretty similar to stock return codes and let's see how your plot looks like.
# Price change within a trading day, calculated by
# [(closing_price)-(closing_price_a_day_ahead)]/(closing_price_a_day_ahead)
stock_change = Df3.apply(lambda x: np.multiply(np.divide(np.subtract(x, x.shift(1)), x.shift(1)),100)) # shift moves dates back by 1.
stock_change.columns = ["Change"]
# Set plot parameters
...
# Adjust plot characteristic parameters
...My plot looks like this and a horizontal line at y=0 is added to show a better comparison for the ups and downs for each day.
Rather than looking at the performance of the past data for each stock, we also need to make some predictions of how the stock will go for the next few days or so. Stock experts created a method to help us get a better prelook into how the stock will behave in the near future and it's called 20-day moving average. This method is implemented by taking the average of the closing price of the stock for the past 20 days and most likely the stock will follow it's trend from the 20-day moving average curve for the next short period. Let's plot this curve on top of the candlestick chart to get a better visualization of how this works.
To do this, we will need to create a window of 20 days on the data and take the average for the numbers we get for these 20 days. Also, we may not be able to get a useful number until we have 20 days to calculate. So for the first 19 days, we will need to drop the date since they will not be useful.
The code for creating candlestick chart is exactly the same as what we show before and we will just need to add one line for the 20-day moving average curve plot.
# Creat Dataframe with slicing by date from the first Dataframe
df4 = df1.loc['20170101':'20171231']
#Calculate 20-day Moving Average from closing price and attach to the original dataframe
df4["20d"] = np.round(df4['Close'].rolling(window = 20, center = False).mean(), 2)
df4 = df4.dropna()
# Create a new DataFrame which includes stock data for each day shown on each candle stick
plotdata = df4.loc[:,["Open", "High", "Low", "Close"]]
# Set plot parameters
...
# Create the candlestick chart, Gain shown in red, Loss shown in green
...
# Add moving averages lines to the Candlestick chart
df4.loc[:,"20d"].plot(ax = ax, color = "blue")
# Adjust plot characteristic parameters
...Here comes our final plot of this project, nice and clean:
Thanks for spending time reading this blog! You finally make it to the end~ 😎 😎
Making stock plots with python is a lot of fun and I hope you enjoyed this whole process. Here are some of the reference examples of this project: Example1 Example2
Welcome to visit my github page or visit my github profile ! 😃 😃