Request fails for more than 10 tickers at a time
Closed this issue · 8 comments
Code to reproduce:
mnem = ['U:DY','@MSFT','U:NAV','U:C','U:JPM','U:IR','U:NEE','U:FE','U:BAC','U:JNJ','U:WMT','U:PH','U:F','U:DIS','U:OI','U:GE','U:NR','U:AXP','U:AOS','U:MRO','U:NKE','U:L']
conn.fetch(mnem,['RI'],'2015-10-01')
I have fixed the issue on my cloned repo. Please either allow me to push into a separate branch to create a PR or change line 331 in pydtastream.py to
fields = [re.sub(r'(_\d*)', '', x) for x in fields if not any([y in x for y in meta_fields])]
Hello @asuhas and apologies for the late response.
Unfortunately I don't have access to TR DWE at the moment, so I could neither check the issue nor test the solution that it does not break anything else. From just looking at the function and trying to remember how the response looks like, it suggests that the suffix in the line 285 should be fixed instead. But as I said, I have no means to check this now, so I will keep the issue open until I could verify the fix.
BTW, are there any problems in creating PR?
Hi,
Thanks for your reply. The issue is that the suffix is created correctly, but when you debug the line I am proposing to change, you will realize that it substitutes the number(in the suffix) with an empty string. So when there is a request with <=10 tickers it works as intended because it will make FIELD_1 into FIELD but when you have FIELD_11 it will generate FIELD1 because suffix is _1. This throws the exception.
My proposed solution achieves this by using regex to sub out any number with a lagging _ char. with blank, which works on my local machine (I've written a test suite around the library which I could share)
As for the PR issue, I managed to get it working. Some issue with my cred manager locally.
Think I might be running into the same issue, but unfortunately @asuhas fix is not working on my side.
Trying to call DWE.fetch for a number of tickers and fields and I'm getting the following error:
Example of my call:
code = ['R:AGLJ', 'R:ABGJ', 'R:AMSJ', 'R:APNJ', 'R:BATJ', 'R:BILJ', 'R:BVTJ', 'R:CPIJ', 'R:DSYJ']
sort_list1 = ['WC08001', 'WC01251', 'WC01651', 'WC02005', 'WC02051', 'WC02201', 'WC02649', 'WC02999', 'WC03101', 'WC03151', 'WC03255', 'WC03351', 'WC03495', 'WC03995', 'WC04201', 'WC04551', 'WC04601']
data_in1 = DWE.fetch(code,sort_list1 ,date_from="2005-12-31" ,date_to="2019-05-05" ,freq='Y')
Any ideas?
Hey @MikeMccree ,
I've been working with the changes that I proposed and can confirm that they are working. Would you mind sharing what you changed? Alternatively look at Pull request 20 and test it yourself.
Hi @asuhas, thanks for the reply.
I made the exact changes you suggested in pull request 20 and used the code above I pasted as an example. Is it working on your side (My example)?
Yes. The issue with your example is multi-fold. First, I couldn't find 'R:BILJ' as a valid ticker. Second, it fails because some of the tickers you have requested either
- a) don't have data for one of your fields
- b) don't have data going back as far (B-date issue)
- c) You don't have permission to view them
This request worked pretty well for me:
code = ['@AAPL','@FB','@NFLX','U:AMT','U:GE','@GOOG','U:WMT','U:PFE','U:CVX','U:KO']
sort_list1 = ['WC08001', 'WC01251', 'WC01651', 'WC02005', 'WC02051', 'WC02201', 'WC02649', 'WC02999', 'WC03101', 'WC03151', 'WC03255', 'WC03351', 'WC03495', 'WC03995', 'WC04201', 'WC04551', 'WC04601']
x=conn.fetch(code,sort_list1,'2013-01-01','2019-05-05',freq='Y')
The new version which uses the DSWS API instead of discontinued DWE is already in the pypi.
The original issue does not appear for the new library, and the code
mnem = ['U:DY','@MSFT','U:NAV','U:C','U:JPM','U:IR','U:NEE','U:FE','U:BAC','U:JNJ','U:WMT','U:PH','U:F','U:DIS','U:OI','U:GE','U:NR','U:AXP','U:AOS','U:MRO','U:NKE','U:L']
conn.fetch(mnem,['RI'],'2015-10-01')
works fine.
I'm closing this issue, but if the problem still occurs - feel free to reopen it!