Obfuscated code
Closed this issue · 9 comments
Appreciate the rewrite for this @shuckster. Just worried that with the new code being obfuscated, it's very difficult to see what's going on. Since this scrapes bank details, it would be good to be able to see what the code does.
Also, it's not immediately clear what page one should run the bookmarklet on.
Good points @masterell . All non-obfuscated source lives in src/, and I put together a (very barebones!) ARCH.md document that describes the build-process for turning it into dist/.
Still, since the code is run from dist/ it would indeed be a better default to leave it unobfuscated, especially as it deals with sensitive data. I'll make a global MINIFY_DIST variable for that so we can easily toggle minification on-and-off.
Unfortunately, CORB may mean that OBIS needs to be run locally from now on. This is a bit of a pain, but it does mean that you have to download the full unobfuscated source before running it.
As for the page to run it on, it doesn't matter anymore. I usually run it just from the summary page.
Yes CORB does make this a bit more difficult for the average user. I'm not particularly familiar with Chrome Extensions, so not sure how portable this is, but would this not lend itself quite well to one?
In addition to the obvious cross-browser problems this will invite, I guess the project will have to deal with the occasional breaking extension-API change. Still, it would be a considerable usability improvement if it can overcome the CORB issue.
I have thought many times about turning this into Chrome Extension. This might finally be the motivation to do it.
@masterell - I have a WIP extension now in the dist/ folder.
I've not added it to the Chrome Web Store yet, so if you'd like to try it out follow these steps:
- Navigate to:
chrome://extensions/ - Enable
Developer mode - Click
[ Load unpacked ] - Choose the
dist/extension/folder
The code is left unminified, same as the bookmarklet. The extension is integrated into the build-scripts and will be produced along with the bookmarklet if you'd like to DIY it.
Turns out there's a "Web Extensions" standard I wasn't aware of, and this means that it should work cross-browser. Unfortunately, while the UI loads in Firefox no statements will download, so it's Chrome-only for now.
@shuckster Truly superb. It's a really sorry state of events where someone has to write 12,000 lines of code just for functionality a bank like HSBC should be providing their customers with, but here we are I guess. Really appreciate all the effort and time that has gone into this. The Chrome extension is superb and I think would greatly help a lot of people. It would also appeal to less tech savvy users who may be put off by having to run a local server.
Just as a suggestion for future versions - the checkbox functionality in earlier versions to specify which statements are downloaded and in what formats was very helpful. Currently you download all the statements for a given time period (min 1 year) in every possible format. That said, what you've done here is faultless and I can assure you it's greatly appreciated.
Thank you @masterell , and I'm glad to hear that the extension worked for you. I'll try and get it onto the Chrome Web Store. I've read a little documentation about it now and it requires a developer-account and pre-release curation on Google's part. Not huge hurdles, and actually a positive for less technical users (so long as it gets accepted of course!) I do have my fingers-crossed that HSBC don't notice it... But then, they haven't noticed OBIS for 8 years!
Speaking of which, you put it very well: It is crazy that something like this is necessary at all in 2021 to duck-punch missing functionality. In fact, in the time since the first-commit we seem to have gone backwards: HSBC offer fewer options now for downloading statements than they did back then!
Concerning your suggestions, thank you: I'll put the file-formats checkbox on the TODO list.
As for month-by-month date-ranges, I set them to years since that's seems to be the lowest granularity that HSBC's API accepts. Yes, believe it or not OBIS uses HSBC's own API to pull-down statements. This is why it works on any page and is faster than before. But the only query-parameters accepted are 'Latest' or a range of years.
It's quite possible to do some post-processing on the OBIS-side to offer month-by-month though, so I'll add that as a TODO.
I must admit I'm a bit reticent to do too much more on that side for HSBC, though. The reason is I'm not overly confident that they will keep this API around for much longer. Miraculously, I documented the API early last year, and then shelved the rewrite. Since then HSBC have removed all reference of it from their new PDF statement download pages. If I tried writing OBIS today the same way I couldn't do it. Imagine my astonishment to discover that, after the rewrite, the scraps of documentation I pieced together actually worked!
I hope it works for some time to come - corporate cogs move slowly. But OBIS has broken more than once in its history.
@masterell - I've managed to get the extension on the Chrome Web Store. I thought it would make sense to keep it "private" until it's tested. If you'd like to do that, let me know and I'll add your email to the testers list. There are no functionality changes, so no worries if you'd rather not.
@shuckster That's the most flabbergasting point of all - that OBIS merely makes an API request and processes the response. Yet all HSBC can offer their users is a lousy PDF statement. Having it available on the Chrome web store is a double edged sword. Yes on the one hand HSBC may notice it, but on the other hand - perhaps it'll wake then up and realise there is demand for it. Perhaps one day they'll even incorporate this functionality themselves! I've sent them enough emails on it.
Happy to be added to the tester list. Will have a play around when I get some time.
Super, thank you. I'll add you to the tester list @masterell . I do hope the same as you, and that one day we won't need OBIS anymore. Fingers-crossed on that one!