Q: Is the API behind the webapp open for integration?
Closed this issue · 4 comments
Hi there,
Not an issue, just a curious question. In the aim to not re-invent the wheel, I've noticed you are using ElasticSearch, are there any plans to wrap it in an API for integration?
Thanks,
Francisco
Hi Francisco, currently there are no plans on putting Elasticsearch behind an API. As ES is an API in itself it would be more likely we we expose ES behind a security layer providing authentication and rate limiting, etc. At least until a time when there was a known set of common queries that might make sense to expose as an API endpoint for multiple consumers to make use of.
If you want to use the same (or similar) setup to what we are doing I can explain that in a bit more detail...
When the application is deployed it connects to an ES cluster. That cluster contains data injected on a schedule from the elasticsearch-updater application.
The data the updater injects (gp-data-merged.json) is combined from the publicly accessible 'raw' files hosted in Azure by the profiles-etl-combiner application.
The applications scraping the data sources and generating the pre-combined JSON files are:
- https://github.com/nhsuk/pomi-data-etl
- https://github.com/nhsuk/gp-data-etl
- https://github.com/nhsuk/pharmacy-data-etl
Which is a quite a few moving parts. Ultimately, if you wanted to run ES you would need an instance of ES running into which elasticsearch-updater should be configured to inject data into it.
Hope that helps.
Steve.
P.S. I'll leave the issue open if there is anything else you would like to add to it. When you have the information you need can you close it please? Thanks.
P.P.S. If you want to discuss this any further and feel GitHub issues aren't the best place you can get in touch with me on Twitter @st3v3nhunt or email shunt7 at nhs.net
Hi Steve,
Thank you very much! for your reply, it has been really helpful.
I guess my last of question would be licensing and availability of raw files, e.g.: gp-data-merged.json ? Any restriction I should be aware of?
The data for the raw files comes from either NHS Choices Syndication feeds or NHS Digital's indicator portal. You are best consulting the terms and conditions of those sites for how the data should be handled.
It is worth noting the files hosted in Azure are not supported in anyway and might change in structure or even be removed at any point without any notice. This is unlikely to happen in the immediate future but it is a possibility.
The files are very much available only as a consequence of our open development policy, they are not part of a strategic policy. However, as you can see with the NHS Choices Syndication feeds, there is a strategic policy of opening up the data and making it available to third parties. Ideally, what you see here will at some point become part of that. Whenever possible myself and my colleagues are looking at ways of feeding into the strategic platform with artefacts such as these application and files. And with use cases where we know these artefacts have been used.
Thank you very much again, really appreciated.