Restructure the API endpoints
Closed this issue · 1 comments
Note
This ticket aims at suggesting a structure of the API in the long term, particularly when the Data Connector Server will be its own server, separated from the Private Data API (fetching any data, LLM, search, etc.).
In the short-term, we can take shortcuts and leverage the data API endpoints instead of having dedicated endpoints (for the logs for instance)
Suggested long-term API structure
Considering:
BASE_URL: "http://domain.com"
API_URL: ${BASE_URL}/api/rest/v1
(Using /rest
to be future-proof, in case a /api/graphql
endpoint is necessary)
Data Connections API endpoints
All these endpoints are available under API_URL
Providers
GET /providers
Get all the provider definitions, include the handler definitions
Provider
GET /providers/:providerId
Get a given provider definition
Connections
GET /connections
To get all the connections. Could be a POST as the query (filter, sorting, pagination) is better in the body than in query/search params.
GET /connections/watch
Event Source endpoint. Get updates on all the connections, particularly theirs status. Could be a POST as the query (filter, sorting, pagination) is better in the body than in query/search params.
POST /connections/sync
Sync all the connections at once
GET /connections/logs
Get all the existing logs. Could be a POST to pass a query in the body.
GET /connections/logs/watch
Event Source endpoint. Listen to new logs for all connections. Could be a POST to pass a query in the body.
Connection
GET /connections/:connectionId
Get a connection object by its id
PUT /connections/:connectionId
To update the configuration of the connection. Not all properties of the connection object can be updated though, so the controller will have to carefully extract the allowed field from the body.
DELETE /connections/:connectionId
Disconnect a connection
POST /connections/:connectionId/watch
Event Source endpoint. To listen to updates on the connection
POST /connections/:connectionId/pause
To pause a connection
POST /connections/:connectionId/resume
To resume a paused connection
POST /connections/:connectionId/sync
To trigger the sync operation of the connection. Doesn't wait for the sync to finish to respond, doesn't return anything apart from whether triggering the sync was successful. The actual result of the sync can be checked via the status of the connection (also the /watch endpoint to listen to changes on the status)
GET /connections/:connectionId/logs
Get the logs of a given connection. Could be a POST to pass a query in the body.
GET /connections/:connectionId/logs/watch
Event Source endpoint. Listen to new logs for a given connection. Could be a POST to pass a query in the body.
Data Connections other routes
These routes are available under BASE_URL
, not API_URL
(/api/rest/v1
) because they are not API endpoints, but are actual pages opened by the browser.
/providers/:providerId/connect
To connect to a provider using their authentication/authorisation mechanism (OAuth)
/providers/:providerId/callback
Passed to the provider OAuth mechanism to get the information back and creates the Connection object
Short-term needed updates
In the short-term, the /connections
, the /connections/logs
, the /connections/:connectionId
, and the /connections/:connectionId/logs
endpoints are not needed as the generic /db/query
and /db/get
endpoints can be used instead.
The /.../watch
endpoints to listen to changes would be nice but not a priority as we can refetch the data occasionally.
The /providers/:providerId
is also not needed as there is not a lot of providers, so /providers
is enough for now.
The current GET /sync
sure works, even though a POST /connections/:connectionId/sync
would be more explicit and more RESTFUL.
Same for the GET /provider/disconnect/:provider
, which works but would be more RESTFUL with as a DELETE /connections/:connectionId
.
The only actual current need is the ability to update a connection as well as pause/resume it. When updating we have to be careful on what should be excluded from the update (providerId, handlerId, status, etc.) and what can be updated (basically just the config/options).
Immediate tasks:
- Refactor to use
/api/rest/v1
- Add
/db/watch
and/ds/watch
EventSource endpoints - Move
/providers/:providerId/connect
and/providers/:providerId/callback
outside of/api/rest/v1
and to/
- Move
GET /sync
toPOST /connections/sync
- Add
PUT /connections/:connectionId
to update a connection (only some properties) - Move
GET /provider/disconnect/:provider
toDELETE /connections/:connectionId
- Delete
/sync/logs
- Move
/sync/status
toGET /connections
- Add
GET /providers/:providerId
- Add
POST /connections/:connectionId/sync