Summary AND Description to App DB

Question

Summary AND Description to App DB

Opened this issue 4 months ago · 9 comments

Foxcapades commented 4 months ago

Problems

Presently:

We are only writing the description to the App db control table for a given dataset.
The current description column in the App db is limited to 4k characters.

Proposal

Change the App DB schema: (by @steve-fischer-200)
a. change the type of the description column to CLOB through some series of changes
b. add a new summary column typed as VARCHAR2(4000)
Change the REST API to reject summaries greater than 4k characters in length.
Change the APP DB lib to insert into the new column
Update the client to disallow summaries > 4k in length. - may require @dmfalke

Concerns

Existing summaries greater than 4k length?

Followup

Get the summaries for currently installed datasets into the app dbs, VDI won't know to do this on its own.

Answer 1 · 2024-09-04T16:09:36.000Z

@dmfalke, if you want to tackle the client change, it isn't blocked by anything on the backend, that can go out ahead of time or whenever.

The only issue with it happening after the service change is the UX for when a user gets a 4xx error for the summary being too long.

Answer 2 · 2024-09-04T16:23:34.000Z

Is the length of description unbounded?

Answer 3 · 2024-09-04T16:24:47.000Z

Yes, description is already unbounded in VDI, and is going to be changed to CLOB in the App DB.

Maybe we could have some sanity check limit in place though? 🤔 Like, is there a use case for more than 1M of text for example?

Answer 4 · 2024-09-04T16:26:51.000Z

Maybe we could have some sanity check limit in place though? 🤔 Like, is there a use case for more than 1M of text for example?

We should probably only do this if we know we want to 🙂. Maybe something to consider as user datasets become more central.

Answer 5 · 2024-09-10T20:51:40.000Z

PR for VDI based changes #335

Answer 6 · 2024-11-26T16:53:49.000Z

@steve-fischer-200 did we update the summaries/descriptions for existing datasets in the app dbs? I remember some work being done on this but I don't remember if it was finished.

Answer 7 · 2024-11-26T16:54:00.000Z

@dmfalke have the client changes been made for this already?

Answer 8 · 2024-12-05T19:34:01.000Z

@dmfalke have the client changes been made for this already?

Nope. I have a draft PR that I haven't looked at in a while. I can resurrect it, if needed.

Answer 9 · 2024-12-12T22:09:49.000Z

Nope. I have a draft PR that I haven't looked at in a while. I can resurrect it, if needed.

IDK what the priority on it is/was, I'm pretty sure the backend change is done and in on this already and was wondering if this ticket should be closed.

Maybe the client handling of the response 422 error is good enough?