Help me decouple data format from data display
HonkingGoose opened this issue · 14 comments
Problem
Using Markdown tables to store this data is not really the best way to do things.
The main problem is that each time the table updates, everybody else must to re-do their work.
This is not really motivating for me or others to actually contribute.
So I probably need to switch to a different type of data store.
I know that there's YAML and JSON formats which we could use maybe.
The main problem then is having the format be human readable in the form of a table.
Possible solutions
Use some kind of JSON or YAML format, and create a basic website to fetch data from the data store and display it?
Requirements
- Whatever it is, it should cost me no money to host it for this project.
- It should be easy to use, and easy to maintain.
Can use GitHub Pages most likely. Should meet all requirements.
I would also argue that JSON and YAML not much less readable for a technical person. And tables in their current shape don't fit into my screen anyway.
There is another issue: after moving tables into separate repo, automatic upstream issue links are lost. JS that will fetch the data might also handle that.
Can use GitHub Pages most likely. Should meet all requirements.
Yeah that should work for hosting the page.
I would also argue that JSON and YAML not much less readable for a technical person.
The Markdown tables have the benefit that it's quite easy to understand what to do. But once the JSON or YAML migration is done, it should be quite easy to see what you need to add to create a new entry. 😄
And tables in their current shape don't fit into my screen anyway.
By decoupling the data from the display we can fix that as well by styling the table.
There is another issue: after moving tables into separate repo, automatic upstream issue links are lost. JS that will fetch the data might also handle that.
Yes, that's also a negative of the current way things are setup.
However because all links follow the same format, it should be doable to program a small utility program that takes the issue numbers in the JSON/YAML and replaces it with a fully qualified link to the upstream issue.
What do you all think of this JSON file?
The idea is to only allow one item per issue, that item can get updates via pull requests.
There is only one JSON file for all entries, that way there is a single source of truth.
It's important to get this right the first time, as making further changes is a breaking change in the API somebody else maybe builds around this...
I have added some new boolean fields for game_starts, video_has_visual and video_has_audio, as those can be helpful data to have at a glance for the Proton developers (at least I think).
There's also a field that takes a short Proton log snippet if that is relevant to the report.
Let me know if this seems OK, or if I'm barking up the wrong tree with this.
Any edits/corrections/tips are most welcome!
{
"array": {
"items": [
{
"issue": 1,
"frameworks": ["gstreamer"],
"proton_version": "5.13-1",
"codecs": ["mpeg-1"],
"game_starts": false,
"video_has_visual": false,
"video_has_audio": false,
"proton_log_snippet": "",
"remarks": ""
},
{
"issue": 2,
"frameworks": ["quartz"],
"proton_version": "5.13-1",
"codecs": ["unknown"],
"game_starts": true,
"video_has_visual": true,
"video_has_audio": true,
"proton_log_snippet": "",
"remarks": ""
}
]
}
}Hi @aeikum,
Can you tell me what data fields and data types you need? I made a basic starter JSON template just above this post.
I'm thinking of migrating the data to a single .json file. Somebody else can than use that .json file as input for their website/parser. Because I'm then somebody's API, I need to get this right the first time, as I don't have much liberty to make breaking changes to the data format later.
I hope I'm not bothering you too much with this. I I'm bothering you too much, please tell me, and I will stop badgering you. 😉
Greetings,
HonkingGoose
I don't know that we need all that much infrastructure. Just a plaintext list of titles affected is fine, our QA and devs have used it as a list of titles to check on the status of video support and find new bugs to fix.
Thanks for responding @aeikum! 😄
Nice to hear that we don't need a lot of infrastructure. 😄 😌
Just a plaintext list of titles affected is fine.
So you want something like this then?
https://github.com/ValveSoftware/Proton/issues/44
https://github.com/ValveSoftware/Proton/issues/90
https://github.com/ValveSoftware/Proton/issues/137The filename would then denote the framework used, so the quartz.md file covers Quartz games, etc.
Let me know if this is/isn't what you want, and I'll go rejig the files one last time. 👍
What you have here: https://github.com/HonkingGoose/proton_codecs_media_support_tracker/blob/main/tables/quartz.md looks fine to me. The only change I'd make is to make the issue numbers into links back to the issues in the Proton tracker (please don't do this by hand, it is easy to automate).
Apologies if I am misunderstanding what you are asking from me.
The only change I'd make is to make the issue numbers into links back to the issues in the Proton tracker (please don't do this by hand, it is easy to automate).
Well let's keep the tables then, and see where we run into issues later. 👍
How do you recommend that I automate the links, do you have a quick-tip or direction I should follow?
Use a programmable text editor! In vim, this command will swap all of the issue numbers in the file with links:
%s!^| \s*\(\d\+\)!| [\1](https://github.com/ValveSoftware/Proton/issues/\1)!
It matches any line beginning with | (bar-space), and replaces the digits it finds there (\d\+) with the issue link, replacing the digits in the text and the URL with the digits that it found (\1).
You can find more info searching for "vim search and replace" and "vim regular expressions".
I'll go try that! Thank you! 😄
Closing this for now, as the main developers at Proton are happy with the current table format, so I see no urgent need to fix this.
We now also have links back to the upstream project in the files anyways, so that part is fixed now (see pull #16 ).
Thank you Andrew for your help with the GitHub issue links! I've credited you in the commit that introduced the fix.
We can revisit this issue if we run into actual problems with the contributing workflow.
Oh, and thank you for your input @KillyMXI! Even though I'm not using it for now, thank you for trying to help! 😄
No problem.
While it definitely seems like overkill, I think it wouldn't have taken much time to make a simple page and GitHub Action for anyone with a bit more relevant experience than me. I would've given it a spin just to have something like this done, if I weren't trying to free up my schedule and focus...
And I just realized GitHub has another tool to group relevant issues - Projects.
A project for media support can be created in the Proton repo, with tables per framework...
Although I'm not sure how people can add or request issue addition to a board.
While it definitely seems like overkill, I think it wouldn't have taken much time to make a simple page and GitHub Action for anyone with a bit more relevant experience than me. I would've given it a spin just to have something like this done, if I weren't trying to free up my schedule and focus...
Yeah making the simple page and GitHub Actions should be something that can be done quickly by somebody with proper web-dev experience.
I know that Vue/Angular/React have a component system. It could be as "easy" as plugging in the Markdown tables into separate components, and then loading those components at static page build time.
Of course having structured data (like JSON or YAML) would be better for actually making those components, as then you can assign variables to the different names. But like I said, until the upstream Proton project is shouting at me that I'm doing it wrong, let's keep the Markdown tables. 😄