Suggestion: Remove table-views-functions.org file from repo
Closed this issue · 3 comments
Historically, we wrote the snoopdb in a "literate" fashion. All the sql was defined in the file table-views-functions.org. We also defined all our python utility functions in this same file.
With the wisdom of time, I no longer think this is a good idea. For one, we are committing both the org file and the tangled output of this file. This makes searching the repo for a function or a call hard, as there is noise of all our org files interfering with the actual code references. It also means that PR's require comitting both the org file and the files, and so every function gets reviewed twice. It's noisy. It also makes it hard to collaborate. If you are not diligent and writing new code only in the org file and tangling it out, then there will be drift where the "tangled" files contain newer code than what's in the authoratitive "literate" document.
One solution would be to have git ignore any of our generated files and commit only the tables-views-functions.org. I don't think this would be a good idea, though. For one, it would mean that the CI job that builds new images of snoopdb would need to be able to run org-mode tangling functions before it could build the image. This is non-trivial, but could likely be done.
More importantly, though, it is not as pleasant to code in the org file as it is in the actual sql and python files. Syntax highlighting doesn't always work and the indentation changes depending on someone's emacs configuration, meaning you can accidentally indent all the files in yr commit if you don't tangle just right. Yolu also don't get lsp support, and the experience is notably slower than editing in the actual mode of the language yr writing in.
Because of this, I think we should abandon the literate approach for our files and remove this file from the repo so it is easier to find the relevant code we need.
I think literate programming has a better place in explorations, onboarding, and documentation. I think once things get as evolved as this it's likely fine to drop the literate style.
Would there be a documentation value to update the table-view-functions.org as an index with working hyperlinks to the previously-tangled files?
i think the initdb directory, and the comments in the sql files, acts as the best TOC and documentation.
Removed this file in #714