Investigate Docusaurus (revisited)
Opened this issue · 38 comments
Prerequisites
- I have written a descriptive issue title
- I have searched existing issues to ensure the feature has not already been requested
🚀 Feature Proposal
Related: #100 and fastify/fastify#722.
I propose revisiting migrating the docs to Docusaurus, especially after significant improvements to their tech since previous discussions.
I would love to draft up a proof of concept for the team if there's serious interest, and help with efforts to migrate if the demo proves viable.
Motivation
My team recently spent some time investigating alternatives to Express and some elements of the Fastify docs gave us pause. The most pressing for us was the lack of order and headers in the sidebar which made getting started more cumbersome than necessary. Additionally, something about the "system defaults" approach to Fastify's docs makes the site feel less professional than, say, Create React App or React Redux.
I think Docusaurus's default styling, creature comforts, and additional features make migrating a compelling option. Recent (since #100) support for markdown will make migrating far less troublesome than it once was.
Potential benefits:
- Order pages in the sidebar
- More intuitive table-of-contents (potentially subjective)
- Separating the table of contents and sidebar could prevent confusion on what's a page and what's a page's content (and allows for more nested docs)
- Improved code blocks
- Copy button
- Built in titles
- Better styling (subjective)
- Better docs-wide styling
- Large number of ready-to-use components (ex. tabs, admonitions)
- Dark mode
- "Trendier" default design
- Familiarity with developers
- Docusaurus is widely used across the React ecosystem and, for me at least, instills a degree of confidence when I see it
- Existing familiarity translates to comfort and ease of use
- Internationalization support
- Adopting this feature would obviously incur many additional challenges, so it's benefits are diminished
- Off-load documentation maintenance to a well-established industry standard
- Using a tool such as Docusaurus could save the core Fastify team some dev time
Example
No response
In general, I think the fastify website tech stack is out of date and need an upgrade.
I think the "must have" features are:
- Fastify's version dropdown to see the right documentation based on the version (not the last one only)
- The fastify repository's
docs/folder is the main source of truth
Nice to have:
- runkit integration (I saw it somewhere and it is just amazing)
- site preview on PRs
I'm not against any new framework (Docusaurus, Gatsby, GitBook or others), I prefer the one that fits the fastify needs.
EDIT: forgot to say: we need someone that would like to push this initiative. I'm happy to give feedback, but I don't have a time slot to help implement this
I recognize the problems in the documentation and current website. We truly need somebody that will champion it in the long run.
This is a significant endeavor. I'm not convinced in moving to Docusaurus is the right solution as it felt too "strict" in the past with how they wanted docs.
My company is adopting Fastify and I would love to do what I can to lead new documentation improvements (content, design, infra, etc).
I suppose my next question is what do you feel would be the best next steps? I'm confident in Docusaurus as the best tool for the job though I could draft up some comparisons if it'd be beneficial.
I can build out a proof of concept for new docs as a jumping off / talking point for further conversations on what this would entail.
Edit: made it a bit more clear that I'm interested in leading documentation improvements.
With the recent addition of support for guides in the documentation I intend to do a reorganization. I just haven't had time to get started yet. Might have some time this weekend. Not sure.
As for the proposed package: I would rather stay far away from Facebook tech.
Regarding the Facebook tool: I'm generally all for anti-facebook sentiments but I think we should pursue the best tool for the job. Avoiding Facebook also means avoiding Next, Gatsby, and React.
That said I have some ideas for alternatives:
- GitBook
- Really solid from a design & usability standpoint
- It's proprietary and doesn't leave a lot of room for customizing (they've changed pricing models in significant ways in the past)
- Flip side of that is they handle all the tech & deployments which means even less developer maintenance
- Next.js with additional code to handle markdown and styling
- Still uses Facebook tech (react)
- Allows for pretty much whatever design / functionality we want
- I personally have a lot of experience with the framework
- The MUI docs are an incredible example of this approach
- VuePress
- I've heard good things and it's used frequently in the Vue community, but as I have almost no experience in Vue I wouldn't be able to assist here
It seems to me that next.js and vue could end up requiring too much developer time, which would leave GitBook and Docusaurus as popular feature-rich options. I prefer the open-source nature of Docusaurus to GitBook, and I like the Next.js solution if the engineering time ends up being low enough.
Avoiding Facebook also means avoiding Next, Gatsby, and React.
This is completely acceptable to me.
I'm not convinced we need to change the tooling. I will try to put together my proposal in short order. It amounts to:
- Split
/docsinto/docs/guidesand/docs/reference - Add
/docs/index.md - Add
/docs/reference/index.md - Move appropriate docs into the correct locations
A subsequent PR(s) would address the actual text of the documents. All reference docs should match the style of https://www.fastify.io/docs/latest/Encapsulation/ and https://www.fastify.io/docs/latest/Hooks/ .
Hi, I've been directed here from fastify/fastify#2877. The key focus of my comment there was improving the integration of TypeScript documentation so that it is alongside and "equal" to the JavaScript documentation, rather than in its own separate page.
One of my suggestions as part of this improvement was implementing toggles for code examples between JS and TS, citing Redux Toolkit's docs as an example, which I've since learnt actually uses Docusaurus. I know @Xhale1 has mentioned improved code blocks as one of the benefits, and I see this toggle functionality as the next-step on top of that and another pro for migration.
In terms of the React ecosystem, at this point it simply has the best tools for frontend development, and I believe migrating to Docusaurus would ensure the tech supporting Fastify's documentation is future-proofed. I've also just checked and found that Metalsmith hasn't pushed any updates since 2016.
As someone with a fair bit of time on my hands at the moment, I'd be keen to work on a Docusaurus project for Fastify - let me know if this is something you'd be interested in.
I feel strongly that avoiding react because it't facebook tech is a poor decision. The same logic could be applied to avoiding nodejs, typescript, github, npm, yarn, or a great many leading tools that make developer lives so much better.
A massive amount of the top tech sites in the world use React, and a healthy majority of top open source projects use Docusaurus. The project is fully open source and we could fork it to avoid facebook influence if necessary.
Adopting an industry standard tool makes first-time contributions far easier, and can ultimately let us focus on writing incredible documentation instead of re-inventing the technology itself. The benefits we would get out of the box seem extremely compelling to me, and they seem compelling enough to a large number of other open source projects.
I greatly appreciate the work that has gone into the existing documentation site and the Fastify ecosystem as a whole. That said, I believe docusaurus could benefit the community in a number of ways and is the best tool for the job.
Examples of incredible tools using docusaurus:
Edit: I want to readdress and refocus on the core pain points I think we're looking to address here.
Documentation features
Improving the in-house implementation will inevitably lead to a game of catch-up with tools dedicated to docs, and it will pull developer focus away from other parts of the Fastify ecosystem. Existing dedicated solutions give us new features to write amazing docs out-of-the-box, for free. We only pay the cost of migration.
Time commitment
Dedicated tools bring with them their own knowledge base and documentation on how to use them. Adopting a dedicated tool means our community benefits from the existing knowledge on how to use it. Docusaurus's own docs, for example, lower the barrier to entry to working on any docusaurus site and alleviates the struggle of learning the quirks of an home brewed system. In my opinion community members wanting to contribute to improving the docs site would have an easier time with docusaurus or similar than they would with the current system.
There is a fundamental issue with docusaurus: versioning. Versioning in Docusaurus is very far from how things should be structured. Docusaurus support this as an afterthought by keeping all the data on the same repository (ref https://docusaurus.io/docs/versioning). The approach that this website currently has is great for this as it allows multiple versions to coexists (they are assembled from different releases).
IMHO we should just move this to elevently. It's more up to date than the current stack but it's still true to the same principles.
Note that the majority of the work is needed on the documentation itself.
Note that the majority of the work is needed on the documentation itself.
This is correct. Unfortunately, I was just too out of it over the weekend to work on https://github.com/fastify/website/issues/291#issuecomment-971901732 (I started a local branch and that's as far as I got). Thursday and Friday are a holiday for me, so I hope to get some work done at that time.
In terms of the React ecosystem, at this point it simply has the best tools for frontend development, and I believe migrating to Docusaurus would ensure the tech supporting Fastify's documentation is future-proofed.
This statement is a matter of opinion. If you ask the folks that like Vue, Marko, or other things, they would say the same sort of thing about those technologies.
I feel strongly that avoiding react because it't facebook tech is a poor decision. The same logic could be applied to avoiding nodejs, typescript, github, npm, yarn, or a great many leading tools that make developer lives so much better.
It is my opinion that using Facebook tech is tacit approval of that company's practices and tactics on a myriad of issues around privacy and data collection. I am but one vote amongst many, but my vote will always be against using Facebook tech.
Adopting an industry standard tool makes first-time contributions far easier, and can ultimately let us focus on writing incredible documentation instead of re-inventing the technology itself.
On this we agree. I just want to see a different technology used to fill that role. Matteo is suggesting https://www.11ty.dev. I use https://gohugo.io for my personal sites and find it fantastic.
Remember, fastify.io is more than just documentation. It is a full website that needs to support things like fastify/website#194 in addition to the documentation. The requirements set forth by https://github.com/fastify/website/issues/291#issuecomment-971494581 are what should be focused on in any project that seeks to update the website generation. For example, when I do get my proposal PR up for the documentation, I will have no way of knowing if it works since we do not have a way to generate test sites from PRs to the main repository.
Just a few thoughts on the versioning issue in Docusaurus - wouldn't it be possible to just use the current strategy of clone all releases, and then copy the docs of old versions into /versioned_docs, and the latest release into /docs?
Also to clarify my view, I would like to see something more than the current site's features, including those mentioned by @Xhale1 up the top of this page. Docusaurus seems to be the easiest way to get these out-of-the-box and with less manual work and also has wide adoption. However, if you guys would be more comfortable with something like 11ty or Hugo, I'm not fussed as long as they can be expanded to add this functionality.
I'm also just wondering about the benefits of having separate doc versions for each minor release - most sites I've seen just split it into major. Is there a reason why this format is used on Fastify's website? I'm updating on v3 whenever I notice a new version, and haven't had any breaking changes (yet).
Thanks in advance - this is the first time I've been invested enough in an open source project to contribute to discussion, and I'm really looking forward to seeing how Fastify grows :)
Please see fastify/fastify#3474 for my proposal as a starting point.
I'm also just wondering about the benefits of having separate doc versions for each minor release - most sites I've seen just split it into major. Is there a reason why this format is used on Fastify's website? I'm updating on v3 whenever I notice a new version, and haven't had any breaking changes (yet).
Personally, I would be willing to cull all of the old documentation and stick with the current supported LTS releases. Either at the semver major level or only the most recent release in the LTS line. It would greatly simplify a lot of site generation logic.
I fully agree with @lachlancollins on not being attached to docusaurus, whatever tool gets us the best docs would be great. Apologies if I seem like I'm just pushing a single tool.
Regarding versioning: I also agree that we should drop the patch (and I think minor) versions of the docs. I propose having 1.x, 2.x, 3.x, and next (or similar). This is also the way docusaurus is built to handle versioning, with a next branch and then a set of docs for each major version. I actually really like how they handle it, but to each their own.
I also want to throw Vue Press into the mix for consideration as an alternative tool. I don't have very much experience with it but it's used extensively in the vue ecosystem.
I think the best path forward is to start new repository, say new-website, that starts from scratch (but imports all necessary resources) as a proof-of-concept. This repo could then be promoted to the main repo upon completion.
This new project must:
- Follow https://github.com/fastify/website/issues/291#issuecomment-971494581
- Support robust cross referencing as I outline in fastify/fastify#3474 (comment)
@jsumners I found the cross-referencing you mentioned in Hugo very cool and agree it is essential - I checked and found that Docusaurus also checks for broken links and fails with errors during build.
In terms of the framework of choice for new-website, I'm a bit uncertain on the verdict at this point - is Docusaurus entirely ruled out as an option because it is part of the React ecosystem?
EDIT: Another thought has come to me, to use Hugo's cross-referencing would we need to use its syntax in the main repo's /docs folder? I know @Eomm mentioned that this folder should remain the primary source earlier.
This new project must:
- Follow Investigate Docusaurus (revisited) #291 (comment)
- Support robust cross referencing as I outline in Start of docs reorganization fastify#3474 (comment)
I'm not sure this fully covers the grievances listed here.
It seems to me that the consensus is that the current Fastify docs tech stack is out of date and would require too much dev work when existing solutions are so readily available and prevalent in the open source community.
Some of the headlining issues appear to be
- Code blocks lack popular features
- ex. tabs to allow switching between js and ts, npm and yarn, etc
- Ordered sidebar
- Easier for first-timers to contribute
- Easier to maintain the tech stack
- Improved deploying workflow (seems this could be handled rather well by a host such as Vercel or Netlify, regardless of framework)
- Automatic checks for broken links / assets
- Styling improvement / face lift (this is a personal opinion, not gonna hold to tightly to it)
- Better nesting support for docs pages
- ex. a dropdown for ecosystem which lists some popular plugins and a quick guide on how to use them would be neat
Additional issues related to the docs in general:
- Better organization
- Only keep versions for the latest LTS / major versions
Please let me know if I'm missing something or if you believe one of these issues to be irrelevant.
That said, I again want too point out that it seems to me that Docusaurus would solve every issue here while also bringing additional improvements to the table, all without the needed developer lift of engineering new in-house solutions. The primary issues listed here with docusaurus are its versioning system and that it's owned by facebook. It seems that the versioning problem wouldn't be an issue for us and I'm not sold that avoiding tech due to its parent company is reason enough to avoid a solution if it provides enough value.
Regarding versioning: I also agree that we should drop the patch (and I think minor) versions of the docs. I propose having 1.x, 2.x, 3.x, and next (or similar)
Some users opened bugs for not working features, but the issue was a wrong minor version. I think a good approach would be tracking only major versions, but having a UI that shows something like added in 2.34.0. That would be cool!
EDIT: Another thought has come to me, to use Hugo's cross-referencing would we need to use its syntax in the main repo's /docs folder? I know @Eomm mentioned that this folder should remain the primary source earlier.
I mean: we should use that documents' content, but it is not denied to update them to be processed by the new-site (for example adding markdown metadata).
Please let me know if I'm missing something or if you believe one of these issues to be irrelevant.
Your list is good
I'm not sold that avoiding tech due to its parent company is reason enough to avoid a solution if it provides enough value.
I'm a dumb programmer: right now, I read only PROs for Docusaurus 😄
Many questionable companies pay fantastic people to craft great tools that we are still using today since humanity invented wars. I think it is a matter of "do we use that tool to do good or evil?"
I think the versioning support in Docusarus is so bad that even them are not using it: if you flip that switch at the top it redirects to a different netlify URL. I can only see Docusaurus as problematic long term.
A better approach would be to have:
- a "product" website that includes:
- guides
- ecosystem
- tutorials
- companies using it
- good first issues
- a rendered version of the reference documentation from fastify/fastify for each individual version
This will keep the build time short and free us from using a single technology for everything.
The versioning on docusaurus's site is weird - they only directly build the current (beta 9) and previous beta (beta 8) in the live site. I imagine this is to keep their bundle size small while they iterate through so many small beta releases. However if you look at sites such as https://socket.io/ they don't have such problems, and the solution is very clean in my opinion.
If their specific versioned docs strategy doesn't work for our use case, we don't actually need to use it - it's totally possible for us to just split it into docs/latest/, docs/2.x/, etc. within the one folder, like how the current site handles it.
Would you be willing for either myself or @Xhale1 to at least build a quick demo to try to prove versioning is a non-issue and that docusaurus could be used across the whole website's infrastructure?
I have invited both of you to a repository "website-next".
Having gone through the struggle of working with the current tech stack when implementing the nested documentation, I totally agree with @Eomm. As a contributor I ended up spending a lot time trying to understand the current build process, and I think this can become a huge barrier when seeking help and contributions.
IMO, Jamstack is the go to solution, and as suggested by @mcollina Eleventy would be a good approach. Moving the current site would be less painful than adopting solutions like Next.js, Gatsby (or any other React framework), and having to atomize the whole website. Besides, Eleventy already supports the templating language we are already using (Nunjucks)
As for the linking problem described by @jsumners, I haven't had the time to get hands on this subject, but I already found some documentation in the eleventy page.
Hi @luisorbaiceta, I agree that something like Eleventy is a much closer match to the current site's Metalsmith and would have very few cons - however, from what I can tell it wouldn't actually add much value to the user-facing side without additional work. In terms of Docusaurus, it brings docs enhancements that either work-of-the-box or with minimal configuration, including those mentioned by @Xhale1.
Also, Docusaurus falls within the Jamstack family :) As someone with a lot of experience with Next.js I agree that it would be more work compared to Eleventy, but have no doubt that won't be the case for Docusaurus.
@lachlancollins I haven't had the chance to look at docusaurus, but for what I've read in this thread It has some issues with version handling. Is that correct? Is there any workaround?
@luisorbaiceta yep we have discussed the versioning, and I explained why I'm not worried about it here.
To summarise, https://socket.io/ is a great example of using Docusaurus's versioning system, but there is no obligation for us to use that system, and we can keep organising it into subfolders of /docs/ based on release as the current site does.
Hey @lachlancollins, I kinda jumped the gun and added a first pass of the docusaurus demo over at https://github.com/fastify/website-next. Worked on it while traveling and I didn't have wifi, hence no communication.
It's just the getting-started page for now which ended up being a good demo page. We're able to show off the use of tabs for NPM / Yarn and ESM / CommonJS, code block file naming (sooo nice imo), a long page with a table of contents, and basic formatting.
I think a good next step is getting raw opinions and doing a quick demo of versioning. Thoughts?
Hi @Xhale1, I just realised we have access to the main branch - I've been working on the docusaurus-demo branch and have done quite a bit :) just trying to merge our work now!
@lachlancollins oh no, I'm so sorry, I'll check out your work now. If you want feel free to scrap my stuff (I only put an hour or so into it).
@Xhale1 all good, it seems like you prioritised the config and I prioritised the docs so it worked out well :) just checking that the merge had no problems right now - if you notice anything missing please add it back in!
Everything is building successfully, the big things which haven't been customised are the non-docs pages and the order of docs. Everything else seems pretty small. Should we make an issue/discussion over on https://github.com/fastify/website-next for progress instead of continuing on here? I might write a GitHub Action to let people browse the demo, @mcollina is this okay? If so can you handle the settings changes to enable GitHub Pages?
I have enabled gh-pages. Please open issues on that repo if you'd like to gather feedback about that, this issue is already way too long.
@mcollina final question, what is the URL for gh-pages? https://fastify.github.io/website-next/ doesn't work because it isn't at the root path.
@lachlancollins please open a separate issue on that repo, describe Exactly what you need and let's keep this for high-level coordination/discussion. I'm going to hide this and a few of the previuous posts.
I'm fine with docusaurs, as long as we can have some custom pages like we are having right now.
A plus of docusaurs is that it's a well know tech, and maintaining it shouldn't be that hard.
Still, we don't have experience with it, so if we want to commit to moving to docusaurs, I would kindly ask @Xhale1 @lachlancollins @luisorbaiceta if you are willing to help us maintain this for the foreseeable future.
If not, then move to eleventy and figure out how to fix the link it's the best strategy IMHO.
I can't really commit to a tech I am not familiar with either. I will migrate and help with the scripts and docs processing
Hi @delvedor, I'm in a position right now where I'd be very happy to commit to maintaining the new website with Docusaurus! Also, I'd be more than happy to work alongside @Xhale1 if he's interested, and of course glad to have @luisorbaiceta helping with the scripting :)
Before we continue working on this, is there anyone else who needs to give this change the thumbs up? Thanks!
I can commit to maintaining the docs site as well, super happy to work with @lachlancollins. The nice thing is most of the tech is largely maintained by the greater docusaurus community letting us focus on the docs themselves (which I'm also really interested in improving).
Hi there,
I found this repo through GH's "Used By" section on a metalsmith plugin.
There are numerous elaborate efforts linked to this repo to switch to vitepress, fastify+vite or docusaurus, which obviously provide more out-of-the-box features than Metalsmith. As the late maintainer of the latter (since sept. 2021), I can of course only encourage you to continue using metalsmith and support its development through PR's where you find features lacking or new ecosystem plugins. FWIW I also co-maintain the underlying jstransformer dependencies used by some ms plugins.
For a "quick win update" though (= eliminating security issues, better editor intellisense, ESM/CJS builds, TS support and extra, backwards-compatible features) you could switch metalsmith-layouts to @metalsmith/layouts, and metalsmith-sass to @metalsmith/sass. New plugins have been added too (@metalsmith/js-bundle an esbuild js bundler, @metalsmith/table-of-contents, @metalsmith/postcss).
Most issues listed in https://github.com/fastify/website/issues/291#issuecomment-976212036 can be fairly easily implemented with Metalsmith plugins/ templating too. I have been experimenting with docs in metalsmith for https://metalsmith.io/docs. For inspiration the source code is at https://github.com/metalsmith/metalsmith.io/. A.o. there are:
- codeblocks with tabs
- using
@metalsmith/in-placewith 2 transformers (marked & nunjucks) to be able to use nunjucks "components" in markdown files similar to.mdx(call it.mdnjk) - using a nunjucks include that contains named markdown links to be used in every page where it's included (see https://github.com/metalsmith/metalsmith.io/blob/master/lib/views/partials/doc-mdlinks.njk)
Whichever route you choose to go, thank you for having chosen metalsmith in the past and up until now 😄