python/docs-community

Research ways to improve SEO

Mariatta opened this issue · 8 comments

Mariatta commented

How to improve docs.python.org's SEO ranking?

Come up with action items on how the SEO can be improved.

Mariatta commented

I did a quick reading on a few blogs about SEO.
Here are some things I learned:

hugovk commented

Google Chrome has something in devtools called Lighthouse that runs an audit on a webpage:

  • Right click page > Inspect > Lighthouse > can uncheck the non-SEO categories for speed > Analyze page load

There's also an online version: https://web.dev/measure/

Results for an arbitrary page (https://docs.python.org/3/library/csv.html) are pretty good to begin with:

image

image

image

Links to those recommendations:

The first one should be pretty easy, and we should the second. The last is bigger theme work, and we may get a good enough bump from the first one or two.

  • 👍2
hugovk commented

For the first one, https://pypi.org/project/sphinxext-opengraph/ is a Sphinx extension to automatically add OpenGraph metadata.

It doesn't add <meta name="description"> (although there's a feature request: wpilibsuite/sphinxext-opengraph#65), but it would be good to add this anyway and could also help with SEO. (And it improves the previews when sharing on Facebook.)

However, we will need to know the version number of the actual docs build (e.g. (it could be /3/, /3.9/ or /3.9.2/. See python/docsbuild-scripts#137

  • 👍1
ezio-melotti commented

During the last call someone mentioned that "searching for specific functions doesn't bring docs.p.o up as first result".

In the chat I suggested to add the entries that recently got added to the docs ToC in a <meta name="keyword">, but I'm not sure whether that would belong to built-in Sphinx or if it would be better as an extension (@AA-Turner suggested to create a Sphinx issue, so I could start from there).

However It's also been a few years since I last looked into SEO-related stuff, and I have the impression that nowadays the meta keywords are not as important anymore (I might be mistaken, or there might be something equivalent/better).

Some concern was also brought up about search engines indexing version of the docs that shouldn't be indexed, e.g. docs PR previews or outdated versions. This could be solved by adding an explicit <meta robots="noindex; nofollow">, especially to the former group -- Google got much better with the search result not including outdated versions, and there might be old deprecated/removed features that should still be indexed.

hugovk commented

Good suggestions.

However It's also been a few years since I last looked into SEO-related stuff, and I have the impression that nowadays the meta keywords are not as important anymore (I might be mistaken, or there might be something equivalent/better).

Maybe, although I think a good rule of thumb for good Google results is to follow the suggestions by Google's SEO tool :)

  • 👍4
hugovk commented

For the first one, pypi.org/project/sphinxext-opengraph is a Sphinx extension to automatically add OpenGraph metadata.

It doesn't add <meta name="description"> (although there's a feature request: wpilibsuite/sphinxext-opengraph#65), but it would be good to add this anyway and could also help with SEO. (And it improves the previews when sharing on Facebook.)

I contributed some improvements to this extension, including adding <meta name="description">.

We've added it to the devguide, and the SEO score has increased from 91 to 100. It also improves the previews on Facebook, Slack, Discord, etc.

We can look into adding it to the main docs next.

  • 👍1
  • 🎉1
  • 🚀2
hugovk commented

Next up! PR to add the same thing to https://docs.python.org:

python/cpython#99931

  • 🚀2
hugovk commented

This is now on /3/ and /3.12/! 🚀

For example:

(Note: It's not on the index.html or download.html pages, because they're html_additional_pages and not under control of the Sphinx extension.)

Google's Pagespeed Insights, SEO score

Increased from 81 to 88 on mobile, and 83 to 92 on desktop. As expected the meta description now passes. ✅

Details
Old /3.10/ New /3/
SEO mobile: 81, desktop: 83 image SEO mobile: 88, desktop: 92 image

Facebook

image

Here's how the new page looks on Facebook's sharing debugger:

Details image

Interestingly, because old pages like https://docs.python.org/3.10/library/csv.html declare their canonical URL to be the /3/ version, we also get that metadata for free!

<link rel="canonical" href="https://docs.python.org/3/library/csv.html" />

So here's 3.10:

Details image

We have to go back to 2.7 to see how it was before:

Details image

Discord

/3/ is expanded:

image

Slack

Also /3/ is expanded:

image

  • 🎉5
  • 🚀4