WordPress/wordpress-playground

Growth Metric

adamziel opened this issue · 11 comments

adamziel commented

Let's track Playground growth and share the stats publicly. Version 1 would involve measuring the number of started Playground sessions in any context (browser, node, desktop app etc.)

Done is

  • There's a defined a way of measuring new sessions (that doesn't involve any PII)
  • It's clear what does it cover and what are the shortcomings
  • There's a plan of action to address the shortcomings
  • There's a workflow in place to share updates – either manually every month, or automatically in a continuous way

Technical considerations

This requires either:

  • A wasm-level call to capture the fact a new Playground was initialized. There's no other place that would assure reliable logging from custom integrations, or
  • A decision not to track sessions started outside of the official Playground projects.
  • Fixing the WordPress core telemetry call to wp.org. It doesn't work now. The wp_install / wp_blog headers are not set, due to the CORS issues. #1009 https://make.wordpress.org/systems/2024/02/07/cors-requests-for-api-wordpress-org/. After fixing the above, we'd have to ensure the w.org stats looks at those fields + the Playground PHP extension.
    • Some examples of how the API requests would look in the wild would be helpful to determine how the stats could be worked, including once-off and ongoing-use.
    • What would wp-now or Studio API requests look? What’s the unique identifier on each request? etc.
    • Complication: w.org API excludes temporary/not-routable/development sites from the stats to avoid spam.
bgrgicak commented

A decision not to track sessions started outside of the official Playground projects.

I prefer this option. We need to allow opt-out of telemetry/metrics data for sessions that are started outside of our project.
The easiest solution is not to track.

Our data doesn't need to be perfect, it should cover all standard usage behaviors to help us spot trends.

  • 👍1
bgrgicak commented

Should we start sending metrics to the new Playground server?

adamziel commented

@bgrgicak sending them to the new server implies we'll need to figure out wrangling and dashboarding the data. I'd prefer Google Analytics + "share this dashboard" for now, unless there's another canonical approach in WP.org.

bgrgicak commented

Sorry, I haven't explained my thoughts completely.

The current GA implementation is limited to the browser, but if we sent all analytics data to the Playground server we would get support for other Playground apps like CLI and remote.

In that case, the Playground server could be the starting point and data could be sent to another service. Today this is GA, but it could be anything else in the future. (GA seems to support sending events over HTTP.)

  • 👍1
andreilupu commented

Hey, I'm having some fun with Playground and I noticed the GTM tag on my self-hosted version.

I don't mind sending some telemetry data from my own version, but some people might want the GTM tag tracking by default (or maybe an option to disable it?).

Chears

bgrgicak commented

Hey @andreilupu 🙂
I'm glad that you are having fun with Playground!

It's interesting that you see the GTM tag on a self-hosted version.
Our intention was for GTM to run only on https://playground.wordpress.net/.

Can you share more details about your self-hosted setup?

GTM is registered here in the Playground Website. If you are using the website code, you could remove that part from the index.html file.

How would you prefer to disable it?

andreilupu commented

Hey @bgrgicak 👋

I'm making a PR preview app for a private GitHub repository where I build a WordPress plugin. I simply used the npm run build:website command and uploaded the wasm-wordpress-net dir to my server.

I'm not sure what's the best way, today I've learned about nx, maybe there is a way to add a config property in the jest.config / nx.json.

Or even better, we could introduce a .env(which I see it's already .gitignored) variable for the gtag id, and if that's empty, the tracking code should not be printed in HTML.

bgrgicak commented

Thanks for explaining!

There is a Playground PR previewer action, but it only works with themes for now. @erseco suggested we could add plugin preview support.

Or even better, we could introduce a .env(which I see it's already .gitignored) variable for the gtag id, and if that's empty, the tracking code should not be printed in HTML.

That sounds like a good solution. It would enable anyone to change or disable the tracking code.
@andreilupu would you be interested in contributing this feature to Playground?

andreilupu commented

That sounds like a good solution. It would enable anyone to change or disable the tracking code.
@andreilupu would you be interested in contributing this feature to Playground?

Sure, I'm happy to give it a try over the weekend. I just have to go through the documents and get familiar with the build process.

  • ❤️1
erseco commented

Just wanted to share a quick (and dirty) fix I implemented in my plugin pipeline that might be helpful. What I did was take the blueprint.json, replace the default main.zip with the correct branch's ZIP URL, encode it in Base64, and then use it to create a deployment in GitHub.

You can check the implementation here: https://github.com/ateeducacion/wp-decker/blob/main/.github/workflows/ci.yml#L107

How it works:

  1. Replaces main.zip with the actual branch ZIP URL in blueprint.json.
  2. Encodes the updated blueprint.json in Base64.
  3. Creates a GitHub deployment with the encoded Playground URL.

This way, each deployment correctly reflects the branch it's built from. If this helps or sparks any ideas for improvements, let me know!

  • 🚀1
  • 👀1
adamziel commented

Re GTM: It would be useful to only add that tag during the build and only if the target domain is playground.wordpress.net. Alternatively, a <script> in the head could add it if the current domain is playground.wordpress.net.

Re PR previewers: Really cool stuff, thank you for sharing @erseco!