Document how to track metrics to assess success of strategic changes

[knolleary]

We currently document our key metic here: https://github.com/flowforge/handbook/blob/main/company/strategy.md#key-metrics

But beneath that metric there are a number of things we need to pay attention to so we can assess the success of the different changes we make - such as the recently introduced trials.

We have lots of dashboards in PostHog and HubSpot and DataStudio that each serve a slightly different purpose. That makes it hard to know where to look to answer key questions until you know everything that is there.

We risk overloading ourselves with too much data and too many graphs.

When we have a strategic change being made, we should document how we'll track the success of the change.

We should:

1. identify what questions we want to be able to answer from our data
2. identify how we answer those questions

Trials

So applying this to the trials work, we currently have two forms of trial in FF Cloud:

• free-credit : From January 19th - February 16th we offered $15 free credit to any newly created team, but still required CC details upfront
• team-trial : From Feb 16th onwards, we offered 30 days free trial with 1 small project without any need for CC details

In both cases, the basic metrics of users on the platform and projects created will give us a crude measure of success. That will give us a sense of their success in bringing people into the platform to begin with.

We then need to assess the impact on churn. Essentially, how many trial users go on to pay in the second month.

For users that start with free-credit, we need to know if they continue paying in the second month once the credit has been used.
For team-trial users, we can track how many enable billing on their account and continue paying in the second month.

As a list of metrics to track, I think that gives us the list:

1. what: number of new sign-ups
2. what: number of projects created during trial/credit period
3. what: for free-credit users, how many still have a project running after 30 days
4. what: for team-trial users, how many enable billing
5. what: for team-trial users, how many still have a project running after 30 days

The next step is to document the how for each of these metrics.

[ZJvandeWeg]

I suggest we update the title to be trial scoped. As there's a few metrics we might need to add beyond that, either product focussed, engineering, or beyond.

Around trials, the only thing I'd like to also measure is: Users that have access to 1 Node-RED instance (absolute and relative)