Review - DebianOS Install (w/Docker)

[joepavitt]

Given the install instructions available at https://flowfuse.com/docs/install/docker/ go form a clean DebianOS image and install and setup FlowFuse using Docker.

Docker currently represented just shy of 30% of our EE deployments in self-hosted environments, and 35% of CE deployments.

Considerations

• Each step taken should be recorded
• Any frustrations/friction should be highlighted and documented
• The full process should be timed, ideally with rough timestamps along the too so that we understand where time is being spent.

Areas for Improvement

• #4574
• Insert other ideas here from this investigation

[ppawlowski]

FlowFuse platform installation timeline using docker-compose approach:

Installation prerequisites:

• docker and docker-compose present
• domain configured to point to the testing server using an external DNS provider

Installation timeline:

• Download docker-compose - 47s
• Prepare flowfuse.yaml configuration file: 1m 8s
• Prepare docker-compose.yml configuration file: 2m
• Pull Node-RED image: 19s
• Startup: 31s
• Optional
  • Configure https using Let's Encrypt : 4m30s
  • Configure https with own TLS: 3m 35s

Total time:

• without TLS ~5m
• with TLS: ~9-10m

timeline
  title Setup Task Durations with Time in Seconds
  section Initial Steps - 235 seconds
    Download docker-compose release: 47 seconds
    Prepare flowfuse.yaml :  68 seconds
    Prepare docker-compose.yml : 120 seconds
  section HTTPS Configuration (optional)
    Configure https using Let's Encrypt : 270 seconds
    Configure https with own TLS: 215 seconds
  section Run FlowFuse - 50 seconds
    Pull Node-RED image :  19 seconds
    Startup : 31 seconds

Loading

My thoughts:

• The requirement for a properly configured domain is purely technical stuff, this may discourage less technical people from trying out
• the whole configuration is error-prone - during my first attempt, I spent 5-6 minutes debugging the content of the docker-compose.yaml due to the single typo
  • (needs investigation) if the core app could use environmental variables as configuration parameters, this could probably be avoided
  • (needs investigation) for docker-compose.yaml configuration we probably could use interpolation and .env file
  • single, user-interactive binary/script as a wrapper for whole configuration steps can be also a solution
• (needs investigation) once docker-compose stack is up, we could somehow print information similar to the one we output in helm
• in the documentation, replace paragraphs like Digital Ocean or Disabling the default File nodes (very specific and technical) with a drop-down FAQ ; leave installation as clean and simple as possible
• paragraph First Run Setup duplicates with the First Run Setup page
• Wildcard TLS Certificate paragraph is missing detailed information about adjustments needed to flowforge.yaml file

My overall impression is that instructions are overwhelming for someone, who just want to try our software. As discussed, we could transform the docker installation page into a short quick start guide while moving configuration details elsewhere (FAQ).

[ppawlowski]

Additionally, IMO whole installation documentation should be kept in the how-to approach with copy-and-paste commands instead of giving general instructions.
Example:
edit this file, replace example.com with your domain instruction could be replaced with a single sed command.

[ppawlowski]

FlowFuse platform installation timeline using docker-compose approach.

Important note: between the previous and this evaluation, the installation process has been simplified. Due to this fact, approaches cannot be compared 1:1 .

Installation prerequisites:

• docker and docker-compose present
• domain configured to point to the testing server using an external DNS provider

Installation timeline:

• Download docker-compose and env files - 11s
• Prepare configuration files: 37s
• Startup: 1m 45s
  • pulling all container images: 1m
  • startup 45s
• Optional
  • Configure https using Let's Encrypt: 13s
  • Configure https with own TLS: 1m 8s

Total time:

• without TLS: 2m33s (~50% decrease)
• with:
  • automatic TLS: 2m46s (~70% decrease)
  • custom TLS: 3m41s (~60% decrease)

timeline
  title Setup Task Durations with Time in Seconds
  section Initial Steps - 48 seconds
    Download docker-compose release: 11 seconds
    Prepare configuration: 37 seconds 
  section HTTPS Configuration (optional)
    Configure https using Let's Encrypt : 13 seconds
    Configure https with own TLS: 68 seconds
  section Run FlowFuse - 105 seconds
    pulling all container images :  60 seconds
    Startup : 45 seconds

Loading

[joepavitt]

@ppawlowski the "235 seconds" at the start there, where is that time then spent, as the two child items only add to 48 seconds?

Update: Thanks for fixing 😄

[ppawlowski]

@ppawlowski the "235 seconds" at the start there, where is that time then spent, as the two child items only add to 48 seconds?

Forgot to change it while copying from the previous evaluation summary.

[joepavitt]

Interesting that actually, in the purple/yellow stages, you've saved a lot of time, but the Green went up by 2x?

[ppawlowski]

Interesting that actually, in the purple/yellow stages, you've saved a lot of time, but the Green went up by 2x?

On the first run, I considered a Startup as completed once docker compose reported the stack to be up and running. However, today I noticed, that when using automatic TLS certification generation some additional time is needed for certificates to be issued - I added note about it. So on the second run, the Startup period is considered from creating the stack using docker compose till the application is accessible from the browser.
With own TLS or no TLS this time is shorter as there is no need to wait for certificates to generate.

Regarding images pull time - the new approach pulls more images in the background. Also, this part may be affected by the network and disk bandwidth.

BTW, in GH dark mode I see different colours ;)