roots/roots-example-project.com

roots-project-generator

austinpray opened this issue · 5 comments

austinpray commented

I want to start a discussion about making this repo a programmatically generated repo.

Problem

Our projects move fast. This process will fail us as roots-example-project diverges further and further from the base bedrock, sage, and trellis repos.

Proposed solution

We should make a program that takes a set of static transforms and combines and transforms the base repos accordingly. The transforms can be defined in perhaps a yaml file or something.

Design Considerations

Be able to specify versions of each repo

versions:
  trellis: stable
  sage: stable
  bedrock: stable
versions:
  trellis: HEAD
  sage: 8.2.1
  bedrock: stable

For this repo we would probably use stable. Stable is the latest release according to GitHub.

Merge ansible yaml files

Given an ansible yaml file like this: https://raw.githubusercontent.com/roots/roots-example-project.com/06c7f6abf84d58babaeef6a583469611cdd9b6ac/ansible/group_vars/development

If the repo source is this:

mysql_root_password: devpw

web_user: vagrant

wordpress_sites:
  example.com:
    site_hosts:
      - example.dev
    local_path: ../site # path targeting local Bedrock site directory (relative to Ansible root)
    repo: git@github.com:roots/bedrock.git
    site_install: true
    site_title: Example Site
    admin_user: admin
    admin_password: admin
    admin_email: admin@example.dev
    multisite:
      enabled: false
      subdomains: false
    ssl:
      enabled: false
    cache:
      enabled: false
      duration: 30s
    system_cron: true
    env:
      wp_home: http://example.dev
      wp_siteurl: http://example.dev/wp
      wp_env: development
      db_name: example_dev
      db_user: example_dbuser
      db_password: example_dbpassword

php_error_reporting: 'E_ALL'
php_display_errors: 'On'
php_display_startup_errors: 'On'
php_track_errors: 'On'
php_mysqlnd_collect_memory_statistics: 'On'
php_opcache_enable: 0

xdebug_install: false
php_xdebug_remote_enable: true
php_xdebug_remote_connect_back: true
php_xdebug_remote_host: localhost
php_xdebug_remote_port: 9000
php_xdebug_remote_log: /tmp/xdebug.log
php_xdebug_idekey: XDEBUG
php_max_nesting_level: 200

and my transform looked like this:

wordpress_sites:
  roots-example-project.com:
    site_hosts:
      - roots-example-project.dev

My output would be

mysql_root_password: devpw

web_user: vagrant

wordpress_sites:
  roots-example-project.com:
    site_hosts:
      - roots-example-project.dev
    local_path: ../site # path targeting local Bedrock site directory (relative to Ansible root)
    repo: git@github.com:roots/bedrock.git
    site_install: true
    site_title: Example Site
    admin_user: admin
    admin_password: admin
    admin_email: admin@example.dev
    multisite:
      enabled: false
      subdomains: false
    ssl:
      enabled: false
    cache:
      enabled: false
      duration: 30s
    system_cron: true
    env:
      wp_home: http://example.dev
      wp_siteurl: http://example.dev/wp
      wp_env: development
      db_name: example_dev
      db_user: example_dbuser
      db_password: example_dbpassword

php_error_reporting: 'E_ALL'
php_display_errors: 'On'
php_display_startup_errors: 'On'
php_track_errors: 'On'
php_mysqlnd_collect_memory_statistics: 'On'
php_opcache_enable: 0

xdebug_install: false
php_xdebug_remote_enable: true
php_xdebug_remote_connect_back: true
php_xdebug_remote_host: localhost
php_xdebug_remote_port: 9000
php_xdebug_remote_log: /tmp/xdebug.log
php_xdebug_idekey: XDEBUG
php_max_nesting_level: 200

Benefits

Easier to maintain this repo

This would keep our project easy to maintain because we are not manually making and merging changes. Also the transforms needed on top of our base repos are declared explicitly. No git cherry picking. No merge conflicts. Also a clear audit trail of what needs to happen to create the stack. The more transforms we have to do the more indications of low cohesion between our products.

Easier to start projects

On top of making this repo easy to maintain: this could also be used by us as a standalone tool to generate private projects.

Integration testing

versions:
  trellis: HEAD
  sage: HEAD
  bedrock: HEAD

or

versions:
  trellis: 4fff69497270305966d0e9456a546e12def148cb
  sage: HEAD
  bedrock: HEAD

Is basically an integration test. We need to make sure our projects have high cohesion. Again: our projects move so fast that doing this manually is not feasible. I should be able to test the full stack of roots projects easily. What is good for our users is good for us as maintainers. This allows us to easily determine whether a change to a particular repo (bedrock, bedrock-ansible) is a breaking change. It also allows us to go back in time and check for incompatibilities.

Questions

Yaml format?

Not quite sure what would be better:

versions:
  trellis: stable
  sage: stable
  bedrock: stable

repos:
  trellis: roots/trellis
  sage: roots/sage
  bedrock: roots/bedrock

or

sources:
  trellis:
    version: stable
    repo: roots/trellis

or

trellis:
  version: stable
  repo: roots/trellis

The later looks pretty good. Dunno.

Contents of this repo?

This repo is would just be the latest compiled output of the generator. I don't think there is anything wrong with this. That way people can have a consistent static reference as to what the latest and greatest looks like.

Sage generator

How would this interact with our idea of a sage generator? If a sage generator existed this would definitely use that generator instead of manually transforming the repo. Although the generator would have to comply with the design considerations of this project.

Trellis-cli

How would this interact with the trellis CLI? Probably little to no affect.

Implementation language?

Maybe ruby or python? Those are generally pretty easily available to users on the *nix computers. Ruby has lots of really useful yaml gems and such. Go is also an option 😎. Leaning towards ruby so @swalkinshaw could sling some loc with me.

Split CLI and lib

It might be a good idea to split the project into roots-project-generator and roots-project-generator-cli. That way we could perhaps expose roots-project-generator as a web application and keep roots-project-generator-cli as a CLI interface for the library. This will also help in writing tests.

kalenjohnson commented

Great ideas @austinpray

Just to be clear, this "generator" would still just set up a project, but after initial set up and committing everything to your new project's repository, you would still need to add updates manually or cherry pick, correct? The roots-example-project.com would be able to be continuously generated simply because it's now a static project of just the latest releases of each Roots project.

Yaml format: I agree, the latter is better, declare each "project" once. Would require less back and forth, however little that might be.

Implementation: I've been meaning to get my hands dirty with some Ruby. Python could obviously work as well. Tough call actually since just going by what our projects use, majority is PHP, but Vagrant is Ruby, Ansible is Python.... I'm just going to vote Ruby, and downvote Go, just to cheese off @austinpray 😂

Split CLI and lib - sure, sounds good

swalkinshaw commented

YAML format: 2nd or 3rd. Nesting under sources protects this if we add other configuration in the future.

Contents of repo: yes, just generated. It's still nice to actually see a generated example.

Sage generator: no concern for now.

trellis-cli: no connection.

Language: make it easy to install. That means either PHP (since these are PHP projects), Node (since we require npm for Sage), or Go/Rust/whatever language will spit out a nice simple binary.

Project split: sure, whatever you want. warning: be pragmatic to start with.

austinpray commented

YAML format: 2nd or 3rd. Nesting under sources protects this if we add other configuration in the future.

Yeah let's do 3rd.

Language: make it easy to install. That means either PHP (since these are PHP projects), Node (since we require npm for Sage), or Go/Rust/whatever language will spit out a nice simple binary.

Negative on PHP. No desire to write PHP.

[4:44 PM] swalkinshaw: spoiler: i probably wont do anything

Let's just do Go then. Spit out ezpz binaries for windows + mac + linux.

Project split: sure, whatever you want. warning: be pragmatic to start with.

Lib/CLI split is easier to test, so will speed development. Don't want to fight a war on two fronts. Would rather have a tested and solid programmatic API than a sort-of-working CLI. But yes: thin vertical slices.

tobyl commented

I think I'd do more harm than good chiming in on the technical details, but if the end result was even just an up-to-date 'roots-example-project.com' repo then that'd be awesome. Looking forward to fiddling with this. :)