refarch-eda documentation refactoring

[osowski]

Referencing the original proposed refactoring structure laid out by Johanna in MURAL, this issue will be used to drive the refactoring of the gitbooks that are published to https://ibm-cloud-architecture.github.io/refarch-eda/ and https://ibm-cloud-architecture.github.io/refarch-kc/.

Approach going forward

https://ibm-cloud-architecture.github.io/refarch-eda/ will be the entry-point for high-level, conceptual event-driven architectural overview. This will be topical in nature, but also contain content for the more abstract data patterns, agile integration approaches, etc.

https://ibm-cloud-architecture.github.io/refarch-kc/ will be the entry-point for low-level, technical implementation details for our shipping container reference implementation. It will absorb all the implementation details from the existing microservice gitbooks, while we will leave the basic "build & run" instructions in basic markdown README files in each microservice repository.

The goal here is to reduce the overall footprint of our EDA documentation and allow us to curate more focused content.

New GitBook format

As the docs for both refarch-eda and refarch-kc have been ported over to using Gatsby/Carbon via this Issue, we will structure our new GitBooks by referencing the left-nav as our main grouping of sections & content.

Some of this will require moving content from current versions of refarch-eda to refarch-kc (or vice-versa), as well as breaking out information from https://ibm-cloud-architecture.github.io/refarch-reefer-ml/ to make sure the consolidated information is available in one of two GitBooks.

Useful references:

• https://gatsby-theme-carbon.now.sh/
• https://www.gatsbyjs.org/

I have put together a draft of the left-nav structuring that below that we can use to track progress for doc refactoring.

(Current as of 2020/04/29)

refarch-eda

• Introduction
  • What is the purpose of the EDA Reference Architecture?
  • What will you learn
  • Target audiences
• Concepts
  • Terms & Definitions
  • Events versus Messages
• Advantages of EDA
  • Push
  • Scalability
  • Resiliency
• Patterns in EDA
  • CQRS
  • Event Sourcing
  • Long-Running Actions / Sagas
  • Dead Letter Queue
  • Event-driven Data Replication
  • Real-time Analytics
• Methodology
  • Event Storming
  • Domain-Driven Design
• Technology
  • Kafka Overview
  • Kafka Producers & Consumers
  • Kafka Connect
  • Kafka Mirror Maker
  • Kafka Streams
  • Serverless EDA
• Additional reading

refarch-kc

• Business Scenario
  • Scenario Overview
  • Architecture Overview
  • Microservices Overview
  • Quickstart Tutorial
  • Git Repository Overview
• Development Practices & Implementation Details
  • Event Storming Analysis
  • Domain-Driven Design Analysis
  • Schema Registry and Avro
  • Saga patterns
  • Dead Letter Queue implementation
• Microservices Deep-dive
  • Prerequisites
  • Orders UI
  • Order Command
  • Order Query
  • Container Management
  • Simulator
  • Scoring
  • Fleets
  • Voyages
• Integration Tests
  • Overview
  • Happy Path
  • Saga Pattern
  • Order Cancellation
  • Dead Letter Queue
  • Container Anomaly

Moving forward

1. Confirm proposed format above or comment with desired changes. As changes are discussed & resolved inline in this issue, we can update the structure in this comment for visibility.
2. The published GitBooks are currently being generated out of the /docs-gatsby subdirectories in each repository. Below is my plan for where we will work out of in the specific docs directories:
   1. Move existing /docs to /docs-archive
   2. Copy existing /docs-gatsby to /docs
   3. Edit in place in /docs
   4. Once enough initial work and scaffolding has been completed, update build process to build from /docs
   5. Ensure all current content from /docs-gatsby makes it into new /docs sections as appropriate.
3. As sections are being worked on and completed, they can be claimed here via comments and then checked off via the checkboxes in this comment.

[jbcodeforce]

I'm okay with the approach. For reefer-ml it is touchy: as it includes labs for CP4D. So this one may be splitted in 3 repos: one for labs doc, one for cloud-pak assets/code, one for open source implementation assets/code. Few of this content will go back to refarch-kc
Also as we have 230 repos under ibm-cloud-architecture account, may be we can be rename every repo that are part of the reefer container solution with a prefix refarch-kc-

[osowski]

Agreed on both points. For the latter, we can definitely rename the repositories to include all the necessary prefixes to logically separate & group the repositories together. I will create a new issue for that.

For the former, we can potentially take a pass at splitting up refarch-reefer-ml in a separate issue and where that goes in context here as well. I don't want every use case we have to spawn new GitBooks, as the approach is to have everything be available under high-level & low-level GitBooks. If we need to add a more robust and explicit "Use Cases" section under the refarch-kc GitBook to include the base EDA scenario and then the EDA+CP4D/AI/ML, that would make sense to me.

[osowski]

Added @djones6 and @josiemundi to this issue for visibility.

In doing some of the initial scaffolding today, I discovered that Gatsby, the pattern & theme the docs will be displayed in, does not handle left-nav items with a depth greater than 2. This causes issues for our initially planned structure of Advantages and Patterns (with child elements) being under the Concepts topic. I propose we move both the Advantages & Patterns to be a top-level topic and everything else stays pretty simple (as they will be visible for everyone navigating the page).

If folks can comment and/or give thumbs up/down reactions on this, then we'll update the root comment once we get consensus etc.

[osowski]

Based on confirmation above and discussion on calls yesterday, I have merged the PR above with the initial work for scaffolding implementation and layout for refarch-eda/docs

[osowski]

• @osowski is working on Concepts & Methodology sections.
• @jbcodeforce is working on Patterns in EDA sections.
• @jesusmah is working on Technology sections.

[josiemundi]

@osowski I will be starting work on the Business Scenario section this week with Sue Chaplain.

[osowski]

Docs have been switched over to build off of /docs sub-directory and the refactored structure is live.

[osowski]

The first pass at both refarch-eda and refarch-kc have been updated and pushed. Closing this issue in favor of smaller point issue going forward for individual issues in each gitbook/section/topic.