/DataGenerationToolkit

Open Source Community Sprint project focused on building a tool to generate test data.

Primary LanguageJavaScript

Data Generation Toolkit

Data Generation Toolkit Logo featuring Astro in a rain of data

This project aims to create toolkit of tools and documentation for generating test data sets based on admin-selected criteria.

Project Name

Data Generation Tool

Major use cases for these data sets include:

  • QA an org populated with permutations of all, or nearly all, the types of data that are relavent to the project.
  • Be able to reliably and easily load sample data into a given Salesforce org (production, sandbox or scratch).
  • Have a data set for demos, potentially with the ability to add specific sets of data that could be used for story based training materials.
  • Have a data set that ensures the privacy of people represented in the dataset (ie not real names)
  • Have data sets at scales that allow for testing bulk data processing.

Project Vision/Goals

  • Everyone has access to an open process/tool to reliably generate sample data for telling user stories
  • Admin-friendly interface to define requirements for the data generated
  • Data sets generated through this process could be appopriate for medium-sized orgs and can fit in a sandbox
  • Data sets generated through this process could be valid for large data valume (LDV) testing
  • Develop process for how you can build test data sets that follow a story (automated? manual? best practice doc?)
  • Prioritize leadership development/capacity building through project leadership/contributions from customers, partners, devs, etc

Ongoing Projects

Currently the project team has two major efforts for our work:

  • Documentation of existing tools and partial solutions
  • Creation of a community supported tool that will empower easy creation of data sets tailored to an org's needs.

Hub Group Link

https://powerofus.force.com/s/group/0F91E000000brOoSAI/community-project-data-generation

Project Theme

  • theme-community-ecosystem

Project Vertical

Everyone.

This project is aimed at helping ease the process of generating test data for projects. Nearly all Salesforce projects benefit from improving this process.

Current Project Team

Help us give you the thanks you deserve and ensure future contributors know who to contact if they have questions! Please ensure that all contributing members of the team are included.

  • Team Leader(s):
    • Aaron Crosman (Attain)
    • Samantha Shain (William Penn Foundation)
    • Cassie Supilowski (OneGoal)

Project Team Accomplishments

  • [9/23-9/24 Virtual Sprint]
    • Drafted architecture diagrams for a UI to sit on top of Snowfakery. This will be called Snowmakery! There are two proof of concepts that have been initiated. One is inside of the org, the other is in Heroku.
    • Analyzed survey data from 75 community members; updated relevant Personas based on survey insights
    • Established document outline and Admin Story for documentation project that describes how to Generate Data and Move Data between orgs
    • Documented limitations and considerations for Partial Data sampling algorithm and manual steps, third party apps, and code for creating mock data records
    • Documented steps for an Admin-audience to use CCI to move data records from one Dev Sandbox to another Dev Sandbox (or any two persistent orgs)
    • QA-ed documentation steps for CCI steps (referenced above)
  • [3/31 -> 4/1 virtual sprint]
    • Determined that Snowfakery accomplishes many of the original requirements brainstormed at the Philly Sprint (fall 2019) (namely: ability to generate mock data with related tables, random names and values, standard and custom objects, datasets of any size/scale, ability to populate Salesforce orgs)
    • Socialized Snowfakery to community members
    • Onboarded project leadership from multiple orgs and began application for inclusion in Open Source Commons program
    • Refined use cases and differentiated Snowfakery from (1) existing tools in the market (2) proprietary tools at Salesforce (3) Full Sandbox product
    • Brainstormed 2+ potential directions for extending Snowfakery to include an admin-friendly web interface
    • Reviewed documentation and install steps
    • Overhauled ReadMe file

Proof of Concept Code

During the course of this project some ideas have been tested in code, and to ensure that code isn't lost and doesn't confuse other efforts in this repo they are kept in the proofs-of-concept directory.

Currently those include:

  1. A CumulusCI Task created during this project's first sprint to generate permutations of data.
  2. A web-based Snowfakery recipe editor created during the fall 2020 virtual sprint. This was the second tool named Snowmakery and largely confirmed that this project's UI will bare that name.

Past Project Team Members

Full Name Github Username Sprint(s)
Fatama Ahmed ffarag85 2019 Philly Sprint
Michael Beaty mbeaty-sf Virtual Mar. 2020, Virtual Sept. 2020
Andrew Beyer Virtual Sept. 2020
Joe Blodgett Virtual Sept. 2020
Francisco Borges ciscoborges 2019 Philly Sprint
Chris Chen chrisVCH Virtual Mar. 2020
Rebecca Cole Sullivan rcolesullivan Virtual Mar. 2020
Aaron Crosman acrosman 2019 Philly, Virtual Mar. 2020, Virtual Sept. 2020
Andrew Curran andrew-curran 2019 Philly Sprint
Marciana Davis mdportfolio Virtual Sept. 2020
Debbie Duran Virtual Sept. 2020
Michael Kolodner mkolodner 2019 Philly, Virtual Mar. 2020, Virtual Sept. 2020
Jason Lantz jlantz 2019 Philly, Virtual Mar. 2020
Allison Letts allisonletts 2019 Philly, Virtual Mar. 2020
Jung Mun eehjunggnujhee Virtual Sept. 2020
Lawrence Newcombe LawrenceLoz Virtual Sept. 2020
Paul Prescod prescod Virtual Mar. 2020, Virtual Sept. 2020
David Reed davidmreed 2019 Philly, Virtual Mar. 2020
Erin Schroder eschroder Virtual Mar. 2020
Samantha Shain samanthashain Virtual Mar. 2020, Virtual Sept. 2020
Kim Snipes kimsnipes 2019 Philly Sprint
Cassie Supilowski csupilow Virtual Mar. 2020, Virtual Sept. 2020
Amanda Styles demanda Virtual Mar. 2020
Tyler Woebkenberg tylerw-sfdc Virtual Mar. 2020

Useful References