Samsung/qaboard

Open-Source Release Status

Closed this issue · 16 comments

arthur-flam commented

Go!

Welcome to QA-Board! We are working on open-sourcing the project and are targeting an official release for Q3 2020.

What needs to be done before the release

  • Pre-release, as all items marked 🌱 are done.

Open-source process

  • Authorization to release as OSS 🌱
  • Licence (Apache 2) 🌱

Docs

Technical changes

  • qa init creates a functional project
  • Merge the qatools and qaboard internal repositories 🌱
  • Make sure nothing is hardcoded for our infra : pass secrets at least as ENV 🌱variables or config files, and handle mount naming convention with a centralized config.... 🌱
  • Decide on a way to pick a default location where to store the output data. Needs to be mounted by the container and writable easily.
  • Ensure 1min server bringup with e.g. docker-compose
  • Remove USER arthurf from Dockerfile, maybe make it configurable to avoid NFS+squash_root issues.

API changes:

  • qaboard-backend.slamvizapp>backend.backend, qaboard-webapp>webapp
  • qatools.yaml->qaboard.yaml
  • Import as qaboard not qatools
  • Output folder naming conventions with hashes
  • Friendlier run() API (ctx.obj->ctx : RunContext)
  • sed -i s/qatools/qaboard/g *

Features

  • Make it easy to support more async task queues - Wiki
  • Support for at least one common task queue beyond what we have internally at Samsung, and explain how to implement support for others.
  • Create a public CI
  • Support more git servers (e.g. Github). Don't even require Read rights.
  • Make it easier to debug visualizations and metrics with per-batch settings

PR

  • Public demo server
  • All good for a release blog post
amirfru commented

Hey @arthur-flam !
Can I ask which task-runners is it tested for? thanks!

arthur-flam commented

Hi Amir ! Even better, please suggest something you are using!

I planned on adding support for something easy to setup and flexible like celery or python-rq.

For other integrations (unless there is a big demand) I'll write docs, make sure the code is easy to extend, and rely on help from the community.

arthur-flam commented

This week I improved the docs a lot, they are now good enough to be published. I expect to have them online early next week, along with a website.

I also published the CLI wrapper to pypi:

pip install qaboard

As soon as I merge the two repositories composing QAboard today, I'll push the code online

arthur-flam commented

Hi, some updates:

  • Merged the two repos that composed qaboard
  • Added basic support for "secrets" to pass e.g. gitlab credentials.

We're getting closer to a pre-release. What's missing is creating a "master-open-source" branch with a clean history, replacing a few default / docs, and pushing it..!

arthur-flam commented

Update:

  • there is now a fully sanitized version of the code. I'll publish it ASAP now 😌
  • I uploaded the website/docs to the gh-pages branch, but GitHub Pages isn't setup correctly so we'll have to wait for it to be live...
arthur-flam commented

The code is online 🎉🎉🎉 !
There is still work to do, but things feel more concrete now. 🎆

arthur-flam commented

Hi!
I've been making progress refactoring batches to allow more types of runners. More info at:
https://github.com/Samsung/qaboard/wiki/adding-new-runners

@amirfru hi! Any update about what you'd need?

I expect to push the branch in the next few days, possibly directly merge on master.

arthur-flam commented

✨ There is now support for Celery ! https://samsung.github.io/qaboard/docs/celery-integration/
Tuning from the web application is still hardcoded to use our infra... But that will be fixed soon.

arthur-flam commented

Hi! I quick update to say that we're still working on the open-source release, although we've focused more on quality of life features and small fixes this past month. Stay tuned!

arthur-flam commented

Hello!
Last month we've been busy with other priorities, still some notable improvements landed:

  • support for "matrix" batches
  • filtering is now much more performant
  • lossless image viewer support
arthur-flam commented

Hi!
I am finishing adding support for docker-compose:

  • the whole application stack can already be started in 1 command, and there is a structured way to specify custom ENV variables.
  • makes it easier to implement a public CI, host a demo server, or just have users try QA-Board.
  • will help us provide pre-built container images.
  • will provide an easy migration path to more complex orchestration tools (e.g. k8s..).
  • Other benefits: the app now has only seconds of downtime during upgrades, is served from a unique port, no need for cron jobs managed separately, and our internal tweaks are much more isolated.

I expect to push the changes and new docs in the coming days. Bye!

arthur-flam commented

We now use docker-compose!

  • New guide to start a QA-Board server
  • Faster startup/update time, as all services are split
  • Everything is served from a single port, we reviewed the whole mess of reverse proxies.
  • The docs are available from the application at /docs
  • Almost all the code is the same in the OSS and our internal version.
  • We will shortly add in docker-compose.yml a sample cron job doing backups

We also started using github actions to create a public CI:

  • the website is already updated automatically from the docs
  • we're building the docker images and doing a basic smoke test.
  • we're also running the unit tests, type checks and linter.
  • On master we push images to github's docker registry and on releases we upload the CLI package to pypy

The next task is not requiring read-access to the git repositories of the users' projects - or at least support github in addition to gitlab (currently required...)

arthur-flam commented

Hi!

Here are some updates:

  • QA-Board now doesn't require Gitlab anymore, nor read-right to projects' repositories. Using Gitlab enables nice-to-have features like showing user avatars, but we'll work on doing it with Github too.
  • Each batch can now have its own config and metrics, making using with qa --share much more pleasant.
  • If you've missed it, we now support a new visualization, flame graphs.

We're now in new territory: anyone can start using QA-board! We'll check it's as smooth as promised by the docs, and give updates.

arthur-flam commented

There is now a demo server at demo.qamarks.com!

  • It was straightforward to setup just following the guide, so we can now feel at ease looking for users.
  • For now there is a single demo project comparing 2 versions of a map-based algorithm. We'll add more demos and polish it...
arthur-flam commented

Some major updates:

  1. The API was simplified:
  • Instead of the ugly context.obj["configurations"], there is context.configs. And so on.
  • Tuning comes for free, no need to worry anymore about context.obj["extra_parameters"]
  • For convenience there is now context.params
  • The docs were updated and changes are backward compatible.
  1. Users can now save artifacts and outputs at different locations, and it's all a bit simpler to specify. Go to the docs to read more.

Those changes close the TODOs to make QA-Board "public-ready" 🚀 !

I'm closing the issue!