FTL: A Jinja repository from ivan-a-souza

FTL Overview

FTL, Finish The Labs, is currently a student invoked automated lab grading system designed to allow students to self mark their OPENTLC labs to see if they have passed them correctly. Other uses include lab and lab environment validation.

On failure it highlights the failed sections via Instructor supplied error messages. Students have unlimited attempts and can run the grading at any time, without penalty.

On success it:

Informs the student
Updates the LMS (at this point the grade-update-lms role is a stub toc::[]

Changelog

A detailed Changelog can be found here - keep it up to date

Structure

├── README.adoc                 # This document
├── ansible.cfg                 # Default ansible.cfg - can be over-riddden
├── courses/                    # Directory containing course and their graders and solvers
├── docs/                       # Documentation - see below for overview
├── main.yml                    # Main entry point to grade or solve a lab
├── roles/                      # Top level roles directory, can be supplemented at a course/lab level
├── vars/                       # Set global *common* vars reports etc
└── devel                       # developer tools, data, scripts, playbooks

Documentation:

A documentation directory is provided docs/

This README.adoc - A General Overview
Lab Grading Author Guide - Creating Lab Test Playbooks
Deployer Guide - Deploying, Installing, and Executing Lab Tests

Terminology

A small number of terms are defined below

grader host - Host that the student works from, typically bastion or workstation
graders - Graders are playbooks that grade a lab, they are typically invoked via the grade_lab command which is installed at deployment time.
solvers - Solvers are playbooks that solver labs i.e. they apply all the necessary ansible tasks to move complete a lab.
grader roles - grader roles, in the roles directory provide pre-built roles for testing specfic, and common, scenarios e.g.
- grader_check_command_output
- grader_check_file_exists
- grader_check_ocp_node_exists
- grader_check_ocp_route_exists
- grader_check_ocp_svc_exists
- grader_check_package_installed
- grader_check_service_enabled
- grader_check_service_started
- grader_check_user_exists

Workflow - Deployment

FTL can be deployed via push i.e. inserted into a lab at the deployment stage. An obvious example being inserted via Ansible Agnostic Deployer in the Post Software stage of a config.

Note	FTL currently clones the entire repo i.e all tests for all labs so no additional meta data needs to be supplied at deployment time.

Alternatively it could be modified to be deployed via pull i.e. the student clones the FTL repo locally. This will require either a permissions change on the repo or a API token/login to be provided as FTL is a private repo today.

Workflow - Developing a Lab Test

Instructors create a series of lab checking playbooks in the correct sub-directory of the courses directory. Tests are written in ansible and test 1 or more conditions in the lab. For example:

Has package foo been installed
Does user bar exist
Does a file contain a certain line
Does a URI give a certain response e.g. 200 or a certain output

Whilst instructors can create tests some convenience roles have been created to both test condition and to track score, these are detailed in the Lab Grading Author Guide*

Sample of a typical lab test leveraging an existing role

- name: Section 1.5, Check student has installed ansible
  role: grader_check_package_installed
  vars:
    section_number: 1.5
    student_error_message: Package not installed
    package: ansible

Instructors can author custom, and more complex, tests as long as these can be executed in an ansible playbook. See the Documentation for more details.

Usage

Students are supplied with grade_lab and solve_lab wrapper scripts, in /usr/local/bin that typically take 2 arguments, course name and lab number:

$ grade_lab ansible_engine_foundations 03_01

Questions:

Do students invoke the check via:
- CLI on the lab machine
- Via button in the LMS
Does the check action pull in the playbook(s) or are they already there?
- i.e. backed in at lab build time

ivan-a-souza/FTL