Sakai-Scripts

Some scripts that I use to make my life as a Sakai developer easier.

This is a set of scripts to set up a developer instance of Sakai on your system or set up a nighly build. Sakai can run on a 2GB RAM system but is a lot more comfortable with a 4GB or more app server.

The outlie of this document is that you jump to the setup for Docker, Ubuntu, Mac, or WSL and do some pre-steps and then continue with the common steps, below.

Getting started on ubuntu in Docker

A quick way to test all this if you have this is to just make a ubuntu in your docker:

docker run -it -p 8080:8080 ubuntu

You might want to give your docker images 4GB - ubuntu + Sakai does not fit into 2GB.

Getting Started on Ubuntu Linux

Install git using the command

  apt-get update
  apt-get -y install git curl unzip zip vim

Check out sakai-scripts into some folder and then install the rest of the pre-reqs

  cd
  git clone https://github.com/csev/sakai-scripts
  cd sakai-scripts
  bash ubuntu.sh
  source ~/.sdkman/bin/sdkman-init.sh

If this is a fresh setup and you have no database you can do the following:

  apt-get -y install mariadb-server
  apt-get -y install mariadb-client
  service mysql start

Continue with the common steps below.

Getting Started on Mac (as needed)

Install git using the command
```
  xcode-select --install
```
Make sure you have Java-11 JDK (not JRE) Installed
```
  java -version
  javac -version
```

If you need a JDK-11 is is easiest to use using https://sdkman.io/

  curl -s "https://get.sdkman.io" | bash
  source ~/.sdkman/bin/sdkman-init.sh
  sdk install java 11.0.16-tem

Make sure you have Maven (mvn) 3.8 or later installed. If you have Homebrew installed, you can use
```
  sdk install maven
  source ~/.sdkman/bin/sdkman-init.sh
```
Check out sakai-scripts into some folder. I tend to make a folder named "dev" in my home directory:
```
  cd
  mkdir dev  (if needed)
  cd dev
  git clone https://github.com/csev/sakai-scripts
```
If you need a database, install MAMP or something else if you have not already done so

Continue working with the Common steps below.

Getting Started with WSL for Sakai Developers

Install WSL and Ubuntu. There are lots of good sources on the Internet on how to do this but there are some sticking points where you might get stuck:

Make sure Virtualization is enabled in your BIOS - it usually is off to be more secure - but if it is not on WSL2 will break with lots of cryptic messages - just boot into the BIOS and enable it.
Go into Start -> Control Panel -> Programs -> Programs and Features and enable Virtual Machine Platform, Hyber-V (Win11), and Windows Subsystem for Linux - The WSL option might only show up after you install WSL.

You can install WSL and Ubuntu from the Windows Application Store or using the wsl command in the command line (Start -> cmd).

wsl --install

You may need to restart your system, then do Start -> WSL - the first time after install you will need to make a Linux User and Password. Don't forget these. Once you can log into WSL and see a prompt like:

csev@WSLSakaiger:~$

You are in. We are going to install Tsugi in order to get Apache, PHP, MySQL, and PHPMyAdmin installed into your WSL.

sudo bash
cd /root
git clone https://github.com/tsugiproject/tsugi-build.git
cd tsugi-build
bash ubuntu/build-dev.sh

Once the software has been installed you need to configure the installation. This will be a localhost only setup with no access to port 3306 from the Internet.

cd /root
cp tsugi-build/ubuntu/ubuntu-env-dev.sh ubuntu-env.sh
source ubuntu-env.sh
bash /usr/local/bin/tsugi-dev-configure.sh return

Then in a Windows browser navigate to:

http://localhost/phpMyAdmin

Login as root / root And verify there is a tsugi database. You can use this to do MySQL maintenance and queries - because the port is shared between WSL and Windows, this can be done from Windows.

Navigate to

http://localhost/tsugi/admin

The admin unlock is tsugi - you should see the Tsugi Administration Console. This verifies the PHP, MySQL, Tsugi, PHPMyAdmin, and Apache installs are working and accessible from Windows.

Our next step is to install the Sakai scripts and dependencies.

sudo bash
cd /root
git clone https://github.com/csev/sakai-scripts
cd sakai-scripts
bash ubuntu.sh
source ~/.sdkman/bin/sdkman-init.sh

Verify that Java is installed with:

java --version

It should respond with something like:

openjdk 11.0.12 2021-07-20

Continue with the common steps below. Note we are doing all the steps below in WSL as root.

Common Steps For Ubuntu, WSL, and Mac

You can either run your own fork of Sakai, if you want to make PR's or run your own branches. But if you are just getting started you can just checkout the Sakai repo and run it - skip straight to "Compiling and Running Sakai".

Using Your Fork

Skip this section if you just want to checkout the main Sakai repository and get it running.

If you want to run your own fork, create an account on github and go to https://github.com/sakaiproject/sakai and "fork" a copy into your github account

If you want to push commits to your fork, set up git with the folowing commands

  git config --global user.name "John Doe"
  git config --global user.email johndoe@example.com

Open a teminal and navigate to the sakai-scripts directory and:
```
  cp config-dist.sh config.sh
```
Edit config.sh with a text editor of your choice
- Start at "Settings start here"
- Change "sakaiproject" in the GIT REPO variable to be your github account if you want to run your fork
- If you are not running MAMP or MariaDB, edit the MySQL root password, urls, ports, etc

Compiling and Running Sakai

(If you are in WSL make sure to be runnning as root with an sudo bash)

Make a copy of the ditributed sakai.properties and edit with the editor of your choice.

cp sakai-dist.properties sakai.properties
If you don't already have a database set up, make sure your MariaDB, MAMP or XAMPP is running and run bash db.sh to create a database. This script lists the databases so you should see sakai23 as one of the databases if things work. Note: If you run bash db.sh more than once it will drop and re-create the database.
```
  Active MySQL Databases:
  Database
  information_schema
  mysql
  performance_schema
  sakai23

  You will need add / update these lines in your sakai.properties

  username@javax.sql.....
  password@javax.sql.....
  url@javax.sql....
```

At the end of db.sh it prints out three lines that need to insert into sakai.properties.

Make sure to edit your copy of sakai.properties and edit/update the database strings before creating the Tomcat

Run bash na.sh to set up the Tomcat
Run bash co.sh to check the Sakai source code out (which repo is checked out is in config.sh)
Run bash qmv.sh to compile it all - the first time you do this it will take a long time and use a lot of network bandwidth as it downloads a bunch of library code in maven. The first full compile might take 20+ minutes - later full compiles will be about 2-3 minutes and partial compiles are 30 seconds or less. If the compile fails due to a download - just run bash qmv.sh again
Run bash start.sh to start the Tomcat
Run bash tail.sh to watch the logs (Press CTRL-C to stop the tail). Watch for tracebacks as the tail goes by. You can always take a look at apache-tomcat-*version*/logs/catalina.out if the tail output goes by too quickly. One the first start up on a fresh database, you might want to take a look at the tables in the sakai23 database to see Sakai add all the tables on the first start.

Sakai is up when you see a message in the log that looks as follows:
```
  ... AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
```
Navigate to http://localhost:8080/portal
To shut Sakai down, in the tail.sh terminal window, press CTRL-C and run bash stop.sh to shutdown Tomcat
Copy smv.sh into a folder in your PATH and set execute permission so you can recompile any sub-folder in Sakai that has a pom.xml by typing "smv.sh". The smv.sh command simply looks through your path to find the correct value for maven.tomcat.home and sets up some JAVA environment variables.

Description of the Scripts

Here are the scripts to be run in roughly the following order:

ubuntu.sh

This runs a bunch of apt-get commands to load the pre-requisites for Sakai for Ubuntu. If you are running something other than Ubuntu, you can look at this and use whatever package manager that is on the system to install the necessary pre-requisites.

config-dist.sh

This is a configuration script that sets a few parameters. If you want/ need to tweak the parameters simply copy this to config.sh so git does not try to check it in. If there is a config.sh, it will be used instead of config-dist.sh.

Generally you need to change the git repo to be checked out and the root mysql password for non-MAMP MySQL connections.

But there is a lot of flexibility here if you are setting up nightly build servers. Changing things like the HTTP port, shutdown port, and location of the Tomcat logs in config.sh allows you check these scripts out into several directories and make different nightly builds. If you get really tricky, you can override sakai-dist.properties by copying it to sakai.properties (which git will ignore) and change lots of things like switching from MySql to Oracle.

Note that db.sh does not understand oracle so if you are running oracle, you might want to change MYSQL_COMMAND to be "cat" so it does not try to run mysql to create an Oracle database.

db.sh

This sets up a MySql database with an account and password based on the configuration

co.sh

Checkout all of the Sakai source into the folder trunk. This checks out the main Sakai source and a few projects that Sakai depends on in case you want to peruse the source of those projects.

db.sh

Check to see if a database is there and create it if necessary.

na.sh

Download and create a fresh instance of Tomcat, patch configurations, and set up a sakai.properties

qmv.sh

Compile all of the Sakai source bypassing unit tests. This saves a lot of time in the initial compile. The mv.sh will compile with unit test. Once things work and you need to go out for an errand, you can run a mv.sh to compile with all unit tests.

start.sh

Start tomcat. Actually this script calls stop.sh at the begining to shut down any running tomcat on the configured port so it can aso be used as a "restart" to bounce Tomcat.

stop.sh

Stop tomcat

tail.sh

Tail the tomcat log.

Using Sakai

And at this point, you should be able to navigate a browser to:

http://localhost:8080/portal

If you need to recompile, the steps to shut down Sakai are to first abort your tail command with a CTRL-C and then run stop.sh

Usually to keep things clean, I remove all the files in the Tomcat logs folder before I restart Sakai.

Developer Re-Compiles

smv.sh

This is for developers working on one or more of the elements of Sakai. This can be run deep in the code tree at any maven node that can compile (i.e. has a pom.xml). You can actually put this in your ~/bin path and then just type ./smv.sh to do a re-compile into your Tomcat.

A surprising amount of Sakai development can be done without rebooting your Sakai. If you are working on tool code, generally you can recompile without a reboot. Changes to APIs in shared or components/services generally require a Tomcat bounce.

Setting up Nightly Builds

There is a lot of flexibility inherent in creating config.sh and/or sakai.properties that lets you set up a bunch of nightlies running on the same server and then having crontab entries to run things at whatever period(s) you desire. You can set up different databases, different TCP ports, log to some Apache web space, even use Oracle instead of MySql with some effort. The script nightly.sh is pretty much ready to handle the job once the configuration is in place.

You can test all the scripts separately as you are configuring things and then nightly.sh knits them together.

nightly.sh

This script just calls the other scripts in the right order. The script does a checkout, shuts down any tomcat (killing if necessary), creates a new empty database, creates a fresh tomcat, checks out the code you want to compile, compiles the code and starts Tomcat. Since the script stops and starts the right Tomcat automatically, you can run this script over and over interactively or in a cron job.

pravintargaryen/sakai-scripts

Sakai-Scripts

Getting started on ubuntu in Docker

Getting Started on Ubuntu Linux

Getting Started on Mac (as needed)

Getting Started with WSL for Sakai Developers

Common Steps For Ubuntu, WSL, and Mac

Using Your Fork

Compiling and Running Sakai

Description of the Scripts

ubuntu.sh

config-dist.sh

db.sh

co.sh

db.sh

na.sh

qmv.sh

start.sh

stop.sh

tail.sh

Using Sakai

Developer Re-Compiles

smv.sh

Setting up Nightly Builds

nightly.sh