This is a resource for developers new to DBCA, to get to a basic standard of understanding of our development tools and processes. Individuals should feel free to skip elements for which they already have a good level of competency. This page is not prescriptive, but it represents a sensible order of operations for a new hire to go through.
Additional resource documentation pages include the following:
To summarise the recommended technology stack for new web development projects:
- Python - programming/scripting language
- Django - Web application framework
- PostgreSQL - database
- Docker - application containerisation
- Git - source control
Python is a general purpose programming language which is dynamically typed, interpreted, and known for its easy readability. General principles to be aware of when undertaking development of applications using Python:
- Python 2.x/3.x - New projects should always be developed against a recent version of Python 3. Notwithstanding the similarities, Python 2 is no longer maintained for the purposes of features or security.
- Correct tool for the job - developers should be mindful of using Python where its strengths are greatest (rapid prototyping, versatility, portability, maintainability) versus other tools that may have advantages (SQL for database queries, command-line tools for file/text handling, JavaScript for DOM manipulation, etc.)
For installation, most *nix operating systems come with a version of Python installed which will probably be sufficient for learning. For usage instructions, see this page. For installation on Windows, see this page.
A suggested syllabus for learning Python for the purposes of web application development is as follows:
- Install Python 3.x on your PC.
- If required, take a crash course in the Python syntax. This is also a good intro for Python: https://realpython.com/python-first-steps/
- For a more thorough overview of Python syntax, read the Dive Into Python ebook, chapters 0, 1, 2, 3, 4, 7, 11 & 14 (the whole book is valuable, but these chapters will get you basically productive).
- The Hitchhiker's Guide to Python is a well-maintained, opinionated guide of best practices for installing, configuring and using Python: https://docs.python-guide.org
Once you start working on more than one project, managing separate and isolated Python environments for each one becomes a requirement. This topic is slightly "extra credit" (it's most important to get in and start learning the syntax), but it's worth learning sooner rather than later. This is a good primer on Python virtual environments and why you need them. The specifics for how to manage Python virtual environments are very much up to individual developers, but a good starting point is as follows:
- Use pyenv to install and switch between different versions on Python on a host (installation instructions).
- Use Poetry to manage project dependency trees (installation instructions).
Another popular tool for managing Python dependencies is pipenv, as an alternative to Poetry. There is no clear market leader within this topic, and developers are encouraged to experiment and find their own best fit as they gain experience. Additional resources related to managing Python environments:
- Hypermodern Python setup: https://cjolowicz.github.io/posts/hypermodern-python-01-setup/
- pyenv tutorial: https://amaral.northwestern.edu/resources/guides/pyenv-tutorial
- Set up an awesome Python environment: https://towardsdatascience.com/how-to-setup-an-awesome-python-environment-for-data-science-or-anything-else-35d358cc95d5
- Overview of Python dependency management tools: https://modelpredict.com/python-dependency-management-tools
Django is our current tool of choice for non-trivial web applications requiring a significant amount of business logic, direct manipulation of databases, and a large amount of user interaction. It is a "full stack" web framework that includes all of the components required to develop a complex web application for internal or external customers. It also boasts some of the best online documentation to be found.
Examples of web applications developed using Django:
- Planning Referral System - code repository at https://github.com/dbca-wa/prs
- IBMS - code repository at https://github.com/dbca-wa/ibms
To minimise development effort, we typically take the following approach when building a new system:
- Define a good data model with constraints: Django models.
- Add validation for model fields.
- Use the built-in Django admin for testing and validation of the data model.
- Customise the Django admin for internal end users (e.g. hiding fields, add business rules, add related inlines as tables/stacked forms). This should be sufficient for a small, proficient group of users.
- Replace high throughput (lots of users / users with not much scope for training such as the public) interface pages with VueJS and API interactions such as Django Rest Framework if you enjoy the pain of developing in two languages, or with Django's own class-based views for more oldschool request/response forms.
An example of a web application that make heavy use of JavaScript in the UI is ParkStay Bookings.
A suggested syllabus for learning Django is as follows:
- Once you have a good understanding of Python syntax, complete the ENTIRE Django tutorial (really, the whole thing): https://docs.djangoproject.com/en/dev/
- As an alternative/supplement to the Django docs tutorial, Mozilla has an excellent tutorial series for Django: https://developer.mozilla.org/en-US/docs/Learn/Server-side/Django
All modern web applications in the department are built as Docker images and deployed using Kubernetes. This is a whole technical speciality on its own, so we suggest that new developers not already familiar with Docker ease into it over time (concentrate initially on Python, Django and getting set up with our other collaboration tools).
For the enthusiastic, here are some resources to starting learning about Docker, and an intro to Kubernetes below.
Resources that are useful / relevant for internal developers to learn just enough about Kubernetes to be productive.
- Introduction to Kubernetes on Azure - Learning path from Microsoft. Pretty thorough introduction to Docker, containers and AKS in general.
- Kubernetes Documentation - comprehensive, but not the best introductory learning resource. Has a pretty good set of interactive tutorials.
- Introduction to Kustomize - Kustomize provides a solution for customizing Kubernetes resource configuration.
- Lens - a nice client-side management tool for Kubernetes clusters. See this article for an overview.
- Rancher Desktop - an open-source desktop application for Mac and Windows, providing Kubernetes and container management in a desktop installer.
- Git - for version control. Refer to the "Source control" section below.
- Windows Subsystem for Linux - for developing more easily on windows.
- 1Password - a service for managing and sharing organisational secrets.
- Keep sensitive information out of project repositories - don't commit user credentials, passwords, access keys or database dumps to the repository. Get in the habit of creating a
.gitignorefile and adding relevant file patterns to it in order to reduce the chance of this happening. - ISAP - the department runs an Information Security Awareness Program training course that is compulsory. Even if the information seems basic, try to internalise the content and cultivate a security mindset.
- In the case of credentials or keys, the recommended approach is to make your settings/config code read these pieces of information in via environment variables.
- Set up an Ubuntu-based development environment and get to mastering the Linux command line, if you don't already use it. This is a valuable skillset for any IT professional; your career will thank you. In a Windows 10 environment, we have found it to be a good experience to set up an Ubuntu development environment using WSL.
- Undertake a small development project, ideally in collaboration with another new team member (talk to a senior team member for a project specification).
OIM application developers are encouraged to undertake development locally in an environment that is as close as possible to the final deployed application environment in order to reduce the incidence of "works on my machine" problems.
For the best development experience, OIM recommends developers install a variant of Linux on their workstation, either outright or in a dual-boot arrangement with Windows. We've found Ubuntu 20.04 and up to have a good user experience plus up-to-date packages, but the choice of distribution can be entirely up to the user.
Some good additional resources from the internet are as follows:
Microsoft has made it surprisingly easy to run a Linux environment under Windows 10. Windows Subsystem for Linux is a fairly good substitute for a full VM, and is a lot more pleasant for Python development than using the Win32 ported equivalent of the tools. There are a few cases where support is less than optimal (e.g. some complex filesystem interactions fail, meaning you can't run PostgreSQL), but for basic development it provides a good alternative or transitional step to a full UNIX environment.
A decent guide for installing Docker inside the Ubuntu WSL is here (includes instructions for installing the Azure CLI and dotnet core).
Failing that, there's the option of using an emulator like VirtualBox to run a local Linux VM for development purposes but this can consume a fair amount of your computer's resources.
Apple includes a lot of UNIXy tools in the standard OS X environment, but isn't very prompt in keeping the software up to date. You may have some luck using Homebrew, which is styled to be like a Linux package manager. See https://hackercodex.com/guide/mac-development-configuration/
Git is a free and open source version control system. Learning to use Git is non-trivial and can be intimidating. However, it is the most widely-used VCS in the world therefore learning to use it will be a good investment of time for any developer.
All public DBCA source code repositories are located at the department's GitHub account.
GitHub provides this list of introductory resources for getting started with Git.
Here is a very basic workflow of a Git repository and a developer's fork of it:
- Create a fork of a repository (repo) on Github.
- Clone the forked repo to your local machine (your fork on Github becomes the "origin").
- Make changes to your local repository and commit the changes.
- Push your commits to your remote repository (the fork) on Github.
- Open a pull request (PR) from your fork to the original repo (propose changes).
- The project owner reviews and merged your PR into the original repo.
Obviously things become progressively more complicated as we add things such as branching, multiple developers collaborating to update a repo, etc. The specific workflow in use should be settled on in each team; consistency of usage within a team is more important than the workflow chosen. A good, basic workflow is the the Github Flow: https://docs.github.com/en/get-started/quickstart/github-flow
A helpful graphic demonstrating the Git commands to move a repository state between locations is as follows:
Below are some additional resources to assist with learning Git:
- Git tutorials - a good set of tutorials about Git functions by Atlassian.
- Git cheatsheets - reference sheets covering essential Git commands.
- Git Immersion - a guided tour through using Git via a series of lessons.
- Learn Git branching - learn the workflow of branch and merging at the CLI, in your browser.
- Awesome Git - a curated list of resources about Git.
The high-level expectations for source code management are explained in the ACSC guidelines for Software Development. For most multi-contributor projects we use GitHub pull requests as the workflow for contributions/code review.
We recommend the following steps for getting started with a code repository:
- In the GitHub UI, fork the main repository to your own account.
- Add your fork of the repository as the origin
git clone https://github.com/your_username/project_name.git cd project_name - Add the original version of the repository as an "upstream" source:
git remote add upstream https://github.com/dbca-wa/project_name.git
Do all of your work in this fork, make a bunch of commits, push frequently, etc. Once a feature is done, use the GitHub UI to create a pull request from your fork to the upstream repository, so that it may be code reviewed. Developers might wish to work on specific features or fixes in a branch and then merge that branch to the master/main branch as needed, however the specific workflow can be defined by the project owner.
The following ebook is a well-regarded and very readable collection of advice related to software development: Andrew Hunt, David Thomas - The Pragmatic Programmer - your journey to mastery