Test | Coverage |
---|---|
E2E FE Test |
LiteFarm is the world’s first community-led, not-for-profit, digital platform joining farmers and scientists together for participatory assessment of social, environmental and economic outputs of farming systems. LiteFarm is the first application of its kind specifically tailored to the needs of diversified farmers with built-in pathways to provide expert decision support and help them earn additional income through payment for ecological services (PES) schemes and in-app certifications (such as organic). These approaches serve the multiple purposes of incentivizing adoption of sustainable land use practices through the provision of evidence-based decision support, and significantly increasing the amount of data being collected by diversified farming operations around the globe. It was developed with farmers at the center of the design process and built from the ground up with accessibility and approachability in mind. We are proud of our mission:
To meet farmers where they are and equip them with the tools they need to make informed and responsible decisions about the health of their farm, their livelihood, their community, and the planet.
LiteFarm version 1.0.0 was released to the public in July 2020. The LiteFarm app is continually being developed, with farmers, researchers, designers and developers working together to create new localized modules and features into the future. LiteFarm is deployed in Canada, the USA, and Latin America.
If you’re a farmer and would like to join LiteFarm you can sign up today at app.litefarm.org. If you are a researcher or would like to find out more about this project you can contact the UBC Centre for Sustainable Food Systems. If you're a developer, welcome to the team! All the details on how you can contribute to this project are right here.
LiteFarm is comprised of three applications which all reside in this monorepo.
packages/webapp
is the client-facing applicationpackages/api
is the back-end API server with entry pointsrc/server.js
packages/api/src/jobs
is the "jobs scheduler" for certification exports, with entry pointindex.js
- Check to see if you have Node.js installed. We use the version specified in the
.nvmrc
file of each folder with package imports. On a Mac use the commandnode -v
in terminal. If it is installed, the version in use will be reported in the terminal. If not, install it from node.js. - Check to see if you have NVM installed. On a Mac use the command
nvm -v
. If you do not have NVM (Node Version Manager) installed, install it using these instructions: NVM - Check to see if you have pnpm installed. On a Mac use the command
pnpm -v
. If it is installed, the version will be reported. If you do not have it installed, runnpm install -g pnpm
in a terminal. - Clone the repository from Github to your computer. On a Mac, in a Terminal window navigate to the directory you want to put the files in. Then use the command
git clone https://github.com/LiteFarmOrg/LiteFarm.git
. - Install all packages:
- Switch to current version of Node.js by running
nvm use
in the root folder - Navigate to the root folder
/
of the repo and runnpm install
. - Navigate to the
packages/api
folder, and runnpm install
. - Navigate to the
packages/webapp
folder, and runpnpm install
. - Navigate to the
packages/shared
folder, and runnpm install
.
- (Highly recommended) Go to Get Docker | Docker Documentation and install Docker. Docker is the recommended method for setting up development dependencies, but alternative instructions will be provided as well if you are unable to use Docker. There will be extra instructions for each operating system. For example: Windows requires Windows subsystem for linux (WSL). Mac users may need to enable experimental/beta settings for Mac users.
The applications are configured with environment variables stored in .env
files. Configuration information includes secrets like API keys, so the .env
files are not included in this git repository.
This repository contains a .env.default
file for both api and webapp. These files contain directions to acquire the personal keys needed to get LiteFarm running locally. Please note you will want to copy .env.default
and rename the file to .env
. Only after adding the .env
file should you proceed to add the new keys. If you add your api keys to .env.default
you may accidentally expose your keys since this file is tracked on git.
If you have questions about the other api keys, or wish to join the LiteFarm team, please contact community@litefarm.org.
Run docker compose up
in the root directory to configure and start all development dependencies, including the database.
For more information, see services (local development dependencies) below.
Once the database container is running and the .env
files have been configured as described below, in a terminal navigate to the packages/api
folder. Execute npm run migrate:dev:db
to run the migrations that set up the PostgreSQL database used by the app.
Instructions
1. If using Windows, install PostgreSQL by downloading installers or packages from https://www.postgresql.org/download/. Mac and Linux users can use homebrew with the commands shown below (a link for installing Homebrew is below too!). The second command can take up to 10 minutes because it may trigger the compilation of a new binary.In a Terminal window:
# Install homebrew if you don't already have it with the command:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
# Install PostgreSQL.
brew install postgresql
# Start the Database Management Systems (DBMS) service.
brew services start postgresql
-
Set up the PostgreSQL role (account) and databases. Use the
psql
client program. If an installer asks you to choose a password for thepostgres
(superuser) account, usepostgres
for consistency with the contents of.env.default
.-
On a Mac, type "psql" in the terminal to start the client.
If this returns the error, "/... postgresql.plist: service already loaded..." then you need to remove a .pid file that is interfering with the start of the DBMS service. On a Mac, use the terminal command,
rm /usr/local/var/postgres/postmaster.pid
thenbrew services restart postgresql
.Then use the Linux commands below to set the postgres user password, and make two new databases.
-
Linux. In a terminal, start the client with
sudo -u postgres psql
, then execute each of the following commands. (The last command terminates the client session.)ALTER ROLE postgres WITH PASSWORD 'postgres'; CREATE DATABASE "pg-litefarm"; CREATE DATABASE test_farm;
Then exit with, exit;
-
Windows. At the Start menu, type
psql
and the search results will show "SQL Shell (psql)". In the client, execute each of the following commands. (The last command terminates the client session.)CREATE DATABASE "pg-litefarm"; CREATE DATABASE test_farm;
Then exit with, exit;
For Windows, the ALTER ROLE command is not used because the password is set using the wizard installer downloaded.
-
-
Set the value of
DEV_DATABASE_PORT
inpackages/api/.env
to the correct port of your PostgreSQL installation. The default port is5432
-
In a terminal, navigate to the
packages/api
folder. Executenpm run migrate:dev:db
to run the migrations that set up the PostgreSQL database used by the app.
In a terminal, navigate to the packages/api
folder. Run npm run nodemon
to launch the backend application. Nodemon will automatically restart the application when changes are made to the backend code.
In a terminal, navigate to the packages/webapp
folder and run pnpm dev
. This builds the frontend code, and starts a web server that will automatically reflect any changes you make to the frontend.
Load the frontend app in your browser at http://localhost:3000.
Background & Details
The LiteFarm database can be run either directly on your local machine or, for ease of setup, in a pre-configured Docker container. We recommend using Docker, and so the database is included by default in the docker-compose.yml
file. Please note the docker-compose exposes port 5433
to the host machine for connecting your Postgres client of choice (e.g. Postico/pgAdmin4).
If you prefer to install and use Postgres natively, instructions for different operating systems are provided above under Database Setup - Native installation.
In beta and production, images, uploaded files, and certification exports are all stored on Digital Ocean Spaces (AWS S3-SDK compatible buckets). Organic certification document creation is handled by a node.js application that runs separately from the API alongside an image compression microservice called the imaginary, both hosted on Digital Ocean Droplets.
In the local development environment, we use MinIO, a free and open-source drop-in replacement for AWS S3. MinIO, the export server, and the export server's dependency Redis (used to manage its job queue) can all be run in either Docker containers or natively, depending on developer preference. The easier method is to use Docker and let the docker-compose.yml
file handle the configuration.
Run docker compose up
in the root directory of the project.
Make sure that your .env
variables are configured as outlined in the .env.default
files (see section above). When exporting certification documents, both the webapp and API need to be running.
Also see the section below on using Docker.
Instructions for running image, document, and certification services natively
-
Create and run a local Redis database with password "test". Make sure to record the correct port under
REDIS_PORT
inpackages/api/.env
-- the default is 6379 -
Install and configure MinIO. You will want to create a Single-Node Single-Drive (Standalone) MinIO installation. The MinIO website lists instructions for MacOS along with other operating systems. Use the default port.
-
Use the MinIO console to
- create a new bucket called
minio-dev
and set its access policy to "public" - generate an access key. Record both key + secret
- Note: for local use you may instead use the default username and password -- both
myminioadmin
-- in place of keys
- create a new bucket called
-
Download aws-cli and configure it with your MinIO access key + secret directly in the terminal using:
aws configure
Make sure that the region name is either removed from your aws configuration or set up correspondingly in your MinIO admin panel.
-
Connect MinIO to LiteFarm:
in
packages/api/.env
make sure you have the following variables set:MINIO_ENDPOINT=http://localhost:9000 PRIVATE_BUCKET_NAME=minio-dev PUBLIC_BUCKET_NAME=minio-dev
in
packages/webapp/.env
:VITE_DEV_BUCKET_NAME=minio-dev VITE_DEV_ENDPOINT=localhost:9000
-
Make sure both the LiteFarm api and webapp are already running, then run the export server from
packages/api
usingnpm run scheduler
A detailed walkthrough (with screenshots) is also available on the LiteFarm Confluence.
You can also use the same MinIO bucket to store documents uploaded from the Documents view of the webapp. To configure this, set up MinIO as described above, add the access key credentials to /api/.env
, e.g.
DO_SPACES_ACCESS_KEY_ID=myminioadmin
DO_SPACES_SECRET_ACCESS_KEY=myminioadmin
The imaginary only exists as a Docker container (both in production and for development), and we highly recommend running it with docker compose up imaginary
You will need these variables set in your /api/.env
:
IMAGINARY_TOKEN=localonlytoken
LOCAL_IMAGINARY=http://localhost:8088
If you are unable to use Docker, please contact a core team member to get instructions on using the production Imaginary.
To run ESLint checks execute npm run lint
The chai.js and jest libraries automate tests that run real database operations using JWT. The tests use a dedicated database named test_farm
, distinct from the pg-litefarm
database that the app normally uses .
- In a terminal, navigate to the
packages/api
folder. - Execute
npm run migrate:testing:db
to set up the test database. - Execute
npm test
to launch the tests. Or, to generate test coverage information, runnpm test -- --coverage .
and then see thecoverage/index.html
file.
While the tests do attempt to clean up after themselves, it's a good idea to periodically use psql
to DROP
and CREATE
the test_farm
database, followed by the migrations from step 2 above.
To run ESLint checks execute pnpm lint
Since this is a mobile web application, webapp should be viewed in a mobile view in the browser.
You can also test LiteFarm on your actual mobile device using the network adddress returned by vite --host
when you start the webapp in development mode. To do this, also update VITE_API_URL
in your webapp/.env
file from localhost to that address (or your computer's network name) and the appropriate API port. Most of LiteFarm can be tested like this, but please note that Google SSO and some other functionality will not work over the local network.
Please see https://ngrok.com/ for more general information about ngrok.
While not required for most developers, use cases in which we currently utilize ngrok at LiteFarm include:
- Testing local changes on phones or different devices
- Testing local changes when working with other APIs and integrations
- Go to https://ngrok.com/ and sign up for an account
- Install the ngrok CLI (on the Getting Started page after signing up)
- Create a copy of
ngrok/ngrok.default.yml
and call itngrok.yml
. Make sure this file is in thengrok
folder at the root of the repo - Login to your ngrok account and go to https://dashboard.ngrok.com/get-started/your-authtoken
- Add the auth token from there in
ngrok.yml
by replacing the?
These commands can be run from the root of the repo.
npm run ngrok
to forward both backend and frontend ports with ngroknpm run ngrok:setup
to add the ngrok urls to the file .env files (always run after forwarding a port to ngrok)npm run ngrok:api
to forward the backend port with ngroknpm run ngrok:webapp
to forward the frontend port with ngrok
Note: Please make sure to run the commands in the following order:
npm run ngrok
ornpm run ngrok:api
ornpm run ngrok:webapp
npm run ngrok:setup
(in a new terminal)pnpm dev
(in a new terminal from thepackages/webapp
folder)npm run nodemon
(in a new terminal from thepackages/api
folder)
Please see https://docs.docker.com/ for more general information about docker.
Use cases in which we currently utilize docker at LiteFarm include:
- Managing development dependencies in the local development environment, including the database and services for working with images, files, and exports
- Simulating the server environment.
- Building LiteFarm application using docker commands and supporting its components using containers.
- Go to https://docs.docker.com/get-docker/ and install docker in your local system.
- After installation, the docker CLI will be available where you can run the docker commands.
- create a
.env
file at the root directory of the project i.e. LiteFarm - Add key-value pairs in the
.env
by referring to thedocker-compose.[ENV].yml
that contains the docker env keys.
-
On Windows: During
npm run nodemon
of/packages/api
there is an error - possibly with the client variable of knex.The NODE_ENV variable is not being set properly you will need to adjust the nodemon script in
/packages/api/package.json
.We hope to improve our
package.json
scripts in the near future.
These commands can be run from the root of the repo.
docker compose up
to run all the local development services, ORdocker compose up [service names]
to run one or more particular servicesdocker compose down
to stop and remove all containers defined indocker-compose.yml
docker compose down --volumes
to stop and remove all containers and volumes defined indocker-compose.yml
docker-compose -f docker-compose.[ENV].yml up --build -d
to build the docker containers in the detach mode.docker ps
to see the list of docker containers in the running state.docker logs --details [containers name]
to view the logs inside the container.
Notes:
- [service names] are db, minio, redis, export_server, and imaginary
- [container_name] are litefarm-db, litefarm-api and litefarm-web.
- [ENV] are beta and prod
You can use Storybook to visualize and test out the UI components used throughout the app.
- To view Storybook on your local environment, run
pnmp storybook
on thepackages/webapp
directory and navigate to http://localhost:6006 - To view the deployed version of Storybook, which is updated automatically with the latest changes on the integration branch, go to https://65316fc4f177c73a9181a843-yobioprfjv.chromatic.com
Please email: community@litefarm.org for more details.