ExpressionParser

Introduction

ExpressionParser parses and evaluates mathematical and boolean expressions. It utilizes a lexer and parser generated using the Antlr parser generator. The framework is compiled into a JAR file (along with a JavaDoc JAR) in the dist/ directory. Just add these to your classpath or import as a library to your project in your IDE of choice.

Simple Usage

In its simplest terms, all you have to do is provide the Expression class which an expression to evaluate and then call eval() to obtain the result. For example:

Expression e = new Expression("[1,2,3] + [4,5,6];");
Context result = e.eval();

System.out.println(result); //prints [5, 7, 9]

Types

ExpressionParser has support for Scalars, Vectors & Matrices and operations can be changed based on the type of variables that are being operated on. Vector & Matrix values can be defined as Scalar values, variables or expressions which evaluate to a Scalar value.

Scalars

Scalar values are any real number or an expression or variable which can be evaluated to a real number.

Vectors

Vectors are defined using braces where the values are delimited using a comma. Values can be defined from either expressions, variables, functions and real numbers. ('e' can used for exponential notation i.e 45e2 == 4500)

eg. [1,2,3,(2 + 2)] would obviously be evaluated as a Vector with the values 1,2,3,4.

Matrices

Matrices share a similar syntax to Vectors which the only difference being a semi colon to distinguish the end of a row.

eg. [1,2,3; 4,5,6; 7,8,9] would be evaluated as a 3 x 3 matrix.

Matrix & Vector Operations

Operations performed on Matrices and Vectors require that the dimensions of the left and right sizes of the operation agree.

Matrix & Vector Array Access

Matrix & Vector values (including any function or variable which evaluates to a Matrix or Vector) can have its values accessed using using array access notation.

So, for example [1, 2, 3][0] will return the Scalar value 1.0 (Vector and Matrix indices always start from 0).

When working with a Matrix you can provide two values to access a specific row/column index or a single value to return the row vector at index n.

Again, for example [1, 2, 3; 4, 5, 6][0] would return the row vector [1, 2, 3] and [1, 2, 3; 4, 5, 6][0,1] would return the Scalar value 2.0.

As stated above, accessing values using this notation can also be achieved from function results and evaluated variables, so, for example, SIZE([1,2,3; 4,5,6])[0,1], SIZE(A)[0,1] and A[0] are all valid.

Scalar, Vector & Matrix values can all be negated. i.e var e = -[1, 2, 3]; print e; //prints [-1.0, -2.0, -3.0].

Variables

Variables can be referenced in expressions and then added to the Expression class before evaluating the result. Variables can be of any Type (i.e a Scalar, Vector or Matrix) and be referenced by number of letters which are not used by any other keyword or function definition.

for example:

Expression e = new Expression();

Vector x = new Vector(new double[]{ 1,2,3 });
Scalar y = new Scalar(10d);

e.addVariable("x", x).addVariable("y", y);

e.setExpression("x * y;");
Context result = e.eval();

System.out.println(result); //prints [10, 20, 30]

Variables can also be negated using the - operator before the variable name. i.e var e = [1, 2, 3]; print -e; //prints [-1.0, -2.0, -3.0].

Variable type checking

Defined variables can have their type checked as a logical expression (where the evaluated result will be a 1 or 0). This can be done using the instanceof keyword. So for example: var a = [1,2,3]; (a instanceof Scalar); would return 0. As stated above there are 3 different types. Scalar, Vector & Matrix.

Syntax

The syntax conventions copy that of other traditional programming languages so it should be quite familiar to use. An expression script can contain one or more expressions terminated with a ';' (semi-colon).

Import Statements.

We can use import statements in a script to split up functionality across multiple files. Import statements have to be defined at the start of an script. They are essentially the location of the file without the file extension (which is assumed .ex) with the directory seperator as '/'. if the import starts with a leading '/' it is assumed to be an absolute directory path and if the OS is windows and the first directory is the path is a single letter on an absolute path, this is determined to be a drive letter and is suffixed with a trailing ':'.

For example, for a file located at C:/example/file.ex on a windows machine we would have:

import /c/example/file;

.... rest of script file ....

Assigning Variables.

Variables can be assigned during expression evaluation using the 'var' keyword. Variables can be assigned from any valid expression, i.e a function return, evaluated expression or from another variable. For example:

Matrix A = new Matrix(
    new double[][]{
        { 100,  2, 3 },
        {  -2, 34, 2 }
    }
);

Expression e = new Expression("var b = A; SIZE(b)[0] == 2;");
e.addVariable("A", A);

System.out.println(e.eval()); //prints 1.0 (boolean TRUE)

Result Value

The last evaluated expression in the script will be returned as the result of the evaluation, but all expressions in the script will be evaluated. So for example:

Expression e = new Expression("var a = [1, 2, 3]; 1 + 1; a;");
System.out.println(e.eval()); //prints [1, 2, 3]

There are also scenarios in which the evaluated result of the expression can be nothing. An example of this is an if statement which no expressions to evaluate inside or an empty expression or an expression only containing assignment operations. In this case the Context returned is classed as 'empty' and contains a null result value.

Expression e = new Expression("var x = 4; if(4 < 3){ /* empty context. */ }");
System.out.println(e.eval()); //prints [Empty Context] as no return value was determined.

However, even though empty contexts are supported, attempting to use an empty context in an operation or a function will result in an an exception being thrown.

Expression e = new Expression("function hello(arg) {} hello(1) + 2; SIZE(hello(1));");
System.out.println(e.eval()); //throws exception as performing operations on empty contexts is invalid.

You can print out results from other expressions using the print keyword which prints the result to the output listener.

Output Listener

When the print keyword is used, the output listeners print method is triggered. By default it just prints the result of the evaluated expression to the console. This functionality can be overrided using the setOutputListener method in the Expression class.

For Example:

Expression e = new Expression();

e.setOutputListener(
	new OutputListener() {
		@Override
		public void print(Context context) {
			System.out.format("%s = %s\n", context.getExpression(), context.getValue());
		}
	}
);

//using print on the final statement also triggers the listener as well as returning this result.
e.setExpression("var x = 1; var y = 2; print x; print y; print x + y;"); 
//prints:
//x = 1.0
//y = 2.0
//x + y = 3.0

The OutputListener interface also has a variety of other methods which will need to be overriden, these are:

syntaxError() - triggered when a syntax error is found while parsing the expression.
exceptionThrown() - triggered when any exception is thrown in the application.
onReturn() - triggered when a return statement is used in the script. (deprecated)

Commenting

Expressions also support commenting in the form of /* comment */ and //comment which are ignored by the parser.

Updating Variable Values

You can also re-assign values to already defined values, for example var i = [1, 2, 3]; i = [4, i[1], i[2]]; would print [4, 2, 3]. We can also update certain values in data structures using array access notation, for example: var i = [1, 2, 3]; i[0] = 4; would also print [4, 2, 3].

Incrementing/Decrementing Values

You can apply the ++ or -- operators after any Scalar, Vector & Matrix to increment or decrement all values in that type by one. However, because these operators are supported on values which don't nessacarily have a variable name (i.e SIZE([1, 2, 3])++), To store this incremented value as a variable you should use the var keyword (or just the variable name if using an existing variable).

So for example: var n = 0; n = n--; and var b = [1, 2, 3]++; are both valid statements.

The operators that are used in incrementing/decrementing variables are now controlled using FixedOperators which means their functionality can be customised to meet your needs.

However, since for loops use these statements to control the execution of the loop condition, changing the functionality of these variables will also affect this functionality, as for loops are bound to the API.

Control Statements

If Statement

The ExpressionParser also has the capability to handling simple if/else statements. For example:

var r = 2;

if(r < 2 || r > -1) {
	var a = 1;
} elseif(r == 4) {
	var a = 2;
} else {
	var a = 3;
}

print a;

Expression e = new Expression(new File("/path/to/source/file.ex"));
System.out.println(e.eval()); //prints a = 1;

The ternary operator is also available to shorthand if statements. var e = (b > 2) ? TRUE : FALSE.

For & While Loop

ExpressionParser also supports simple for and while loops in its computation. Be aware however that infinite loops can occur from using loop control structures. For and while loop have access to any variable which was defined before the loop declaration, but any variable defined inside the loop scope is only available inside that scope.

While Loop

var i = [1, 2, 3];
var n = 0;
while(n < SIZE(i)) {
	print i + n;
	n = n++; //needed to stop infinite loops.
}

Expression e = new Expression(new File("path/to/source/file.ex"));
System.out.println(e.eval());
//prints:
//i + n = [1, 2, 3]
//i + n = [2, 3, 4]
//i + n = [3, 4, 5]

For Loop

var i = [1, 2, 3];
for(var n = 0; n < SIZE(i); n++) {
	print i + n;
}

Expression e = new Expression(new File("path/to/source/file.ex"));
System.out.println(e.eval());
//prints:
//i + n = [1, 2, 3]
//i + n = [2, 3, 4]
//i + n = [3, 4, 5]

Return Statement

Return statements are supported in script and can be used with the return keyword. When using a return statement a value has to be returned. For example:

function helloWorld(a) {
	if(!(a instanceof Scalar)) {
		return [1,2,3];
	}
	[4,5,6]; //the last evaluated statement is returned.
}

var a = helloWorld(1); //prints [4,5,6];
var b = helloWorld([1]); //prints [1,2,3];

Expression e = new Expression(new File("/path/to/source/file.ex"));
System.out.println(e.eval());

Operations

The core operations that are included are as follows: (Also be aware that these are in the order that the operator takes precedence in the operation.)

Exponents (Powers) '^'
Division '/'
Multiplication '*'
Addition '+'
Subtraction '-'
Modulo '%'
Bitwise '>>' & '<<'
Dot '.'

As stated above, the ExpressionParser also supports logical operators, which are:

Equals '=='
Not Equals '!='
Greater Than '>'
Less Than '<'
Greater Than Or Equal '>='
Less Than Or Equal '<='
And '&&'
Or '||'

The result from using logical operators will either be 1 or 0, i.e boolean TRUE or FALSE.

Adding/Overriding an Operator

In addition to using the core operations, you can also add or override any exisiting operator the ExpressionParser uses.

When implementing an operator, you need to provide it with the symbol which the parser will use to match the operation and also the operation types that will take place, dependant on the left and right types that are provided in the operation.

Operation Types

There are different operation types which can occur in each operation. For example, even though 2 + 2 and [1,2,3,4] + [1,2,3,4] use the same operator '+', they will be evaluated differently based on the left and right types that are provided.

The operation types that can occur are:

Expression Type	Description
EXPRESSION_SCALAR	A Scalar - Scalar operation
EXPRESSION_VECTOR	A Vector - Vector operation
EXPRESSION_MATRIX	A Matrix - Matrix operation
EXPRESSION_SCALAR_VECTOR	A Scalar - Vector operation
EXPRESSION_SCALAR_MATRIX	A Scalar - Matrix operation
EXPRESSION_VECTOR_SCALAR	A Vector - Scalar operation
EXPRESSION_MATRIX_SCALAR	A Matrix - Scalar operation
EXPRESSION_MATRIX_VECTOR	A Matrix - Vector operation
EXPRESSION_VECTOR_MATRIX	A Vector - Matrix operation

When creating a new operator you can define 1 or all of the operation types based on what your operator will support. There are also some other expression types defined in the Operator class to make your life easier in defining operations. These are:

Expression Type	Description
EXPRESSION_ALL	This matches all types of operation stated above.
EXPRESSION_MATRICES	This matches all operations where a Matrix is at the left of the expression.
EXPRESSION_VECTORS	This matches all operations where a Vector is at the left of the expression.
EXPRESSION_SYMMETRIC	This matches operations where the type of the left and right sides are the same.
EXPRESSION_SCALARS	This matches operations where a Scalar is at the left of the expression.

Implementation

So, for example, to add a new operator which overrides the default implementation of the '+' (plus) operation we can do:

Expression e = new Expression();

e.addOperator(
	new Operator("+")
	.addEvaluator(
		//Can provide an int[] of expression types or EXPRESSION_ALL, MATRICES, VECTORS or SYMMETRIC.
		Operator.EXPRESSION_SCALAR,
		new Evaluator() {
			@Override
			public Type eval(Arithmetic left, Arithmetic right) {
				//only take absolute values (sign is disregarded.)
				//the Arithmetic interface provides access to a lot of pre-defined arithmetic methods.
				return left.absolute().plus(right.absolute());
			}
		}
	)
);

//Our '+' operator now only supports Scalar - Scalar expressions.
e.setExpression("-2 + 2;");
Context result = e.eval();

System.out.println(result); //prints 4

Fixed Operations

Fixed Operators can also be added and customised. A fixed operator is where an operator is pre/postfixed to a value (An example is the ++/-- operators.) These are defined in effectively the same way as normal operations except we use a FixedOperator operator and this requires a secondary variable or whether this operator is pre or post fixed to the value.

So for example, if we wanted to add a '++' operator which is prefixed to variables then we can do:

Expression e = new Expression();

e.addOperator(
    new FixedOperator(
        "++", 
        FixedOperator.PREFIX
    ).addEvaluator(
        Operator.EXPRESSION_ALL,
        new Evaluator() {
                @Override
                public Type eval(Arithmetic left, Arithmetic right) {
                    //right is always null.
                    return left.add(Scalar.TWO);
                }
        }
    )
);

e.setExpression("var a = 1; ++a;"); //prints 3.0

Post and prefix operations cannot be carried out simultaneously i.e. var a = 3; --a++; is not valid.

The default functionality of fixed operations includes:

all of the below examples use the variable a assigned at the value 2.

Operator	Prefix	Postfix	Description
++	++a = 4	a++ = 3	prefix adds a to itself, and postfix increments the value by one.
--	--a = 0	a-- = 1	prefix subtracts a from itself and postfix decrements the value by one.

Also Fixed operators only support the expression types EXPRESSION_SCALAR, EXPRESSION_VECTOR, EXPRESSION_MATRIX & EXPRESSION_ALL (for all).

Functions

ExpressionParser also has a collection of functions that are available which can manipulate or generate values. The core functions available are:

Functions which accept Scalars, Vectors & Matrices as parameters.

Function Name	Description
LOG(expression)	Computes the Natural Logarithm for an evaluated expression.
LOG10(expression)	Computes the Base 10 Logarithm for an evaluated expression.
RANDOM(args?)	Generates a random Scalar, Vector or Matrix value.
SIN(expression)	Calculates the Sine from an evaluated expression.
COS(expression)	Calculates the Cosine from an evaluated expression.
TAN(expression)	Calculates the Tangent from an evaluated expression.
ASIN(expression)	Calculates the inverse Sine from an evaluated expression.
ACOS(expression)	Calculates the inverse Cosine from an evaluated expression.
ATAN(expression)	Calculates the inverse Tangent from an evaluated expression.
SINH(expression)	Calculates the hyperbolic Sine from an evaluated expression.
COSH(expression)	Calculates the hyperbolic Cosine from an evaluated expression.
TANH(expression)	Calculates the hyperbolic Tangent from an evaluated expression.
RAD(expression)	Converts an evaluated value to radians.
DEG(expression)	Converts an evaluated value to degrees.
ABS(expression)	Converts an evaluated value to its absolute value (sign is disregarded).
ROUND(expression, precision)	Rounds an evaluated value to the requested precision.
FLOOR(expression)	Rounds an evaluated value towards negative infinity.
CEILING(expression)	Rounds an evaluated value towards positive infinity.

Functions which accept Vectors & Matrices as parameters.

Function Name	Description
MAX(expression)	Returns the largest value from a struture.
MIN(expression)	Returns the smallest value from a structure.
SUM(expression)	Calculates the sum of all values in a structure.
COLUMN(expression1, expression2, index?)	Inserts expression2 as a new column if expression1 is a Matrix or all the values from expression2 into expression1 if expression1 is a Vector at the specified index (or the end if index is omitted).
SLICE(expression, start, end)	Extracts a slice from expression using the start and end indices (start & end must be Vectors when slicing a Matrix and Scalars when slicing a Vector).
SIZE(expression)	Calculates the dimensions of a Structure.

Functions which accept only Matrix values.

Function Name	Description
TRANSPOSE(expression)	Generates the `A^T` transpose matrix from matrix A.
IDENTITY(n)	Generates a n by n identity matrix.
ROW(expression1, expression2, index?)	Inserts expression2 into expression1 as a new row at index (or at the end if index is omitted).
SQUARE(expression)	Determines if a Matrix is square.
DIAGONALLY_DOMINANT(expression)	Determines if a Matrix is diagonally dominant.
DET(expression)	Determines the determinant of the evaluated matrix.
RANK(expression)	Detemines the rank of the evaluated matrix.
SCALAR(m, n)	Generates a Scalar m by m matrix with the n value down the diagonal.

Functions which accept only Scalar values.

Function Name	Description
SQRT(expression)	Calculates the square root of an evaluated expression.

Linear equations

ExpressionParser also has the capability to solve systems of linear equations using typical matrix manipulation methods. All of these functions need a coefficient matrix A and a set of right hand side values b in order to solve Ax = b. The functions include:

Function Name	Description
GAUSSIAN(A, b)	solves the system of linear equations using gaussian elimination.
LU(A, b)	solves the system of linear equations using LU Factorization.
QR(A, b)	solves the system of linear equations using QR Factorization.

For example:

Expression e = new Expression();

//  x + 2y + 3z = 15
//-2x +  y + 2z = 20
// 5x - 2y +  z = 13

Matrix A = new Matrix(
    new double[][]{
        { 1,  2, 3},
        {-2,  1, 2},
        { 5, -2, 1}
    }
);
Vector b = new Vector(
    new double[] {
        15, 20, 13
    }
);

e.addVariable("A", A).addVariable("b", b);
e.setExpression("GAUSSIAN(A,b);");

System.out.println(e.eval()); //prints [-2.769234, -7.846158, 11.15385]

parameters ending in a '?' mean they are optional and can be omitted.

Adding/Overriding a Function

You can also add or override any function to the ExpressionParser. When adding a new function, add you need to do is provide the name of the function, the minimum amount of arguments it expects and an implementation of its eval() method which is triggered when the method is found in the expression.

Implementation.

So, for example, we can add a new function which generates a new Scalar, Vector or Matrix (based on the amount of args) full of zero values called zeros(args?).

Expression e = new Expression();

e.addFunction(new Function("zeros", 0/* minimum args required is none. */){
	@Override
	public Type eval(List<Type> args) {
		//Where a Type object is the generic type for the Scalar, Vector & Matrix objects.
		if(args.isEmpty()) {
			//can be empty, as minimum required args is zero.
			return Scalar.ZERO; //returns a zero Scalar value.
		}
		try {
			Scalar m = (Scalar) args.get(0);
			if(args.size() > 1) {
				//we are generating a zero matrix.
				Scalar n = (Scalar) args.get(1);
				return Matrix.zeroes(m, n); //generates a new Matrix with zero values.
			}
			return Vector.zeroes(m); //generates a new Vector with zero values.
		} catch (ClassCastException e) {
			throw new IllegalArgumentException("invalid parameter types.");
		}
	}
});

System.out.println(e.setExpression("ZEROS()").eval()); //prints 0 
System.out.println(e.setExpression("ZEROS(1)").eval()); //prints [0]
System.out.println(e.setExpression("ZEROS(3,3)").eval()); //prints [0,0,0; 0,0,0; 0,0,0]

Defining Functions in Script.

Functions can also be defined in the actual expression script. Functions defined in the script cannot already be defined as a function so they cannot be used to override functions.

To define a function we use the function keyword followed by a function definition. A function definition is in the form: function _functionName_ (_functionParam1_, ...)? (-> _returnType_)?. For example to define the same function zeros in script:

function zeros(a) {
	//var 'a' should be a Vector.
	var r = 0;
	if(SIZE(a) > 0) {
		var m = a[0];
		if(SIZE(a) > 1) {
			var n = a[1];
			r = new Matrix(m,n);
		} else {
			r = new Vector(m);
		}		
	}
	r; //the last evaluated statement is returned.
}

var z = zeros([2,2]);

z;

Expression e = new Expression(new File("path/to/source/file.ex"));
e.eval(); //prints [0, 0; 0, 0]

Usage Examples

1: Solving a system of linear equations using LU factorization and confirming that the generated `x` values are correct.

/** 
 * The sum of the first row in 'A' multiplied with the computed values of 'x'
 * (in this case using LU Factorization) should equal the first value in 'b'. 
 **/
String expression = "SUM(A[0] * LU(A,b)) == b[0];"; 

//  x + 2y + 3z = 15
//-2x +  34y + 2z = 20
// 5x - 2y +  8z = 13
Matrix A = new Matrix(
    new double[][]{
        { 100,  2,  3 },
        {-2,  34, 2 },
        { 5, -2,  8 }
    }
);
Vector b = new Vector(
    new double[] {
        15, 20, 13
    }
);
Expression e = new Expression();
e.addVariable("A", A).addVariable("b", b).setExpression(expression);

System.out.println(e.eval()); //prints 1.0 (or equivilent to boolean TRUE)

2: Vector addition from a source file.

/* This source file.ex performs vector addition. */
//File saved at path/to/source/file.ex

var x = [100, 45, 3e2]; //should be [100, 45, 300].
var y = [(12 * 12), 8, MAX(x)]; //should be [144, 8, 300].

//print x & y to standard output.
print x; print y;

/* calculate x + y */
x + y;

File source = new File("/path/to/source/file.ex");

Expression e = new Expression(source);

System.out.println(e.eval()); //prints [244, 53, 600]

3: Storing all evaluated expressions in a List.

public class ExpressionLister implements OutputListener {
	private List<Context> expressions = new ArrayList<>();

	/**
	 * Used to add the final context to the list.
	 * @param ctx the final context.
	 */
	public void add(Context ctx) {
		this.expressions.add(ctx);
	}

	@Override
	public void print(Context ctx) {
		this.add(ctx);
	}

	@Override
	public String toString() {
		return this.expressions.toString();
	}

}

Expression e = new Expression();
ExpressionLister l = new ExpressionLister();
e.setOutputListener(l);

//don't need to add last if we place a 'print' statement on the final statement.
e.setExpression("var x = 1; var y = 2; print x + 1; print y + 1; print (x * 2) + y;").eval();

System.out.println(l); //prints [2, 3, 4]

4: Constructing complex data structures to use within Java.

Expression e = new Expression();
e.setExpression(
        "[ MAX([12, 4, 10]), "
        + "SUM([18, 12, 6; 56, 34, -89; 3e2, (2^10 / 2), 45]), "
        + "(10 - 4) + 3 - RANDOM(),"
        + "IDENTITY(5)[0,0] ];");

Vector v = (Vector) e.eval().getValue();

//Manipulate vector 'v'....

System.out.println(v); //prints [12.0, 894.0, 8.260880967445882, 1.0]

5: Importing a function using an import statement.

//file stored at 'an/example/location.ex'
function helloWorld(a) -> Vector {
	if(a instanceof Scalar) {
		return [1,2,3];
	}
	[4,5,6] //last expression is returned.
}

import an/example/location;

var a = helloWorld(2); 
a; //prints [1,2,3];

Expression e = new Expression(new File("another/source/file.ex"));
System.out.println(e.eval()); //prints [1,2,3];

TODO

There is some functionality that is in the works of being implemented. These are:

jacklaaa89/ExpressionParser