Oolong

Oolong - compile-time query generation for document stores.

This library is insipred by Quill. Everything is implemented with Scala 3 macros. Scala 2 is not supported. At the moment MongoDB is the only supported document store.

Community

Join us on Discord!

If you want to contribute please see our guide for contributors.

Overview

All query generation is happening at compile-time. This means:

Zero runtime overhead. You can enjoy the abstraction without worrying about performance.
Debugging is straightforward because generated queries are displayed as compilation messages.

Write your queries as plain Scala lambdas and oolong will translate them into the target representation for your document store:

import org.mongodb.scala.bson.BsonDocument

import oolong.dsl.*
import oolong.mongo.*

case class Person(name: String, address: Address)

case class Address(city: String)

val q: BsonDocument = query[Person](p => p.name == "Joe" && p.address.city == "Amsterdam")

// The generated query will be displayed during compilation:
// {"$and": [{"name": {"$eq": "Joe"}}, {"address.city": {"$eq": "Amsterdam"}}]}
  
// ... Then you run the query by passing the generated BSON to mongo-scala-driver

Updates are also supported:

val q: BsonDocument = update[Person](_
	.set(_.name, "Alice")
	.inc(_.age, 5)
)
// q is {
// 	$set: { "name": "Alice" },
//	$inc: { "age": 5 }
// }

DSL of oolong

Supported MongoDB operators

Query operators

I Comparison query operators

import oolong.dsl.*
import oolong.mongo.*

case class Person(name: String, age: Int, email: Option[String])

val q = query[Person](_.name == "John")
// q is {"name": "John"}

In oolong $eq query is transformed into its implicit form: { field: <value> }, except when a field is queried more than once.

val q = query[Person](_.age > 18)
// q is {"age": {"$gt": 18}}

$gte

val q = query[Person](_.age >= 18)
// q is {"age": {"$gte": 18}}

val q = query[Person](p => List(18, 19, 20).contains(p.age))
// q is {"age": {"$in": [18, 19, 20]}}

val q = query[Person](_.age < 18)
// q is {"age": {"$lt": 18}}

$lte

val q = query[Person](_.age <= 18)
// q is {"age": {"$lte": 18}}

val q = query[Person](_.name != "John")
// q is {"name" : {"$ne": "John"}}

$nin

val q = query[Person](p => !List(18, 19, 20).contains(p.age))
// q is {"age": {"$nin": [18, 19, 20]}}

$type

val q = query[Person](_.age.isInstance[MongoType.INT32])
// q is {"age": { "$type": 16 }}

$mod

val q = query[Person](_.age % 4.5 == 2)
// q is {"age": {"$mod": [4.5, 2]}}

Also $mod is supported if % is defined in extension:

trait NewType[T](using ev: Numeric[T]):
  opaque type Type = T
  given Numeric[Type] = ev
  extension (nt: Type) def value: T = nt

object Number extends NewType[Int]:
  extension (self: Number) def %(a: Int): Int = self.value % a
type Number = Number.Type

case class Human(age: Number)
val q    = query[Human](_.age % 2 == 2)
// q is {"age": {"$mod": [2, 2]}}

II Logical query operators

$and

val q = query[Person](p => p.name == "John" && p.age >= 18)
// q is {"name" : "John", "age": {"$gte": 18}}

If we query different fields the query is simplified as above.

//However, should we query the same field twice, we would observe the form with $and
val q = query[Person](p => p.age != 33 && p.age >= 18)
// q is {"$and": [{"age": {"$ne": 33}}, {"age": {"$gte": 18}]}

val q = query[Person](p => p.age != 33 || p.age >= 18)
// q is {"or": [{"age": {"$ne": 33}}, {"age": {"$gte": 18}]}

$not

val q = query[Person](p => !(p.age < 18))
// q is { "age": { "$not": { "$lt": 18 } } }

III Element Query Operators

$exists

val q = query[Person](_.email.isDefined)
// q is { "email": { "$exists": true } }
val q1 = query[Person](_.email.nonEmpty)
// q1 is { "email": { "$exists": true } }
val q2 = query[Person](_.email.isEmpty)
// q2 is { "email": { "$exists": false } }

IV Evaluation Query Operators

$regex

There are 4 ways to make a $regex query, that are supported in oolong, which are:

import java.util.regex.Pattern

val q = query[Person](_.email.!!.matches("(?ix)^[\\w-\\.]+@([\\w-]+\\.)+[\\w-]{2,4}$"))
//q is {"email": {"$regex": "^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$", "$options": "ix"} 
val q1 = query[Person](p => Pattern.compile("(?ix)^[\\w-\\.]+@([\\w-]+\\.)+[\\w-]{2,4}$").matcher(p.email.!!).matches())
//q1 is {"email": {"$regex": "^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$", "$options": "ix"}
val q2 = query[Person](p => Pattern.compile("^[\\w-\\.]+@([\\w-]+\\.)+[\\w-]{2,4}$", Pattern.CASE_INSENSITIVE | Pattern.COMMENTS).matcher(p.email.!!).matches())
//q2 is {"email": {"$regex": "^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$", "$options": "ix"}
val q3 = query[Person](p => Pattern.matches("^[\\w-\\.]+@([\\w-]+\\.)+[\\w-]{2,4}$", p.email.!!))
//q3 is {"email": {"$regex": "^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$"}

V Array Query Operators

$size

import oolong.dsl.*

case class Course(studentNames: List[String])

val q = query[Course](_.studentNames.size == 20)
val q = query[Course](_.studentNames.length == 20)
// q is {"studentNames": {"$size": 20}}

$elemMatch

import oolong.dsl.*

case class Student(name: String, age: Int)

case class Course(students: List[Student], tutor: String)

val q = query[Course](_.students.exists(_.age == 20)) // $elemMatch ommited when querying single field
// q is {"students.age": 20}

val q = query[Course](course => course.students.exists(st => st.age > 20 && st.name == "Pavel"))
// q is {"students": {"$elemMatch": {"age": {"$gt": 20}, "name": "Pavel"}}}

$all

case class LotteryTicket(numbers: List[Int])

inline def winningNumbers = List(4, 8, 15, 16, 23, 42)

val q    = query[LotteryTicket](lt => winningNumbers.forall(lt.numbers.contains))
// q is { "numbers": { "$all": [4, 8, 15, 16, 23, 42] } }

$all with $elemMatch

case class LotteryTicket(numbers: List[Int], series: Long)

case class LotteryTickets(tickets: Vector[LotteryTicket])

val q = query[LotteryTickets](lts =>
  lts.tickets.exists(_.numbers.size == 20) && lts.tickets.exists(ticket =>
    ticket.numbers.size == 10 && ticket.series == 99L
  )
)
// q is { "tickets": { "$all": [{ "$elemMatch": { "numbers": { "$size": 20 } } }, { "$elemMatch": { "numbers": { "$size": 10 }, "series": 99 } }] } }

Update operators

I Field Update Operators

$inc

import oolong.dsl.*
import oolong.mongo.*

case class Observation(count: Int, result: Long, name: String, threshold: Option[Int])

val q = update[Observation](_.inc(_.count, 1))
// q is {"$set": {"count": 1}}

$min

val q = update[Observation](_.min(_.result, 1))
// q is {"$min": {"result": 1}}

$max

val q = update[Observation](_.max(_.result, 10))
// q is {"$min": {"result": 1}}

$mul

val q = update[Observation](_.mul(_.result, 2))
// q is {"$mul": {"result": 2}}

$rename

val q = update[Observation](_.rename(_.name, "tag"))
// q is {"$rename": {"name": "tag"}}

$set

val q = update[Observation](_.set(_.count, 0))
// q is {"$set": {"count": 0}}

$set

val q = update[Observation](_.set(_.count, 0))
// q is {"$set": {"count": 0}}

$set

val q = update[Observation](_.setOnInsert(_.threshold, 100))
// q is {"$setOnInsert": {"threshold": 100}}

$unset

$unset can be used only to set None on Option fields

val q = update[Observation](_.unset(_.threshold))
// q is {"$unset": {"threshold": ""}}

II Array update operators

$addToSet

case class Student(id: Int, courses: List[Int])

val q = update[Student](_.addToSet(_.courses, 55))
// q is {"$addToSet": {"courses": 55}}

In order to append multiple values to array addToSetAll should be used:

val q = update[Student](_.addToSetAll(_.courses, List(42, 44, 53)))
// q is {"$addToSet": {"courses": {$each: [42, 44, 53] }}}

$pop

case class Student(id: Int, courses: List[Int])

val q = update[Student](_.popHead(_.courses)) // removes the first element
// q is {"$pop": {"courses": -1}}

val q1 = update[Student](_.popLast(_.courses)) // removes the last element
// q1 is {"$pop": {"courses": 1}}

$pull

case class Student(id: Int, courses: List[Int])

val q = update[Student](_.pull(_.courses, _ >= 42)) 
// q is {"$pull": {"courses": {"$gte": 42}}}

$pullAll

case class Student(id: Int, courses: List[Int])

val q = update[Student](_.pullAll(_.courses, List(5, 10, 42)))
// q is {"$pullAll": {"courses": [5, 10, 42]}}

Projection

case class Passport(number: String, issueDate: LocalDate)
case class BirthInfo(country: String, date: LocalDate)
case class Student(name: String, lastName: String, passport: Passport, birthInfo: BirthInfo)

case class StudentDTO(name: String, lastName: String)
case class PassportDTO(number: String, issueDate: LocalDate)
case class BirthDateDTO(country: String, date: LocalDate)

val proj = projection[Student, StudentDTO]
// proj is {"name": 1, "birthInfo.date": 1, "passport": 1, "lastName": 1}

QueryMeta

In order to rename fields in codecs and queries for type T the instance of QueryMeta[T] should be provided in the scope:

import org.mongodb.scala.BsonDocument

import oolong.bson.BsonDecoder
import oolong.bson.BsonEncoder
import oolong.bson.given
import oolong.bson.meta.*
import oolong.bson.meta.QueryMeta
import oolong.dsl.*
import oolong.mongo.*

case class Person(name: String, address: Option[Address]) derives BsonEncoder, BsonDecoder

object Person:
  inline given QueryMeta[Person] = queryMeta(_.name -> "lastName")

case class Address(city: String) derives BsonEncoder, BsonDecoder

val person = Person("Adams", Some(Address("New York")))
val bson: BsonDocument = person.bson.asDocument()
val json = bson.toJson
// json is {"lastName": "Adams", "address": {"city": "New York"}}

//also having QueryMeta[Person] affects filter and update queries:
val q0: BsonDocument = query[Person](_.name == "Johnson")

// The generated query will be:
// {"lastName": "Johnson"}

val q1: BsonDocument = update[Person](_
  .set(_.name, "Brook")
)
// q1 is {
// 	$set: { "lastName": "Brook" },
// }

All QueryMeta instances should be inline given instances to be used in macro. If they are not given their presence will not have any effect on codecs and queries. And if they are not inline the error will be thrown during compilation: Please, add `inline` to given QueryMeta[T]

In addition to manual creation of QueryMeta instances, there are several existing instances of QueryMeta: QueryMeta.snakeCase QueryMeta.camelCase QueryMeta.upperCamelCase

Also they can be combined with manual fields renaming:

import oolong.bson.BsonDecoder
import oolong.bson.BsonEncoder
import oolong.bson.given
import oolong.bson.meta.*
import oolong.bson.meta.QueryMeta

case class Student(firstName: String, lastName: String, previousUniversity: String) derives BsonEncoder, BsonDecoder

object Student:
  inline given QueryMeta[Student] = QueryMeta.snakeCase.withRenaming(_.firstName -> "name")

val s = Student("Alexander", "Bloom", "MSU")
val bson = s.bson
// bson printed form is: {"name": "Alexander", "last_name": "Bloom", "previous_university": "MSU"}

If fields of a class T are not renamed, you don't need to provide any instance, even if some other class U has a field of type T. Macro automatically searches for instances of QueryMeta for all fields, types of which are case classes, and if not found, assumes that fields are not renamed, and then continues doing it recursively

Working with Option[_]

When we need to unwrap an A from Option[A], we don't use map / flatMap / etc. We use !! to reduce verbosity:

case class Person(name: String, address: Option[Address])

case class Address(city: String)

val q = query[Person](_.address.!!.city == "Amsterdam")

Similar to Quill, Oolong provides a quoted DSL, which means that the code you write inside query(...) and update blocks never gets to execute. Since we don't have to worry about runtime exceptions, we can tell the compiler to relax and give us the type that we want.

Raw subquries

If you need to use a feature that's not supported by oolong, you can write the target subquery manually and combine it with the high level query DSL:

val q = query[Person](_.name == "Joe" && unchecked(
  BsonDocument(Seq(
    ("address.city", BsonDocument(Seq(
      ("$eq", BsonString("Amsterdam"))
    )))
  ))
))

Reusing queries

It's possible to reuse a query by defining an 'inline def':

inline def cityFilter(doc: Person) = doc.address.!!.city == "Amsterdam"

val q = query[Person](p => p.name == "Joe" && cityFilter(p))

Coming soon

elasticsearch support
aggregation pipelines for Mongo

leviysoft/oolong