hmrc-mongo

Provides support to use the Official Scala driver for MongoDB

It is designed to make the transition from Simple Reactivemongo as easy as possible.

Main features

It provides a PlayMongoRepository class to help set up a org.mongodb.scala.MongoCollection, registering the domain model codecs, and creating the indices. Then all queries/updates can be carried out on the MongoCollection with the official scala driver directly.

The model objects are mapped to json with Play json. This library will then map the json to mongo BSON.

Other included features are:

Migration

See MIGRATION for migrating from Simple-reactivemongo.

Adding to your build

In your SBT build add:

resolvers += "HMRC-open-artefacts-maven2" at "https://open.artefacts.tax.service.gov.uk/maven2"

libraryDependencies ++= Seq(
  "uk.gov.hmrc.mongo" %% "hmrc-mongo-play-xx" % "[INSERT_VERSION]"
)

Where play-xx is play-26, play-27 or play-28 depending on your version of Play.

PlayMongoRepository

Create a case class to represent the data model to be serialised/deserialised to MongoDB

Create JSON Format to map the data model to JSON.

Extend PlayMongoRepository, providing the collectionName, the mongoComponent, and domainFormat.

The mongoComponent can be injected if you register the PlayMongoModule with play. In application.conf:

play.modules.enabled += "uk.gov.hmrc.mongo.play.PlayMongoModule"

@Singleton
class UserRepository @Inject()(
  mongoComponent: MongoComponent
)(implicit ec: ExecutionContext
) extends PlayMongoRepository[User](
  collectionName = "user",
  mongoComponent = mongoComponent,
  domainFormat   = User.mongoFormat,
  indexes        = Seq(
                     IndexModel(ascending("name"), IndexOptions.name("nameIdx").unique(true))
                   )))
) {
  // queries and updates can now be implemented with the available `collection: org.mongodb.scala.MongoCollection`
  def findAll(): Future[User] =
    collection.find().toFuture
}

Other parameters:

indexes - in the above example, the indices were also provided to the constructor. They will be ensured to be created before the Repository is available, by blocking. Mark Indices as background as appropriate.
optSchema - you may provide a BSONDocument to represent the schema. If provided, all inserts will be validated against this. This may be useful when migrating from simple-reactivemongo to ensure the domainFormat has not changed, if relying on library provided formats (e.g. dates).
replaceIndexes - by default, the indices defined by indexes parameter will be created in addition to any previously defined indices. If an index definition changes, you will get a MongoCommandException for the conflict. If an index definition has been removed, it will still be left in the database. By setting replaceIndexes to true, it will remove any previously but no longer defined indices, and replace any indices with changed definition. Please check how reindexing affects your application before turning on

Lock

This is a utility that prevents multiple instances of the same application from performing an operation at the same time. This can be useful for example when a REST api has to be called at a scheduled time. Without this utility every instance of the application would call the REST api.

There are 2 variants that can be used to instigate a lock, LockService for locking for a particular task and ExclusiveTimePeriodLock to lock exclusively for a given time period (i.e. stop other instances executing the task until it stops renewing the lock).

LockService

Inject MongoLockRepository and create an instance of LockService.

The ttl timeout allows other apps to release and get the lock if it was stuck for some reason.

withLock[T](body: => Future[T]): Future[Option[T]] accepts anything that returns a Future[T] and will return the result in an Option. If it was not possible to acquire the lock, None is returned.

It will execute the body only if the lock can be obtained, and the lock is released when the action has finished (both successfully or in failure).

e.g.

@Singleton
class LockClient @Inject()(mongoLockRepository: MongoLockRepository) {
  val lockService = LockService(mongoLockRepository, lockId = "my-lock", ttl = 1.hour)

  // now use the lock
  lockService.withLock {
    Future { /* do something */ }
  }.map {
    case Some(res) => logger.debug(s"Finished with $res. Lock has been released.")
    case None      => logger.debug("Failed to take lock")
  }
}

TimePeriodLockService

The ttl timeout allows other apps to claim the lock if it is not renewed for this period.

withRenewedLock[T](body: => Future[T]): Future[Option[T]] accepts anything that returns a Future[T] and will return the result in an Option. If it was not possible to acquire the lock, None is returned.

It will execute the body only if no lock is already taken, or the lock is already owned by this service instance. It is not released when the action has finished (unless it ends in failure), but is held onto until it expires.

@Singleton
class LockClient @Inject()(mongoLockRepository: MongoLockRepository) {
  val lockService = TimePeriodLockService(mongoLockRepository, lockId = "my-lock", ttl = 1.hour)

  // now use the lock
  lockService.withRenewedLock {
    Future { /* do something */ }
  }.map {
    case Some(res) => logger.debug(s"Finished with $res. Lock has been renewed.")
    case None      => logger.debug("Failed to take lock")
  }
}

Cache

This is a utility to cache generic JSON data in Mongo DB.

The variants are:

MongoCacheRepository for storing json data into a composite object which expires as a unit.
SessionCacheRepository ties the composite object to the session.
EntityCache which makes it easier to work with a single data type.

`MongoCacheRepository`

e.g.

@Singleton
class MyCacheRepository @Inject()(
  mongoComponent  : MongoComponent,
  configuration   : Configuration,
  timestampSupport: TimestampSupport
)(implicit ec: ExecutionContext
) extends MongoCacheRepository(
  mongoComponent   = mongoComponent,
  collectionName   = "mycache",
  ttl              = configuration.get[FiniteDuration]("cache.expiry")
  timestampSupport = timestampSupport, // Provide a different one for testing
  cacheIdType      = CacheIdType.SimpleCacheId // Here, CacheId to be represented with `String`
)

The functions exposed by this class are:

put[A: Writes](cacheId: CacheId)(dataKey: DataKey[A], data: A): Future[CacheItem]

This upserts data into the cache.

Data inserted using this method has a time-to-live (TTL) that applies per CacheId. The amount of time is configured by the ttl parameter when creating the class. Any modifications of data for an CacheId will reset the TTL.

Calling put[String](cacheId)(DataKey("key1"), "value1) and put[String](cacheId)(DataKey("key2"), "value2) with the same cacheId will create the following JSON structure in Mongo:
```
{
  "_id": "cacheId",
  "data": {
    "key1": "value1",
    "key2": "value2"
  }
}
```
This structure allows caching multiple keys against a CacheId. As cached values expire per CacheId, this provides a way to expire related data at the same time.

See EntityCache for a simpler use-case, where key is hardcoded to a constant, to provide a cache of CacheId to value.

get[A: Reads](cacheId: CacheId)(dataKey: DataKey[A]): Future[Option[A]]

This retrieves the data stored under the dataKey.

The DataKey has a phantom type to indicate the type of data stored. e.g.

implicit val stepOneDataFormat: Format[StepOneData] = ...
val stepOneDataKey = DataKey[StepOneData]("stepOne")

for {
  _       <- cacheRepository.put(cacheId)(stepOneDataKey, StepOneData(..))
  optData <- cacheRepository.get(cacheId)(stepOneDataKey) // inferred as Option[StepOneData]
} yield println(s"Found $optData")

delete[A](cacheId: CacheId)(dataKey: DataKey[A]): Future[Unit]

Deletes the data stored under the dataKey.
deleteEntity(cacheId: CacheId): Future[Unit]

Deletes the whole entity (rather than waiting for the ttl).
findById(cacheId: CacheId): Future[Option[CacheItem]]

Returns the whole entity.

SessionCacheRepository

A variant of MongoCacheRepository which uses the sessionId for cacheId. This helps, for example, store data from different steps of a flow against the same session.

It exposes the functions putSession, getFromSession and deleteFromSession which require the Request rather than a CacheId.

EntityCache

A variant which makes it easier to work with a single data type, which all expire independently.

License

This code is open source software licensed under the Apache 2.0 License.

DavidGregory084/hmrc-mongo