💎 Trove

⚠️ This package is still in development

A Composable Search Library for Ecto.

Installation

If available in Hex, the package can be installed by adding trove to your list of dependencies in mix.exs:

def deps do
  [
    {:trove, "~> 0.1.0"}
  ]
end

Documentation can be generated with ExDoc and published on HexDocs. Once published, the docs can be found at https://hexdocs.pm/trove.

Example

Log
  |> Trove.search(%{message: "Find me"})
  |> Repo.all()

Development

Run docker-compose up -d to start the database.

Run mix deps.get to install dependencies.

Run mix test to run tests.

Features

1.0

Search by fields on schema
Preload relationships
Sorting
Paginated results

1.x

Search with ilike, gte and other modifiers

1.x

Search by relations fields
Search one relation level deep

1.x

Search "infinite" levels deep

1.x

HTTP params transform util function

1.x

Schema information caching (ie Cache available_filters per module/schema)
~~Add custom filters to search~~ can be added to the returned query
^ OR filters validation built from macros

Reference

https://yos.io/2016/04/28/writing-and-publishing-elixir-libraries/
https://hexdocs.pm/ecto/Ecto.Schema.html#module-reflection
https://hexdocs.pm/ecto_shorts/EctoShorts.html

Notes

I'd like to eventually add support for better field searching to match this api: https://hexdocs.pm/ecto_shorts/EctoShorts.html#module-actions

The ideal api would look something like:

# would it be possible to put this on the Person and if the schema model doesn't have it throw a validation error
# pass :all to allow all fields
# override-able in the search function?
# or just make it the only way to set the allowed fields
@allowed_fields Trove.allowed_fields(Person, [:first_name])

 search_terms = %{
    first_name: %{ilike: "Scott"},
    vehicle_make: "Rivian",
    vehicle: %{
      make: "Rivian"
      year: %{gte: 2022}
    }


# utility function (this is run in Trove.search before the query is created)
Trove.validate_search(Person, search_terms)

...
# Trove.search! pattern
Person
  |> Trove.search!(
    search_terms,
    page: 1,
    limit: 10,
    preloads: [vehicle: :parking_reservation]
    sort: [first_name: :asc]
    allowed_fields: allowed_fields
  )
  |> Repo.all()

# Trove.search pattern
case Trove.search(
  Person,
  search_terms,
  allowed_fields,
  pagination: %{page: 1, limit: 10}
  page: 1,
  limit: 10,
  preloads: [vehicle: :parking_reservation]
}) do
  {:ok, search} -> Repo.all()
  {:error, {:{type}, message}} -> message
end

...
total_count = Person
  |> Trove.search(search_terms)
  |> Repo.aggregate(:count)

Challenges

getting compile time helpers like ilike, gte, date between working

Should be solved by EctoShorts
could be generating unoptimized queries

It may create marginally slower queries for basic searches but this is not meant to be used to replace report type queries
avoid making api params directly assignable to search terms

Solved by validation input
validating user input
avoiding infinitely recursive relation queries (especially for many-to-many)

Could be solved with a configurable limit
error handling -> bubbling errors up to user.

Trove should handle as much as possible

Contributing

Test Fixtures

https://drawsql.app/teams/tad-scritchfield/diagrams/trove

smartrent/trove