Component for loading osu! data into Elasticsearch.
Currently limited to user high scores.
- .NET Core 2.1
The project reads configuration in the following order:
appsettings.json
appsettings.{env}.json
- Environment
where {env}
is specified by the APP_ENV
environment variable and defaults to development
.
Settings should be set in appsettings
or environment appropriate to the platform, e.g.
appsettings.json
{
"elasticsearch": {
"host": "http://localhost:9200"
}
}
Linux / MacOS
# note the double underscore
elasticsearch__host=http://localhost:9200 dotnet run
Number of chunks from the database to read-ahead and buffer.
Defaults to 5
Don't change this.
Defaults to 4
Standard .NET Connection String to the database.
Elasticsearch host.
Assigns a prefix to the indices used.
Game modes to index in a comma separated list.
Available modes are "osu,fruits,mania,taiko"
.
Forces the indexer to always create a new index.
new
and resume_from
incompatible and should not be used together.
Batch size when querying from the database.
Defaults to 10000
Cursor value of where to resume reading from.
Sets the program into watch mode. In watch mode, the program will keep polling for updates to index.
The time between watch polls.
Index aliases are used to support zero-downtime index switches. When creating new indices, the new index is suffixed with a timestamp. At the end of the indexing process, the alias is updated to point to the new index and the old index is closed.
The program keeps track of progress information by writing to a index_meta
index. When starting, it will try to resume from the last known position.
Setting resume_from=0
will force the indexer to being reading from the beginning.
new
and resume_from
incompatible and should not be used together.
The indexer supports running in watch mode while a different indexer process is creating a new index. Once the new index is complete, the watching indexer will automatically switch to updating the new index.
- Option to cleanup closed indices.