Event store (estore) is an important th2 component responsible for storing events into Cradle. Please refer to [Cradle repository] (https://github.com/th2-net/cradleapi/blob/master/README.md) for more details. This component has a pin for listening events via MQ.
The infra-operator adds the special MQ pin with the "event" and "publish" attributes , to every box described via infra-schema.
Event is a base entity of th2. Information related to the work of every component, the executed tests, and the problems that happened are presented as events hierarchy. Every event contains important parts:
- book - name of the book
- id - unique identifier within th2
- parent id - optional link to parent event
- description - set of fields for short description
- body - useful data in JSON format
- attachedMessageIDs - the list of message IDs that are linked to the event
Infra schema can only contain one estore box description. It consists of one required option - docker image . Pin configuration is generated and managed by infra-operator.
General view of the component will look like this:
apiVersion: th2.exactpro.com/v1
kind: Th2Estore
metadata:
name: estore
spec:
image-name: ghcr.io/th2-net/th2-estore
image-version: <image version>
extended-settings:
service:
enabled: false
envVariables:
JAVA_TOOL_OPTIONS: "-XX:+ExitOnOutOfMemoryError -Ddatastax-java-driver.advanced.connection.init-query-timeout=\"5000 milliseconds\""
resources:
limits:
memory: 500Mi
cpu: 200m
requests:
memory: 100Mi
cpu: 20m
Configuration is provided as custom.json
file
{
"maxTaskCount" : 256,
"maxTaskDataSize" : 133169152,
"maxRetryCount" : 1000000,
"retryDelayBase" : 5000
}
- maxTaskCount - maximum number of events that will be processed simultaneously
- maxTaskDataSize - maximum total data size of events during parallel processing
- maxRetryCount - maximum number of retries that will be done in case of event persistence failure
- retryDelayBase - constant that will be used to calculate next retry time(ms): retryDelayBase * retryNumber
If some of these parameters are not provided, estore will use default(undocumented) value. If maxTaskCount or maxTaskDataSize limits are reached during processing, estore will pause processing new events until some events are processed
This is a list of supported features provided by libraries.
CradleMaxEventBatchSize - this option defines the maximum event batch size in bytes. Please see more details about this feature via link
- Migrated to th2 gradle plugin
0.0.8
- Updated bom:
4.6.1
- Updated common:
5.12.0-dev
- Updated common-utils:
2.2.3-dev
- Updated cradle api:
5.3.0-dev
- Updated bom:
4.6.0
- Updated common:
5.9.1-dev
- Updated cradle api:
5.2.0-dev
- Updated common:
5.8.0-dev
- Estore persists event with aggregated statistics about internal errors into cradle periodically
- Updated common:
5.6.0-dev
- Added common-utils:
2.2.2-dev
- Migrated to the cradle version with fixed load pages where
removed
field is null problem. - Updated cradle:
5.1.4-dev
-
Fixed the lost messages problem when estore is restarted under a load
-
Updated common:
5.4.2-dev
- Updated bom:
4.5.0-dev
- Updated common:
5.4.0-dev
- Updated cradle 5.1.1-dev
- Updated common 5.3.0-dev
- Migration to books/pages cradle 5.0.0
- Using Cradle 3.1.4 with a fix for degraded performance while persisting events
- Sending manual acknowledgements only after successfully saving event
- Changed default configuration to
{
"maxTaskCount" : 256,
"maxRetryCount" : 1000000,
"retryDelayBase" : 5000
}
- Added metrics collection for Prometheus
- Limiting simultaneously processed events by number and content size
- Retrying saving events in case of failure
- Updated cradle version from
3.1.1
to3.1.3
. This requires data migration
- Update common version from
3.35.0
to3.39.3
- Fixed problem with unconfirmed deliveries
- Cradle version is updated from
2.21.0
to3.1.1
(please, note that the new version of Cradle is not compatible with Cradle2.*
and requires additional schema migration if you want to keep the stored data) - Messages' IDs that are attached to the event are stored along with event itself
- Update common version from
3.31.6
to3.35.0
- Update Cradle version from
2.20.2
to2.21.0
- Update common version from
3.30.0
to3.31.6
- Update Cradle version from
2.20.0
to2.20.2
- Added util methods from store-common
- Removed dependency to store-common
- Update common version from
3.18.0
to3.29.0
- Update store-common version from
3.1.0
to3.2.0
- Disable waiting for connection recovery when closing the
SubscribeMonitor
- Update Cradle version from
2.9.1
to2.13.0
- Rework logging for incoming and outgoing messages
- Resets embedded log4j configuration before configuring it from a file
- Compressed metadata for events
- Use async methods for storing events to the Cradle