Explain engine architecture

[gitspeaks]

Please provide additional docs explaining the engine internal architecture with specific focus on topics such as the data access algorithm, why it is inherently better then btree for ssd’s , why the need for a new kernel module as apposed to using standard file syscalls using O_DIRECT.

Additionally, please provide high level code review for several key CRUD scenarios referencing key functions and structures

[sherlockwu]

Plus, it would be great if the team could clarify:

1. Is the standalone Mpool module who handles the management of heterogenous devices(say two layers of devices)?
2. If so, how does Mpool works in high level?

Thanks

[smoyergh]

We anticipate publishing an overview whitepaper, though I can't comment on the timing.

For HSE API usage examples, see the API documentation in hse.h, and the sample programs.

For mpool concepts see the mpool wiki.

[sherlockwu]

We anticipate publishing an overview whitepaper, though I can't comment on the timing.

For HSE API usage examples, see the API documentation in hse.h, and the sample programs.

For mpool concepts see the mpool wiki.

Oh, I see that Mpool is just an API to allocate objects in different layers.

Could you point to the documentation or code piece in HSE that related to how it decides which storage layer the "key-value" pair will goes to? Thanks

[davidboles]

Could you point to the documentation or code piece in HSE that related to how it decides which storage layer the "key-value" pair will goes to? Thanks

Where HSE places the key and value of a given instance of a KV pair is determined by the expected lifetime of each, how each is expected to be accessed, and the available media classes. Placement of key and value are independent and a given instance may migrate between media classes over time. This is all under the control of the hse/src/cn component.