Canner/wren-engine

Wren Engile Local run

yb-sid opened this issue · 5 comments

I have cloned the repository of wren-engine and would like to run and debug this as a standalone java application.

Can someone point me towards main file and steps to run and test the application.
I would like to understand the working of wren-engine and maybe simplify the code for my use-case.
How can I connect it to my own metadata and interact with wren-ai-service which I have cloned as well.

Hi @yb-sid, there're some guide for wren-engine maybe you can refer to them. It includes how to start a local wren-engine, how to set up the data source and how wren engine processes data modeling.
https://docs.getwren.ai/engine/get_started/quickstart

The entry point of wren engine is preview api.

public void preview(
PreviewDto previewDto,
@Suspended AsyncResponse asyncResponse)
{
WrenMDL mdl;
if (previewDto.getManifest() == null) {
mdl = wrenManager.getAnalyzedMDL().getWrenMDL();
}
else {
mdl = WrenMDL.fromManifest(previewDto.getManifest());
}
previewService.preview(
mdl,
previewDto.getSql(),
Optional.ofNullable(previewDto.getLimit()).orElse(100L))
.whenComplete(bindAsyncResponse(asyncResponse));

WrenAI submits query to wren-engine through this API.

GET /v1/mdl/preview

I had a post in discord to introduce the query flow of wren engine.
https://discord.com/channels/1227143286951514152/1238353430985179208/1238372569812566079

It may help you to understand the query flow.

Much thanks @goldmedal
I had explored the wren-engine docs. I did not want to run the pre-configured docker-compose, Rather I was looking for the entry-point and wanted to debug this application.

Thanks for pointing me towards discord. The discord invite at the bottom of www.getwren.ai shows invalid (you might want to update it). I will explore the post.

Maybe one more help , if you could point me towards the tests I could leverage for running the wren-engine.
I'm quite interested how this engine is designed and it's internal workings.

Much thanks @goldmedal I had explored the wren-engine docs. I did not want to run the pre-configured docker-compose, Rather I was looking for the entry-point and wanted to debug this application.

The endpoint is WrenServer. I think you can just run it with VM options to point where is your config file if you used an IDE.

 -Dconfig=etc-pg/config.properties

Or you can build the project with

mvn clean package -DskipTests -P exec-jar

You can get an executable jar in wren-server/target. When starting it, you also need to provide the config file.
The config list can be found in https://docs.getwren.ai/engine/reference/config
You can check how it work in
https://github.com/Canner/wren-engine/blob/main/docker/entrypoint.sh

Thanks for pointing me towards discord. The discord invite at the bottom of www.getwren.ai shows invalid (you might want to update it). I will explore the post.

hmm. I'm not sure which link you means but I try the link in the footer. It work.
https://discord.gg/5DvshJqG8Z

Maybe one more help , if you could point me towards the tests I could leverage for running the wren-engine. I'm quite interested how this engine is designed and it's internal workings.

You can check the tests case in wren-tests/src/test/java/io/wren/testing which include all the integration tests.

  • TesMDLResource#testPreview is used to test the API I mentioned above.
  • TestWrenWithDuckDB is easy to be executed at local.
  • All of the integration test will create a TestingWrenServer. You might want to check it.

I hope they're helpful to you.

@goldmedal , thank you so much for a quick reply I'm all set to go!

As for the discord the invite is not available for me.

Screenshot 2024-05-27 at 8 39 07 PM

hmm.. It's weird. It should be a public link. I'll check it with our discord manager tomorrow. Thanks for your report!
I repost the query flow graph here. I think it maybe helpful to you.

wren-engine-archi drawio_1