Auto include page content for page references in prompts

[zefhemel]

Since in many contexts of using AI (in chat for sure) it's super useful to give the LLM more information for it to use, I think any page reference in a prompt should have that page's content automatically be added (appended) to the prompt.

I don't have a strong opinion on how this applies generally, but since I'm primarily a chat user so far what I'd hope for is something like this:

**user**: Does [[John]] deserve a raise?

Which would submit the following actual prompt:

**user**: Does [[John]] deserve a raise?

Base your answer on the content of the following referenced pages (referenced above using the [[page name]] format). In these listings ~~~ is used to mark the page's content start and end:

Content of the [[John]] page:
~~~
<<CONTENT OF John PAGE HERE>>
~~~

or something similar. Note that I'm using this "Content of X using delimiter" purposefully here, I've seen some OpenAI prompts that use this format and it seems to work.

[justyns]

I took a first pass at this and used almost the exact formatting in this issue and looks like this:

Now I'm wondering if this should be recursive? If [[John]] links to [[Susan]] inside its page, should we go ahead and include both pages? It'd be cool, but would definitely need a limit to how deep we can go.

I'd also like to extend this to support non-wikilink urls too. Gemini seems to download urls automatically, but gpt4/local models do not. Being able to say "Look at the docs on this url and summarize them" would be nice to have built-in

[zefhemel]

Nice! Recursion would conceptually be cool, but I'd indeed imagine that could quickly blow up even accidentally. Personally I'd just look at the directly linked pages.

[justyns]

I changed the prompt a little bit to try and encourage the llm to ask for a direct link if it's missing context. gpt4 seems to get it:

but gemini and mistral just get confused. I'll try to improve the prompt more later, but I think this is pretty good to start with

[zefhemel]

It's cool that it even asks for that and with page link syntax no less.

[justyns]

This is merged in for 0.0.8, let me know if you have any issues with it or ideas for improving it

[zefhemel]

This is so great. This is what I just did: create an empty page and ask "Tell me about [[page link to 1-1 page with a certain person]]" and it started to summarize their history, various things they've focused on. Feels like magic.