[bug] Summaries are Gibberish

[DmacMcgreg]

describe the bug
When I open the app, its difficult to find any use, since the summaries it gives me are just walls of gibberish. (I've blurred some Personal information)

to reproduce

1. click the default mail summary button when opening the app
2. check the list of results
3. open any of the results

expected behavior
The text should be formatted, correct and useful. Right now, I'm having a hard time finding any value in this setup.

system info

• macos 14.6
• screenpipe version: Version 0.7.6 (20241025.162038)

additional context

[linear]

MED-251 [bug] Summaries are Gibberish

[DmacMcgreg]

The way I see the app having value is that if you were to focus on having a timeline feature like rewind, or ScreenMemory, you could provide key points where you were using apps, and highlight them on the timeline. The same with searches. If you search gmail, then you highlight all the points on the timeline, where you could easily scroll back to those main points on the timeline to see what gmail was doing. Right now, the focus on OCR text is providing too much noise. Also since it splits the video files arbitrarily into small chunks, its hard to actually see what the purpose of the search results are since they are split up among many sections that have a lot of duplicate data.

[louis030195]

@DmacMcgreg thanks for the feedback!

I created a bounty here #609

would love any more context on it, how to make it great

regarding OCR we're going to release today a new alternative data source which extracts text natively from the UI and will be unique and highly structured

[DmacMcgreg]

This is great you're opening a bounty for this one.
Some ideas:

• Daily stats on used apps
• Daily stats on projects - might be useful for agencies or freelancers that need time tracking on projects they worked on. User could provide name and description of projects they are working on, and the tool will keep track based on what was on their screen.
• Searching for keywords highlights points on the timeline. Perhaps even highlighting the words on the screen of the timeline
• Highlight a length in time, and ask for a summary from the LLM, and other LLM tasks, extracting text
• Export screenshot
• Option to Highlight meetings on the timeline, which when clicking on a meeting you can view the transcript window with ability to summarize, etc. can also be used to build a more robust meetings dashboard where you can group them by project, by attendees.