Set up frequency analysis

[amilner42]

The branch frequencies has the stubs in place to create a new graphql query to get the analysis of the frequencies to display to the user how we are doing.

You can already do lein repl within the /api directory to open up a repl for that project (it reads project.clj) and then that will start in the default user namespace, so you can within your repl run (start) and it will start the server. From there you get a graphiql server which you can play with on localhost:8888, go type in the query:

query {
  get_frequency_stats
}

You will get 5 back because the stubs in place return 5. You'll wan to change:

(defn get-frequency-stats
  "TODO - Implement + Document"
  [habit_ids]
  5)

You have to decide on what analysis you actually want to return back and you'll have to update the return type for the new query in the schema.edn:

           :get_frequency_stats {:type Int   ; Here
                                 :description "TODO - Describe this endpoint."
                                 :args {:habit_ids {:type (list (non-null ID))}}
                                 :resolve :query/get-frequency-stats}}

To represent the new type that you decide on in graphql, you'll have to update the objects which are located in that same schema.edn file under the top level key :objects, you can see how I've done this in my queries. Remeber if something is not supposed to be null then wrap it in (non-null ... ) because they allow null by default.

To get the habits and habit data which you'll need to calculate all this you can use my existing functions in db.clj that query for that data, you do not need to write mongo queries.

[amilner42]

As discussed, you'll want to return frequency stats for each habit, that can be a map like:

{ "habit1ID": <stats>
, "habit2ID": <stats>
}

Per-habit stats can look like:

{ "totalFragments": Int
, "successfulFragments": Int
,  "total": Int
,  "fragmentStreak": Int
}

This way on the frontend I can figure out successful fragment ratio (successfulFragments/totalFragments) and I can also figure out average per fragment (total/totalFragments).

Lastly, fragmentStreak will allow the frontend to render streaks which is always nice

[amilner42]

Also don't worry about requesting frequency stats for specific dates, the args from the first comment will be good :args {:habit_ids {:type (list (non-null ID))}}, just the habits you want the frequencies for, and I think it's nice to allow null and assume that means "get me the frequencies for all habits", so I don't have to explicitly pass the ID of every habit if I want that data.

If you really want to have the option to select a specific start date you can, but I don't think it's essential. If anything, I just realized having an arg called compress which defaults to true would probably be nice, if you compress (or don't pass that arg), it returns it like

{ "totalFragments": Int
, "successfulFragments": Int
,  "total": Int
,  "fragmentStreak": Int
}

but if you pass compress: false, then you return an array for each habit with the actual fragments:

[ { "startDate": Date, "endDate": Date, "count": Int},
...
]

This would allow the frontend to render graphs for specific portions without re-querying the backend every time the user changes the dates.

Code-wise, I recommend getting the stats in the un-compressed form (the array of fragments), and then returing that if compress = false and if compress = true/null then compress that data. This will probably be the way to code it with least resistance (as opposed data -> uncompressed and data -> compressed, it makes more sense to do: data -> uncompressed -> compressed)

[alythobani]

That all sounds good to me. I'm assuming the "total" output refers to the sum of all counts over all days?

[amilner42]

Yup

[alythobani]

Performance stats aka frequency stats are now displayed on the Today's Progress panel, with some colour coding and sorting by those stats as well:

TODO: Still need to add performance stats to the Browse and Edit History panel as well—currently it just displays "N/A" on the side of each habit and treats each habit as failed. I'll want to change the call to get_frequency_stats to pass in the date of the historical habit data that the user is looking at, so they see what the performance stats looked like at the time. (Otherwise they'll see performance stats like "7 out of 5 ___" that don't change when the user changes the amount for that day.)

[alythobani]

Also, as for sorting habits, the code currently sorts first by whether or not the current goal has been achieved for each habit, and then by the number of days left in the current fragment (i.e. urgency of habit). You can see this in the screenshot above, e.g. incomplete good habits are purple and they are shown first.

TODO: For any two habits (of the same kind—both good or both bad—since good and bad habits are sorted separately): If they are of equal urgencies + current-fragment-completeness, the code should further sort them by how much they have been done so far relative to their goals. E.g. if I've played guitar for 15 out of 20 minutes (75%), and I've meditated 20 out of 30 minutes (66%), and I only have 1 day left to complete both goals since they're daily habits, I should see Meditation show up before Chill Guitar, because I'm farther from my meditation goal (66% < 75%). This also encourages balance between habits so that I'm not doing a lot poorer on one habit than another.

[alythobani]

Performance stats are now visible on the Browse and Edit History panel as well, and those habits are sorted / colour-coded / etc. by the user's performance on the selected date. This is super convenient for when it's past midnight and you're editing habits from "yesterday" (but really it's the same day, just late at night... maybe you just brushed your teeth at 12:30am before going to bed). Now when you edit that data, you can immediately see your relevant performance stats change. If you brushed your teeth 2 out of 2 times, "yesterday's" box now turns green, even if "today's" box is still purple because you haven't brushed your teeth yet on what is, technically speaking, today.

[alythobani]

Habits are now further sorted by progress in their current fragment! This encourages working on straggler habits more, and worst case just gets rid of some randomness in the sorting (with respect to when two habits are both complete/incomplete and of equal urgency).

[alythobani]

Currently there's still some randomness in the sorting when you've done 0 out of X units for multiple habits, because then it evaluates the progress percentage to be 0% for each and considers them equal. I can see this happen when I update a habit, and the other habits that I've still done 0 of seemingly randomly change their order.

TODO: If the progress percentages (current amount * 100 ÷ current goal) of two habits are equal, then further sort by how much left the user needs to do. E.g. if the user has done 0 out of 30 minutes of meditation, and 0 out of 5 minutes of guitar scales, show meditation first because the user is further behind on that habit.

This will be kind of an Eat That Frog approach: encourage the user to tackle the most challenging habits first. It's definitely flawed because units don't always tell you how hard something is (e.g. 1 workout is harder and takes more time than 5 minutes of guitar) but it's better than nothing.

[alythobani]

TODO: Update frequency stats on page when the date changes

[alythobani]

Update: Habits are now sorted by the Eat That Frog approach described above, and frequency stats now update automatically when the date changes