Integrate with Atuin

[jankatins]

Atuin has shell history sync across computers. mcfly has (for my taste and uses) the nicer UI and more intelligent sorting. Having it combined would be a dream.

Atuin also has a sqlite DB as a backend, but of course the schema is different and it's managed by sqlx.

https://github.com/atuinsh/atuin/blob/aa8e5f5c04524d3d5c6f1d5b6c4616dbdb8d40be/atuin-client/src/history.rs#L23-L44

pub struct History {
    /// A client-generated ID, used to identify the entry when syncing.
    ///
    /// Stored as `client_id` in the database.
    pub id: String,
    /// When the command was run.
    pub timestamp: chrono::DateTime<Utc>,
    /// How long the command took to run.
    pub duration: i64,
    /// The exit code of the command.
    pub exit: i64,
    /// The command that was run.
    pub command: String,
    /// The current working directory when the command was run.
    pub cwd: String,
    /// The session ID, associated with a terminal session.
    pub session: String,
    /// The hostname of the machine the command was run on.
    pub hostname: String,
    /// Timestamp, which is set when the entry is deleted, allowing a soft delete.
    pub deleted_at: Option<chrono::DateTime<Utc>>,
}

[cantino]

I'm open to ideas on how this would be done, but it might be best as a separate script that integrates / synchronizes the two.

[jankatins]

Building it as a sync script would be quite hacky: it would need both atuin and mcfly run in the background (instead of one) and periodically a script would sync the atuin DB (or at least all entries from hostnames not our own) into the mcfly DB: Keeping the state in sync might be tricky, given that two syncs happen one after another, but should be doable.

I had some snooping around in the source code of both atuin and mcfly and I think the most reliable would be to mirror the sync step with using the mcfly specifc database by using their api client and reimplementing the download and upload steps on the mcfly side (or probably just the way it's saved into/ queried from the DB)...

What would be a bit of a pain is that both sides have unique things and in this case atuin misses two fields completely: old_dir and selected and mcfly needs hostname. I'm unsure how that influence the computation of the best matches (haven't understand the matching code yet :-(). On the other hand, the atuin server only sees most of this as encrypted blob, so one could just dump it all in, you just wouldn't be able to us the atuin client side anymore...

-- mcflys commands table which corresponds to the above History table, with atuins name in the comment
CREATE TABLE commands( 
id INTEGER PRIMARY KEY AUTOINCREMENT, -- different types: int (mcfly) vs string/UUID, so "save" to sync across multiple hosts (atuin)!
cmd TEXT NOT NULL, -- command
cmd_tpl TEXT, -- missing
session_id TEXT NOT NULL, --session
when_run INTEGER NOT NULL, --timestamp
exit_code INTEGER NOT NULL, -- exit
selected INTEGER NOT NULL, --missing in atuin
dir TEXT, -- cwd
old_dir TEXT -- missing in atuin
-- hostname: missing in mcfly
); 

Assuming one would simply use the atuin server as a "sync server" (and drop all of mcflys data into the "data blob") mcfly would need to gain a new hostname field and a new primary key (e.g. a UUID or use compound PK of id+hostname). And integrate the hostname into the matching/scoring process.

No idea how the atuin server (and their maintainer) would take a different client (mis)using the server as "sync server" only.

Maybe the atuin maintainer have some idea how to integrate the two worlds: @ellie @conradludgate

(There is also https://github.com/ddworken/hishtory, written in Go, but that has even less overlap :-/)

[conradludgate]

Hi! We'd love to colab with mcfly!

We've actually been working on our new sync engine for our history which is actually agnostic to what it stores. We haven't yet enabled it for our history. Currently it's "just" a key-value store as an MVP but it can be an anything store as it has a version and tag field on the record which makes it custom for your application.

I've been building a demo journal app into it to show off how to use it as a library, but I haven't got it finished just yet.

However, it's currently still an MVP that were beta testing so I wouldn't suggest rushing out anything using it right now

[ellie]

I'm also totally happy for McFly to use our sync server (api.atuin.sh), so long as there's some attribution!

[ellie]

For some more context, might be worth skimming our latest release notes, checking out the PR introducing the new sync, and perhaps specifically the KV store which is currently the reference usage of it. Happy to discuss further!