pikers/piker

Position accounting refinements, bug fixes, src-asset-pnls, `ledgerd` daemon

goodboy opened this issue · 0 comments

goodboy commented

The major summary follow up to the .accounting subsys introduced in
#489 this is the list of outstanding but needed features and bugfixes
that should be resolved asap!

The existing #478 was the older not-as-specific issue for this stuff..

The new set of top level issue-tasks are:

  • new toml libs: #496
  • accounting schema changes: #510
  • ledger CLI and offline audit system: #512
  • paper engine clearing models: #476
  • dark and alert testing: #463
  • dark / alert offline persistence: #463
  • order mode UI outstanding bugs: #479
  • missing order mode features:

The high level sections here are:

  • missing accounting calcs and documentation

    • (eg. ppu, bep)
    • cross asset pnl valuations

  • UI components to display the above ^

  • service daemon ledgerd or accountd to do bookkeeping out of thread

Missing critical calcs and accounting:

  • as per #510, make Position.clears -> Position.events as per
    #510 to handle market specific accounting calc modifying occurrences
    (like stock splits)!

  • use transfers/withdrawals to modify the Position.ppu

    • particularly important for cexes such that ppus are calcuated
      properly and graphically correct on chart!

  • add a Position.bep:

    • TODO: white board math is as follows:
Pertaining notes and tasks from legacy issues:

  • #263, the original kraken position tracking way back before we had
    anything 😂

  • #373, more kraken accounting todos from the comments:

    • for wallet transfers out of an account we get
      a Transaction-like entry in our account.<>.<>.toml of the form
      (note this is a non-real example):

      [xmr/eur.kraken]
size = 78.31125778
ppu = 0.3222278400000044
bsuid = "xmr"
clears = [
 { dt = "2022-12-21T07:20:16.316061+00:00", ppu = 0.3222278400000044, accum_size = 778.33125778, price = 0.3212, size = 778.33125778, cost = 0.4, tid = "TJKLRL-7UE2L-BX2UE5" },
 { dt = "2022-12-21T07:26:56+00:00", ppu = 0.3222278400000044, accum_size = 78.31125778, price = "NaN", size = -700.02, cost = 0, tid = "1503A92E24BAA1E8B8E7B78E79A78AAF3E6C3F3E35227EF7D2C13754876B37D5" },
]
      • ^ Notice the price field has a Nan value
      • this transaction record should contribute to the rolling ppu
        calc in the sense that on the next clear the weighting of the
        position should be more right?
        • well that would be the case where you presume the withdrawn
          funds take no losses when (eventually) used, which seems like
          the sanest way to copy with tracking this account's pp right?
        • what happens if you make further trades in this market-pair and
          then bring back the transferred out funds and begin to sell
          them?
      • how do we handle buying more of xmr from say another pair in
        terms of the current next ppu weighting in this position?
        • For eg. if you bought xmr/btc how do we project that
          contribution to this position?
          • say you wanted to project it back to some "main fiat" like EUR
            in this case?
          • the EUR value would come from the original (recursive)
            projection of btc/eur, which would provide the source/base
            pnl from preceding trades as they contribute to the projection
            to EUR; seems like the calculation needs to be
            recursive and rely heavily on time stamps?

    • we aren't tabulating nor using the widthdrawal fee
      schedule       in our cost field in each ledger entry, obviously this should
      resolve any accounting discrepancies a transfer-as-sell clear
      should have a cost (that kraken takes) as part of the position calc.

UI stuff we'll probably be innovators of..

How about historical position line graphics on charts which show
breakeven price changes overlayed with actual underlyings, aka pp
liftetime "lines" more or less as super simple FSP style graphics that
show the beginning, changes, and end to a given pp such that users can
easily see past trades and their state changes on top of historical data
🏄

  • as per #220 and much idea-ing during #489
  • bep and ppu historical lines over every chart with
    real-time update with flat lines.
  • on order staging dotted-line project both the bep and ppu
    given the current level line value B)
    • idea is that for any order you're planning to make you see how it
      will change both your ppu and bep!
    • would change in rt based on user's cursor price placement
    • what to do about submitted live order?
      • some kinda line projection kept up to date like the ppu / bep
        lines?
  • will likely require that we also compute and store arrays of
    breakeven or lifo prices per clear in the account<broker>.<account>.toml
  • might be handy to show actual LIFO style "contextual" pricing if
    configured which could make it easy to enter contextual short term
    trades despite an overall breakeven price, egs:
    • enters long but bep-pnl is red but there's a dump and you want to
      play a bounce so you complete a bounce trade and make a profit and
      then your be price goes back to what it was before the scalp trade
    • same but say you have a green pnl (in this case entering above
      your be would bring it's mean price up) and want to show your
      original longer term be_price after the trade completes
  • adding "ghost data" for paper engine fills which we can
    display as a curve overlay to the actual price history?
    • for example filling in (lesser colored) line/bar to show
      the theoretical paper engine clearing history for something like
      this:
      screenshot-2023-06-14_14-41-46

Add a ledgerd actor-daemon?

Should we consider a separate actor-service: ledgerd which can do all
ledger record monitoring (via connections to emsd) and writing
out-of-process from brokerd/datad such that all other actors only
deal with position data via msgs?

Currently the fs IO is directly serving latency for broker backends
which call this code in thread during startup and possibly on clearing
/ updates.

Premises:

  • it's way more handy to have real-time updates as a user, also
    obvsly for testing..

Solutions:

  • a singleton-style actor that listens to msgs from the ems and
    does all the work of computing positions and recording them to
    whatever backing storage sys we have (for now
    account.broker.account.toml files) and thus can operate as the
    writer in a readers-writer style lock around file access.

  • we should add support for immediate config file
    (ledger or pps.toml) writing

  • can we make a better incremental update system;

    • right now we have lots of file writes going on in both the paper
      engine, per clear, and in various brokers. backends inside the
      trades_dialogue() endpoint.
    • using a timestamp scanning technique in TOML and/or a different
      filesystem format (like parquet? something binary?) we can probably
      accomplish this rather easily.
    • only do ledger file writes on teardown/api-block exit by default