Heroku can't keep up with current market volume

Question

Heroku can't keep up with current market volume

badgerwithagun opened this issue 5 years ago · 5 comments

badgerwithagun commented 5 years ago

Significant lag occurring when BTC/USD pairs are subscribed, causing memory issues on the box and delayed data.

Two problems here:

The basic Heroku box is too slow
We are backing up messages rather than dropping them where we can afford to do so

Need to investigate

Answer 1 · 2019-05-14T14:59:33.000Z

Tried Azure?

…

________________________________ From: Graham Crockford <notifications@github.com> Sent: Tuesday, May 14, 2019 3:51:31 PM To: gruelbox/orko Cc: Subscribed Subject: [gruelbox/orko] Heroku can't keep up with current market volume (#360) Significant lag occurring when BTC/USD pairs are subscribed, causing memory issues on the box and delayed data. Two problems here: * The basic Heroku box is too slow * We are backing up messages rather than dropping them where we can afford to do so Need to investigate — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#360?email_source=notifications&email_token=AAZNAVAS24NUA3F25JZLKHTPVLGXHA5CNFSM4HM2YG52YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GTWKCIA>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAZNAVB4R3WZH5VUB4ZYH7LPVLGXHANCNFSM4HM2YG5Q>.

Answer 2 · 2019-05-14T22:14:48.000Z

Not yet, but I doubt I'll bother. This thing is working constantly all day. It needs 24 hours of compute per day. Looks like 55.43 GBP/Month on Azure, and I'd have to set up the pipeline myself. Nope!

If I'm going to go to the effort of setting up a CD pipeline to a cloud provider, it'll probably be AWS.

Answer 3 · 2019-05-16T08:57:26.000Z

(Y)

…

On Tue, 14 May 2019 at 23:14, Graham Crockford ***@***.***> wrote: Not yet, but I doubt I'll bother. This thing is working constantly all day. It needs 24 hours of compute per day. Looks like 55.43 GBP/Month on Azure, and I'd have to set up the pipeline myself. Nope! If I'm going to go to the effort of setting up a CD pipeline to a cloud provider, it'll probably be AWS. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#360?email_source=notifications&email_token=AAZNAVCG3BZ7HZU3SH4ESRDPVM2VRA5CNFSM4HM2YG52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVM6HFQ#issuecomment-492430230>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAZNAVEVBO5KXE6AUNU2ILLPVM2VRANCNFSM4HM2YG5Q> .

Answer 4 · 2019-05-25T16:48:49.000Z

Promising performance from a $10/month Vultr server. I'm still working on the deployment scripting. I plan to open source that too.

Answer 5 · 2019-11-12T11:10:21.000Z

Full replacement of Heroku includes the full CD pipeline, infrastructure-as-code and provisioning. I have this all mostly working now but am working through getting it bomb-proof.