Bot trades with loss all the time.
krilson opened this issue · 70 comments
Gave this a try, exchange poloniex, default settings, live trading (not sim).
The bot consistently without exception buys at price X and sells at a lower price. It is doing the opposite of what it should be doing. ( I am not kidding, every trade was with a loss, even in this crazy bull market we have now)
Even tried disabling c.sell_stop_pct = 0 but still it persists in doing this.
Example:
Buy NXT at 0.00003540 at 2017-05-24 02:13:05
Sell NXT at 0.00003449 at 2017-05-24 13:02:01
I will leave it running and keep up with the updates if the behavior changes maybe.
Edit: To also mention, I have it running on 10 pairs. Every trade that happened on any of the pairs was this way. Sold at a lower price. And it is running now for about 5 days. Maybe the long term result will be different.
I'm sorry you experienced some rough trades. It appears that (using GMT timezone?) it bought on the first big green candle here and sold on the first big red one:
That's a whipsaw, a bad signal followed by a corrective signal. At least the sell saved you from holding during the down period at the end there. All trading systems will have bad signals like this and there's no great way to filter them all out. The default strategy tries to filter them with a std. deviation (--neutral_rate=auto) and you can tweak that parameter to set a hard threshold for trend detection.
If you haven't run sims, I'd really recommend doing so... the default settings are a rough starting point. Some altcoins that have 20-50% change per day, it's very risky to trade, manually or with a bot. I would focus on low volatility markets with strong uptrends, and avoid trading on downtrends.
Also the graph could've easily gone the other way and the first buy would've returned profit, but there's no way to code a strategy that can predict the future :) I'm very open to ideas about improving the strategy. The higher the volatility the higher chance for bad signals I think.
Thanks for the reply. I was already testing zenbot3 and I got kinda used to it. Still trying to grasp this one.
But is it normal behavior that it sells at a lower price if the c.sell_stop_pct is set to 0 (disabled)?
And how far does the bots memory go between restarts? Is it aware of a trade it did three days ago if I restarted it twice in the meantime? If he does not know he bought something at a certain price it would explain the selling at a lower price. (but with the NXT example there were no restarts).
Strategy wise I am just playing around for now. For now I am trying to achieve that the bot buys, if price goes higher sell (80%), if it goes lower dont sell. I have the spend percentage set to 30% on the buy side and 80% on the sell side. So I always hold some coins on both sides. This actually saved me through this, if I gave it 99% the loses would be much higher.
Performance wise it works great btw if you dont count poloniex having issues all the time...
Edit: And no need to be sorry, we all read the disclaimer, use at your own risk :)
Edit2: Would setting this c.max_sell_loss_pct to 0 prevent it from selling at a loss?
how far does the bots memory go between restarts?
the profit figure is the only thing that's carried over (if the balance is the same as the end of last session), it starts fresh with the my_trades, since the gap between stopping and starting the bot could cause unwanted stops to trigger.
Would setting this c.max_sell_loss_pct to 0 prevent it from selling at a loss?
No, that one is meant to be protection in case the price slips in a huge way during order execution, (and it actually holds despite sell signal) if the order takes too long or the exchange has maintenance (prices sometimes do wild stuff after maintenance). setting it to 0 disables that protection, but setting it to 0.1 would keep holding if any loss from last buy. That's dangerous because holding during a downtrend loses profit. If you set --stop_loss_pct you can prevent holding during downtrends, but that's dangerous also since setting it too close to the usual price variation will sell at a loss often.
I have same case.
I lose my btc anytime even if my result of sim is good.
sold at lower price and bought at higher price
@nao0811ta Still the same for me too, have a few more days of data now.
Example ETC:
Buy 0.00797516 2017-05-25 10:00:39
Sell 0.00656001 2017-05-26 07:00:40
Buy 0.00720500 2017-05-26 13:00:30
Buy 0.07610011 2017-05-26 13:00:48
Sell 0.00654845 2017-05-26 19:00:35
Buy 0.00669999 2017-05-27 16:00:23
At this pattern I assume it will sell again around 0.0062 😄
Pulled the code updates now and restarted all the bots. Also set c.max_sell_loss_pct to 0.1 like suggested by @carlos8f to prevent it from selling at a loss. Will report in a few days what happens.
It seems my results are kinda the same as well.
This is live trading on gdax.ETH-BTC with --period=5m --trend_ema=15 --max_sell_loss_pct=10.
Simulations seemed fine so I decided to give it a shot.
It seems to be consistently selling low, buying high, unless I'm missing something.
2017-05-29 01:31:00: SOLD at 0.07779 BTC
2017-05-29 01:47:02: BOUGHT at 0.07830 BTC
2017-05-29 02:53:10: SOLD at 0.07819 BTC
2017-05-29 03:56:09: BOUGHT at 0.07819 BTC
2017-05-29 04:40:58: SOLD at 0.07761 BTC
2017-05-29 04:42:04: SOLD at 0.07768 BTC
2017-05-29 05:07:51: BOUGHT at 0.07769 BTC
2017-05-29 05:11:13: SOLD at 0.07767 BTC
2017-05-29 05:48:20: BOUGHT at 0.07793 BTC
2017-05-29 05:53:17: SOLD at 0.07793 BTC
2017-05-29 05:55:29: BOUGHT at 0.07800 BTC
2017-05-29 06:11:51: SOLD at 0.07777 BTC
2017-05-29 08:05:50: BOUGHT at 0.07779 BTC
2017-05-29 08:09:20: BOUGHT at 0.07774 BTC
2017-05-29 09:23:03: SOLD at 0.07797 BTC
2017-05-29 09:29:01: BOUGHT at 0.07815 BTC
2017-05-29 09:51:05: BOUGHT at 0.07847 BTC
2017-05-29 10:20:58: SOLD at 0.07824 BTC
2017-05-29 10:55:08: BOUGHT at 0.07861 BTC
2017-05-29 11:31:18: SOLD at 0.07828 BTC
2017-05-29 11:50:28: BOUGHT at 0.07892 BTC
I have tried both the default settings which had the same results, as well as the ones pasted above.
Both resulted in 6-8% loss after 12 hours.
@carlos8f thank you for all your work on zenbot; this is not criticism or a complaint, just trying to figure out how we can make this better! :) losing is part of the fun ;)
With c.max_sell_loss_pct set to 0.1 I have seen a few positive trades, but still more exceptions than a rule. Plenty of trades where it buys higher and sells lower. Still an improvement, before not a single trade was positive.
@carlos8f Shouldn't it hold no mater what if set to 0.1? Or did I misunderstand?
@carlos8f For improving the strategy, I would suggest:
Looking at the console output, the bot would make a decent profit if would do the exact opposite of what it is doing now, change the buy signals to sells and sell signals to buys:
Current behaviour:
2017-05-28 17:00:00 0.19860200 XRP-USDT -8.4% 3.1m -- -0.6157 0.2109 sell
2017-05-28 18:00:00 0.19300002 XRP-USDT -2.8% 2.76m -- -0.7452 0.2414 sell
2017-05-28 19:00:00 0.20010000 XRP-USDT +3.7% 1.64m - -0.4908 0.2999 sell
2017-05-28 20:00:00 0.19850000 XRP-USDT -0.8% 905.88k -- -0.5093 0.3233 sell
2017-05-28 21:00:00 0.20490000 XRP-USDT +3.2% 872.12k - -0.2863 0.3405
2017-05-28 22:00:00 0.20140000 XRP-USDT -1.7% 822.16k - -0.3742 0.3425 sell
2017-05-28 23:00:00 0.19505002 XRP-USDT -3.2% 598.37k -- -0.5437 0.3380 sell
2017-05-29 00:00:00 0.20362669 XRP-USDT +4.4% 392.32k - -0.2502 0.3496
2017-05-29 01:00:00 0.21900999 XRP-USDT +7.6% 1.05m + 0.2350 0.3450
2017-05-29 02:00:00 0.21626223 XRP-USDT -1.3% 542.16k + 0.1356 0.3317
2017-05-29 03:00:00 0.21255057 XRP-USDT -1.7% 345.17k - 0.0138 0.3182
2017-05-29 04:00:00 0.20899996 XRP-USDT -1.7% 1.13m - -0.0951 0.2854
2017-05-29 05:00:00 0.20708824 XRP-USDT -0.9% 909.19k - -0.1472 0.2745
2017-05-29 06:00:00 0.20500001 XRP-USDT -1.0% 1.88m - -0.2016 0.2683
2017-05-29 07:00:00 0.20298180 XRP-USDT -1.0% 328.84k - -0.2506 0.2647
2017-05-29 08:00:00 0.21083519 XRP-USDT +3.9% 272.5k - 0.0055 0.2613
2017-05-29 09:00:00 0.21099996 XRP-USDT +0.1% 430.08k - 0.0102 0.2535
2017-05-29 10:00:00 0.21199999 XRP-USDT +0.5% 874.55k + 0.0401 0.2219
2017-05-29 11:00:00 0.21140197 XRP-USDT -0.3% 377.85k - 0.0192 0.2126
2017-05-29 12:00:00 0.21881808 XRP-USDT +3.5% 557.72k + 0.2449 0.1949 buy
2017-05-29 13:00:00 0.21950000 XRP-USDT +0.3% 1.51m + 0.2494 0.2069 buy
2017-05-29 14:00:00 0.21700000 XRP-USDT -1.1% 604.79k + 0.1566 0.2052
2017-05-29 15:00:00 0.21959016 XRP-USDT +1.2% 457.42k + 0.2250 0.1621 buy
2017-05-29 16:00:00 0.21481973 XRP-USDT -2.2% 791.3k + 0.0653 0.1541
2017-05-29 17:00:00 0.22300001 XRP-USDT +3.8% 1.08m + 0.3090 0.1471 buy
2017-05-29 18:00:00 0.22691937 XRP-USDT +1.8% 3.37m + 0.4067 0.1608 buy
2017-05-29 19:00:00 0.22797344 XRP-USDT +0.5% 3.19m ++ 0.4106 0.1827 buy
2017-05-29 20:00:00 0.23000000 XRP-USDT +0.9% 1.13m ++ 0.4433 0.1947 buy
Buy when ema value bottoms. On a 1h period for example, track last 12hours. If ema current is negative and smaller then ema n-1 DO NOTHING. If ema current is equal or is getting larger than n-1 then BUY.
Sell when ema tops. If ema current is positive and larger than ema n-1 DO NOTHING. If ema current is equal or getting smaller than ema n-1 SELL.
Of course evening it out a bit, looking at a few hours of ema values back, not just the last one to kinda catch the trend.
What do you think?
@krilson I will try experimenting with that, however I'm a little confused:
ema n-1is this a comparison of 12 periods ago, or 1 period? (how are last 12 hours "tracked"?)- Comparing the ema's themselves, or the ema change % from 1 period ago (which is what's displayed)?
smaller/getting smaller- do you mean closer to 0/crossed the other way? or further negative?
Also set c.max_sell_loss_pct to 0.1 like suggested by @carlos8f to prevent it from selling at a loss.
@carlos8f Shouldn't it hold no mater what if set to 0.1? Or did I misunderstand?
To be clear I don't recommend that as a setting (yes it should hold, unless you restart the bot in which case the trade memory resets)
I should also note that
- even in very profitable simulations, error rate (% that trades were unprofitable) is normally in the 60-70% range.
- So usually, about 20% of the trades result in 80% of the profit (80-20 rule). When the market is going sideways and lots of whipsaws occur, error rate is usually higher, but once a trend emerges you'll be on the good side.
- In general the higher the volatility of a market, the quicker the period required. However:
- On Poloniex the trade fees can hurt during sideways markets especially with quicker period, so running a 10m or 5m period may be much more profitable on a market like GDAX that has no maker fees.
Live trading with Bitfinex gets similar negative results. Paper trading gives possitive with the same settings.
hey guys, I posted a readme update for "known issues" relating to losses seen in live trading. I've been busy lately so it may take some time for improvement. Thanks!
https://github.com/carlos8f/zenbot#known-issues-and-current-status
Hi, I'm not an expert but just thinking out loud here. Would it be nice if the --period goes to, let's say from 30 to 10 minutes as soon as a buy was initiated? In this case the bot sells sooner?
@rondijkstra this hold period is one that v3 had, it can end either good or bad.
Ok!
I know it's stupid, but try to test this with your setting. I've got better results in sim and paper with error rate below 30% (even 0%) - it seems that standard setting gives profit only in specific market conditions.
I did some testing (SIM) with DGB that was rising pretty fast yesterday. The sim of standard setting gave me 42.6% profit.
The newly strategy, thus with the 'reversed trading', gave me -9.36% :(
(5 day sim gave +103.97% VS 4.52% profit..)
Results for about 30 hours paper simulation:
./zenbot.sh trade poloniex.BCN-BTC --paper --days=3 --period=2m --min_periods=3 --trend_ema=2 --oversold_rsi_periods 26 --oversold_rsi 30 --neutral_rate 0.7
2017-06-05 18:24:00 0.00000164 BCN-BTC +0.0% 3.79m + 0.1005 0.7000 +0.6% 0.00 BCN 0.02604 BTC +30.2% +19.9%
Results was also positive for the two, live, 15 minutes sessions.
...but
For "reversed buy/sell signals" the most important parameters are neutral_rate and period - those parameters are correlated (higher period - higher neutral_rate).
Setting "--neutral_rate auto" doesn't work well - usually leads to losses or marginal profit, the same with setting "neutral_rate" too low - bot tries to trade too often. Setting too high from the other side, leads to no trades at all. Performance is also affected by an up-trend and down-trend of the market, because of neutral_rate setting symmetry - could be corrected by adding "trend factor", positive for uptrend, negative for downtrend.
[Summary]
Reversed buy/sell with properly set period and neutral_rate can perform pretty well on stable, low volatile pairs (even 1% price changes) especially for very short periods (even 15s).
I am getting positive results with this: --period=60m --min_periods=60 --trend-ema=40 --neutral_rate=auto
conf.js
// sell if price drops below this % of bought price (0 to disable)
c.sell_stop_pct = 0
// buy if price surges above this % of sold price (0 to disable)
c.buy_stop_pct = 0
// enable trailing sell stop when reaching this % profit (0 to disable)
c.profit_stop_enable_pct = 0
// maintain a trailing stop this % below the high-water mark of profit
c.profit_stop_pct = 1
// avoid trading at a slippage above this pct
c.max_slippage_pct = 5
// buy with this % of currency balance
c.buy_pct = 20
// sell with this % of asset balance
c.sell_pct = 90
// ms to adjust non-filled order after
c.order_adjust_time = 30000
// avoid selling at a loss below this pct
c.max_sell_loss_pct = 0.1
// ms to poll order status
c.order_poll_time = 5000
// ms to wait for settlement (after an order cancel)
c.wait_for_settlement = 5000
// ms to wait for settlement (after a funds on hold error)
c.wait_more_for_settlement = 60000
// % to mark up or down price for orders
c.markup_pct = 0
Gave it only 20% buy balance, to make it buy several times, otherwise it would spend the whole balance on one buy. This way it spreads the buys and usually gets a better price on average.
I don't even bother with the simulation, the results are too different.
My idea was along the line that the bot captures the curve of the ema and buy at bottom, sell at top.
Both the number of ema captures and the period should be configurable by the user. A good period would be about 10/20 minutes from my experience.
Also a hard limit on when to buy should be placed. Buy only when ema is bellow -0.15 and sell only if its above 0.15 (for example).
An example when to buy, considering 12 periods:
ema1: -0.12
ema2: -0.14
ema3: -0.12
ema4: -0.16
ema5: -0.18
ema6: -0.20
ema7: -0.18
ema8: -0.18 buy
ema9: -0.17
ema10: -0.17
ema11: -0.14
ema12: -0.12
Here a buy would happen at ema8.
My criteria in this example is:
- ema is bellow -0.15
- last three emas are the basically the same value (meaning in most cases, that it bottomed)
To sum up, while the ema is decreasing do not buy, wait for it to start evening out. Then buy. And the opposite for sell, do not sell until it is rising, sell when it starts evening out.
Implementing this behavior would give good results from my experience. Fine tuning would be left to the user. Like what kind of margin to be allowed for the ema bottoming, period length, hard min/max ema to buy, last n number of emas to watch for the average. My test result at a 10min period, looking at past 5 values to determine a buy/sell is optimal.
A live example of what I mean. BTC-ETC pair on poloniex:
I have it set reverse now. The default signal would be buy here, but I changed it to sell. I am trying to show where the bot should sell, most of the sell signals here are too soon. I made the ones that fit my above mentioned strategy bold and the sell signal I would use is upper case "SELL HERE".
You can see that the ema values go high, than stay in the relatively same value 4 periods, on the fifth period the bot would sell.
2017-06-11 08:40:00 0.00655029 ETC-BTC -0.6% 7749 + 0.0377 0.0392
2017-06-11 08:50:00 0.00653000 ETC-BTC -0.3% 7717 + 0.0206 0.0376
2017-06-11 09:00:00 0.00651233 ETC-BTC -0.3% 6292 + 0.0064 0.0385
2017-06-11 09:10:00 0.00651197 ETC-BTC 0.0% 5843 + 0.0058 0.0421
2017-06-11 09:20:00 0.00657543 ETC-BTC +1.0% 10973 + 0.0531 0.0444 sell
2017-06-11 09:30:00 0.00660900 ETC-BTC +0.5% 7473 ++ 0.0756 0.0444 sell
2017-06-11 09:40:00 0.00662250 ETC-BTC +0.2% 12908 ++ 0.0820 0.0415 sell
2017-06-11 09:50:00 0.00666756 ETC-BTC +0.7% 9569 ++ 0.1117 0.0413 sell
2017-06-11 10:00:00 0.00665172 ETC-BTC -0.2% 13074 ++ 0.0943 0.0376 sell
2017-06-11 10:10:00 0.00674998 ETC-BTC +1.5% 17893 ++ 0.1630 0.0317 sell
2017-06-11 10:20:00 0.00673994 ETC-BTC -0.1% 17284 ++ 0.1473 0.0370 sell
2017-06-11 10:30:00 0.00680000 ETC-BTC +0.9% 17256 ++ 0.1846 0.0400 sell
2017-06-11 10:40:00 0.00682967 ETC-BTC +0.4% 26883 +++ 0.1973 0.0459 sell
2017-06-11 10:50:00 0.00676891 ETC-BTC -0.9% 23673 ++ 0.1423 0.0515 sell
2017-06-11 11:00:00 0.00675000 ETC-BTC -0.3% 14911 ++ 0.1211 0.0522 sell
2017-06-11 11:10:00 0.00681995 ETC-BTC +1.0% 23652 ++ 0.1668 0.0526 sell
2017-06-11 11:20:00 0.00684999 ETC-BTC +0.4% 12245 ++ 0.1806 0.0548 sell
2017-06-11 11:30:00 0.00689299 ETC-BTC +0.6% 20764 +++ 0.2032 0.0575 sell
2017-06-11 11:40:00 0.00700000 ETC-BTC +1.6% 28229 +++ 0.2716 0.0614 sell
2017-06-11 11:50:00 0.00714998 ETC-BTC +2.1% 73299 +++ 0.3677 0.0708 sell
2017-06-11 12:00:00 0.00700001 ETC-BTC -2.1% 50678 ++ 0.2388 0.0871 sell
2017-06-11 12:10:00 0.00720000 ETC-BTC +2.9% 56566 +++ 0.3725 0.0878 sell
2017-06-11 12:20:00 0.00743215 ETC-BTC +3.2% 57938 ++++ 0.5217 0.0971 sell
2017-06-11 12:30:00 0.00735000 ETC-BTC -1.1% 78121 +++ 0.4343 0.1174 sell
2017-06-11 12:40:00 0.00738300 ETC-BTC +0.4% 34716 +++ 0.4350 0.1215 sell
2017-06-11 12:50:00 0.00744400 ETC-BTC +0.8% 51042 +++ 0.4557 0.1256 SELL HERE
2017-06-11 13:00:00 0.00740011 ETC-BTC -0.6% 26878 +++ 0.4002 0.1297 sell
2017-06-11 13:10:00 0.00723002 ETC-BTC -2.3% 33544 ++ 0.2584 0.1282 sell
2017-06-11 13:10:05 0.00722960 ETC-BTC 0.0% 160 ++ 0.2448 0.1238
I am having issues coding this, if any help would be appreciated.
I was thinking about making a sub signal, that would then help to make the final buy/sell signal.
Every period would have a value calculated from the previous period:
(ema current - ema previous = close to 0) = subsignal 1 (S1)
(ema current - ema previous = positive) = subsignal 2 (S2) - ema going up
(ema current - ema previous = negative) = subsignal 3 (S3) - ema going down
A strategy could be built from these kind of subsignals like this for example:
if last 3 periods are S1 and the preceding periods (lets say period n-4 to n-10, so 6 periods before) were S2 then sell now.
And same for buying, if last 3 periods are S1 and the preceding periods were S3 then buy now.
Does this make any sense to anyone? 😄
I'm testing macd strategy and default parameters on poloniex
It's not bad..
But I want that it can buy during uptrend when I start.
I gave the bot a try on default settings for a few days, clearly there's something wrong, it just flips the sell and buy (or so it seems).
Trying the settings above from @krilson ... let's see what happens?
By the way, does flipping sell and buy works?
For what it's worth, I've left it running for about 3 hours with a small amount of ETC trading back and forth to ETH and I've made a not bad return.
based on the comments above is it safe to say the simulations are not currently accurate?
@CreativeBacon Well, at the risk of stating the obvious, they are called simulations for a reason.
A small update to the trend_ema strategy,
This simple change: krilson@66fc7fa#diff-c081f5fa7634cfc3a1254e982a748f9c
...reversing the buy/sell signals is making a consistent profit for the last week.
My current setting: ./zenbot.sh trade --period=10m --min_periods=60 --trend_ema=30 --neutral_rate=0.15 poloniex.REP-ETH
neutral_rate at 0.20 is also good, but if the market calms down to much the bot wont make a trade.
I did not go around to calculating the exact profit %. But from the poloniex trading history I make out around 100% profit...
Also buy pct is set to 50, and sell pct to 90.
Tested on 4 pairs simultaniously, all are making a profit.
Is it safe to say this issue is closed?
@krilson With yo reversal strategies, and settings I tried 2 different pairs and the bot won't even attemp to trade.
Do you have any guidance on which pair it might be profitable (stable coin or volatile coin) ?
never got positive results, either simulate or paper trade
why ? :(
If one makes profit and shares the parameters it might be just fluke ... . I tried for weeks now and the around 100 dollar profit were eaten up completely by the fees. If trading without fees (Gdax maker) then the execution is so much delayed that I don't end in positive results.
Bottom line I'm not 100% sure if all the smart trader strategies make any sense at all compared with buy and hold.
But I keep trying.....
Been lurking and testing for a few weeks now. I agree with @marco4444 on this. There is two parts. With default settings the bot trades with major loss. Took a few days to tinker with the settings to get it to trade with profit via paper trading. In real trading on GDAX with maker the bot can't sell/buy fast enough. So by the time the bot buys/sells... The profit is already under water.
The 30 second loop is killing the bot. I've tried all sorts of settings trying to get it to trade faster. When I do it seems like I'm hitting GDAX's rate limting... Sure you might get profit during a major upswing.. However, during a sawtooth, the bot will blow through cash faster than a rock star in Vegas.
https://docs.gdax.com/#rate-limits
Perhaps add a feature to buy with maker but then sell with taker?
@krilson also tried your settings, but the bot doesn't trade, I guess is better then loosing hehe :)
Really looking forward for this to be fixed as well. I currently can't use Zenbot 3 because I'm on Windows.
Just like me - I can't even make zenbot 3 work with kraken :(
Any updates on this issue?
So does reversing the buy/sell signals really works better ?
any update?
Closing due to:
- inactivity
- devoid of solutions and mostly random questions that will most likely never get an answer here
Feel free to open new tickets for your questions if you feel the question was not adequately handled.
Is this still an ongoing issue with zenbot ? Live trading not doing what is intended and always trading at a loss ? Any feedback, thanks.
Seems so.
Default settings and seeing same thing here, its opposite of what it should do, sells when it goes low and buys when its high..and this is on --paper trades too... if it was opposite of what its doing it might actually work good,
Any ideas anyone?
@DeviaVir: A bit weird to close this as @carlos8f says this is it's highest priority on the README.md (linking to this exact issue):
- Many people are reporting losses in live trading even if the simulation results and/or paper trading is positive.
- This is my highest priority right now, since an unprofitable bot is not worth much, but please understand that reliably making profit is hard, and so is making a realistic simulator.
Not sure if the bot is the issue or the exchanges. They react so slowly (especially when there a big changes with many trades), that a simulation has no comparison with the reality anymore.
Probably quite a few people (like me) download the bot thinking it's a turn key solution when it's not. As pointed out by the OP.. "The default settings trade with a loss". You have to tune the thing. Almost on a daily basis to get it to work properly. I'd love to see a feature added that runs sims like the genetic backtester.. Then imports them into the bot on a scheduled interval. That way the bot can adapt to the current market changes.
If anything the README.md should say in big bold neon red letters.. "THIS IS NOT A MINING RIG. A un-tuned bot will trade with a loss".
@interbiznw I did exactly that. Just reversed the signals on the default trading strategy and its working "ok", still more room for improvement, but I am short on time to play with this, I just left it running for months now, and it didnt run out of money, it is slowly accumulating more. I would say it is right 70% of the time.
Considering this crazy bull run in bitcoin its hard to say that it is making a profit, since it would be better to just leave it all in bitcoin, but I dont consider this a normal environment...
Is really that simple to solve this problem by reversing buy/sell signals? Simply too easy to be true :p
@dukye Sure, here it is, very simple modification: krilson@66fc7fa#diff-c081f5fa7634cfc3a1254e982a748f9c
My conf looks like this:
// Optional stop-order triggers:
// sell if price drops below this % of bought price (0 to disable)
c.sell_stop_pct = 0
// buy if price surges above this % of sold price (0 to disable)
c.buy_stop_pct = 0
// enable trailing sell stop when reaching this % profit (0 to disable)
c.profit_stop_enable_pct = 0
// maintain a trailing stop this % below the high-water mark of profit
c.profit_stop_pct = 1
// Order execution rules:
// avoid trading at a slippage above this pct
c.max_slippage_pct = 5
// buy with this % of currency balance
c.buy_pct = 50
// sell with this % of asset balance
c.sell_pct = 90
// ms to adjust non-filled order after
c.order_adjust_time = 30000
// avoid selling at a loss below this pct
c.max_sell_loss_pct = 0.1
// ms to poll order status
c.order_poll_time = 5000
// ms to wait for settlement (after an order cancel)
c.wait_for_settlement = 5000
// ms to wait for settlement (after a funds on hold error)
c.wait_more_for_settlement = 60000
// % to mark up or down price for orders
c.markup_pct = 0
// Misc options:
// default # days for backfill and sim commands
c.days = 14
// ms to poll new trades at
c.poll_trades = 60000
// amount of currency to start simulations with
c.currency_capital = 1000
// amount of asset to start simulations with
c.asset_capital = 0
// for sim, reverse time at the end of the graph, normalizing buy/hold to 0
c.symmetrical = false
// number of periods to calculate RSI at
c.rsi_periods = 14
// period to record balances for stats
c.balance_snapshot_period = '15m'
And I start the bot with this:
./zenbot.sh trade --period=2h --min_periods=60 --trend_ema=30 --neutral_rate=0.25 poloniex.LTC-XMR
I am experimenting now one pair with a 24h period. Experiment with the neutral rate a bit, depends how soon you want it to buy/sell.
Thank you for sharing @krilson Would you mind sharing your statistics? My bot doesn't seem to be trading with very much with these settings
@krilson when I install zenbot for the first time, are your changes already included or do I have to edit them in myself? Setting up a Raspi fresh, so asking in advance. thanks!
@krilson thanks :)
@ether-btc yourself check krilson's prior post 9 days ago :) he join the diff link for trend_ema strategie.
will the fix be merged?
I was going to try this bot but then noticed this issue. Checking the code made me wonder, in the file mentioned above (extensions/strategies/trend_ema/strategy.js), the description at the top says:
Buy when (EMA - last(EMA) > 0) and sell when (EMA - last(EMA) < 0).
(I'm assuming s.period.trend_ema_rate represents EMA and s.period.trend_ema_stddev represents last(EMA) here. Please correct me if I'm wrong..)
- On line 51,
s.period.trend_ema_rate > s.period.trend_ema_stddevdoes match(EMA - last(EMA) > 0)(→ Buy)
However, the code at line 59 doesn't seem to reflect this statement and it seems a logic error was made while putting it into code:
s.period.trend_ema_rate < (s.period.trend_ema_stddev * -1)does not seem to match(EMA - last(EMA) < 0)(→ Sell)
Let's say EMA = s.period.trend_ema_rate = a and last(EMA) = s.period.trend_ema_stddev = b
In this case, the description says we should sell when a - b < 0, however it's clear that when we rewrite it to match the current format in code, this does not add up:
a - b < 0
(a - b) + b < 0 + b
a < b, while in code, it currently saysa < -b
So, I'm going to change line 59 to the following and then give this bot a spin!
else if (!s.cancel_down && s.period.trend_ema_rate < s.period.trend_ema_stddev) {
Edit: Commit on hotfix branch: mathiasaerts@59eed42#diff-c081f5fa7634cfc3a1254e982a748f9c
Edit: Commit on hotfix branch: mathiasaerts/zenbot@59eed42#diff-c081f5fa7634cfc3a1254e982a748f9c
With this change in the simulation mode, the bot shows the worst result ...
@mathiasaerts Great!. Did you already make some simulations comparing the result before and after your fix (same period, same options)? Can you do it ?
(I'm assuming s.period.trend_ema_rate represents EMA and s.period.trend_ema_stddev represents last(EMA) here. Please correct me if I'm wrong..)
Nope, you're wrong trend_ema_rate is the trend_ema expressed as a rate:
(EMA - last(EMA)) / last(EMA) * 100 https://github.com/carlos8f/zenbot/blob/55d877a1998e65171c3efa544d8b3476ee11e61f/extensions/strategies/trend_ema/strategy.js#L30
trend_ema_rate is positive if EMA - last(EMA) > 0 so the signal should be buy, negative if EMA - last(EMA) < 0 so the signal should be sell
trend_ema_stddev is a threshold value that avoid trades if -trend_ema_stddev < trend_ema_rate < trend_ema_stddev, by default is the standard deviation of the trend_ema on the last 10 periods, but it could be overridden by a static value passing the option neutral_rate.
So https://github.com/carlos8f/zenbot/blob/55d877a1998e65171c3efa544d8b3476ee11e61f/extensions/strategies/trend_ema/strategy.js#L51-L64
conditions are correct, the issue is not here.
What I don't get is s.acted_on_trend variable, looking at the code it should always be false or undefined so the ternary operator
https://github.com/carlos8f/zenbot/blob/55d877a1998e65171c3efa544d8b3476ee11e61f/extensions/strategies/trend_ema/strategy.js#L64
always resolve the first expression -> it always acts. This does not sound right to me, I'll have a look and get back.
@smarzola Thanks for clearing that up. I was obviously drawing a conclusion way too fast.
@kazdimenator Thanks for trying it out anyway! It seems like I need to take a better look first..
@smarzola The "acted_on_trend" part of the code does not make sense to me either. It seems like the code is trying to say "if the trend in the last period was not up, then we did not act on it", ie "act" on the first period with a buy signal. The way the code currently is, it seems like the acted_on_trend variable is a dummy variable that is not really doing anything.
In live trading, the moment the buy threshold is hit (emarate > trend_ema_stddev), the console output signals a "buy"; then on the next period if emarate>trend_ema_stddev still holds, it says "buying" and attempts to make the trade until it says "bought". (similar for "sell", "selling", "sold" statuses). Not sure if this helps you in any way, just trying to spark some ideas.
@krilson it would be very interesting to see this play out so that the bot does not whipsaw as much and does not enters/exit so suddenly. i may try playing with this
(ema current - ema previous = close to 0) = subsignal 1 (S1)
(ema current - ema previous = positive) = subsignal 2 (S2) - ema going up
(ema current - ema previous = negative) = subsignal 3 (S3) - ema going down
Buy on consecutive S2 signals, or when pattern goes from S3-->S1-->S2 (reversal)
Sell on consecutive S3 signals, or when the pattern goes from S2-->S1-->S3 (reversal)
Observations for neutral_rate: For using small periods like 2m or 5m, neutral_rate="auto" works well, but causes too many whipsaws. For larger periods (15m, 1h), neutral_rate="auto" does not trigger often thus some set the neutral_rate to an arbitrary .1 or .2 (i guess because this .1 or .2 is bigger than the trend_ema_stddev using a long period). I wonder if multiplying the trend_ema_stddev by a variable based on period size would work instead of setting it manually. To generalize, since neutral_rate and period are correlated, perhaps there is some variable "k" such that k*trend_ema_stddev smoothes the whipsaws, where k is larger for small periods and smaller for large periods. This would help filter out whipsaws for small-period trials and help pick up signals better for large periods trials using a variable neutral_rate.
I hope the above is helpful in some way. If i implement the new trend or new neutral_rate strategy mentioned above I will post the code snippets (despite having ever coded in node.js). Thank you for devoting the time to developing and improving Zenbot, very amazing tool.
EDIT (results): I tested multiplying the stddev by a constant - while it filters out whipsaws a bit for low periods, it makes the buy/sell trigger peaks less accurate, so there will always be that tradeoff; concluded that the default neutral_rate is great without the multiplier for low periods and ideally an arbitrary constant close to 0 for larger periods. Also briefly tested krilson's strategy - one can accomplish this using
s.period.trend_ema_rate_1 = (s.lookback[0].trend_ema - s.lookback[1].trend_ema) / s.lookback[1].trend_ema * 100
s.period.trend_ema_rate_2 = (s.lookback[1].trend_ema - s.lookback[2].trend_ema) / s.lookback[2].trend_ema * 100
.......... (for t periods, depending on how sensitive you want the signal to be)
Then for all of them do: if s.period.trend_ema_rate_t > neutral_rate, S2, if s.period.trend_ema_rate_t < -neutral_rate, S3, else S1
Store these rates in an array {rate_1, rate_2, .... , rate_t}
And code the strategy: for example, if {S3 S3 S1 S2 S2} then buy, if {S2 S2 S1 S3 S3} then sell
I personally got too lazy to implement all the variations for this last step; if someone has a proposition to nail these combinations, perhaps I can finish the code. I imagine this working very well at picking up momentum in low-period (1m, 2m) simulations
this might just be a layperson's observations of what's going on behind the scenes, but this is what i'm observing.
current:
- backfill via api/websocket for purposes of backtesting
- trade paper/live: backfill to get minimum periods satisfied in order to calculate technical components of strategy.
- once technical components are calculated, switch to websocket (if available) to collect data until queried based on strategy timers set in conf.js, then dump cached data to db.
- trade data analyzed during current period and compared to previous periods for buy/sell orders.
- if buy/sell logic triggered, trade onPeriod
the way it SHOULD be:
- backfill for genetic/backtest if so desired.
- trade in paper/live: backfill/websocket for minimum periods per strategy requirements.
- calculate strategy technicals for previous trade data
- engage websocket and start caching to ram.
- trade data for current period calculated at 500ms intervals (or based on some calculation of exchange trade data volume/trade volume) in aggregate as ohlc for a minimum of 10% of period_length duration or another % which allows for minimal amount of data to provide accurate comparison to historic period technical calculations.
- act on buy/sell or out of band signals as soon as triggers activate, but not before minimal amount of data has been collected.
- once period concludes, dump websocket ram cache to DB and calculate previous period data for next period technical comparison.
other thoughts:
this is less of a concern for people using strategies that are <5m in period_length, but if you're over 5/10m, let's say 45m, and current period data indicates an oversold/buy/sell 10m into your current period, it's gonna wait 35m to act on that?...that don't work.
strategies are also the wrong approach. strategies should be replaced with a library of technical indicators which one should be allowed to use in order to create strategies via simple equation logic. maybe provide a handful of out of the box "strategies" to show how they are utilized, but not depend upon them for continued or base utility.
No news about this?
Hey guys! I'm working on a new bot: Bot18, which will be more flexible, better execution, not TA-dependent. Hopefully will lead to much better profitability! However, exchange support will take a while to port over and implement for the new socket-based, realtime approach. Stay tuned!
~Carlos
@shr00mie, those are my thoughts entirely, and I've implemented that approach already in Bot18. It's gonna be 🔥 !
@carlos8f so it sounds like this is going to be a lot more event driven than the on_period nature of zenbot (yay!). the approach i'm currently working around is standardization of api/websocket components to hopefully allow plug and play exchange connectivity. commands and URLs would be the variables while standardization of API/websocket/order workflow would be ubiquitous.
what would you be replacing TA with any why? not following on the desire or move away from a standardized computational lib. something hand made or custom seems like it would be considerably more error prone.
looking forward to seeing what you're doing with as well as how data is going to be stored compared to zenbot (which was pretty painful). i feel like the DB could have been considerably more utilized for things like trade history, user account storage (user app login&pass, encrypted API keys, etc).
@shr00mie yes, bot18 will need to standardize the ws API workflow. bot18 doesn't have exchange abstraction yet, so i can focus on the maximum potential of the gdax api, my primary target. abstraction will come as soon as i get the foundation laid.
I'm not replacing TA with anything, just moving TA to the background, one potential way to generate orders. The engine will be more low-level, so basically the commands are:
- "set limit order at X price X size", then "cancel when conditions are..."
- "maintain limit order at X price depth", cancel when...
- "execute taker order with X funds or X asset"
Then, if we're mimicking what Zenbot 4 does by default, the strategy says every 2m,
- "load me the latest candles for X granularity"
- "apply the TA stuff" (ta-lib)
- "does my strategy give a signal change?"
- if strategy likes maker order, apply command 2, cancel when enough asset is traded to satisfy signal
- else, execute command 3
That way, the engine input is changed to 3 mix-and-match commands (you can now buy and sell at the same time with multiple orders on each side), instead of a vague "signal" which invalidates signals before it and doesn't differentiate between command 2 and 3 (you have to provide taker or maker as an arg)
And the addition of command 1, which doesn't exist in Zenbot yet, will make market-making possible, and strategies will be able to maintain trailing stops directly.