Wrong error raised by BTBSession on too many errors
csala opened this issue · 0 comments
csala commented
- BTB version > 0.3.5
Description
There is an error in the way the remaining options for the BTBSession are evaluate which makes the session crash with a misleading exception when it has actually run out of options.
What I Did
The erro can be provoked by using a single scorer that raises an exception.
After the max_errors
is reached, the expected StopTuning
exception is not being raised, but rather a misleading ValueError
:
In [1]: tunables = {
...: 'a': {
...: 'a': {'type': 'int', 'default': 0, 'range': [0, 100]},
...: }
...: }
...:
...: def scorer(name, hyperparameters):
...: raise Exception()
...:
...: from btb.session import BTBSession
...:
...: session = BTBSession(tunables, scorer, verbose=True)
...:
...: session.run(5)
...:
ValueError Traceback (most recent call last)
<ipython-input-1-34ba33972a06> in <module>
12 session = BTBSession(tunables, scorer, verbose=True)
13
---> 14 session.run(5)
~/Projects/MIT/BTB/btb/session.py in run(self, iterations)
296 for _ in iterator:
297 self.iterations += 1
--> 298 tunable_name, config = self.propose()
299
300 try:
~/Projects/MIT/BTB/btb/session.py in propose(self)
199
200 else:
--> 201 tunable_name = self._selector.select(self._normalized_scores)
202 tuner = self._tuners[tunable_name]
203 try:
~/Projects/MIT/BTB/btb/selection/selector.py in select(self, choice_scores)
63 choice_rewards[choice] = self.compute_rewards(scores)
64
---> 65 return self.bandit(choice_rewards)
~/Projects/MIT/BTB/btb/selection/ucb1.py in bandit(self, choice_rewards)
46 return average_reward + error
47
---> 48 return max(self._shuffle(choice_rewards), key=ucb1)
ValueError: max() arg is an empty sequence