how could you train so fast for MoveToBeacon?

[hechenghai]

These are some of my results of training MoveToBeacon.
I see your results that the score could reach up to 20 when episode is 500.

NSWindowEdgeResizing is off!
2017-12-10 22:33:43.740567

batch 0

saving model to _files/models/my_beacon_model, step 0
episode 0 ended. Score 1.000000
episode 1 ended. Score 1.000000
episode 2 ended. Score 1.000000
episode 3 ended. Score 0.000000
episode 4 ended. Score 0.000000
episode 5 ended. Score 2.000000
episode 6 ended. Score 2.000000
episode 7 ended. Score 0.000000
episode 8 ended. Score 0.000000
episode 9 ended. Score 0.000000
episode 10 ended. Score 1.000000
episode 11 ended. Score 0.000000
episode 12 ended. Score 1.000000
episode 13 ended. Score 1.000000
episode 14 ended. Score 0.000000
episode 15 ended. Score 1.000000
2017-12-10 22:39:20.393286

batch 500

episode 16 ended. Score 0.000000
episode 17 ended. Score 0.000000
episode 18 ended. Score 0.000000
episode 19 ended. Score 0.000000
episode 20 ended. Score 2.000000
episode 21 ended. Score 1.000000
episode 22 ended. Score 1.000000
episode 23 ended. Score 1.000000
episode 24 ended. Score 0.000000
episode 25 ended. Score 0.000000
episode 26 ended. Score 0.000000
episode 27 ended. Score 1.000000
episode 28 ended. Score 0.000000
episode 29 ended. Score 0.000000
episode 30 ended. Score 0.000000
episode 31 ended. Score 2.000000
episode 32 ended. Score 0.000000
2017-12-10 22:44:49.787661

batch 1000

episode 33 ended. Score 0.000000
episode 34 ended. Score 0.000000
episode 35 ended. Score 1.000000
episode 36 ended. Score 0.000000
episode 37 ended. Score 0.000000
episode 38 ended. Score 0.000000
episode 39 ended. Score 2.000000
episode 40 ended. Score 0.000000
episode 41 ended. Score 0.000000
episode 42 ended. Score 0.000000
episode 43 ended. Score 0.000000
episode 44 ended. Score 1.000000
episode 45 ended. Score 1.000000
episode 46 ended. Score 1.000000
episode 47 ended. Score 0.000000
episode 48 ended. Score 0.000000
episode 49 ended. Score 0.000000
2017-12-10 22:52:10.556408

batch 1500

episode 50 ended. Score 1.000000
episode 51 ended. Score 0.000000
episode 52 ended. Score 2.000000
episode 53 ended. Score 1.000000
episode 54 ended. Score 0.000000
episode 55 ended. Score 1.000000
episode 56 ended. Score 0.000000
episode 57 ended. Score 0.000000
episode 58 ended. Score 1.000000
episode 59 ended. Score 1.000000
episode 60 ended. Score 0.000000
episode 61 ended. Score 0.000000
episode 62 ended. Score 0.000000
episode 63 ended. Score 3.000000
episode 64 ended. Score 0.000000
episode 65 ended. Score 0.000000
2017-12-10 22:58:03.336561

batch 2000

episode 66 ended. Score 1.000000
episode 67 ended. Score 0.000000
episode 68 ended. Score 1.000000
episode 69 ended. Score 0.000000
episode 70 ended. Score 0.000000
episode 71 ended. Score 0.000000
episode 72 ended. Score 1.000000
episode 73 ended. Score 0.000000
episode 74 ended. Score 0.000000
episode 75 ended. Score 0.000000
episode 76 ended. Score 1.000000
episode 77 ended. Score 0.000000
episode 78 ended. Score 0.000000
episode 79 ended. Score 2.000000
episode 80 ended. Score 0.000000
episode 81 ended. Score 1.000000
episode 82 ended. Score 2.000000
2017-12-10 23:02:32.007840

batch 2500

episode 83 ended. Score 0.000000
episode 84 ended. Score 1.000000
episode 85 ended. Score 0.000000
episode 86 ended. Score 0.000000
episode 87 ended. Score 1.000000
episode 88 ended. Score 0.000000
episode 89 ended. Score 1.000000
episode 90 ended. Score 0.000000
episode 91 ended. Score 2.000000
episode 92 ended. Score 0.000000
episode 93 ended. Score 0.000000
episode 94 ended. Score 0.000000
episode 95 ended. Score 1.000000
episode 96 ended. Score 0.000000
episode 97 ended. Score 0.000000
episode 98 ended. Score 0.000000
episode 99 ended. Score 2.000000
2017-12-10 23:06:58.782262

batch 3000

episode 100 ended. Score 1.000000
episode 101 ended. Score 1.000000
episode 102 ended. Score 0.000000
episode 103 ended. Score 1.000000
episode 104 ended. Score 1.000000
episode 105 ended. Score 1.000000
episode 106 ended. Score 0.000000
episode 107 ended. Score 0.000000
episode 108 ended. Score 2.000000
episode 109 ended. Score 0.000000
episode 110 ended. Score 2.000000
episode 111 ended. Score 2.000000
episode 112 ended. Score 3.000000
episode 113 ended. Score 0.000000
episode 114 ended. Score 0.000000
episode 115 ended. Score 1.000000
2017-12-10 23:11:25.523182

batch 3500

episode 116 ended. Score 3.000000
episode 117 ended. Score 0.000000
episode 118 ended. Score 0.000000
episode 119 ended. Score 1.000000
episode 120 ended. Score 1.000000
episode 121 ended. Score 1.000000
episode 122 ended. Score 1.000000
episode 123 ended. Score 0.000000
episode 124 ended. Score 1.000000
episode 125 ended. Score 0.000000
episode 126 ended. Score 1.000000
episode 127 ended. Score 1.000000
episode 128 ended. Score 0.000000
episode 129 ended. Score 0.000000
episode 130 ended. Score 0.000000
episode 131 ended. Score 2.000000
episode 132 ended. Score 0.000000
2017-12-10 23:15:52.268392

batch 4000

saving model to _files/models/my_beacon_model, step 4000
episode 133 ended. Score 0.000000
episode 134 ended. Score 0.000000
episode 135 ended. Score 0.000000
episode 136 ended. Score 0.000000
episode 137 ended. Score 0.000000
episode 138 ended. Score 1.000000
episode 139 ended. Score 0.000000
episode 140 ended. Score 0.000000
episode 141 ended. Score 0.000000
episode 142 ended. Score 0.000000
episode 143 ended. Score 0.000000
episode 144 ended. Score 1.000000
episode 145 ended. Score 0.000000
episode 146 ended. Score 0.000000
episode 147 ended. Score 1.000000
episode 148 ended. Score 1.000000
episode 149 ended. Score 1.000000
2017-12-10 23:22:51.106148

batch 4500

episode 150 ended. Score 1.000000
episode 151 ended. Score 0.000000
episode 152 ended. Score 0.000000
episode 153 ended. Score 0.000000
episode 154 ended. Score 0.000000
episode 155 ended. Score 1.000000
episode 156 ended. Score 2.000000
episode 157 ended. Score 1.000000
episode 158 ended. Score 1.000000
episode 159 ended. Score 0.000000
episode 160 ended. Score 0.000000
episode 161 ended. Score 0.000000
episode 162 ended. Score 0.000000
episode 163 ended. Score 1.000000
episode 164 ended. Score 1.000000
episode 165 ended. Score 0.000000
2017-12-10 23:28:30.229832

batch 5000

episode 166 ended. Score 0.000000
episode 167 ended. Score 1.000000
episode 168 ended. Score 1.000000
episode 169 ended. Score 2.000000
episode 170 ended. Score 1.000000
episode 171 ended. Score 1.000000
episode 172 ended. Score 1.000000
episode 173 ended. Score 0.000000
episode 174 ended. Score 0.000000
episode 175 ended. Score 0.000000
episode 176 ended. Score 0.000000
episode 177 ended. Score 0.000000
episode 178 ended. Score 1.000000
episode 179 ended. Score 2.000000
episode 180 ended. Score 0.000000
episode 181 ended. Score 0.000000
episode 182 ended. Score 0.000000
2017-12-10 23:33:05.909882

batch 5500

episode 183 ended. Score 1.000000
episode 184 ended. Score 1.000000
episode 185 ended. Score 2.000000
episode 186 ended. Score 0.000000
episode 187 ended. Score 0.000000
episode 188 ended. Score 1.000000
episode 189 ended. Score 0.000000
episode 190 ended. Score 0.000000
episode 191 ended. Score 0.000000
episode 192 ended. Score 1.000000
episode 193 ended. Score 2.000000
episode 194 ended. Score 1.000000
episode 195 ended. Score 2.000000
episode 196 ended. Score 0.000000
episode 197 ended. Score 0.000000
episode 198 ended. Score 3.000000
episode 199 ended. Score 2.000000
2017-12-10 23:39:06.014119

batch 6000

episode 200 ended. Score 0.000000
episode 201 ended. Score 0.000000
episode 202 ended. Score 0.000000
episode 203 ended. Score 1.000000
episode 204 ended. Score 1.000000
episode 205 ended. Score 1.000000
episode 206 ended. Score 1.000000
episode 207 ended. Score 1.000000
episode 208 ended. Score 1.000000
episode 209 ended. Score 1.000000
episode 210 ended. Score 0.000000
episode 211 ended. Score 0.000000
episode 212 ended. Score 1.000000
episode 213 ended. Score 1.000000
episode 214 ended. Score 1.000000
episode 215 ended. Score 2.000000
2017-12-10 23:45:45.587434

batch 6500

episode 216 ended. Score 2.000000
episode 217 ended. Score 0.000000
episode 218 ended. Score 0.000000
episode 219 ended. Score 1.000000
episode 220 ended. Score 1.000000
episode 221 ended. Score 1.000000
episode 222 ended. Score 0.000000
episode 223 ended. Score 2.000000
episode 224 ended. Score 0.000000
episode 225 ended. Score 0.000000
episode 226 ended. Score 0.000000
episode 227 ended. Score 0.000000
episode 228 ended. Score 0.000000
episode 229 ended. Score 0.000000
episode 230 ended. Score 0.000000
episode 231 ended. Score 0.000000
episode 232 ended. Score 1.000000
2017-12-10 23:52:25.792195

batch 7000

episode 233 ended. Score 0.000000
episode 234 ended. Score 1.000000
episode 235 ended. Score 0.000000
episode 236 ended. Score 1.000000
episode 237 ended. Score 0.000000
episode 238 ended. Score 1.000000
episode 239 ended. Score 0.000000
episode 240 ended. Score 1.000000
episode 241 ended. Score 0.000000
episode 242 ended. Score 0.000000
episode 243 ended. Score 1.000000
episode 244 ended. Score 0.000000
episode 245 ended. Score 0.000000
episode 246 ended. Score 1.000000
episode 247 ended. Score 0.000000
episode 248 ended. Score 1.000000
episode 249 ended. Score 2.000000
2017-12-10 23:59:06.497571

batch 7500

episode 250 ended. Score 0.000000
episode 251 ended. Score 0.000000
episode 252 ended. Score 1.000000
episode 253 ended. Score 1.000000
episode 254 ended. Score 0.000000
episode 255 ended. Score 2.000000
episode 256 ended. Score 0.000000
episode 257 ended. Score 0.000000
episode 258 ended. Score 1.000000
episode 259 ended. Score 0.000000
episode 260 ended. Score 0.000000
episode 261 ended. Score 1.000000
episode 262 ended. Score 1.000000
episode 263 ended. Score 2.000000
episode 264 ended. Score 2.000000
episode 265 ended. Score 0.000000
2017-12-11 00:05:47.152910

batch 8000

saving model to _files/models/my_beacon_model, step 8000
episode 266 ended. Score 1.000000
episode 267 ended. Score 0.000000
episode 268 ended. Score 0.000000
episode 269 ended. Score 1.000000
episode 270 ended. Score 0.000000
episode 271 ended. Score 0.000000
episode 272 ended. Score 2.000000
episode 273 ended. Score 0.000000
episode 274 ended. Score 0.000000
episode 275 ended. Score 0.000000
episode 276 ended. Score 0.000000
episode 277 ended. Score 1.000000
episode 278 ended. Score 1.000000
episode 279 ended. Score 1.000000
episode 280 ended. Score 0.000000
episode 281 ended. Score 0.000000
episode 282 ended. Score 1.000000
2017-12-11 00:12:41.691547

batch 8500

episode 283 ended. Score 0.000000
episode 284 ended. Score 0.000000
episode 285 ended. Score 0.000000
episode 286 ended. Score 0.000000
episode 287 ended. Score 0.000000
episode 288 ended. Score 0.000000
episode 289 ended. Score 2.000000
episode 290 ended. Score 0.000000
episode 291 ended. Score 2.000000
episode 292 ended. Score 0.000000
episode 293 ended. Score 1.000000
episode 294 ended. Score 0.000000
episode 295 ended. Score 0.000000
episode 296 ended. Score 1.000000
episode 297 ended. Score 0.000000
episode 298 ended. Score 0.000000
episode 299 ended. Score 0.000000
2017-12-11 00:19:25.455596

batch 9000

episode 300 ended. Score 1.000000
episode 301 ended. Score 2.000000
episode 302 ended. Score 0.000000
episode 303 ended. Score 0.000000
episode 304 ended. Score 1.000000
episode 305 ended. Score 1.000000
episode 306 ended. Score 0.000000
episode 307 ended. Score 0.000000
episode 308 ended. Score 0.000000
episode 309 ended. Score 0.000000
episode 310 ended. Score 1.000000
episode 311 ended. Score 0.000000
episode 312 ended. Score 0.000000
episode 313 ended. Score 0.000000
episode 314 ended. Score 1.000000
episode 315 ended. Score 2.000000
2017-12-11 00:26:08.259561

batch 9500

episode 316 ended. Score 0.000000
episode 317 ended. Score 1.000000
episode 318 ended. Score 0.000000
episode 319 ended. Score 0.000000
episode 320 ended. Score 0.000000
episode 321 ended. Score 0.000000
episode 322 ended. Score 0.000000
episode 323 ended. Score 0.000000
episode 324 ended. Score 0.000000
episode 325 ended. Score 0.000000
episode 326 ended. Score 1.000000
episode 327 ended. Score 0.000000
episode 328 ended. Score 0.000000
episode 329 ended. Score 1.000000
episode 330 ended. Score 0.000000
episode 331 ended. Score 0.000000
episode 332 ended. Score 0.000000
2017-12-11 00:32:50.869950

batch 10000

episode 333 ended. Score 0.000000
episode 334 ended. Score 0.000000
episode 335 ended. Score 0.000000
episode 336 ended. Score 1.000000
episode 337 ended. Score 0.000000
episode 338 ended. Score 1.000000
episode 339 ended. Score 0.000000
episode 340 ended. Score 0.000000
episode 341 ended. Score 1.000000
episode 342 ended. Score 2.000000
episode 343 ended. Score 0.000000
episode 344 ended. Score 0.000000
episode 345 ended. Score 1.000000
episode 346 ended. Score 1.000000
episode 347 ended. Score 2.000000
episode 348 ended. Score 1.000000
episode 349 ended. Score 0.000000
2017-12-11 00:39:34.598646

batch 10500

episode 350 ended. Score 0.000000
episode 351 ended. Score 0.000000
episode 352 ended. Score 1.000000
episode 353 ended. Score 0.000000
episode 354 ended. Score 1.000000
episode 355 ended. Score 0.000000
episode 356 ended. Score 2.000000
episode 357 ended. Score 0.000000
episode 358 ended. Score 1.000000
episode 359 ended. Score 0.000000
episode 360 ended. Score 2.000000
episode 361 ended. Score 0.000000
episode 362 ended. Score 0.000000
episode 363 ended. Score 2.000000
episode 364 ended. Score 1.000000
episode 365 ended. Score 2.000000
2017-12-11 00:46:17.711384

batch 11000

episode 366 ended. Score 0.000000
episode 367 ended. Score 0.000000
episode 368 ended. Score 1.000000
episode 369 ended. Score 1.000000
episode 370 ended. Score 0.000000
episode 371 ended. Score 3.000000
episode 372 ended. Score 0.000000
episode 373 ended. Score 0.000000
episode 374 ended. Score 1.000000
episode 375 ended. Score 0.000000
episode 376 ended. Score 0.000000
episode 377 ended. Score 0.000000
episode 378 ended. Score 0.000000
episode 379 ended. Score 1.000000
episode 380 ended. Score 4.000000
episode 381 ended. Score 0.000000
episode 382 ended. Score 0.000000
2017-12-11 00:53:01.373379

batch 11500

episode 383 ended. Score 2.000000
episode 384 ended. Score 2.000000
episode 385 ended. Score 0.000000
episode 386 ended. Score 0.000000
episode 387 ended. Score 3.000000
episode 388 ended. Score 1.000000
episode 389 ended. Score 0.000000
episode 390 ended. Score 1.000000
episode 391 ended. Score 0.000000
episode 392 ended. Score 1.000000
episode 393 ended. Score 3.000000
episode 394 ended. Score 1.000000
episode 395 ended. Score 0.000000
episode 396 ended. Score 2.000000
episode 397 ended. Score 2.000000
episode 398 ended. Score 1.000000
episode 399 ended. Score 0.000000
2017-12-11 00:59:44.721279

batch 12000

saving model to _files/models/my_beacon_model, step 12000
episode 400 ended. Score 1.000000
episode 401 ended. Score 0.000000
episode 402 ended. Score 0.000000
episode 403 ended. Score 1.000000
episode 404 ended. Score 0.000000
episode 405 ended. Score 0.000000
episode 406 ended. Score 1.000000
episode 407 ended. Score 1.000000
episode 408 ended. Score 1.000000
episode 409 ended. Score 0.000000
episode 410 ended. Score 0.000000
episode 411 ended. Score 3.000000
episode 412 ended. Score 1.000000
episode 413 ended. Score 0.000000
episode 414 ended. Score 1.000000
episode 415 ended. Score 1.000000
2017-12-11 01:06:27.963973

batch 12500

episode 416 ended. Score 0.000000
episode 417 ended. Score 0.000000
episode 418 ended. Score 1.000000
episode 419 ended. Score 1.000000
episode 420 ended. Score 1.000000
episode 421 ended. Score 0.000000
episode 422 ended. Score 0.000000
episode 423 ended. Score 1.000000
episode 424 ended. Score 0.000000
episode 425 ended. Score 2.000000
episode 426 ended. Score 2.000000
episode 427 ended. Score 1.000000
episode 428 ended. Score 0.000000
episode 429 ended. Score 1.000000
episode 430 ended. Score 0.000000
episode 431 ended. Score 0.000000
episode 432 ended. Score 1.000000
2017-12-11 01:12:59.229277

batch 13000

episode 433 ended. Score 0.000000
episode 434 ended. Score 0.000000
episode 435 ended. Score 0.000000
episode 436 ended. Score 0.000000
episode 437 ended. Score 2.000000
episode 438 ended. Score 0.000000
episode 439 ended. Score 1.000000
episode 440 ended. Score 1.000000
episode 441 ended. Score 2.000000
episode 442 ended. Score 0.000000
episode 443 ended. Score 0.000000
episode 444 ended. Score 0.000000
episode 445 ended. Score 1.000000
episode 446 ended. Score 1.000000
episode 447 ended. Score 1.000000
episode 448 ended. Score 0.000000
episode 449 ended. Score 1.000000
2017-12-11 01:19:31.117036

batch 13500

episode 450 ended. Score 2.000000
episode 451 ended. Score 0.000000
episode 452 ended. Score 0.000000
episode 453 ended. Score 0.000000
episode 454 ended. Score 0.000000
episode 455 ended. Score 0.000000
episode 456 ended. Score 0.000000
episode 457 ended. Score 0.000000
episode 458 ended. Score 1.000000
episode 459 ended. Score 3.000000
episode 460 ended. Score 1.000000
episode 461 ended. Score 0.000000
episode 462 ended. Score 0.000000
episode 463 ended. Score 1.000000
episode 464 ended. Score 0.000000
episode 465 ended. Score 0.000000
2017-12-11 01:26:03.445053

batch 14000

episode 466 ended. Score 1.000000
episode 467 ended. Score 0.000000
episode 468 ended. Score 1.000000
episode 469 ended. Score 0.000000
episode 470 ended. Score 1.000000
episode 471 ended. Score 1.000000
episode 472 ended. Score 2.000000
episode 473 ended. Score 0.000000
episode 474 ended. Score 0.000000
episode 475 ended. Score 0.000000
episode 476 ended. Score 0.000000
episode 477 ended. Score 3.000000
episode 478 ended. Score 0.000000
episode 479 ended. Score 1.000000
episode 480 ended. Score 3.000000
episode 481 ended. Score 1.000000
episode 482 ended. Score 0.000000
2017-12-11 01:32:37.656397

batch 14500

episode 483 ended. Score 1.000000
episode 484 ended. Score 2.000000
episode 485 ended. Score 0.000000
episode 486 ended. Score 0.000000
episode 487 ended. Score 2.000000
episode 488 ended. Score 0.000000
episode 489 ended. Score 0.000000
episode 490 ended. Score 0.000000
episode 491 ended. Score 0.000000
episode 492 ended. Score 1.000000
episode 493 ended. Score 0.000000
episode 494 ended. Score 0.000000
episode 495 ended. Score 0.000000
episode 496 ended. Score 0.000000
episode 497 ended. Score 2.000000
episode 498 ended. Score 1.000000
episode 499 ended. Score 0.000000
2017-12-11 01:39:15.536213

batch 15000

episode 500 ended. Score 0.000000
episode 501 ended. Score 2.000000
episode 502 ended. Score 0.000000
episode 503 ended. Score 1.000000
episode 504 ended. Score 0.000000
episode 505 ended. Score 1.000000
episode 506 ended. Score 2.000000
episode 507 ended. Score 0.000000
episode 508 ended. Score 1.000000
episode 509 ended. Score 3.000000
episode 510 ended. Score 1.000000
episode 511 ended. Score 1.000000
episode 512 ended. Score 0.000000
episode 513 ended. Score 0.000000
episode 514 ended. Score 1.000000
episode 515 ended. Score 1.000000
2017-12-11 01:45:54.685208

batch 15500

episode 516 ended. Score 0.000000
episode 517 ended. Score 0.000000
episode 518 ended. Score 0.000000
episode 519 ended. Score 1.000000
episode 520 ended. Score 0.000000
episode 521 ended. Score 1.000000
episode 522 ended. Score 0.000000
episode 523 ended. Score 0.000000
episode 524 ended. Score 0.000000
episode 525 ended. Score 1.000000
episode 526 ended. Score 1.000000
episode 527 ended. Score 0.000000
episode 528 ended. Score 0.000000
episode 529 ended. Score 0.000000
episode 530 ended. Score 0.000000
episode 531 ended. Score 0.000000
episode 532 ended. Score 3.000000
2017-12-11 01:52:34.131092

batch 16000

saving model to _files/models/my_beacon_model, step 16000
episode 533 ended. Score 0.000000
episode 534 ended. Score 2.000000
episode 535 ended. Score 1.000000
episode 536 ended. Score 0.000000
episode 537 ended. Score 0.000000
episode 538 ended. Score 2.000000
episode 539 ended. Score 0.000000
episode 540 ended. Score 1.000000
episode 541 ended. Score 1.000000
episode 542 ended. Score 2.000000
episode 543 ended. Score 0.000000
episode 544 ended. Score 1.000000
episode 545 ended. Score 2.000000
episode 546 ended. Score 1.000000
episode 547 ended. Score 1.000000
episode 548 ended. Score 1.000000
episode 549 ended. Score 0.000000
2017-12-11 01:59:26.186323

batch 16500

episode 550 ended. Score 0.000000
episode 551 ended. Score 1.000000
episode 552 ended. Score 1.000000
episode 553 ended. Score 1.000000
episode 554 ended. Score 1.000000
episode 555 ended. Score 1.000000
episode 556 ended. Score 1.000000
episode 557 ended. Score 0.000000
episode 558 ended. Score 4.000000
episode 559 ended. Score 0.000000
episode 560 ended. Score 0.000000
episode 561 ended. Score 0.000000
episode 562 ended. Score 1.000000
episode 563 ended. Score 1.000000
episode 564 ended. Score 1.000000
episode 565 ended. Score 0.000000
2017-12-11 02:06:07.168191

batch 17000

episode 566 ended. Score 1.000000
episode 567 ended. Score 2.000000
episode 568 ended. Score 3.000000
episode 569 ended. Score 0.000000
episode 570 ended. Score 0.000000
episode 571 ended. Score 1.000000
episode 572 ended. Score 2.000000
episode 573 ended. Score 0.000000
episode 574 ended. Score 0.000000
episode 575 ended. Score 1.000000
episode 576 ended. Score 0.000000
episode 577 ended. Score 1.000000
episode 578 ended. Score 0.000000
episode 579 ended. Score 1.000000
episode 580 ended. Score 0.000000
episode 581 ended. Score 0.000000
episode 582 ended. Score 1.000000
2017-12-11 02:12:48.351526

batch 17500

episode 583 ended. Score 0.000000
episode 584 ended. Score 1.000000
episode 585 ended. Score 1.000000
episode 586 ended. Score 1.000000
episode 587 ended. Score 0.000000
episode 588 ended. Score 0.000000
episode 589 ended. Score 1.000000
episode 590 ended. Score 1.000000
episode 591 ended. Score 0.000000
episode 592 ended. Score 2.000000
episode 593 ended. Score 1.000000
episode 594 ended. Score 2.000000
episode 595 ended. Score 0.000000
episode 596 ended. Score 0.000000
episode 597 ended. Score 3.000000
episode 598 ended. Score 0.000000
episode 599 ended. Score 1.000000
2017-12-11 02:19:30.493351

batch 18000

episode 600 ended. Score 1.000000
episode 601 ended. Score 3.000000
episode 602 ended. Score 0.000000
episode 603 ended. Score 1.000000
episode 604 ended. Score 0.000000
episode 605 ended. Score 2.000000
episode 606 ended. Score 2.000000
episode 607 ended. Score 1.000000
episode 608 ended. Score 1.000000
episode 609 ended. Score 1.000000
episode 610 ended. Score 0.000000
episode 611 ended. Score 2.000000
episode 612 ended. Score 0.000000
episode 613 ended. Score 1.000000
episode 614 ended. Score 0.000000
episode 615 ended. Score 0.000000
2017-12-11 02:26:12.317057

batch 18500

episode 616 ended. Score 0.000000
episode 617 ended. Score 0.000000
episode 618 ended. Score 1.000000
episode 619 ended. Score 0.000000
episode 620 ended. Score 0.000000
episode 621 ended. Score 0.000000
episode 622 ended. Score 1.000000
episode 623 ended. Score 0.000000
episode 624 ended. Score 0.000000
episode 625 ended. Score 0.000000
episode 626 ended. Score 1.000000
episode 627 ended. Score 0.000000
episode 628 ended. Score 0.000000
episode 629 ended. Score 0.000000
episode 630 ended. Score 2.000000
episode 631 ended. Score 1.000000
episode 632 ended. Score 0.000000
2017-12-11 02:32:55.671106

batch 19000

episode 633 ended. Score 1.000000
episode 634 ended. Score 0.000000
episode 635 ended. Score 1.000000
episode 636 ended. Score 0.000000
episode 637 ended. Score 1.000000
episode 638 ended. Score 1.000000
episode 639 ended. Score 0.000000
episode 640 ended. Score 1.000000
episode 641 ended. Score 0.000000
episode 642 ended. Score 1.000000
episode 643 ended. Score 0.000000
episode 644 ended. Score 0.000000
episode 645 ended. Score 0.000000
episode 646 ended. Score 2.000000
episode 647 ended. Score 2.000000
episode 648 ended. Score 1.000000
episode 649 ended. Score 0.000000
2017-12-11 02:39:38.990077

batch 19500

episode 650 ended. Score 1.000000
episode 651 ended. Score 1.000000
episode 652 ended. Score 0.000000
episode 653 ended. Score 0.000000
episode 654 ended. Score 2.000000
episode 655 ended. Score 2.000000
episode 656 ended. Score 0.000000
episode 657 ended. Score 2.000000
episode 658 ended. Score 1.000000
episode 659 ended. Score 0.000000
episode 660 ended. Score 0.000000
episode 661 ended. Score 0.000000
episode 662 ended. Score 0.000000
episode 663 ended. Score 1.000000
episode 664 ended. Score 2.000000
episode 665 ended. Score 0.000000
2017-12-11 02:46:22.402119

batch 20000

saving model to _files/models/my_beacon_model, step 20000
episode 666 ended. Score 0.000000
episode 667 ended. Score 2.000000
episode 668 ended. Score 0.000000
episode 669 ended. Score 2.000000
episode 670 ended. Score 0.000000
episode 671 ended. Score 1.000000
episode 672 ended. Score 2.000000
episode 673 ended. Score 0.000000
episode 674 ended. Score 3.000000
episode 675 ended. Score 1.000000
episode 676 ended. Score 1.000000
episode 677 ended. Score 1.000000
episode 678 ended. Score 1.000000
episode 679 ended. Score 0.000000
episode 680 ended. Score 3.000000
episode 681 ended. Score 2.000000
episode 682 ended. Score 0.000000
2017-12-11 02:53:09.307516

batch 20500

episode 683 ended. Score 0.000000
episode 684 ended. Score 1.000000
episode 685 ended. Score 1.000000
episode 686 ended. Score 1.000000
episode 687 ended. Score 1.000000
episode 688 ended. Score 0.000000
episode 689 ended. Score 0.000000
episode 690 ended. Score 1.000000
episode 691 ended. Score 0.000000
episode 692 ended. Score 0.000000
episode 693 ended. Score 1.000000
episode 694 ended. Score 2.000000
episode 695 ended. Score 0.000000
episode 696 ended. Score 2.000000
episode 697 ended. Score 0.000000
episode 698 ended. Score 1.000000
episode 699 ended. Score 0.000000
2017-12-11 03:00:02.190510

batch 21000

episode 700 ended. Score 0.000000
episode 701 ended. Score 1.000000
episode 702 ended. Score 0.000000
episode 703 ended. Score 1.000000
episode 704 ended. Score 1.000000
episode 705 ended. Score 0.000000
episode 706 ended. Score 2.000000
episode 707 ended. Score 1.000000
episode 708 ended. Score 0.000000
episode 709 ended. Score 1.000000
episode 710 ended. Score 0.000000
episode 711 ended. Score 1.000000
episode 712 ended. Score 2.000000
episode 713 ended. Score 0.000000
episode 714 ended. Score 0.000000
episode 715 ended. Score 3.000000
2017-12-11 03:06:46.466250

batch 21500

episode 716 ended. Score 0.000000
episode 717 ended. Score 2.000000
episode 718 ended. Score 0.000000
episode 719 ended. Score 0.000000
episode 720 ended. Score 1.000000
episode 721 ended. Score 2.000000
episode 722 ended. Score 0.000000
episode 723 ended. Score 1.000000
episode 724 ended. Score 0.000000
episode 725 ended. Score 0.000000
episode 726 ended. Score 2.000000
episode 727 ended. Score 1.000000
episode 728 ended. Score 0.000000
episode 729 ended. Score 2.000000
episode 730 ended. Score 0.000000
episode 731 ended. Score 0.000000
episode 732 ended. Score 3.000000
2017-12-11 03:13:30.011923

batch 22000

episode 733 ended. Score 1.000000
episode 734 ended. Score 0.000000
episode 735 ended. Score 1.000000
episode 736 ended. Score 1.000000
episode 737 ended. Score 0.000000
episode 738 ended. Score 2.000000
episode 739 ended. Score 1.000000
episode 740 ended. Score 0.000000
episode 741 ended. Score 2.000000
episode 742 ended. Score 0.000000
episode 743 ended. Score 2.000000
episode 744 ended. Score 1.000000
episode 745 ended. Score 0.000000
episode 746 ended. Score 1.000000
episode 747 ended. Score 1.000000
episode 748 ended. Score 4.000000
episode 749 ended. Score 1.000000
2017-12-11 03:20:14.699293

batch 22500

episode 750 ended. Score 1.000000
episode 751 ended. Score 2.000000
episode 752 ended. Score 1.000000
episode 753 ended. Score 2.000000
episode 754 ended. Score 0.000000
episode 755 ended. Score 0.000000
episode 756 ended. Score 2.000000
episode 757 ended. Score 0.000000
episode 758 ended. Score 2.000000
episode 759 ended. Score 2.000000
episode 760 ended. Score 1.000000
episode 761 ended. Score 1.000000
episode 762 ended. Score 3.000000
episode 763 ended. Score 0.000000
episode 764 ended. Score 2.000000
episode 765 ended. Score 1.000000
2017-12-11 03:26:58.509201

batch 23000

episode 766 ended. Score 1.000000
episode 767 ended. Score 1.000000
episode 768 ended. Score 0.000000
episode 769 ended. Score 1.000000
episode 770 ended. Score 0.000000
episode 771 ended. Score 2.000000
episode 772 ended. Score 2.000000
episode 773 ended. Score 0.000000
episode 774 ended. Score 0.000000
episode 775 ended. Score 0.000000
episode 776 ended. Score 1.000000
episode 777 ended. Score 0.000000
episode 778 ended. Score 0.000000
episode 779 ended. Score 0.000000
episode 780 ended. Score 1.000000
episode 781 ended. Score 1.000000
episode 782 ended. Score 1.000000
2017-12-11 03:33:41.907210

batch 23500

episode 783 ended. Score 2.000000
episode 784 ended. Score 1.000000
episode 785 ended. Score 2.000000
episode 786 ended. Score 0.000000
episode 787 ended. Score 2.000000
episode 788 ended. Score 2.000000
episode 789 ended. Score 2.000000
episode 790 ended. Score 1.000000
episode 791 ended. Score 0.000000
episode 792 ended. Score 1.000000
episode 793 ended. Score 3.000000
episode 794 ended. Score 2.000000
episode 795 ended. Score 0.000000
episode 796 ended. Score 1.000000
episode 797 ended. Score 2.000000
episode 798 ended. Score 1.000000
episode 799 ended. Score 0.000000
2017-12-11 03:40:24.988531

batch 24000

saving model to _files/models/my_beacon_model, step 24000
episode 800 ended. Score 2.000000
episode 801 ended. Score 0.000000
episode 802 ended. Score 1.000000
episode 803 ended. Score 0.000000
episode 804 ended. Score 3.000000
episode 805 ended. Score 0.000000
episode 806 ended. Score 0.000000
episode 807 ended. Score 0.000000
episode 808 ended. Score 1.000000
episode 809 ended. Score 0.000000
episode 810 ended. Score 0.000000
episode 811 ended. Score 2.000000
episode 812 ended. Score 0.000000
episode 813 ended. Score 0.000000
episode 814 ended. Score 1.000000
episode 815 ended. Score 0.000000
2017-12-11 03:47:11.926494

batch 24500

episode 816 ended. Score 2.000000
episode 817 ended. Score 1.000000
episode 818 ended. Score 0.000000
episode 819 ended. Score 2.000000
episode 820 ended. Score 0.000000
episode 821 ended. Score 2.000000
episode 822 ended. Score 2.000000
episode 823 ended. Score 1.000000
episode 824 ended. Score 2.000000
episode 825 ended. Score 0.000000
episode 826 ended. Score 0.000000
episode 827 ended. Score 0.000000
episode 828 ended. Score 1.000000
episode 829 ended. Score 1.000000
episode 830 ended. Score 0.000000
episode 831 ended. Score 0.000000
episode 832 ended. Score 0.000000
2017-12-11 03:54:38.449725

batch 25000

episode 833 ended. Score 0.000000
episode 834 ended. Score 3.000000
episode 835 ended. Score 1.000000
episode 836 ended. Score 2.000000
episode 837 ended. Score 0.000000
episode 838 ended. Score 1.000000
episode 839 ended. Score 1.000000
episode 840 ended. Score 0.000000
episode 841 ended. Score 3.000000
episode 842 ended. Score 1.000000
episode 843 ended. Score 1.000000
episode 844 ended. Score 0.000000
episode 845 ended. Score 1.000000
episode 846 ended. Score 0.000000
episode 847 ended. Score 2.000000
episode 848 ended. Score 1.000000
episode 849 ended. Score 0.000000
2017-12-11 04:03:15.597892

batch 25500

episode 850 ended. Score 1.000000
episode 851 ended. Score 0.000000
episode 852 ended. Score 1.000000
episode 853 ended. Score 2.000000
episode 854 ended. Score 2.000000
episode 855 ended. Score 0.000000
episode 856 ended. Score 1.000000
episode 857 ended. Score 0.000000
episode 858 ended. Score 2.000000
episode 859 ended. Score 1.000000
episode 860 ended. Score 1.000000
episode 861 ended. Score 0.000000
episode 862 ended. Score 1.000000
episode 863 ended. Score 1.000000
episode 864 ended. Score 1.000000
episode 865 ended. Score 2.000000
2017-12-11 07:06:41.215039

batch 26000

episode 866 ended. Score 2.000000
episode 867 ended. Score 1.000000
episode 868 ended. Score 1.000000
episode 869 ended. Score 0.000000
episode 870 ended. Score 3.000000
episode 871 ended. Score 0.000000
episode 872 ended. Score 0.000000
episode 873 ended. Score 1.000000
episode 874 ended. Score 3.000000
episode 875 ended. Score 0.000000
episode 876 ended. Score 0.000000
episode 877 ended. Score 2.000000
episode 878 ended. Score 4.000000
episode 879 ended. Score 0.000000
episode 880 ended. Score 4.000000
episode 881 ended. Score 0.000000
episode 882 ended. Score 1.000000
2017-12-11 07:11:33.630323

batch 26500

episode 883 ended. Score 2.000000
episode 884 ended. Score 2.000000
episode 885 ended. Score 3.000000
episode 886 ended. Score 1.000000
episode 887 ended. Score 1.000000
episode 888 ended. Score 1.000000
episode 889 ended. Score 1.000000
episode 890 ended. Score 0.000000
episode 891 ended. Score 1.000000
episode 892 ended. Score 2.000000
episode 893 ended. Score 2.000000
episode 894 ended. Score 1.000000
episode 895 ended. Score 0.000000
episode 896 ended. Score 1.000000
episode 897 ended. Score 0.000000
episode 898 ended. Score 0.000000
episode 899 ended. Score 0.000000
2017-12-11 07:15:59.849538

batch 27000

episode 900 ended. Score 1.000000
episode 901 ended. Score 1.000000
episode 902 ended. Score 4.000000
episode 903 ended. Score 1.000000
episode 904 ended. Score 0.000000
episode 905 ended. Score 1.000000
episode 906 ended. Score 2.000000
episode 907 ended. Score 3.000000
episode 908 ended. Score 1.000000
episode 909 ended. Score 2.000000
episode 910 ended. Score 2.000000
episode 911 ended. Score 1.000000
episode 912 ended. Score 0.000000
episode 913 ended. Score 2.000000
episode 914 ended. Score 2.000000
episode 915 ended. Score 1.000000
2017-12-11 07:20:26.476639

batch 27500

episode 916 ended. Score 1.000000
episode 917 ended. Score 1.000000
episode 918 ended. Score 1.000000
episode 919 ended. Score 2.000000
episode 920 ended. Score 0.000000
episode 921 ended. Score 3.000000
episode 922 ended. Score 0.000000
episode 923 ended. Score 1.000000
episode 924 ended. Score 1.000000
episode 925 ended. Score 0.000000
episode 926 ended. Score 1.000000
episode 927 ended. Score 1.000000
episode 928 ended. Score 0.000000
episode 929 ended. Score 4.000000
episode 930 ended. Score 1.000000
episode 931 ended. Score 0.000000
episode 932 ended. Score 1.000000
2017-12-11 07:24:53.851987

batch 28000

saving model to _files/models/my_beacon_model, step 28000
episode 933 ended. Score 1.000000
episode 934 ended. Score 0.000000
episode 935 ended. Score 1.000000
episode 936 ended. Score 0.000000
episode 937 ended. Score 1.000000
episode 938 ended. Score 1.000000
episode 939 ended. Score 0.000000
episode 940 ended. Score 2.000000
episode 941 ended. Score 2.000000
episode 942 ended. Score 0.000000
episode 943 ended. Score 1.000000
episode 944 ended. Score 0.000000
episode 945 ended. Score 1.000000
episode 946 ended. Score 0.000000
episode 947 ended. Score 1.000000
episode 948 ended. Score 2.000000
episode 949 ended. Score 1.000000
2017-12-11 08:35:08.906809

batch 28500

episode 950 ended. Score 1.000000
episode 951 ended. Score 1.000000
episode 952 ended. Score 3.000000
episode 953 ended. Score 0.000000
episode 954 ended. Score 0.000000
episode 955 ended. Score 1.000000
episode 956 ended. Score 1.000000
episode 957 ended. Score 0.000000
episode 958 ended. Score 0.000000
episode 959 ended. Score 3.000000
episode 960 ended. Score 0.000000
episode 961 ended. Score 3.000000
episode 962 ended. Score 1.000000
episode 963 ended. Score 1.000000
episode 964 ended. Score 1.000000
episode 965 ended. Score 2.000000
2017-12-11 08:41:48.113830

batch 29000

episode 966 ended. Score 2.000000
episode 967 ended. Score 0.000000
episode 968 ended. Score 1.000000
episode 969 ended. Score 1.000000
episode 970 ended. Score 3.000000
episode 971 ended. Score 1.000000
episode 972 ended. Score 2.000000
episode 973 ended. Score 1.000000
episode 974 ended. Score 0.000000
episode 975 ended. Score 0.000000
episode 976 ended. Score 1.000000
episode 977 ended. Score 3.000000
episode 978 ended. Score 0.000000
episode 979 ended. Score 4.000000
episode 980 ended. Score 1.000000
episode 981 ended. Score 1.000000
episode 982 ended. Score 0.000000
2017-12-11 08:48:13.598087

batch 29500

episode 983 ended. Score 2.000000
episode 984 ended. Score 1.000000
episode 985 ended. Score 2.000000
episode 986 ended. Score 0.000000
episode 987 ended. Score 3.000000
episode 988 ended. Score 2.000000
episode 989 ended. Score 0.000000
episode 990 ended. Score 0.000000
episode 991 ended. Score 1.000000
episode 992 ended. Score 1.000000
episode 993 ended. Score 0.000000
episode 994 ended. Score 0.000000
episode 995 ended. Score 2.000000
episode 996 ended. Score 1.000000
episode 997 ended. Score 0.000000
episode 998 ended. Score 1.000000
episode 999 ended. Score 0.000000
2017-12-11 08:54:31.584907

batch 30000

episode 1000 ended. Score 0.000000
episode 1001 ended. Score 1.000000
episode 1002 ended. Score 2.000000
episode 1003 ended. Score 1.000000
episode 1004 ended. Score 2.000000
episode 1005 ended. Score 3.000000
episode 1006 ended. Score 4.000000
episode 1007 ended. Score 2.000000
episode 1008 ended. Score 1.000000
episode 1009 ended. Score 0.000000
episode 1010 ended. Score 2.000000
episode 1011 ended. Score 4.000000
episode 1012 ended. Score 1.000000
episode 1013 ended. Score 2.000000
episode 1014 ended. Score 0.000000
episode 1015 ended. Score 1.000000
2017-12-11 09:00:55.994843

batch 30500

episode 1016 ended. Score 1.000000
episode 1017 ended. Score 0.000000
episode 1018 ended. Score 1.000000
episode 1019 ended. Score 0.000000
episode 1020 ended. Score 0.000000
episode 1021 ended. Score 3.000000
episode 1022 ended. Score 1.000000
episode 1023 ended. Score 3.000000
episode 1024 ended. Score 1.000000
episode 1025 ended. Score 3.000000
episode 1026 ended. Score 3.000000
episode 1027 ended. Score 2.000000
episode 1028 ended. Score 1.000000
episode 1029 ended. Score 0.000000
episode 1030 ended. Score 1.000000
episode 1031 ended. Score 2.000000
episode 1032 ended. Score 1.000000
2017-12-11 09:07:30.480580

batch 31000

episode 1033 ended. Score 1.000000
episode 1034 ended. Score 1.000000
episode 1035 ended. Score 1.000000
episode 1036 ended. Score 0.000000
episode 1037 ended. Score 0.000000
episode 1038 ended. Score 3.000000
episode 1039 ended. Score 3.000000
episode 1040 ended. Score 0.000000
episode 1041 ended. Score 0.000000
episode 1042 ended. Score 1.000000
episode 1043 ended. Score 1.000000
episode 1044 ended. Score 0.000000
episode 1045 ended. Score 2.000000
episode 1046 ended. Score 1.000000
episode 1047 ended. Score 0.000000
episode 1048 ended. Score 3.000000
episode 1049 ended. Score 1.000000
2017-12-11 09:14:14.015406

batch 31500

episode 1050 ended. Score 0.000000
episode 1051 ended. Score 0.000000
episode 1052 ended. Score 1.000000
episode 1053 ended. Score 0.000000
episode 1054 ended. Score 0.000000
episode 1055 ended. Score 3.000000
episode 1056 ended. Score 0.000000
episode 1057 ended. Score 0.000000
episode 1058 ended. Score 1.000000
episode 1059 ended. Score 1.000000
episode 1060 ended. Score 1.000000
episode 1061 ended. Score 4.000000
episode 1062 ended. Score 1.000000
episode 1063 ended. Score 4.000000
episode 1064 ended. Score 1.000000
episode 1065 ended. Score 1.000000
2017-12-11 09:20:57.431567

batch 32000

saving model to _files/models/my_beacon_model, step 32000
episode 1066 ended. Score 2.000000
episode 1067 ended. Score 0.000000
episode 1068 ended. Score 0.000000
episode 1069 ended. Score 1.000000
episode 1070 ended. Score 1.000000
episode 1071 ended. Score 1.000000
episode 1072 ended. Score 2.000000
episode 1073 ended. Score 0.000000
episode 1074 ended. Score 3.000000
episode 1075 ended. Score 0.000000
episode 1076 ended. Score 0.000000
episode 1077 ended. Score 1.000000
episode 1078 ended. Score 0.000000
episode 1079 ended. Score 3.000000
episode 1080 ended. Score 2.000000
episode 1081 ended. Score 0.000000
episode 1082 ended. Score 0.000000
2017-12-11 09:27:49.562850

batch 32500

episode 1083 ended. Score 0.000000
episode 1084 ended. Score 2.000000
episode 1085 ended. Score 0.000000
episode 1086 ended. Score 1.000000
episode 1087 ended. Score 0.000000
episode 1088 ended. Score 1.000000
episode 1089 ended. Score 1.000000
episode 1090 ended. Score 2.000000
episode 1091 ended. Score 1.000000
episode 1092 ended. Score 2.000000
episode 1093 ended. Score 3.000000
episode 1094 ended. Score 0.000000
episode 1095 ended. Score 1.000000
episode 1096 ended. Score 1.000000
episode 1097 ended. Score 0.000000
episode 1098 ended. Score 0.000000
episode 1099 ended. Score 1.000000
2017-12-11 09:35:03.605197

batch 33000

episode 1100 ended. Score 0.000000
episode 1101 ended. Score 3.000000
episode 1102 ended. Score 0.000000
episode 1103 ended. Score 1.000000
episode 1104 ended. Score 1.000000
episode 1105 ended. Score 1.000000
episode 1106 ended. Score 1.000000
episode 1107 ended. Score 1.000000
episode 1108 ended. Score 0.000000
episode 1109 ended. Score 1.000000
episode 1110 ended. Score 1.000000
episode 1111 ended. Score 2.000000
episode 1112 ended. Score 1.000000
episode 1113 ended. Score 1.000000
episode 1114 ended. Score 0.000000
episode 1115 ended. Score 1.000000
2017-12-11 09:41:49.401205

batch 33500

episode 1116 ended. Score 3.000000
episode 1117 ended. Score 2.000000
episode 1118 ended. Score 4.000000
episode 1119 ended. Score 0.000000
episode 1120 ended. Score 0.000000
episode 1121 ended. Score 1.000000
episode 1122 ended. Score 1.000000
episode 1123 ended. Score 2.000000
episode 1124 ended. Score 0.000000
episode 1125 ended. Score 1.000000
episode 1126 ended. Score 1.000000
episode 1127 ended. Score 1.000000
episode 1128 ended. Score 0.000000
episode 1129 ended. Score 1.000000
episode 1130 ended. Score 2.000000
episode 1131 ended. Score 0.000000
episode 1132 ended. Score 2.000000
2017-12-11 09:47:54.555739

batch 34000

episode 1133 ended. Score 3.000000
episode 1134 ended. Score 1.000000
episode 1135 ended. Score 0.000000
episode 1136 ended. Score 0.000000
episode 1137 ended. Score 2.000000
episode 1138 ended. Score 2.000000
episode 1139 ended. Score 1.000000
episode 1140 ended. Score 0.000000
episode 1141 ended. Score 1.000000
episode 1142 ended. Score 2.000000
episode 1143 ended. Score 0.000000
episode 1144 ended. Score 0.000000
episode 1145 ended. Score 2.000000
episode 1146 ended. Score 0.000000
episode 1147 ended. Score 2.000000
episode 1148 ended. Score 1.000000
episode 1149 ended. Score 2.000000
2017-12-11 09:52:42.937184

batch 34500

episode 1150 ended. Score 2.000000
episode 1151 ended. Score 3.000000
episode 1152 ended. Score 3.000000
episode 1153 ended. Score 4.000000
episode 1154 ended. Score 4.000000
episode 1155 ended. Score 0.000000
episode 1156 ended. Score 2.000000
episode 1157 ended. Score 1.000000
episode 1158 ended. Score 0.000000
episode 1159 ended. Score 2.000000
episode 1160 ended. Score 2.000000
episode 1161 ended. Score 2.000000
episode 1162 ended. Score 2.000000
episode 1163 ended. Score 0.000000
episode 1164 ended. Score 0.000000
episode 1165 ended. Score 1.000000
2017-12-11 09:57:29.448653

batch 35000

episode 1166 ended. Score 3.000000
episode 1167 ended. Score 0.000000
episode 1168 ended. Score 3.000000
episode 1169 ended. Score 2.000000
episode 1170 ended. Score 2.000000
episode 1171 ended. Score 0.000000
episode 1172 ended. Score 1.000000
episode 1173 ended. Score 2.000000
episode 1174 ended. Score 3.000000
episode 1175 ended. Score 0.000000
episode 1176 ended. Score 1.000000
episode 1177 ended. Score 3.000000
episode 1178 ended. Score 1.000000
episode 1179 ended. Score 0.000000
episode 1180 ended. Score 2.000000
episode 1181 ended. Score 0.000000
episode 1182 ended. Score 1.000000
2017-12-11 10:02:16.368836

batch 35500

episode 1183 ended. Score 1.000000
episode 1184 ended. Score 1.000000
episode 1185 ended. Score 1.000000
episode 1186 ended. Score 5.000000
episode 1187 ended. Score 2.000000
episode 1188 ended. Score 1.000000
episode 1189 ended. Score 3.000000
episode 1190 ended. Score 0.000000
episode 1191 ended. Score 0.000000
episode 1192 ended. Score 3.000000
episode 1193 ended. Score 0.000000
episode 1194 ended. Score 1.000000
episode 1195 ended. Score 1.000000
episode 1196 ended. Score 0.000000
episode 1197 ended. Score 2.000000
episode 1198 ended. Score 0.000000
episode 1199 ended. Score 0.000000
2017-12-11 10:07:03.138362

batch 36000

saving model to _files/models/my_beacon_model, step 36000
episode 1200 ended. Score 1.000000
episode 1201 ended. Score 0.000000
episode 1202 ended. Score 0.000000
episode 1203 ended. Score 2.000000
episode 1204 ended. Score 0.000000
episode 1205 ended. Score 2.000000
episode 1206 ended. Score 3.000000
episode 1207 ended. Score 0.000000
episode 1208 ended. Score 0.000000
episode 1209 ended. Score 1.000000
episode 1210 ended. Score 5.000000
episode 1211 ended. Score 0.000000
episode 1212 ended. Score 1.000000
episode 1213 ended. Score 2.000000
episode 1214 ended. Score 4.000000
episode 1215 ended. Score 2.000000
2017-12-11 10:11:45.686354

batch 36500

episode 1216 ended. Score 2.000000
episode 1217 ended. Score 1.000000
episode 1218 ended. Score 1.000000
episode 1219 ended. Score 2.000000
episode 1220 ended. Score 1.000000
episode 1221 ended. Score 1.000000
episode 1222 ended. Score 0.000000
episode 1223 ended. Score 0.000000
episode 1224 ended. Score 2.000000
episode 1225 ended. Score 0.000000
episode 1226 ended. Score 1.000000
episode 1227 ended. Score 1.000000
episode 1228 ended. Score 1.000000
episode 1229 ended. Score 0.000000
episode 1230 ended. Score 0.000000
episode 1231 ended. Score 0.000000
episode 1232 ended. Score 3.000000
2017-12-11 10:16:32.931718

batch 37000

episode 1233 ended. Score 2.000000
episode 1234 ended. Score 1.000000
episode 1235 ended. Score 1.000000
episode 1236 ended. Score 0.000000
episode 1237 ended. Score 1.000000
episode 1238 ended. Score 5.000000
episode 1239 ended. Score 0.000000
episode 1240 ended. Score 0.000000
episode 1241 ended. Score 2.000000
episode 1242 ended. Score 2.000000
episode 1243 ended. Score 1.000000
episode 1244 ended. Score 1.000000
episode 1245 ended. Score 0.000000
episode 1246 ended. Score 0.000000
episode 1247 ended. Score 0.000000
episode 1248 ended. Score 1.000000
episode 1249 ended. Score 0.000000
2017-12-11 10:22:11.837050

batch 37500

episode 1250 ended. Score 1.000000
episode 1251 ended. Score 0.000000
episode 1252 ended. Score 1.000000
episode 1253 ended. Score 0.000000
episode 1254 ended. Score 1.000000
episode 1255 ended. Score 1.000000
episode 1256 ended. Score 0.000000
episode 1257 ended. Score 3.000000
episode 1258 ended. Score 0.000000
episode 1259 ended. Score 1.000000
episode 1260 ended. Score 0.000000
episode 1261 ended. Score 2.000000
episode 1262 ended. Score 0.000000
episode 1263 ended. Score 2.000000
episode 1264 ended. Score 1.000000
episode 1265 ended. Score 0.000000
2017-12-11 10:28:59.191740

batch 38000

episode 1266 ended. Score 2.000000
episode 1267 ended. Score 1.000000
episode 1268 ended. Score 0.000000
episode 1269 ended. Score 0.000000
episode 1270 ended. Score 0.000000
episode 1271 ended. Score 1.000000
episode 1272 ended. Score 2.000000
episode 1273 ended. Score 0.000000
episode 1274 ended. Score 1.000000
episode 1275 ended. Score 1.000000
episode 1276 ended. Score 1.000000
episode 1277 ended. Score 0.000000
episode 1278 ended. Score 1.000000
episode 1279 ended. Score 1.000000
episode 1280 ended. Score 1.000000
episode 1281 ended. Score 3.000000
episode 1282 ended. Score 1.000000
2017-12-11 10:35:46.531160

batch 38500

episode 1283 ended. Score 1.000000
episode 1284 ended. Score 3.000000
episode 1285 ended. Score 1.000000
episode 1286 ended. Score 1.000000
episode 1287 ended. Score 0.000000
episode 1288 ended. Score 0.000000
episode 1289 ended. Score 2.000000
episode 1290 ended. Score 1.000000
episode 1291 ended. Score 0.000000
episode 1292 ended. Score 4.000000
episode 1293 ended. Score 1.000000
episode 1294 ended. Score 0.000000
episode 1295 ended. Score 0.000000
episode 1296 ended. Score 1.000000
episode 1297 ended. Score 2.000000
episode 1298 ended. Score 2.000000
episode 1299 ended. Score 0.000000
2017-12-11 10:42:33.006855

batch 39000

episode 1300 ended. Score 1.000000
episode 1301 ended. Score 1.000000
episode 1302 ended. Score 1.000000
episode 1303 ended. Score 1.000000
episode 1304 ended. Score 1.000000
episode 1305 ended. Score 3.000000
episode 1306 ended. Score 1.000000
episode 1307 ended. Score 2.000000
episode 1308 ended. Score 2.000000
episode 1309 ended. Score 0.000000
episode 1310 ended. Score 0.000000
episode 1311 ended. Score 4.000000
episode 1312 ended. Score 3.000000
episode 1313 ended. Score 1.000000
episode 1314 ended. Score 1.000000
episode 1315 ended. Score 2.000000
2017-12-11 10:49:18.150385

batch 39500

episode 1316 ended. Score 1.000000
episode 1317 ended. Score 1.000000
episode 1318 ended. Score 1.000000
episode 1319 ended. Score 4.000000
episode 1320 ended. Score 1.000000
episode 1321 ended. Score 2.000000
episode 1322 ended. Score 1.000000
episode 1323 ended. Score 1.000000
episode 1324 ended. Score 1.000000
episode 1325 ended. Score 0.000000
episode 1326 ended. Score 0.000000
episode 1327 ended. Score 2.000000
episode 1328 ended. Score 0.000000
episode 1329 ended. Score 4.000000
episode 1330 ended. Score 0.000000
episode 1331 ended. Score 0.000000
episode 1332 ended. Score 1.000000
2017-12-11 10:55:25.040698

batch 40000

saving model to _files/models/my_beacon_model, step 40000
episode 1333 ended. Score 2.000000
episode 1334 ended. Score 1.000000
episode 1335 ended. Score 2.000000
episode 1336 ended. Score 2.000000
episode 1337 ended. Score 3.000000
episode 1338 ended. Score 1.000000
episode 1339 ended. Score 0.000000
episode 1340 ended. Score 3.000000
episode 1341 ended. Score 2.000000
episode 1342 ended. Score 0.000000
episode 1343 ended. Score 1.000000
episode 1344 ended. Score 1.000000
episode 1345 ended. Score 2.000000
episode 1346 ended. Score 0.000000
episode 1347 ended. Score 1.000000
episode 1348 ended. Score 0.000000
episode 1349 ended. Score 0.000000
2017-12-11 11:00:17.405407

batch 40500

episode 1350 ended. Score 3.000000
episode 1351 ended. Score 2.000000
episode 1352 ended. Score 2.000000
episode 1353 ended. Score 5.000000
episode 1354 ended. Score 2.000000
episode 1355 ended. Score 2.000000
episode 1356 ended. Score 1.000000
episode 1357 ended. Score 1.000000
episode 1358 ended. Score 2.000000
episode 1359 ended. Score 0.000000
episode 1360 ended. Score 1.000000
episode 1361 ended. Score 3.000000
episode 1362 ended. Score 3.000000
episode 1363 ended. Score 2.000000
episode 1364 ended. Score 0.000000
episode 1365 ended. Score 5.000000
2017-12-11 11:06:45.348664

batch 41000

episode 1366 ended. Score 0.000000
episode 1367 ended. Score 0.000000
episode 1368 ended. Score 1.000000
episode 1369 ended. Score 2.000000
episode 1370 ended. Score 3.000000
episode 1371 ended. Score 1.000000
episode 1372 ended. Score 2.000000
episode 1373 ended. Score 0.000000
episode 1374 ended. Score 2.000000
episode 1375 ended. Score 1.000000
episode 1376 ended. Score 2.000000
episode 1377 ended. Score 1.000000
episode 1378 ended. Score 1.000000
episode 1379 ended. Score 1.000000
episode 1380 ended. Score 0.000000
episode 1381 ended. Score 2.000000
episode 1382 ended. Score 3.000000
2017-12-11 11:13:21.287452

batch 41500

episode 1383 ended. Score 3.000000
episode 1384 ended. Score 1.000000
episode 1385 ended. Score 1.000000
episode 1386 ended. Score 2.000000
episode 1387 ended. Score 1.000000
episode 1388 ended. Score 1.000000
episode 1389 ended. Score 0.000000
episode 1390 ended. Score 1.000000
episode 1391 ended. Score 1.000000
episode 1392 ended. Score 0.000000
episode 1393 ended. Score 2.000000
episode 1394 ended. Score 2.000000
episode 1395 ended. Score 1.000000
episode 1396 ended. Score 0.000000
episode 1397 ended. Score 0.000000
episode 1398 ended. Score 2.000000
episode 1399 ended. Score 2.000000
2017-12-11 11:18:01.175829

batch 42000

episode 1400 ended. Score 0.000000
episode 1401 ended. Score 2.000000
episode 1402 ended. Score 0.000000
episode 1403 ended. Score 0.000000
episode 1404 ended. Score 0.000000
episode 1405 ended. Score 0.000000
episode 1406 ended. Score 1.000000
episode 1407 ended. Score 1.000000
episode 1408 ended. Score 0.000000
episode 1409 ended. Score 1.000000
episode 1410 ended. Score 1.000000
episode 1411 ended. Score 2.000000
episode 1412 ended. Score 1.000000
episode 1413 ended. Score 1.000000
episode 1414 ended. Score 0.000000
episode 1415 ended. Score 2.000000
2017-12-11 11:22:24.634603

batch 42500

episode 1416 ended. Score 0.000000
episode 1417 ended. Score 1.000000
episode 1418 ended. Score 0.000000
episode 1419 ended. Score 0.000000
episode 1420 ended. Score 0.000000
episode 1421 ended. Score 3.000000
episode 1422 ended. Score 2.000000
episode 1423 ended. Score 0.000000
episode 1424 ended. Score 3.000000
episode 1425 ended. Score 1.000000
episode 1426 ended. Score 0.000000
episode 1427 ended. Score 2.000000
episode 1428 ended. Score 2.000000
episode 1429 ended. Score 1.000000
episode 1430 ended. Score 2.000000
episode 1431 ended. Score 1.000000
episode 1432 ended. Score 2.000000
2017-12-11 11:26:47.714639

batch 43000

episode 1433 ended. Score 2.000000
episode 1434 ended. Score 2.000000
episode 1435 ended. Score 2.000000
episode 1436 ended. Score 0.000000
episode 1437 ended. Score 1.000000
episode 1438 ended. Score 3.000000
episode 1439 ended. Score 1.000000
episode 1440 ended. Score 2.000000
episode 1441 ended. Score 3.000000
episode 1442 ended. Score 1.000000
episode 1443 ended. Score 1.000000
episode 1444 ended. Score 0.000000
episode 1445 ended. Score 3.000000
episode 1446 ended. Score 0.000000
episode 1447 ended. Score 0.000000
episode 1448 ended. Score 1.000000
episode 1449 ended. Score 4.000000
2017-12-11 11:31:15.495303

batch 43500

episode 1450 ended. Score 2.000000
episode 1451 ended. Score 2.000000
episode 1452 ended. Score 1.000000
episode 1453 ended. Score 0.000000
episode 1454 ended. Score 1.000000
episode 1455 ended. Score 0.000000
episode 1456 ended. Score 0.000000
episode 1457 ended. Score 2.000000
episode 1458 ended. Score 0.000000
episode 1459 ended. Score 1.000000
episode 1460 ended. Score 1.000000
episode 1461 ended. Score 3.000000
episode 1462 ended. Score 2.000000
episode 1463 ended. Score 2.000000
episode 1464 ended. Score 2.000000
episode 1465 ended. Score 2.000000
2017-12-11 11:36:30.148549

batch 44000

saving model to _files/models/my_beacon_model, step 44000
episode 1466 ended. Score 3.000000
episode 1467 ended. Score 1.000000
episode 1468 ended. Score 0.000000
episode 1469 ended. Score 2.000000
episode 1470 ended. Score 1.000000
episode 1471 ended. Score 2.000000
episode 1472 ended. Score 0.000000
episode 1473 ended. Score 0.000000
episode 1474 ended. Score 1.000000
episode 1475 ended. Score 0.000000
episode 1476 ended. Score 0.000000
episode 1477 ended. Score 1.000000
episode 1478 ended. Score 1.000000
episode 1479 ended. Score 1.000000
episode 1480 ended. Score 0.000000
episode 1481 ended. Score 1.000000
episode 1482 ended. Score 1.000000
2017-12-11 11:43:23.140227

batch 44500

episode 1483 ended. Score 1.000000
episode 1484 ended. Score 0.000000
episode 1485 ended. Score 0.000000
episode 1486 ended. Score 0.000000
episode 1487 ended. Score 0.000000
episode 1488 ended. Score 2.000000
episode 1489 ended. Score 2.000000
episode 1490 ended. Score 3.000000
episode 1491 ended. Score 2.000000
episode 1492 ended. Score 2.000000
episode 1493 ended. Score 1.000000
episode 1494 ended. Score 0.000000
episode 1495 ended. Score 1.000000
episode 1496 ended. Score 0.000000
episode 1497 ended. Score 2.000000
episode 1498 ended. Score 0.000000
episode 1499 ended. Score 0.000000
2017-12-11 11:50:13.667521

batch 45000

episode 1500 ended. Score 2.000000
episode 1501 ended. Score 1.000000
episode 1502 ended. Score 0.000000
episode 1503 ended. Score 1.000000
episode 1504 ended. Score 2.000000
episode 1505 ended. Score 2.000000
episode 1506 ended. Score 0.000000
episode 1507 ended. Score 1.000000
episode 1508 ended. Score 0.000000
episode 1509 ended. Score 1.000000
episode 1510 ended. Score 2.000000
episode 1511 ended. Score 3.000000
episode 1512 ended. Score 1.000000
episode 1513 ended. Score 3.000000
episode 1514 ended. Score 2.000000
episode 1515 ended. Score 1.000000
2017-12-11 11:57:11.390003

batch 45500

episode 1516 ended. Score 0.000000
episode 1517 ended. Score 5.000000
episode 1518 ended. Score 2.000000
episode 1519 ended. Score 0.000000
episode 1520 ended. Score 1.000000
episode 1521 ended. Score 0.000000
episode 1522 ended. Score 2.000000
episode 1523 ended. Score 0.000000
episode 1524 ended. Score 3.000000
episode 1525 ended. Score 1.000000
episode 1526 ended. Score 2.000000
episode 1527 ended. Score 1.000000
episode 1528 ended. Score 1.000000
episode 1529 ended. Score 0.000000
episode 1530 ended. Score 3.000000
episode 1531 ended. Score 2.000000
episode 1532 ended. Score 0.000000
2017-12-11 12:03:59.481848

batch 46000

episode 1533 ended. Score 2.000000
episode 1534 ended. Score 0.000000
episode 1535 ended. Score 1.000000
episode 1536 ended. Score 1.000000
episode 1537 ended. Score 1.000000
episode 1538 ended. Score 2.000000
episode 1539 ended. Score 0.000000
episode 1540 ended. Score 2.000000
episode 1541 ended. Score 3.000000
episode 1542 ended. Score 2.000000
episode 1543 ended. Score 2.000000
episode 1544 ended. Score 0.000000
episode 1545 ended. Score 1.000000
episode 1546 ended. Score 1.000000
episode 1547 ended. Score 1.000000
episode 1548 ended. Score 1.000000
episode 1549 ended. Score 1.000000
2017-12-11 12:10:48.087831

batch 46500

episode 1550 ended. Score 0.000000
episode 1551 ended. Score 1.000000
episode 1552 ended. Score 3.000000
episode 1553 ended. Score 2.000000
episode 1554 ended. Score 0.000000
episode 1555 ended. Score 1.000000
episode 1556 ended. Score 0.000000
episode 1557 ended. Score 1.000000
episode 1558 ended. Score 1.000000
episode 1559 ended. Score 1.000000
episode 1560 ended. Score 2.000000
episode 1561 ended. Score 1.000000
episode 1562 ended. Score 1.000000
episode 1563 ended. Score 0.000000
episode 1564 ended. Score 1.000000
episode 1565 ended. Score 4.000000
2017-12-11 12:17:36.615626

batch 47000

episode 1566 ended. Score 0.000000
episode 1567 ended. Score 0.000000
episode 1568 ended. Score 0.000000
episode 1569 ended. Score 0.000000
episode 1570 ended. Score 2.000000
episode 1571 ended. Score 2.000000
episode 1572 ended. Score 1.000000
episode 1573 ended. Score 4.000000
episode 1574 ended. Score 0.000000
episode 1575 ended. Score 2.000000
episode 1576 ended. Score 0.000000
episode 1577 ended. Score 2.000000
episode 1578 ended. Score 1.000000
episode 1579 ended. Score 0.000000
episode 1580 ended. Score 1.000000
episode 1581 ended. Score 1.000000
episode 1582 ended. Score 0.000000
2017-12-11 12:24:25.035177

batch 47500

episode 1583 ended. Score 1.000000
episode 1584 ended. Score 0.000000
episode 1585 ended. Score 1.000000
episode 1586 ended. Score 3.000000
episode 1587 ended. Score 0.000000
episode 1588 ended. Score 1.000000
episode 1589 ended. Score 1.000000
episode 1590 ended. Score 1.000000
episode 1591 ended. Score 2.000000
episode 1592 ended. Score 0.000000
episode 1593 ended. Score 0.000000
episode 1594 ended. Score 3.000000
episode 1595 ended. Score 0.000000
episode 1596 ended. Score 1.000000
episode 1597 ended. Score 0.000000
episode 1598 ended. Score 0.000000
episode 1599 ended. Score 0.000000
2017-12-11 12:31:12.756250

batch 48000

saving model to _files/models/my_beacon_model, step 48000
episode 1600 ended. Score 0.000000
episode 1601 ended. Score 1.000000
episode 1602 ended. Score 0.000000
episode 1603 ended. Score 0.000000
episode 1604 ended. Score 0.000000
episode 1605 ended. Score 1.000000
episode 1606 ended. Score 0.000000
episode 1607 ended. Score 0.000000
episode 1608 ended. Score 2.000000
episode 1609 ended. Score 2.000000
episode 1610 ended. Score 1.000000
episode 1611 ended. Score 0.000000
episode 1612 ended. Score 1.000000
episode 1613 ended. Score 1.000000
episode 1614 ended. Score 0.000000
episode 1615 ended. Score 2.000000
2017-12-11 12:37:56.283383

batch 48500

episode 1616 ended. Score 2.000000
episode 1617 ended. Score 0.000000
episode 1618 ended. Score 4.000000
episode 1619 ended. Score 4.000000
episode 1620 ended. Score 0.000000
episode 1621 ended. Score 1.000000
episode 1622 ended. Score 1.000000
episode 1623 ended. Score 0.000000
episode 1624 ended. Score 2.000000
episode 1625 ended. Score 1.000000
episode 1626 ended. Score 4.000000
episode 1627 ended. Score 2.000000
episode 1628 ended. Score 0.000000
episode 1629 ended. Score 0.000000
episode 1630 ended. Score 2.000000
episode 1631 ended. Score 2.000000
episode 1632 ended. Score 1.000000
2017-12-11 12:44:40.894095

batch 49000

episode 1633 ended. Score 0.000000
episode 1634 ended. Score 3.000000
episode 1635 ended. Score 0.000000
episode 1636 ended. Score 2.000000
episode 1637 ended. Score 1.000000
episode 1638 ended. Score 1.000000
episode 1639 ended. Score 2.000000
episode 1640 ended. Score 0.000000
episode 1641 ended. Score 3.000000
episode 1642 ended. Score 1.000000
episode 1643 ended. Score 2.000000
episode 1644 ended. Score 0.000000
episode 1645 ended. Score 3.000000
episode 1646 ended. Score 0.000000
episode 1647 ended. Score 2.000000
episode 1648 ended. Score 1.000000
episode 1649 ended. Score 0.000000
2017-12-11 12:51:32.382602

batch 49500

episode 1650 ended. Score 0.000000
episode 1651 ended. Score 0.000000
episode 1652 ended. Score 1.000000
episode 1653 ended. Score 2.000000
episode 1654 ended. Score 1.000000
episode 1655 ended. Score 0.000000
episode 1656 ended. Score 1.000000
episode 1657 ended. Score 0.000000
episode 1658 ended. Score 1.000000
episode 1659 ended. Score 1.000000
episode 1660 ended. Score 1.000000
episode 1661 ended. Score 3.000000
episode 1662 ended. Score 3.000000
episode 1663 ended. Score 0.000000
episode 1664 ended. Score 0.000000
episode 1665 ended. Score 0.000000
2017-12-11 12:56:34.390393

batch 50000

episode 1666 ended. Score 1.000000
episode 1667 ended. Score 2.000000
episode 1668 ended. Score 1.000000
episode 1669 ended. Score 5.000000
episode 1670 ended. Score 3.000000
episode 1671 ended. Score 0.000000
episode 1672 ended. Score 2.000000
episode 1673 ended. Score 1.000000
episode 1674 ended. Score 0.000000
episode 1675 ended. Score 1.000000
episode 1676 ended. Score 0.000000
episode 1677 ended. Score 1.000000
episode 1678 ended. Score 0.000000
episode 1679 ended. Score 0.000000
episode 1680 ended. Score 3.000000
episode 1681 ended. Score 0.000000
episode 1682 ended. Score 1.000000
2017-12-11 13:01:00.450463

batch 50500

episode 1683 ended. Score 1.000000
episode 1684 ended. Score 2.000000
episode 1685 ended. Score 3.000000
episode 1686 ended. Score 2.000000
episode 1687 ended. Score 1.000000
episode 1688 ended. Score 0.000000
episode 1689 ended. Score 1.000000
episode 1690 ended. Score 1.000000
episode 1691 ended. Score 2.000000
episode 1692 ended. Score 1.000000
episode 1693 ended. Score 0.000000
episode 1694 ended. Score 1.000000
episode 1695 ended. Score 0.000000
episode 1696 ended. Score 0.000000
episode 1697 ended. Score 1.000000
episode 1698 ended. Score 0.000000
episode 1699 ended. Score 1.000000
2017-12-11 13:05:27.318491

batch 51000

episode 1700 ended. Score 2.000000
episode 1701 ended. Score 1.000000
episode 1702 ended. Score 1.000000
episode 1703 ended. Score 3.000000
episode 1704 ended. Score 1.000000
episode 1705 ended. Score 1.000000
episode 1706 ended. Score 0.000000
episode 1707 ended. Score 5.000000
episode 1708 ended. Score 2.000000
episode 1709 ended. Score 1.000000
episode 1710 ended. Score 1.000000
episode 1711 ended. Score 2.000000
episode 1712 ended. Score 1.000000
episode 1713 ended. Score 0.000000
episode 1714 ended. Score 1.000000
episode 1715 ended. Score 3.000000
2017-12-11 13:09:53.326746

batch 51500

episode 1716 ended. Score 2.000000
episode 1717 ended. Score 1.000000
episode 1718 ended. Score 1.000000
episode 1719 ended. Score 3.000000
episode 1720 ended. Score 1.000000
episode 1721 ended. Score 1.000000
episode 1722 ended. Score 1.000000
episode 1723 ended. Score 0.000000
episode 1724 ended. Score 0.000000
episode 1725 ended. Score 1.000000
episode 1726 ended. Score 2.000000
episode 1727 ended. Score 1.000000
episode 1728 ended. Score 4.000000
episode 1729 ended. Score 0.000000
episode 1730 ended. Score 0.000000
episode 1731 ended. Score 1.000000
episode 1732 ended. Score 2.000000
2017-12-11 13:14:46.858930

batch 52000

saving model to _files/models/my_beacon_model, step 52000
episode 1733 ended. Score 0.000000
episode 1734 ended. Score 1.000000
episode 1735 ended. Score 2.000000
episode 1736 ended. Score 1.000000
episode 1737 ended. Score 0.000000
episode 1738 ended. Score 0.000000
episode 1739 ended. Score 1.000000
episode 1740 ended. Score 1.000000
episode 1741 ended. Score 1.000000
episode 1742 ended. Score 2.000000
episode 1743 ended. Score 1.000000
episode 1744 ended. Score 0.000000
episode 1745 ended. Score 1.000000
episode 1746 ended. Score 2.000000
episode 1747 ended. Score 1.000000
episode 1748 ended. Score 0.000000
episode 1749 ended. Score 1.000000
2017-12-11 13:21:23.493233

batch 52500

episode 1750 ended. Score 1.000000
episode 1751 ended. Score 5.000000
episode 1752 ended. Score 2.000000
episode 1753 ended. Score 1.000000
episode 1754 ended. Score 1.000000
episode 1755 ended. Score 0.000000
episode 1756 ended. Score 2.000000
episode 1757 ended. Score 0.000000
episode 1758 ended. Score 3.000000
episode 1759 ended. Score 1.000000
episode 1760 ended. Score 2.000000
episode 1761 ended. Score 1.000000
episode 1762 ended. Score 0.000000
episode 1763 ended. Score 2.000000
episode 1764 ended. Score 0.000000
episode 1765 ended. Score 2.000000
2017-12-11 13:28:10.196112

batch 53000

episode 1766 ended. Score 1.000000
episode 1767 ended. Score 3.000000
episode 1768 ended. Score 0.000000
episode 1769 ended. Score 0.000000
episode 1770 ended. Score 0.000000
episode 1771 ended. Score 2.000000
episode 1772 ended. Score 2.000000
episode 1773 ended. Score 2.000000
episode 1774 ended. Score 0.000000
episode 1775 ended. Score 2.000000
episode 1776 ended. Score 0.000000
episode 1777 ended. Score 1.000000
episode 1778 ended. Score 1.000000
episode 1779 ended. Score 0.000000
episode 1780 ended. Score 1.000000
episode 1781 ended. Score 1.000000
episode 1782 ended. Score 2.000000
2017-12-11 13:35:17.061333

batch 53500

episode 1783 ended. Score 1.000000
episode 1784 ended. Score 0.000000
episode 1785 ended. Score 1.000000
episode 1786 ended. Score 2.000000
episode 1787 ended. Score 2.000000
episode 1788 ended. Score 2.000000
episode 1789 ended. Score 1.000000
episode 1790 ended. Score 1.000000
episode 1791 ended. Score 0.000000
episode 1792 ended. Score 2.000000
episode 1793 ended. Score 1.000000
episode 1794 ended. Score 3.000000
episode 1795 ended. Score 1.000000
episode 1796 ended. Score 2.000000
episode 1797 ended. Score 1.000000
episode 1798 ended. Score 2.000000
episode 1799 ended. Score 1.000000
2017-12-11 13:42:05.667778

batch 54000

episode 1800 ended. Score 0.000000
episode 1801 ended. Score 1.000000
episode 1802 ended. Score 1.000000
episode 1803 ended. Score 0.000000
episode 1804 ended. Score 3.000000
episode 1805 ended. Score 0.000000
episode 1806 ended. Score 1.000000
episode 1807 ended. Score 0.000000
episode 1808 ended. Score 1.000000
episode 1809 ended. Score 2.000000
episode 1810 ended. Score 0.000000
episode 1811 ended. Score 2.000000
episode 1812 ended. Score 1.000000
episode 1813 ended. Score 0.000000
episode 1814 ended. Score 0.000000
episode 1815 ended. Score 0.000000
2017-12-11 13:49:10.736920

batch 54500

episode 1816 ended. Score 2.000000
episode 1817 ended. Score 1.000000
episode 1818 ended. Score 3.000000
episode 1819 ended. Score 0.000000
episode 1820 ended. Score 1.000000
episode 1821 ended. Score 1.000000
episode 1822 ended. Score 1.000000
episode 1823 ended. Score 1.000000
episode 1824 ended. Score 0.000000
episode 1825 ended. Score 0.000000
episode 1826 ended. Score 2.000000
episode 1827 ended. Score 0.000000
episode 1828 ended. Score 0.000000
episode 1829 ended. Score 0.000000
episode 1830 ended. Score 0.000000
episode 1831 ended. Score 0.000000
episode 1832 ended. Score 0.000000
2017-12-11 13:56:03.100812

batch 55000

episode 1833 ended. Score 1.000000
episode 1834 ended. Score 1.000000
episode 1835 ended. Score 1.000000
episode 1836 ended. Score 0.000000
episode 1837 ended. Score 0.000000
episode 1838 ended. Score 0.000000
episode 1839 ended. Score 3.000000
episode 1840 ended. Score 1.000000
episode 1841 ended. Score 0.000000
episode 1842 ended. Score 1.000000
episode 1843 ended. Score 2.000000
episode 1844 ended. Score 0.000000
episode 1845 ended. Score 0.000000
episode 1846 ended. Score 1.000000
episode 1847 ended. Score 2.000000
episode 1848 ended. Score 0.000000
episode 1849 ended. Score 0.000000
2017-12-11 14:02:49.476358

batch 55500

episode 1850 ended. Score 0.000000
episode 1851 ended. Score 0.000000
episode 1852 ended. Score 1.000000
episode 1853 ended. Score 0.000000
episode 1854 ended. Score 2.000000
episode 1855 ended. Score 1.000000
episode 1856 ended. Score 2.000000
episode 1857 ended. Score 3.000000
episode 1858 ended. Score 2.000000
episode 1859 ended. Score 2.000000
episode 1860 ended. Score 3.000000
episode 1861 ended. Score 0.000000
episode 1862 ended. Score 3.000000
episode 1863 ended. Score 0.000000
episode 1864 ended. Score 1.000000
episode 1865 ended. Score 1.000000
2017-12-11 14:10:15.041116

batch 56000

saving model to _files/models/my_beacon_model, step 56000
episode 1866 ended. Score 1.000000
episode 1867 ended. Score 0.000000
episode 1868 ended. Score 1.000000
episode 1869 ended. Score 1.000000
episode 1870 ended. Score 1.000000
episode 1871 ended. Score 0.000000
episode 1872 ended. Score 2.000000
episode 1873 ended. Score 1.000000
episode 1874 ended. Score 2.000000
episode 1875 ended. Score 2.000000
episode 1876 ended. Score 0.000000
episode 1877 ended. Score 4.000000
episode 1878 ended. Score 1.000000
episode 1879 ended. Score 1.000000
episode 1880 ended. Score 1.000000
episode 1881 ended. Score 1.000000
episode 1882 ended. Score 0.000000
2017-12-11 14:17:13.362974

batch 56500

episode 1883 ended. Score 0.000000
episode 1884 ended. Score 2.000000
episode 1885 ended. Score 1.000000
episode 1886 ended. Score 0.000000
episode 1887 ended. Score 2.000000
episode 1888 ended. Score 0.000000
episode 1889 ended. Score 0.000000
episode 1890 ended. Score 4.000000
episode 1891 ended. Score 1.000000
episode 1892 ended. Score 1.000000
episode 1893 ended. Score 1.000000
episode 1894 ended. Score 1.000000
episode 1895 ended. Score 0.000000
episode 1896 ended. Score 0.000000
episode 1897 ended. Score 1.000000
episode 1898 ended. Score 1.000000
episode 1899 ended. Score 1.000000
2017-12-11 14:24:17.444634

batch 57000

episode 1900 ended. Score 0.000000
episode 1901 ended. Score 1.000000
episode 1902 ended. Score 2.000000
episode 1903 ended. Score 1.000000
episode 1904 ended. Score 1.000000
episode 1905 ended. Score 0.000000
episode 1906 ended. Score 0.000000
episode 1907 ended. Score 0.000000
episode 1908 ended. Score 2.000000
episode 1909 ended. Score 0.000000
episode 1910 ended. Score 1.000000
episode 1911 ended. Score 2.000000
episode 1912 ended. Score 2.000000
episode 1913 ended. Score 1.000000
episode 1914 ended. Score 2.000000
episode 1915 ended. Score 2.000000
2017-12-11 14:31:02.201421

batch 57500

episode 1916 ended. Score 1.000000
episode 1917 ended. Score 1.000000
episode 1918 ended. Score 1.000000
episode 1919 ended. Score 1.000000
episode 1920 ended. Score 2.000000
episode 1921 ended. Score 1.000000
episode 1922 ended. Score 2.000000
episode 1923 ended. Score 2.000000
episode 1924 ended. Score 3.000000
episode 1925 ended. Score 2.000000
episode 1926 ended. Score 4.000000
episode 1927 ended. Score 2.000000
episode 1928 ended. Score 3.000000
episode 1929 ended. Score 1.000000
episode 1930 ended. Score 0.000000
episode 1931 ended. Score 0.000000
episode 1932 ended. Score 1.000000
2017-12-11 14:37:59.421694

batch 58000

episode 1933 ended. Score 1.000000
episode 1934 ended. Score 1.000000
episode 1935 ended. Score 1.000000
episode 1936 ended. Score 2.000000
episode 1937 ended. Score 0.000000
episode 1938 ended. Score 1.000000
episode 1939 ended. Score 0.000000
episode 1940 ended. Score 1.000000
episode 1941 ended. Score 0.000000
episode 1942 ended. Score 1.000000
episode 1943 ended. Score 2.000000
episode 1944 ended. Score 0.000000
episode 1945 ended. Score 1.000000
episode 1946 ended. Score 3.000000
episode 1947 ended. Score 1.000000
episode 1948 ended. Score 1.000000
episode 1949 ended. Score 3.000000
2017-12-11 14:44:52.837262

batch 58500

episode 1950 ended. Score 0.000000
episode 1951 ended. Score 2.000000
episode 1952 ended. Score 3.000000
episode 1953 ended. Score 2.000000
episode 1954 ended. Score 0.000000
episode 1955 ended. Score 1.000000
episode 1956 ended. Score 3.000000
episode 1957 ended. Score 1.000000
episode 1958 ended. Score 1.000000
episode 1959 ended. Score 2.000000
episode 1960 ended. Score 3.000000
episode 1961 ended. Score 0.000000
episode 1962 ended. Score 1.000000
episode 1963 ended. Score 1.000000
episode 1964 ended. Score 2.000000
episode 1965 ended. Score 1.000000
2017-12-11 14:51:43.079207

batch 59000

episode 1966 ended. Score 1.000000
episode 1967 ended. Score 3.000000
episode 1968 ended. Score 1.000000
episode 1969 ended. Score 1.000000
episode 1970 ended. Score 3.000000
episode 1971 ended. Score 1.000000
episode 1972 ended. Score 1.000000
episode 1973 ended. Score 0.000000
episode 1974 ended. Score 0.000000
episode 1975 ended. Score 0.000000
episode 1976 ended. Score 2.000000
episode 1977 ended. Score 2.000000
episode 1978 ended. Score 3.000000
episode 1979 ended. Score 1.000000
episode 1980 ended. Score 1.000000
episode 1981 ended. Score 1.000000
episode 1982 ended. Score 0.000000
2017-12-11 14:58:27.411569

batch 59500

episode 1983 ended. Score 2.000000
episode 1984 ended. Score 0.000000
episode 1985 ended. Score 1.000000
episode 1986 ended. Score 0.000000
episode 1987 ended. Score 0.000000
episode 1988 ended. Score 0.000000
episode 1989 ended. Score 1.000000
episode 1990 ended. Score 2.000000
episode 1991 ended. Score 1.000000
episode 1992 ended. Score 1.000000
episode 1993 ended. Score 1.000000
episode 1994 ended. Score 0.000000
episode 1995 ended. Score 0.000000
episode 1996 ended. Score 3.000000
episode 1997 ended. Score 2.000000
episode 1998 ended. Score 1.000000
episode 1999 ended. Score 2.000000
2017-12-11 15:05:12.555454

batch 60000

saving model to _files/models/my_beacon_model, step 60000
episode 2000 ended. Score 0.000000
episode 2001 ended. Score 1.000000
episode 2002 ended. Score 1.000000
episode 2003 ended. Score 0.000000
episode 2004 ended. Score 2.000000
episode 2005 ended. Score 0.000000
episode 2006 ended. Score 2.000000
episode 2007 ended. Score 1.000000
episode 2008 ended. Score 1.000000
episode 2009 ended. Score 2.000000
episode 2010 ended. Score 3.000000
episode 2011 ended. Score 2.000000
episode 2012 ended. Score 2.000000
episode 2013 ended. Score 1.000000
episode 2014 ended. Score 2.000000
episode 2015 ended. Score 0.000000
2017-12-11 15:12:10.104631

batch 60500

episode 2016 ended. Score 1.000000
episode 2017 ended. Score 3.000000
episode 2018 ended. Score 1.000000
episode 2019 ended. Score 1.000000
episode 2020 ended. Score 2.000000
episode 2021 ended. Score 1.000000
episode 2022 ended. Score 3.000000
episode 2023 ended. Score 2.000000
episode 2024 ended. Score 1.000000
episode 2025 ended. Score 2.000000
episode 2026 ended. Score 0.000000
episode 2027 ended. Score 2.000000
episode 2028 ended. Score 2.000000
episode 2029 ended. Score 3.000000
episode 2030 ended. Score 0.000000
episode 2031 ended. Score 2.000000
episode 2032 ended. Score 1.000000
2017-12-11 15:18:54.969219

batch 61000

episode 2033 ended. Score 1.000000
episode 2034 ended. Score 1.000000
episode 2035 ended. Score 1.000000
episode 2036 ended. Score 2.000000
episode 2037 ended. Score 0.000000
episode 2038 ended. Score 2.000000
episode 2039 ended. Score 1.000000
episode 2040 ended. Score 3.000000
episode 2041 ended. Score 2.000000
episode 2042 ended. Score 1.000000
episode 2043 ended. Score 1.000000
episode 2044 ended. Score 0.000000
episode 2045 ended. Score 2.000000
episode 2046 ended. Score 2.000000
episode 2047 ended. Score 1.000000
episode 2048 ended. Score 3.000000
episode 2049 ended. Score 2.000000
2017-12-11 15:24:53.183937

batch 61500

episode 2050 ended. Score 0.000000
episode 2051 ended. Score 0.000000
episode 2052 ended. Score 1.000000
episode 2053 ended. Score 0.000000
episode 2054 ended. Score 1.000000
episode 2055 ended. Score 1.000000
episode 2056 ended. Score 3.000000
episode 2057 ended. Score 2.000000
episode 2058 ended. Score 3.000000
episode 2059 ended. Score 3.000000
episode 2060 ended. Score 0.000000
episode 2061 ended. Score 0.000000
episode 2062 ended. Score 1.000000
episode 2063 ended. Score 2.000000
episode 2064 ended. Score 2.000000
episode 2065 ended. Score 0.000000
2017-12-11 15:31:53.889381

batch 62000

episode 2066 ended. Score 1.000000
episode 2067 ended. Score 0.000000
episode 2068 ended. Score 1.000000
episode 2069 ended. Score 2.000000
episode 2070 ended. Score 1.000000
episode 2071 ended. Score 1.000000
episode 2072 ended. Score 1.000000
episode 2073 ended. Score 1.000000
episode 2074 ended. Score 3.000000
episode 2075 ended. Score 0.000000
episode 2076 ended. Score 2.000000
episode 2077 ended. Score 0.000000
episode 2078 ended. Score 0.000000
episode 2079 ended. Score 2.000000
episode 2080 ended. Score 0.000000
episode 2081 ended. Score 0.000000
episode 2082 ended. Score 3.000000
2017-12-11 15:37:48.264338

batch 62500

episode 2083 ended. Score 3.000000
episode 2084 ended. Score 1.000000
episode 2085 ended. Score 0.000000
episode 2086 ended. Score 1.000000
episode 2087 ended. Score 0.000000
episode 2088 ended. Score 1.000000
episode 2089 ended. Score 0.000000
episode 2090 ended. Score 1.000000
episode 2091 ended. Score 1.000000
episode 2092 ended. Score 1.000000
episode 2093 ended. Score 2.000000
episode 2094 ended. Score 3.000000
episode 2095 ended. Score 1.000000
episode 2096 ended. Score 2.000000
episode 2097 ended. Score 1.000000
episode 2098 ended. Score 1.000000
episode 2099 ended. Score 0.000000
2017-12-11 15:43:06.432630

batch 63000

episode 2100 ended. Score 1.000000
episode 2101 ended. Score 3.000000
episode 2102 ended. Score 1.000000
episode 2103 ended. Score 0.000000
episode 2104 ended. Score 1.000000
episode 2105 ended. Score 2.000000
episode 2106 ended. Score 2.000000
episode 2107 ended. Score 0.000000
episode 2108 ended. Score 2.000000
episode 2109 ended. Score 1.000000
episode 2110 ended. Score 1.000000
episode 2111 ended. Score 4.000000
episode 2112 ended. Score 1.000000
episode 2113 ended. Score 1.000000
episode 2114 ended. Score 2.000000
episode 2115 ended. Score 2.000000
2017-12-11 15:50:53.162990

batch 63500

episode 2116 ended. Score 3.000000
episode 2117 ended. Score 1.000000

[pekaalto]

There is lot's of variance when it eventually "gets" it. It can easily take several thousand episodes.

Based on your training log it looks like it'ss just about to learn. I think if you give it 1000 more episodes you will see high increase in scores.

[hechenghai]

Thank you.
I still don't get satisfied results. I may need to try it for a few more times.
Besides, which software do you use to plot the graph?

[pekaalto]

I tried to train the MoveToBeacon with the parameters in the current commit (b6a581b) and n_envs 64. These are the episodes numbers when first time reached episode score 27:

4674
3079
2355
1231
6358

So as you can see there is some variation. (But you are right that the 500 episodes seems to be quite optimistic. I will add note in the README.)

[pekaalto]

The graphs are from tensorboard https://www.tensorflow.org/get_started/summaries_and_tensorboard.

You can see them by doing tensorboard --logdir _files/summaries (or whatever summary-path you specified) and then navigating in browser to the url the tensorboard gives you.