数据采集不成功问题

Question

数据采集不成功问题

Ezail3 opened this issue 6 years ago · 10 comments

Ezail3 commented 6 years ago

crontab日志看，每分钟都有执行
但通过snapshot里面记录的processlist时间戳来看，经常会出现没执行成功的情况，不知如何解决

Answer 1 · 2018-10-16T14:06:11.000Z

建议按README中的说明在配置文件中将日志级别调为7（Debug），然后看一下mymon自己打的日志中运行的时间及状态信息，方便的话可以把日志上传上来，也好帮忙定位异常原因。

Answer 2 · 2018-10-17T04:40:58.000Z

建议按README中的说明在配置文件中将日志级别调为7（Debug），然后看一下mymon自己打的日志中运行的时间及状态信息，方便的话可以把日志上传上来，也好帮忙定位异常原因。

没采集到的时间点，mymon的报错类似这种，辛苦帮忙看下，谢谢

Answer 3 · 2018-10-17T04:46:29.000Z

建议按README中的说明在配置文件中将日志级别调为7（Debug），然后看一下mymon自己打的日志中运行的时间及状态信息，方便的话可以把日志上传上来，也好帮忙定位异常原因。

没采集到的时间点，mymon的报错类似这种，辛苦帮忙看下，谢谢

root@dpsvstadbs03 12:45:34:mymon$ cat myMon.log |grep 'Error: strconv.Atoi'
2018/10/16 21:40:01.766 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 21:45:01.376 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 21:50:01.807 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 21:55:01.633 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:00:01.305 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:05:01.873 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:15:01.713 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:20:01.379 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:25:01.654 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:35:01.386 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:45:01.094 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:50:01.670 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:52:01.275 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 22:55:01.262 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:00:01.679 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:05:01.253 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:15:01.167 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:20:01.645 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:25:01.215 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:30:01.548 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:40:01.761 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:45:01.574 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:46:01.917 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:50:01.278 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:54:01.494 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/16 23:55:01.931 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 00:00:02.024 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 00:10:01.446 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 00:20:01.431 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 00:26:01.137 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 00:30:01.412 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 00:45:01.209 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 00:55:01.147 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 01:00:01.699 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 01:10:01.589 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 01:30:01.763 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 01:32:01.220 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 01:33:01.425 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 01:40:01.647 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 01:49:01.719 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 01:50:02.004 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 01:51:01.356 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 02:00:01.359 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:05:01.207 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:10:01.581 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:15:01.437 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:25:01.428 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:30:01.700 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:35:01.313 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:40:01.806 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:45:01.401 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:50:01.447 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 02:55:01.325 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 03:00:01.505 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 03:10:01.961 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 03:14:01.175 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 03:20:01.168 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 03:30:01.226 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 03:36:01.194 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 03:40:01.295 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 04:00:01.984 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 04:10:02.088 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 04:15:01.957 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 04:20:01.829 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 04:32:01.356 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 04:40:01.311 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 05:09:01.283 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 05:15:01.411 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 05:20:01.259 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 05:34:01.709 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 06:00:01.880 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 06:04:01.193 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 06:05:01.972 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 06:08:01.812 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 06:10:01.373 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 06:12:01.178 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 06:15:01.279 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 07:10:01.475 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 07:15:01.993 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 07:35:01.655 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 07:36:01.130 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 08:05:01.383 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 08:15:01.533 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 08:30:01.332 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 08:45:01.293 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 08:50:01.057 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 08:55:01.985 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 09:00:01.650 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 09:05:01.868 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 09:12:01.629 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 09:14:01.300 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 09:15:01.681 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 09:17:01.144 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 09:20:01.597 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 09:25:01.377 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 09:30:01.740 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 09:45:01.156 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 10:00:01.885 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 10:01:01.110 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 10:15:01.372 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 10:32:01.118 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 10:36:01.063 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 10:40:01.157 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 10:45:01.669 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 10:55:01.087 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:00:01.799 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:03:01.995 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:10:01.176 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 11:14:01.387 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:20:01.109 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:27:01.347 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:30:01.285 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:35:01.609 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:44:01.215 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:50:01.761 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:54:01.977 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 11:55:01.288 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 12:00:01.218 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 12:05:01.665 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 12:08:01.669 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 12:15:01.639 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 12:17:01.401 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 1": invalid syntax
2018/10/17 12:20:01.419 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 12:22:01.470 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 12:24:01.337 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
2018/10/17 12:25:01.595 [E] [main.go:88] Error: strconv.Atoi: parsing "(PREPARED) 0": invalid syntax
出现次数太频繁了

Answer 4 · 2018-10-17T06:05:04.000Z

从日志上看应该是show.go中处理longest_transaction报的错。这种问题在最近的几次提交中应该fix了，请帮忙确认一下mymon的版本，以及mysql的版本。
mymon -v以及mysql > select @@Version;

如果git pull到最新版没有类似报错了，就可以继续使用啦。

Answer 5 · 2018-10-17T06:24:20.000Z

从日志上看应该是show.go中处理longest_transaction报的错。这种问题在最近的几次提交中应该fix了，请帮忙确认一下mymon的版本，以及mysql的版本。
mymon -v以及mysql > select @@Version;

如果git pull到最新版没有类似报错了，就可以继续使用啦。

奇怪的是我监控的机器里面有一台几乎没有报错，有两台大批量报错

Answer 6 · 2018-10-17T07:08:32.000Z

从日志上看应该是show.go中处理longest_transaction报的错。这种问题在最近的几次提交中应该fix了，请帮忙确认一下mymon的版本，以及mysql的版本。
mymon -v以及mysql > select @@Version;
如果git pull到最新版没有类似报错了，就可以继续使用啦。

奇怪的是我监控的机器里面有一台几乎没有报错，有两台大批量报错

已经换了最新的mymon，还是有问题，但是比之前稍微好一点

Answer 7 · 2018-10-17T08:00:46.000Z

这个机器一小时之内出现了30次没抓到数据，a bit 尴尬，23333333

Answer 8 · 2018-10-17T16:18:04.000Z

这个问题的原因找到了，确认是longest_transaction处理时产生的异常，之前有位朋友也反馈过同样的问题，已经在一个fork中修复了，但尚未提交PR合并到主分支。稍晚些时候我们会再提交这个问题FIX的PR。

之所以频率不同与应用程序使用的事务或SQL类型有关，strconv.Atoi处理的时候容错做得存在问题。

感谢您的积极反馈。

Answer 9 · 2018-10-17T16:44:18.000Z

另外还有个问题，我这边个别机器的监控数据有是有但是dashbord上老是不及时展示，一直不更新，我重启falcon才能看到完整的数据，个别机器

Answer 10 · 2018-10-18T02:14:21.000Z

dashboard的问题可以在https://github.com/open-falcon/falcon-plus 或 https://github.com/open-falcon/dashboard 中咨询一下，在这里回复不合适。

PR已经提交#40, 等待管理员同学合并。