/bench-docker

各言語のベンチマーク計測

Primary LanguageGo

bench

各言語のベンチマーク計測

計測方法

  1. 1万行のCSVを読み込んでDBに登録
  2. 登録したDBのデータをCSVに吐き出す
  3. 入力CSVと出力CSVを突合して確認
  4. 計10回実行してその平均を出す

結果

  1. Python
  2. PHP
  3. Ruby
  4. Go
MySQL 5.7 + Python 3.9.0
平均秒数：7.586802

MySQL 5.7 + PHP 7.4.12
平均秒数：11.787367296219

MySQL 5.7 + Ruby 2.7.2
平均秒数：11.840328

MySQL 5.7 + Go 1.15.5
平均秒数：15.42766124

考察

各言語で一番速度が出た処理を比較してみる

# python
2020-12-04 18:48:09.914944
2020-12-04 18:48:09.935228 import CSV start
2020-12-04 18:48:16.725192 import CSV end
2020-12-04 18:48:16.876930 export CSV start
2020-12-04 18:48:17.105134 export CSV end
2020-12-04 18:48:17.105216 compare CSV start
2020-12-04 18:48:17.208220 compare CSV end
2020-12-04 18:48:17.215074
7.300130

#CSV読み込み＋DBインサート
6.789964

DBセレクト
0.151738

CSV書き込み
0.228204

CSV読み込み＋突合
0.103004

# php
2020-12-04 18:38:18.282300
2020-12-04 18:38:18.299000 import CSV start
2020-12-04 18:38:24.610800 import CSV end
2020-12-04 18:38:24.692000 export CSV start
2020-12-04 18:38:28.719200 export CSV end
2020-12-04 18:38:28.719200 compare CSV start
2020-12-04 18:38:28.865900 compare CSV end
2020-12-04 18:38:28.868100
10.585803985596

CSV読み込み＋DBインサート
6.311800

DBセレクト
0.081200

CSV書き込み
4.027200

CSV読み込み＋突合
0.146700


# ruby
"2020-12-04 18:43:08.038825"
"2020-12-04 18:43:08.053449 import CSV start"
"2020-12-04 18:43:16.653697 import CSV end"
"2020-12-04 18:43:16.683815 export CSV start"
"2020-12-04 18:43:17.118668 export CSV end"
"2020-12-04 18:43:17.118718 compare CSV start"
"2020-12-04 18:43:19.045548 compare CSV end"
"2020-12-04 18:43:19.045579"
"11.006754"

CSV読み込み＋DBインサート
8.600248

DBセレクト
0.030118

CSV書き込み
0.434853

CSV読み込み＋突合
1.926830


# go
2020-12-04 18:33:26.164199
2020-12-04 18:33:26.180394 import CSV start
2020-12-04 18:33:35.627523 import CSV end
2020-12-04 18:33:35.628442 export CSV start
2020-12-04 18:33:40.295815 export CSV end
2020-12-04 18:33:40.295867 compare CSV start
2020-12-04 18:33:40.321120 compare CSV end
2020-12-04 18:33:40.321268
14.1570699 (0x0,0x0)

CSV読み込み＋DBインサート
9.447129

DBセレクト
0.000919

CSV書き込み
4.667373

CSV読み込み＋突合
0.025253

まとめ

  • Pythonはファイル操作に長けている。全体的に処理も早い。ただ他の言語と比較してselectが若干遅い
  • PHPはファイル読み込みは速いが、書き込みは遅い
  • Rubyはファイル読み込みは遅いが、書き込みは速い
  • Goは処理速度は非常に速いが、ファイル処理は非常に遅い

Goは単純な処理は早いがファイルIOが遅い。バッチ処理するならPythonの方が良さそう。 RubyとPHP使うなら、PHPの書き込み速度がネックになりそうなので、Rubyの方が良さそう。

以下処理結果ログ

MySQL 5.7 + Python 3.9.0

平均秒数：7.586802

Python 3.9.0 (default, Nov 25 2020, 02:36:55) 
[GCC 9.3.0]
2020-12-04 18:48:09.914944
2020-12-04 18:48:09.935228 import CSV start
2020-12-04 18:48:16.725192 import CSV end
2020-12-04 18:48:16.876930 export CSV start
2020-12-04 18:48:17.105134 export CSV end
2020-12-04 18:48:17.105216 compare CSV start
2020-12-04 18:48:17.208220 compare CSV end
2020-12-04 18:48:17.215074
7.300130
2020-12-04 18:48:17.215234
2020-12-04 18:48:17.231558 import CSV start
2020-12-04 18:48:24.170420 import CSV end
2020-12-04 18:48:24.322432 export CSV start
2020-12-04 18:48:24.539213 export CSV end
2020-12-04 18:48:24.539270 compare CSV start
2020-12-04 18:48:24.640157 compare CSV end
2020-12-04 18:48:24.647255
7.432021
2020-12-04 18:48:24.647339
2020-12-04 18:48:24.660681 import CSV start
2020-12-04 18:48:31.513771 import CSV end
2020-12-04 18:48:31.661461 export CSV start
2020-12-04 18:48:31.880751 export CSV end
2020-12-04 18:48:31.880851 compare CSV start
2020-12-04 18:48:31.982702 compare CSV end
2020-12-04 18:48:31.989244
7.341905
2020-12-04 18:48:31.989320
2020-12-04 18:48:32.004352 import CSV start
2020-12-04 18:48:38.953776 import CSV end
2020-12-04 18:48:39.115304 export CSV start
2020-12-04 18:48:39.336671 export CSV end
2020-12-04 18:48:39.336729 compare CSV start
2020-12-04 18:48:39.440064 compare CSV end
2020-12-04 18:48:39.446754
7.457434
2020-12-04 18:48:39.446884
2020-12-04 18:48:39.460438 import CSV start
2020-12-04 18:48:46.680622 import CSV end
2020-12-04 18:48:46.845855 export CSV start
2020-12-04 18:48:47.071742 export CSV end
2020-12-04 18:48:47.071842 compare CSV start
2020-12-04 18:48:47.180303 compare CSV end
2020-12-04 18:48:47.187811
7.740927
2020-12-04 18:48:47.187978
2020-12-04 18:48:47.199864 import CSV start
2020-12-04 18:48:54.077490 import CSV end
2020-12-04 18:48:54.242379 export CSV start
2020-12-04 18:48:54.466550 export CSV end
2020-12-04 18:48:54.466608 compare CSV start
2020-12-04 18:48:54.579028 compare CSV end
2020-12-04 18:48:54.586011
7.398033
2020-12-04 18:48:54.586152
2020-12-04 18:48:54.601874 import CSV start
2020-12-04 18:49:01.765982 import CSV end
2020-12-04 18:49:01.939747 export CSV start
2020-12-04 18:49:02.163829 export CSV end
2020-12-04 18:49:02.163930 compare CSV start
2020-12-04 18:49:02.276158 compare CSV end
2020-12-04 18:49:02.283455
7.697303
2020-12-04 18:49:02.283594
2020-12-04 18:49:02.297535 import CSV start
2020-12-04 18:49:09.510598 import CSV end
2020-12-04 18:49:09.691481 export CSV start
2020-12-04 18:49:09.911890 export CSV end
2020-12-04 18:49:09.912001 compare CSV start
2020-12-04 18:49:10.031758 compare CSV end
2020-12-04 18:49:10.039325
7.755731
2020-12-04 18:49:10.039711
2020-12-04 18:49:10.052662 import CSV start
2020-12-04 18:49:17.193991 import CSV end
2020-12-04 18:49:17.383815 export CSV start
2020-12-04 18:49:17.620209 export CSV end
2020-12-04 18:49:17.620323 compare CSV start
2020-12-04 18:49:17.740548 compare CSV end
2020-12-04 18:49:17.747949
7.708238
2020-12-04 18:49:17.748085
2020-12-04 18:49:17.762492 import CSV start
2020-12-04 18:49:25.248833 import CSV end
2020-12-04 18:49:25.417406 export CSV start
2020-12-04 18:49:25.659209 export CSV end
2020-12-04 18:49:25.659332 compare CSV start
2020-12-04 18:49:25.777076 compare CSV end
2020-12-04 18:49:25.784383
8.036298
平均秒数：7.586802

MySQL 5.7 + PHP 7.4.12

平均秒数：11.787367296219

PHP 7.4.12
2020-12-04 18:38:18.282300
2020-12-04 18:38:18.299000 import CSV start
2020-12-04 18:38:24.610800 import CSV end
2020-12-04 18:38:24.692000 export CSV start
2020-12-04 18:38:28.719200 export CSV end
2020-12-04 18:38:28.719200 compare CSV start
2020-12-04 18:38:28.865900 compare CSV end
2020-12-04 18:38:28.868100
10.585803985596
2020-12-04 18:38:28.868200
2020-12-04 18:38:28.882700 import CSV start
2020-12-04 18:38:35.479400 import CSV end
2020-12-04 18:38:35.557000 export CSV start
2020-12-04 18:38:39.653300 export CSV end
2020-12-04 18:38:39.653300 compare CSV start
2020-12-04 18:38:39.806300 compare CSV end
2020-12-04 18:38:39.808800
10.940591096878
2020-12-04 18:38:39.808800
2020-12-04 18:38:39.823300 import CSV start
2020-12-04 18:38:46.457900 import CSV end
2020-12-04 18:38:46.535800 export CSV start
2020-12-04 18:38:50.731800 export CSV end
2020-12-04 18:38:50.731800 compare CSV start
2020-12-04 18:38:50.886400 compare CSV end
2020-12-04 18:38:50.889400
11.080587148666
2020-12-04 18:38:50.889500
2020-12-04 18:38:50.904600 import CSV start
2020-12-04 18:38:57.589400 import CSV end
2020-12-04 18:38:57.665800 export CSV start
2020-12-04 18:39:02.105100 export CSV end
2020-12-04 18:39:02.105200 compare CSV start
2020-12-04 18:39:02.275100 compare CSV end
2020-12-04 18:39:02.277800
11.388316869736
2020-12-04 18:39:02.277900
2020-12-04 18:39:02.291100 import CSV start
2020-12-04 18:39:09.488200 import CSV end
2020-12-04 18:39:09.569100 export CSV start
2020-12-04 18:39:14.799000 export CSV end
2020-12-04 18:39:14.799100 compare CSV start
2020-12-04 18:39:14.964100 compare CSV end
2020-12-04 18:39:14.967000
12.689137935638
2020-12-04 18:39:14.967100
2020-12-04 18:39:14.981200 import CSV start
2020-12-04 18:39:22.239800 import CSV end
2020-12-04 18:39:22.320300 export CSV start
2020-12-04 18:39:26.854400 export CSV end
2020-12-04 18:39:26.854400 compare CSV start
2020-12-04 18:39:27.016000 compare CSV end
2020-12-04 18:39:27.018700
12.05156493187
2020-12-04 18:39:27.018700
2020-12-04 18:39:27.032500 import CSV start
2020-12-04 18:39:34.412000 import CSV end
2020-12-04 18:39:34.492700 export CSV start
2020-12-04 18:39:39.111500 export CSV end
2020-12-04 18:39:39.111500 compare CSV start
2020-12-04 18:39:39.275300 compare CSV end
2020-12-04 18:39:39.277700
12.259013175964
2020-12-04 18:39:39.277800
2020-12-04 18:39:39.292200 import CSV start
2020-12-04 18:39:46.841400 import CSV end
2020-12-04 18:39:46.921200 export CSV start
2020-12-04 18:39:51.522900 export CSV end
2020-12-04 18:39:51.523000 compare CSV start
2020-12-04 18:39:51.686700 compare CSV end
2020-12-04 18:39:51.689100
12.411306858063
2020-12-04 18:39:51.689200
2020-12-04 18:39:51.703500 import CSV start
2020-12-04 18:39:59.070800 import CSV end
2020-12-04 18:39:59.149200 export CSV start
2020-12-04 18:40:03.809000 export CSV end
2020-12-04 18:40:03.809200 compare CSV start
2020-12-04 18:40:03.974900 compare CSV end
2020-12-04 18:40:03.977800
12.288660049438
2020-12-04 18:40:03.977900
2020-12-04 18:40:03.992100 import CSV start
2020-12-04 18:40:11.318800 import CSV end
2020-12-04 18:40:11.398800 export CSV start
2020-12-04 18:40:15.989500 export CSV end
2020-12-04 18:40:15.989500 compare CSV start
2020-12-04 18:40:16.154000 compare CSV end
2020-12-04 18:40:16.156600
12.178690910339
平均秒数：11.787367296219

MySQL 5.7 + Ruby 2.7.2

平均秒数：11.840328

"Ruby 2.7.2"
"2020-12-04 18:43:08.038825"
"2020-12-04 18:43:08.053449 import CSV start"
"2020-12-04 18:43:16.653697 import CSV end"
"2020-12-04 18:43:16.683815 export CSV start"
"2020-12-04 18:43:17.118668 export CSV end"
"2020-12-04 18:43:17.118718 compare CSV start"
"2020-12-04 18:43:19.045548 compare CSV end"
"2020-12-04 18:43:19.045579"
"11.006754"
"2020-12-04 18:43:19.045597"
"2020-12-04 18:43:19.059034 import CSV start"
"2020-12-04 18:43:27.713840 import CSV end"
"2020-12-04 18:43:27.743303 export CSV start"
"2020-12-04 18:43:28.170391 export CSV end"
"2020-12-04 18:43:28.170440 compare CSV start"
"2020-12-04 18:43:30.270370 compare CSV end"
"2020-12-04 18:43:30.270421"
"11.224824"
"2020-12-04 18:43:30.270443"
"2020-12-04 18:43:30.284968 import CSV start"
"2020-12-04 18:43:39.172574 import CSV end"
"2020-12-04 18:43:39.201146 export CSV start"
"2020-12-04 18:43:39.655885 export CSV end"
"2020-12-04 18:43:39.655931 compare CSV start"
"2020-12-04 18:43:41.750329 compare CSV end"
"2020-12-04 18:43:41.750426"
"11.479984"
"2020-12-04 18:43:41.750587"
"2020-12-04 18:43:41.765985 import CSV start"
"2020-12-04 18:43:50.794890 import CSV end"
"2020-12-04 18:43:50.823680 export CSV start"
"2020-12-04 18:43:51.273359 export CSV end"
"2020-12-04 18:43:51.273473 compare CSV start"
"2020-12-04 18:43:53.387215 compare CSV end"
"2020-12-04 18:43:53.387289"
"11.636703"
"2020-12-04 18:43:53.387634"
"2020-12-04 18:43:53.403591 import CSV start"
"2020-12-04 18:44:02.679752 import CSV end"
"2020-12-04 18:44:02.708117 export CSV start"
"2020-12-04 18:44:03.200543 export CSV end"
"2020-12-04 18:44:03.200570 compare CSV start"
"2020-12-04 18:44:05.416933 compare CSV end"
"2020-12-04 18:44:05.416986"
"12.029352"
"2020-12-04 18:44:05.417084"
"2020-12-04 18:44:05.433885 import CSV start"
"2020-12-04 18:44:14.360653 import CSV end"
"2020-12-04 18:44:14.393191 export CSV start"
"2020-12-04 18:44:14.846919 export CSV end"
"2020-12-04 18:44:14.846964 compare CSV start"
"2020-12-04 18:44:17.096839 compare CSV end"
"2020-12-04 18:44:17.096873"
"11.679788"
"2020-12-04 18:44:17.096890"
"2020-12-04 18:44:17.115601 import CSV start"
"2020-12-04 18:44:26.344854 import CSV end"
"2020-12-04 18:44:26.375012 export CSV start"
"2020-12-04 18:44:26.924188 export CSV end"
"2020-12-04 18:44:26.924239 compare CSV start"
"2020-12-04 18:44:29.146173 compare CSV end"
"2020-12-04 18:44:29.146230"
"12.049340"
"2020-12-04 18:44:29.146314"
"2020-12-04 18:44:29.161678 import CSV start"
"2020-12-04 18:44:38.720743 import CSV end"
"2020-12-04 18:44:38.753229 export CSV start"
"2020-12-04 18:44:39.245264 export CSV end"
"2020-12-04 18:44:39.245313 compare CSV start"
"2020-12-04 18:44:41.506689 compare CSV end"
"2020-12-04 18:44:41.506787"
"12.360473"
"2020-12-04 18:44:41.507112"
"2020-12-04 18:44:41.521976 import CSV start"
"2020-12-04 18:44:51.120442 import CSV end"
"2020-12-04 18:44:51.151450 export CSV start"
"2020-12-04 18:44:51.659697 export CSV end"
"2020-12-04 18:44:51.659790 compare CSV start"
"2020-12-04 18:44:53.925933 compare CSV end"
"2020-12-04 18:44:53.925989"
"12.418878"
"2020-12-04 18:44:53.926027"
"2020-12-04 18:44:53.942581 import CSV start"
"2020-12-04 18:45:03.745834 import CSV end"
"2020-12-04 18:45:03.781359 export CSV start"
"2020-12-04 18:45:04.277777 export CSV end"
"2020-12-04 18:45:04.277939 compare CSV start"
"2020-12-04 18:45:06.443145 compare CSV end"
"2020-12-04 18:45:06.443212"
"12.517185"
"平均秒数：11.840328"

MySQL 5.7 + Go 1.15.5

平均秒数：15.42766124

Go go1.15.5
2020-12-04 18:32:56.731407
2020-12-04 18:32:56.770553 import CSV start
2020-12-04 18:33:06.746475 import CSV end
2020-12-04 18:33:06.747334 export CSV start
2020-12-04 18:33:11.803909 export CSV end
2020-12-04 18:33:11.803947 compare CSV start
2020-12-04 18:33:11.835890 compare CSV end
2020-12-04 18:33:11.835999
15.1045939 (0x0,0x0)
2020-12-04 18:33:11.836135
2020-12-04 18:33:11.864054 import CSV start
2020-12-04 18:33:21.292816 import CSV end
2020-12-04 18:33:21.293667 export CSV start
2020-12-04 18:33:26.140855 export CSV end
2020-12-04 18:33:26.140887 compare CSV start
2020-12-04 18:33:26.163898 compare CSV end
2020-12-04 18:33:26.164127
14.3279929 (0x0,0x0)
2020-12-04 18:33:26.164199
2020-12-04 18:33:26.180394 import CSV start
2020-12-04 18:33:35.627523 import CSV end
2020-12-04 18:33:35.628442 export CSV start
2020-12-04 18:33:40.295815 export CSV end
2020-12-04 18:33:40.295867 compare CSV start
2020-12-04 18:33:40.321120 compare CSV end
2020-12-04 18:33:40.321268
14.1570699 (0x0,0x0)
2020-12-04 18:33:40.321443
2020-12-04 18:33:40.340157 import CSV start
2020-12-04 18:33:50.046656 import CSV end
2020-12-04 18:33:50.049821 export CSV start
2020-12-04 18:33:54.967411 export CSV end
2020-12-04 18:33:54.967458 compare CSV start
2020-12-04 18:33:54.997644 compare CSV end
2020-12-04 18:33:54.998142
14.6766999 (0x0,0x0)
2020-12-04 18:33:54.998539
2020-12-04 18:33:55.019787 import CSV start
2020-12-04 18:34:05.519525 import CSV end
2020-12-04 18:34:05.522320 export CSV start
2020-12-04 18:34:10.696589 export CSV end
2020-12-04 18:34:10.696631 compare CSV start
2020-12-04 18:34:10.722159 compare CSV end
2020-12-04 18:34:10.722387
15.7238489 (0x0,0x0)
2020-12-04 18:34:10.722584
2020-12-04 18:34:10.739626 import CSV start
2020-12-04 18:34:21.561216 import CSV end
2020-12-04 18:34:21.561748 export CSV start
2020-12-04 18:34:26.668071 export CSV end
2020-12-04 18:34:26.668182 compare CSV start
2020-12-04 18:34:26.694219 compare CSV end
2020-12-04 18:34:26.694356
15.9717729 (0x0,0x0)
2020-12-04 18:34:26.694437
2020-12-04 18:34:26.712859 import CSV start
2020-12-04 18:34:36.890619 import CSV end
2020-12-04 18:34:36.891380 export CSV start
2020-12-04 18:34:41.626598 export CSV end
2020-12-04 18:34:41.627004 compare CSV start
2020-12-04 18:34:41.657552 compare CSV end
2020-12-04 18:34:41.657670
14.9632329 (0x0,0x0)
2020-12-04 18:34:41.657765
2020-12-04 18:34:41.674880 import CSV start
2020-12-04 18:34:51.565415 import CSV end
2020-12-04 18:34:51.566141 export CSV start
2020-12-04 18:34:56.747330 export CSV end
2020-12-04 18:34:56.747368 compare CSV start
2020-12-04 18:34:56.773639 compare CSV end
2020-12-04 18:34:56.774093
15.1163289 (0x0,0x0)
2020-12-04 18:34:56.776348
2020-12-04 18:34:56.793556 import CSV start
2020-12-04 18:35:08.849634 import CSV end
2020-12-04 18:35:08.850730 export CSV start
2020-12-04 18:35:14.480905 export CSV end
2020-12-04 18:35:14.480942 compare CSV start
2020-12-04 18:35:14.510017 compare CSV end
2020-12-04 18:35:14.510196
17.7338489 (0x0,0x0)
2020-12-04 18:35:14.510872
2020-12-04 18:35:14.532623 import CSV start
2020-12-04 18:35:25.683906 import CSV end
2020-12-04 18:35:25.684857 export CSV start
2020-12-04 18:35:30.976413 export CSV end
2020-12-04 18:35:30.976596 compare CSV start
2020-12-04 18:35:31.011995 compare CSV end
2020-12-04 18:35:31.012100
16.5012289 (0x0,0x0)
平均秒数：15.42766124 (0x0,0x0)