shilad/wikibrain

Building ensemble from master branch gives the following error:

Closed this issue · 1 comments

Alan502 commented

Just fyi! I am trying to build ensemble from a freshly cloned master branch and it does not complete successfully. It gives the following error:

Jun 07, 2014 7:07:54 PM org.wikibrain.sr.SRBuilder deleteDataDirectories
INFO: deleting metric directory ./dat/sr/outlink/simple
Jun 07, 2014 7:07:54 PM org.wikibrain.sr.SRBuilder deleteDataDirectories
INFO: deleting metric directory ./dat/sr/category/simple
Jun 07, 2014 7:07:54 PM org.wikibrain.sr.SRBuilder build
INFO: building metric ensemble
LOADING LANGUAGES (SIMPLE)
Jun 07, 2014 7:07:55 PM org.wikibrain.sr.vector.ESAGenerator setConcepts
WARNING: installed 47036 concepts for Simple English
Jun 07, 2014 7:07:55 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./db/matrix/local-link/links.matrix: initializing sparse matrix with file length 34115088
Jun 07, 2014 7:07:56 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./db/matrix/local-link/links-transpose.matrix: initializing sparse matrix with file length 30548896
LOADING LANGUAGES (SIMPLE)
Jun 07, 2014 7:07:57 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/inlinknotrain/simple does not exist; cannot read files
Jun 07, 2014 7:07:57 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric inlinknotrain
Jun 07, 2014 7:07:57 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/ESA/simple does not exist; cannot read files
Jun 07, 2014 7:07:57 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric ESA
Jun 07, 2014 7:07:57 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/inlink/simple does not exist; cannot read files
Jun 07, 2014 7:07:57 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric inlink
Jun 07, 2014 7:07:57 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/outlink/simple does not exist; cannot read files
Jun 07, 2014 7:07:57 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric outlink
Jun 07, 2014 7:07:58 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/category/simple does not exist; cannot read files
Jun 07, 2014 7:07:58 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric category
Jun 07, 2014 7:07:58 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/ensemble/simple does not exist; cannot read files
Jun 07, 2014 7:07:58 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric ensemble
Jun 07, 2014 7:07:58 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric ESA
Jun 07, 2014 7:07:59 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:08:03 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:08:04 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:08:04 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:08:04 PM org.wikibrain.sr.utils.SrNormalizers trainSimilarity
INFO: trained similarity normalizer: percentile normalizer: 0%: 0.0, 5%: 0.0, 10%: 0.0010428927597552194, 15%: 0.002797699130562242, 20%: 0.004964782901075486, 25%: 0.006638796583043898, 30%: 0.00944920620140384, 35%: 0.012077226625145043, 40%: 0.014214023691574672, 45%: 0.022498378338432098, 50%: 0.029367082282675754, 55%: 0.03575526436975566, 60%: 0.04270472471026882, 65%: 0.049782773633454395, 70%: 0.06037381496579908, 75%: 0.07884587648414584, 80%: 0.09268273456332982, 85%: 0.12105819488153641, 90%: 0.1431009186443515, 95%: 0.20107537409473883, 100%: 1.0,
Jun 07, 2014 7:08:07 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: writing matrix to ./dat/sr/ESA/simple/feature.matrix
Jun 07, 2014 7:08:07 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: writing body to tmp file at ./.tmp/matrix7800368465423501340.tmp
Jun 07, 2014 7:08:07 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 104075
Jun 07, 2014 7:08:56 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 10001 of 104075
Jun 07, 2014 7:10:03 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 20001 of 104075
Jun 07, 2014 7:11:09 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 30001 of 104075
Jun 07, 2014 7:12:19 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 40001 of 104075
Jun 07, 2014 7:13:23 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 50001 of 104075
Jun 07, 2014 7:13:49 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 60001 of 104075
Jun 07, 2014 7:14:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 70001 of 104075
Jun 07, 2014 7:15:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 80001 of 104075
Jun 07, 2014 7:16:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 90001 of 104075
Jun 07, 2014 7:18:12 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 100001 of 104075
Jun 07, 2014 7:18:47 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: wrote 330839312 bytes in body of matrix
Jun 07, 2014 7:18:47 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: generating header
Jun 07, 2014 7:18:48 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: wrote 332085588 bytes to ./dat/sr/ESA/simple/feature.matrix
Jun 07, 2014 7:18:48 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/ESA/simple/feature.matrix: initializing sparse matrix with file length 332085588
Jun 07, 2014 7:18:48 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: writing matrix to ./dat/sr/ESA/simple/featureTranspose.matrix
Jun 07, 2014 7:18:48 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: writing body to tmp file at ./.tmp/matrix5879504889432268355.tmp
Jun 07, 2014 7:18:50 PM org.wikibrain.matrix.SparseMatrixTransposer countCellsPerColumn
INFO: found 47036 unique column ids in matrix
Jun 07, 2014 7:18:50 PM org.wikibrain.matrix.SparseMatrixTransposer accumulateBatch
INFO: processing 47036 columns in batch (total=47036 of 47036)
Jun 07, 2014 7:19:24 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: wrote 329971616 bytes in body of matrix
Jun 07, 2014 7:19:24 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: generating header
Jun 07, 2014 7:19:25 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: wrote 330536064 bytes to ./dat/sr/ESA/simple/featureTranspose.matrix
Jun 07, 2014 7:19:25 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/ESA/simple/featureTranspose.matrix: initializing sparse matrix with file length 330536064
Jun 07, 2014 7:19:25 PM org.wikibrain.sr.vector.CosineSimilarity setMatrices
INFO: building cached matrix information
Jun 07, 2014 7:19:26 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:19:33 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:19:41 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:19:47 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:19:53 PM org.wikibrain.sr.utils.SrNormalizers trainMostSimilar
INFO: trained most similar normalizer: percentile normalizer: 0%: 0.039710421894079244, 5%: 0.07805758392035389, 10%: 0.0972991947067744, 15%: 0.11115491020697257, 20%: 0.12801012866804198, 25%: 0.1527133022859758, 30%: 0.1772639508986551, 35%: 0.21573848805651752, 40%: 0.23651190260630336, 45%: 0.2657220325140341, 50%: 0.2807600917725961, 55%: 0.29431183080880924, 60%: 0.31422721808705195, 65%: 0.3526099324369326, 70%: 0.3797810837397683, 75%: 0.40676972767607233, 80%: 0.43111122924723294, 85%: 0.47847058925126845, 90%: 0.5778927600383529, 95%: 0.999999980387662, 100%: 1.000000017449774,
Jun 07, 2014 7:19:53 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric inlink
Jun 07, 2014 7:19:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:19:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:19:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:19:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:19:54 PM org.wikibrain.sr.utils.SrNormalizers trainSimilarity
INFO: trained similarity normalizer: percentile normalizer: 0%: -0.1959001790179249, 5%: 0.0, 10%: 0.0, 15%: 0.0, 20%: 0.0, 25%: 0.0, 30%: 0.2090231768812829, 35%: 0.2986806914206396, 40%: 0.34692589727836176, 45%: 0.3876536072672815, 50%: 0.43002469459993864, 55%: 0.44991621926451897, 60%: 0.46847167486286745, 65%: 0.4847155367419187, 70%: 0.5160922856806431, 75%: 0.5358847244967025, 80%: 0.5614463113354178, 85%: 0.5982364416362632, 90%: 0.6357638696189806, 95%: 0.7164758980904506, 100%: 1.0,
Jun 07, 2014 7:19:54 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: writing matrix to ./dat/sr/inlink/simple/feature.matrix
Jun 07, 2014 7:19:54 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: writing body to tmp file at ./.tmp/matrix8333668034329938935.tmp
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 10001 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 20001 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 30001 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 40001 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 50001 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 60001 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 70001 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 80001 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 90001 of 104075
Jun 07, 2014 7:19:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 100001 of 104075
Jun 07, 2014 7:19:56 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: wrote 29368576 bytes in body of matrix
Jun 07, 2014 7:19:56 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: generating header
Jun 07, 2014 7:19:56 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: wrote 30545684 bytes to ./dat/sr/inlink/simple/feature.matrix
Jun 07, 2014 7:19:56 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/inlink/simple/feature.matrix: initializing sparse matrix with file length 30545684
Jun 07, 2014 7:19:56 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: writing matrix to ./dat/sr/inlink/simple/featureTranspose.matrix
Jun 07, 2014 7:19:56 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: writing body to tmp file at ./.tmp/matrix4364318854016415626.tmp
Jun 07, 2014 7:19:56 PM org.wikibrain.matrix.SparseMatrixTransposer countCellsPerColumn
INFO: found 221718 unique column ids in matrix
Jun 07, 2014 7:19:56 PM org.wikibrain.matrix.SparseMatrixTransposer accumulateBatch
INFO: processing 221718 columns in batch (total=221718 of 221718)
Jun 07, 2014 7:19:59 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: wrote 31453120 bytes in body of matrix
Jun 07, 2014 7:19:59 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: generating header
Jun 07, 2014 7:19:59 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: wrote 34113752 bytes to ./dat/sr/inlink/simple/featureTranspose.matrix
Jun 07, 2014 7:19:59 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/inlink/simple/featureTranspose.matrix: initializing sparse matrix with file length 34113752
Jun 07, 2014 7:19:59 PM org.wikibrain.sr.vector.GoogleSimilarity setMatrices
INFO: building cached matrix information
found 98091 and 221718
Jun 07, 2014 7:20:00 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:20:01 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:20:01 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:20:02 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:20:02 PM org.wikibrain.sr.utils.SrNormalizers trainMostSimilar
INFO: trained most similar normalizer: percentile normalizer: 0%: 0.3469614319125206, 5%: 0.43859467475372627, 10%: 0.4495926291070297, 15%: 0.45943337355473135, 20%: 0.4798782942618601, 25%: 0.4976877595830803, 30%: 0.5114978171818008, 35%: 0.5268075864326501, 40%: 0.5406302023667577, 45%: 0.5538667023446656, 50%: 0.5717227792883663, 55%: 0.5861906340082671, 60%: 0.5982364416362632, 65%: 0.6190201548304881, 70%: 0.6441773642626449, 75%: 0.6560069293090957, 80%: 0.6846390044045136, 85%: 0.7317678171723871, 90%: 0.7889446627666747, 95%: 1.0, 100%: 1.0,
Jun 07, 2014 7:20:02 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric outlink
Jun 07, 2014 7:20:02 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:20:03 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:20:03 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:20:03 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:20:03 PM org.wikibrain.sr.utils.SrNormalizers trainSimilarity
INFO: trained similarity normalizer: percentile normalizer: 0%: 0.0, 5%: 0.0, 10%: 0.0, 15%: 0.0, 20%: 0.0, 25%: 0.0, 30%: 0.0, 35%: 0.0, 40%: 0.0, 45%: 0.0, 50%: 0.0, 55%: 0.0010634160873798635, 60%: 0.014066008354108853, 65%: 0.028227379351713443, 70%: 0.042185959642105725, 75%: 0.05433789210629009, 80%: 0.07424672494408206, 85%: 0.09719121719433076, 90%: 0.13814993163480477, 95%: 0.19457648600407124, 100%: 1.0,
Jun 07, 2014 7:20:04 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: writing matrix to ./dat/sr/outlink/simple/feature.matrix
Jun 07, 2014 7:20:04 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: writing body to tmp file at ./.tmp/matrix2340047494279487583.tmp
Jun 07, 2014 7:20:04 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 104075
Jun 07, 2014 7:20:04 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 10001 of 104075
Jun 07, 2014 7:20:04 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 20001 of 104075
Jun 07, 2014 7:20:04 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 30001 of 104075
Jun 07, 2014 7:20:04 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 40001 of 104075
Jun 07, 2014 7:20:05 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 50001 of 104075
Jun 07, 2014 7:20:05 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 60001 of 104075
Jun 07, 2014 7:20:05 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 70001 of 104075
Jun 07, 2014 7:20:05 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 80001 of 104075
Jun 07, 2014 7:20:06 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 90001 of 104075
Jun 07, 2014 7:20:06 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 100001 of 104075
Jun 07, 2014 7:20:06 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: wrote 27293400 bytes in body of matrix
Jun 07, 2014 7:20:06 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: generating header
Jun 07, 2014 7:20:06 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: wrote 28537540 bytes to ./dat/sr/outlink/simple/feature.matrix
Jun 07, 2014 7:20:06 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/outlink/simple/feature.matrix: initializing sparse matrix with file length 28537540
Jun 07, 2014 7:20:06 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: writing matrix to ./dat/sr/outlink/simple/featureTranspose.matrix
Jun 07, 2014 7:20:06 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: writing body to tmp file at ./.tmp/matrix1084244034074078675.tmp
Jun 07, 2014 7:20:06 PM org.wikibrain.matrix.SparseMatrixTransposer countCellsPerColumn
INFO: found 93378 unique column ids in matrix
Jun 07, 2014 7:20:06 PM org.wikibrain.matrix.SparseMatrixTransposer accumulateBatch
INFO: processing 93378 columns in batch (total=93378 of 93378)
Jun 07, 2014 7:20:08 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: wrote 27166640 bytes in body of matrix
Jun 07, 2014 7:20:08 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: generating header
Jun 07, 2014 7:20:08 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: wrote 28287192 bytes to ./dat/sr/outlink/simple/featureTranspose.matrix
Jun 07, 2014 7:20:08 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/outlink/simple/featureTranspose.matrix: initializing sparse matrix with file length 28287192
Jun 07, 2014 7:20:08 PM org.wikibrain.sr.vector.CosineSimilarity setMatrices
INFO: building cached matrix information
Jun 07, 2014 7:20:09 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:20:09 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:20:09 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:20:09 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:20:10 PM org.wikibrain.sr.utils.SrNormalizers trainMostSimilar
INFO: trained most similar normalizer: percentile normalizer: 0%: 0.030855840901534034, 5%: 0.04110007219482489, 10%: 0.04593538530864042, 15%: 0.050970457961573463, 20%: 0.05653285607726325, 25%: 0.0686338188991258, 30%: 0.07315602376860292, 35%: 0.07717649718790542, 40%: 0.08895880189587935, 45%: 0.09686243493383082, 50%: 0.10379925796653774, 55%: 0.11161746229842354, 60%: 0.12748385562354853, 65%: 0.13772847145232867, 70%: 0.1520982576715497, 75%: 0.16297590731398653, 80%: 0.18572832119370072, 85%: 0.3146454495260231, 90%: 0.571854205228905, 95%: 1.000000000297556, 100%: 1.0000000529764081,
Jun 07, 2014 7:20:10 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric category
Jun 07, 2014 7:20:10 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:20:11 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:20:11 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:20:12 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:20:12 PM org.wikibrain.sr.utils.SrNormalizers trainSimilarity
INFO: trained similarity normalizer: percentile normalizer: 0%: -0.30198376259386606, 5%: -0.12954166044106635, 10%: -0.09391426274532406, 15%: -0.045402049939197185, 20%: -0.023648584900022668, 25%: 0.0, 30%: 0.013088447377283863, 35%: 0.036813271748164005, 40%: 0.05405495888465583, 45%: 0.07871312050884502, 50%: 0.10093011370505459, 55%: 0.15845949055331351, 60%: 0.21768473095040666, 65%: 0.28256897707432455, 70%: 0.41550028905275715, 75%: 0.46870829471860054, 80%: 0.535348294419165, 85%: 0.655518886621537, 90%: 0.8821017938213499, 95%: 1.0, 100%: 1.0,
Jun 07, 2014 7:20:12 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:20:12 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:20:13 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:20:13 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:20:13 PM org.wikibrain.sr.utils.SrNormalizers trainMostSimilar
INFO: trained most similar normalizer: percentile normalizer: 0%: 0.07497417717899862, 5%: 0.301016130011458, 10%: 0.41550028905275704, 15%: 0.42698548174632983, 20%: 0.4619158598633485, 25%: 0.4772185474453734, 30%: 0.5161414587637939, 35%: 0.5349571090407734, 40%: 0.5674961881895051, 45%: 0.6236876512425283, 50%: 0.655518886621537, 55%: 0.6599831009037894, 60%: 0.7008729187348733, 65%: 0.8238009084839519, 70%: 0.9879905789567505, 75%: 1.0, 80%: 1.0, 85%: 1.0, 90%: 1.0, 95%: 1.0, 100%: 1.0,
Jun 07, 2014 7:20:13 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric milnewitten
Jun 07, 2014 7:20:14 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:20:14 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:20:14 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:20:14 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Exception in thread "main" org.apache.commons.math3.exception.NonMonotonicSequenceException: points 0 and 1 are not strictly increasing (-∞ >= -∞)
at org.apache.commons.math3.util.MathArrays.checkOrder(MathArrays.java:271)
at org.apache.commons.math3.util.MathArrays.checkOrder(MathArrays.java:288)
at org.apache.commons.math3.util.MathArrays.checkOrder(MathArrays.java:299)
at org.apache.commons.math3.analysis.interpolation.LinearInterpolator.interpolate(LinearInterpolator.java:56)
at org.wikibrain.sr.normalize.PercentileNormalizer.makeInterpolater(PercentileNormalizer.java:46)
at org.wikibrain.sr.normalize.PercentileNormalizer.observationsFinished(PercentileNormalizer.java:33)
at org.wikibrain.sr.utils.SrNormalizers.trainSimilarity(SrNormalizers.java:127)
at org.wikibrain.sr.BaseMonolingualSRMetric.trainSimilarity(BaseMonolingualSRMetric.java:196)
at org.wikibrain.sr.vector.VectorBasedMonoSRMetric.trainSimilarity(VectorBasedMonoSRMetric.java:210)
at org.wikibrain.sr.milnewitten.MilneWittenMetric.trainSimilarity(MilneWittenMetric.java:61)
at org.wikibrain.sr.SRBuilder.buildMetric(SRBuilder.java:205)
at org.wikibrain.sr.SRBuilder.build(SRBuilder.java:124)
at org.wikibrain.sr.SRBuilder.main(SRBuilder.java:499)
alan@alan-Lenovo-G470:~/.wikibrain$ git branch

  • fast-matrix
    fast-sql-links
    master
    alan@alan-Lenovo-G470:/.wikibrain$ git checkout master
    Switched to branch 'master'
    Your branch is ahead of 'origin/master' by 120 commits.
    (use "git push" to publish your local commits)
    alan@alan-Lenovo-G470:    /.wikibrain$ git branch
    fast-matrix
    fast-sql-links

  • master
    alan@alan-Lenovo-G470:~/.wikibrain$ ./wb-java.sh org.wikibrain.sr.SRBuilder -m ensemble
    Wikapidia environment settings follow. WB_CLASSPATH is updated again later

    WB_CLASSPATH='/home/alan/.wikibrain/lib/*'
    WB_CONF_PATHS='
    WB_DIR=/home/alan/.wikibrain
    WB_JAVA_BIN=java
    WB_JAVA_OPTS=' -server -ea'
    WB_LIB=/home/alan/.wikibrain/lib
    WB_MVN_TARGETS='compile install'
    WB_POM_PATHS='

Using maven pom /home/alan/.wikibrain/pom.xml
Jun 07, 2014 7:22:09 PM org.jooq.tools.JooqLogger info
INFO: License parameters
Jun 07, 2014 7:22:09 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:09 PM org.jooq.tools.JooqLogger info
INFO: Thank you for using jOOQ and jOOQ's code generator
Jun 07, 2014 7:22:09 PM org.jooq.tools.JooqLogger info
INFO:
Jun 07, 2014 7:22:09 PM org.jooq.tools.JooqLogger info
INFO: Database parameters
Jun 07, 2014 7:22:09 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: dialect : H2
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: target dir : /home/alan/.wikibrain/wikibrain-core/src/generated/jooq-h2
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: target package : org.wikibrain.core.jooq
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: includes : [.]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: excludes : []
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: includeExcludeColumns : false
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO:
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: DefaultGenerator parameters
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: strategy : class org.jooq.util.DefaultGeneratorStrategy
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: deprecated : false
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: generated annotation : true
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: JPA annotations : false
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: validation annotations : false
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: instance fields : true
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: records : false
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: pojos : false
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: immutable pojos : false
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: interfaces : false
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: daos : false
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: relations : true
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: global references : true
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO:
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generation remarks
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Emptying : /home/alan/.wikibrain/wikibrain-core/src/generated/jooq-h2/org/wikibrain/core/jooq
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating schemata : Total: 1
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating schema : Public.java
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Sequences fetched : 3 (3 included, 0 excluded)
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Tables fetched : 16 (16 included, 0 excluded)
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: UDTs fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating sequences
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: ARRAYs fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Enums fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Sequences generated : Total: 350.661ms
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating tables
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : WikidataEntityLabels.java [input=WIKIDATA_ENTITY_LABELS, output=WIKIDATA_ENTITY_LABELS, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : WikidataEntityDescriptions.java [input=WIKIDATA_ENTITY_DESCRIPTIONS, output=WIKIDATA_ENTITY_DESCRIPTIONS, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : WikidataEntityAliases.java [input=WIKIDATA_ENTITY_ALIASES, output=WIKIDATA_ENTITY_ALIASES, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : WikidataStatement.java [input=WIKIDATA_STATEMENT, output=WIKIDATA_STATEMENT, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : UniversalSkeletalLink.java [input=UNIVERSAL_SKELETAL_LINK, output=UNIVERSAL_SKELETAL_LINK, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : UniversalPage.java [input=UNIVERSAL_PAGE, output=UNIVERSAL_PAGE, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : UniversalLink.java [input=UNIVERSAL_LINK, output=UNIVERSAL_LINK, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : TableModified.java [input=TABLE_MODIFIED, output=TABLE_MODIFIED, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : Redirect.java [input=REDIRECT, output=REDIRECT, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : RawPage.java [input=RAW_PAGE, output=RAW_PAGE, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : Pageview.java [input=PAGEVIEW, output=PAGEVIEW, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : MetaInfo.java [input=META_INFO, output=META_INFO, pk=CONSTRAINT_6]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : LocalPage.java [input=LOCAL_PAGE, output=LOCAL_PAGE, pk=CONSTRAINT_6C]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : LocalLink.java [input=LOCAL_LINK, output=LOCAL_LINK, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : Ill.java [input=ILL, output=ILL, pk=N/A]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table : CategoryMembers.java [input=CATEGORY_MEMBERS, output=CATEGORY_MEMBERS, pk=CONSTRAINT_1]
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Tables generated : Total: 602.974ms, +252.312ms
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating table references
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Table refs generated : Total: 607.502ms, +4.527ms
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Generating Keys
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Keys generated : Total: 612.371ms, +4.869ms
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Routines fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: Packages fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:10 PM org.jooq.tools.JooqLogger info
INFO: GENERATION FINISHED! : Total: 621.763ms, +9.392ms
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: License parameters
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Thank you for using jOOQ and jOOQ's code generator
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO:
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Database parameters
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: dialect : H2
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: target dir : /home/alan/.wikibrain/wikibrain-core/src/generated/jooq-h2
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: target package : org.wikibrain.core.jooq
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: includes : [.]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: excludes : []
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: includeExcludeColumns : false
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO:
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: DefaultGenerator parameters
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: strategy : class org.jooq.util.DefaultGeneratorStrategy
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: deprecated : false
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: generated annotation : true
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: JPA annotations : false
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: validation annotations : false
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: instance fields : true
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: records : false
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: pojos : false
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: immutable pojos : false
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: interfaces : false
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: daos : false
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: relations : true
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: global references : true
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO:
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generation remarks
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Emptying : /home/alan/.wikibrain/wikibrain-core/src/generated/jooq-h2/org/wikibrain/core/jooq
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating schemata : Total: 1
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating schema : Public.java
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: ----------------------------------------------------------
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Sequences fetched : 3 (3 included, 0 excluded)
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Tables fetched : 16 (16 included, 0 excluded)
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: UDTs fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating sequences
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: ARRAYs fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Enums fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Sequences generated : Total: 31.954ms
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating tables
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : WikidataEntityLabels.java [input=WIKIDATA_ENTITY_LABELS, output=WIKIDATA_ENTITY_LABELS, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : WikidataEntityDescriptions.java [input=WIKIDATA_ENTITY_DESCRIPTIONS, output=WIKIDATA_ENTITY_DESCRIPTIONS, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : WikidataEntityAliases.java [input=WIKIDATA_ENTITY_ALIASES, output=WIKIDATA_ENTITY_ALIASES, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : WikidataStatement.java [input=WIKIDATA_STATEMENT, output=WIKIDATA_STATEMENT, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : UniversalSkeletalLink.java [input=UNIVERSAL_SKELETAL_LINK, output=UNIVERSAL_SKELETAL_LINK, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : UniversalPage.java [input=UNIVERSAL_PAGE, output=UNIVERSAL_PAGE, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : UniversalLink.java [input=UNIVERSAL_LINK, output=UNIVERSAL_LINK, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : TableModified.java [input=TABLE_MODIFIED, output=TABLE_MODIFIED, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : Redirect.java [input=REDIRECT, output=REDIRECT, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : RawPage.java [input=RAW_PAGE, output=RAW_PAGE, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : Pageview.java [input=PAGEVIEW, output=PAGEVIEW, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : MetaInfo.java [input=META_INFO, output=META_INFO, pk=CONSTRAINT_6]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : LocalPage.java [input=LOCAL_PAGE, output=LOCAL_PAGE, pk=CONSTRAINT_6C]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : LocalLink.java [input=LOCAL_LINK, output=LOCAL_LINK, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : Ill.java [input=ILL, output=ILL, pk=N/A]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table : CategoryMembers.java [input=CATEGORY_MEMBERS, output=CATEGORY_MEMBERS, pk=CONSTRAINT_1]
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Tables generated : Total: 149.098ms, +117.144ms
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating table references
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Table refs generated : Total: 154.036ms, +4.937ms
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Generating Keys
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Keys generated : Total: 158.764ms, +4.728ms
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Routines fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: Packages fetched : 0 (0 included, 0 excluded)
Jun 07, 2014 7:22:16 PM org.jooq.tools.JooqLogger info
INFO: GENERATION FINISHED! : Total: 163.914ms, +5.149ms
final WB_CLASSPATH is /home/alan/.wikibrain/wikibrain-phrases/target/classes:/home/alan/.wikibrain/wikibrain-cookbook/target/classes:/home/alan/.wikibrain/wikibrain-parser/target/classes:/home/alan/.wikibrain/wikibrain-loader/target/classes:/home/alan/.wikibrain/wikibrain-sr/target/classes:/home/alan/.wikibrain/wikibrain-matrix/target/classes:/home/alan/.wikibrain/wikibrain-core/target/classes:/home/alan/.wikibrain/wikibrain-wikidata/target/classes:/home/alan/.wikibrain/wikibrain-spatial/target/classes:/home/alan/.wikibrain/wikibrain-download/target/classes:/home/alan/.wikibrain/wikibrain-pageview/target/classes:/home/alan/.wikibrain/wikibrain-mapper/target/classes:/home/alan/.wikibrain/wikibrain-lucene/target/classes:/home/alan/.wikibrain/wikibrain-utils/target/classes:/home/alan/.wikibrain/lib/*
executing "java" -cp "/home/alan/.wikibrain/wikibrain-phrases/target/classes:/home/alan/.wikibrain/wikibrain-cookbook/target/classes:/home/alan/.wikibrain/wikibrain-parser/target/classes:/home/alan/.wikibrain/wikibrain-loader/target/classes:/home/alan/.wikibrain/wikibrain-sr/target/classes:/home/alan/.wikibrain/wikibrain-matrix/target/classes:/home/alan/.wikibrain/wikibrain-core/target/classes:/home/alan/.wikibrain/wikibrain-wikidata/target/classes:/home/alan/.wikibrain/wikibrain-spatial/target/classes:/home/alan/.wikibrain/wikibrain-download/target/classes:/home/alan/.wikibrain/wikibrain-pageview/target/classes:/home/alan/.wikibrain/wikibrain-mapper/target/classes:/home/alan/.wikibrain/wikibrain-lucene/target/classes:/home/alan/.wikibrain/wikibrain-utils/target/classes:/home/alan/.wikibrain/lib/*" -server -ea org.wikibrain.sr.SRBuilder -m ensemble
Jun 07, 2014 7:22:42 PM org.wikibrain.conf.Configurator registerProviders
INFO: configurator installed 71 providers for 39 classes
Jun 07, 2014 7:22:42 PM org.wikibrain.core.cmd.Env
INFO: using baseDir /home/alan/.wikibrain/.
Jun 07, 2014 7:22:42 PM org.wikibrain.core.cmd.Env
INFO: using max vm heapsize of 1763MB
SLF4J: The requested version 1.6.99 by your slf4j binding is not compatible with [1.5.5, 1.5.6, 1.5.7, 1.5.8, 1.5.9, 1.5.10]
SLF4J: See http://www.slf4j.org/codes.html#version_mismatch for further details.
Jun 07, 2014 7:22:45 PM org.wikibrain.core.lang.Language loadAllLanguages
INFO: loaded 285 languages
Jun 07, 2014 7:22:45 PM org.wikibrain.core.cmd.Env
INFO: using languages (SIMPLE)
Jun 07, 2014 7:22:45 PM org.wikibrain.core.cmd.Env
INFO: using maxThreads 4
Jun 07, 2014 7:22:45 PM org.wikibrain.core.cmd.Env
INFO: using tmpDir ./.tmp
Jun 07, 2014 7:22:45 PM org.wikibrain.sr.SRBuilder deleteDataDirectories
INFO: deleting metric directory ./dat/sr/ESA/simple
Jun 07, 2014 7:22:45 PM org.wikibrain.sr.SRBuilder deleteDataDirectories
INFO: deleting metric directory ./dat/sr/inlink/simple
Jun 07, 2014 7:22:45 PM org.wikibrain.sr.SRBuilder deleteDataDirectories
INFO: deleting metric directory ./dat/sr/outlink/simple
Jun 07, 2014 7:22:45 PM org.wikibrain.sr.SRBuilder deleteDataDirectories
INFO: deleting metric directory ./dat/sr/category/simple
Jun 07, 2014 7:22:45 PM org.wikibrain.sr.SRBuilder build
INFO: building metric ensemble
LOADING LANGUAGES (SIMPLE)
Jun 07, 2014 7:22:46 PM org.wikibrain.sr.vector.ESAGenerator setConcepts
WARNING: installed 47036 concepts for Simple English
Jun 07, 2014 7:22:46 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./db/matrix/local-link/links.matrix: initializing sparse matrix with file length 34115088
Jun 07, 2014 7:22:46 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./db/matrix/local-link/links-transpose.matrix: initializing sparse matrix with file length 30548896
LOADING LANGUAGES (SIMPLE)
Jun 07, 2014 7:22:48 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/inlinknotrain/simple does not exist; cannot read files
Jun 07, 2014 7:22:48 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric inlinknotrain
Jun 07, 2014 7:22:48 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/ESA/simple does not exist; cannot read files
Jun 07, 2014 7:22:48 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric ESA
Jun 07, 2014 7:22:48 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/inlink/simple does not exist; cannot read files
Jun 07, 2014 7:22:48 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric inlink
Jun 07, 2014 7:22:48 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/outlink/simple does not exist; cannot read files
Jun 07, 2014 7:22:48 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric outlink
Jun 07, 2014 7:22:49 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/category/simple does not exist; cannot read files
Jun 07, 2014 7:22:49 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric category
Jun 07, 2014 7:22:49 PM org.wikibrain.sr.BaseMonolingualSRMetric read
WARNING: directory ./dat/sr/ensemble/simple does not exist; cannot read files
Jun 07, 2014 7:22:49 PM org.wikibrain.sr.BaseMonolingualSRMetric configureBase
INFO: finished base configuration of metric ensemble
Jun 07, 2014 7:22:49 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric ESA
Jun 07, 2014 7:22:50 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:22:54 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:22:54 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:22:55 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:22:55 PM org.wikibrain.sr.utils.SrNormalizers trainSimilarity
INFO: trained similarity normalizer: percentile normalizer: 0%: 0.0, 5%: 0.0, 10%: 0.0010428927597552194, 15%: 0.002797699130562242, 20%: 0.004964782901075486, 25%: 0.006638796583043898, 30%: 0.00944920620140384, 35%: 0.012077226625145043, 40%: 0.014214023691574672, 45%: 0.022498378338432098, 50%: 0.029367082282675754, 55%: 0.03575526436975566, 60%: 0.04270472471026882, 65%: 0.049782773633454395, 70%: 0.06037381496579908, 75%: 0.07884587648414584, 80%: 0.09268273456332982, 85%: 0.12105819488153641, 90%: 0.1431009186443515, 95%: 0.20107537409473883, 100%: 1.0,
Jun 07, 2014 7:22:57 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: writing matrix to ./dat/sr/ESA/simple/feature.matrix
Jun 07, 2014 7:22:57 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: writing body to tmp file at ./.tmp/matrix6902607913041582598.tmp
Jun 07, 2014 7:22:57 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 104075
Jun 07, 2014 7:23:56 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 10001 of 104075
Jun 07, 2014 7:25:00 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 20001 of 104075
Jun 07, 2014 7:26:24 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 30001 of 104075
Jun 07, 2014 7:27:40 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 40001 of 104075
Jun 07, 2014 7:28:43 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 50001 of 104075
Jun 07, 2014 7:29:19 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 60001 of 104075
Jun 07, 2014 7:30:30 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 70001 of 104075
Jun 07, 2014 7:31:40 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 80001 of 104075
Jun 07, 2014 7:32:49 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 90001 of 104075
Jun 07, 2014 7:34:07 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 100001 of 104075
Jun 07, 2014 7:34:42 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: wrote 330839312 bytes in body of matrix
Jun 07, 2014 7:34:42 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: generating header
Jun 07, 2014 7:34:43 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/feature.matrix: wrote 332085588 bytes to ./dat/sr/ESA/simple/feature.matrix
Jun 07, 2014 7:34:43 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/ESA/simple/feature.matrix: initializing sparse matrix with file length 332085588
Jun 07, 2014 7:34:43 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: writing matrix to ./dat/sr/ESA/simple/featureTranspose.matrix
Jun 07, 2014 7:34:43 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: writing body to tmp file at ./.tmp/matrix3987284775785542502.tmp
Jun 07, 2014 7:34:45 PM org.wikibrain.matrix.SparseMatrixTransposer countCellsPerColumn
INFO: found 47036 unique column ids in matrix
Jun 07, 2014 7:34:46 PM org.wikibrain.matrix.SparseMatrixTransposer accumulateBatch
INFO: processing 47036 columns in batch (total=47036 of 47036)
Jun 07, 2014 7:35:15 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: wrote 329971616 bytes in body of matrix
Jun 07, 2014 7:35:15 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: generating header
Jun 07, 2014 7:35:16 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/ESA/simple/featureTranspose.matrix: wrote 330536064 bytes to ./dat/sr/ESA/simple/featureTranspose.matrix
Jun 07, 2014 7:35:16 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/ESA/simple/featureTranspose.matrix: initializing sparse matrix with file length 330536064
Jun 07, 2014 7:35:16 PM org.wikibrain.sr.vector.CosineSimilarity setMatrices
INFO: building cached matrix information
Jun 07, 2014 7:35:18 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:35:26 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:35:33 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:35:38 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:35:44 PM org.wikibrain.sr.utils.SrNormalizers trainMostSimilar
INFO: trained most similar normalizer: percentile normalizer: 0%: 0.03806834001287851, 5%: 0.07230208870354823, 10%: 0.09729919426873596, 15%: 0.11382369656471823, 20%: 0.1486585819643594, 25%: 0.15456552508987131, 30%: 0.17497071320555754, 35%: 0.2037664466614659, 40%: 0.2301053658970053, 45%: 0.25247812827177424, 50%: 0.2770597830449147, 55%: 0.2913874323425757, 60%: 0.30812548297052356, 65%: 0.3510441710344808, 70%: 0.37547960438188677, 75%: 0.3991878529587802, 80%: 0.42902745259406816, 85%: 0.47847058925126845, 90%: 0.5296081068281957, 95%: 0.999999980387662, 100%: 1.000000017449774,
Jun 07, 2014 7:35:44 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric inlink
Jun 07, 2014 7:35:44 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:35:44 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:35:44 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:35:44 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:35:45 PM org.wikibrain.sr.utils.SrNormalizers trainSimilarity
INFO: trained similarity normalizer: percentile normalizer: 0%: -0.1959001790179249, 5%: 0.0, 10%: 0.0, 15%: 0.0, 20%: 0.0, 25%: 0.0, 30%: 0.2090231768812829, 35%: 0.2986806914206396, 40%: 0.34692589727836176, 45%: 0.3876536072672815, 50%: 0.43002469459993864, 55%: 0.44991621926451897, 60%: 0.46847167486286745, 65%: 0.4847155367419187, 70%: 0.5160922856806431, 75%: 0.5358847244967025, 80%: 0.5614463113354178, 85%: 0.5982364416362632, 90%: 0.6357638696189806, 95%: 0.7164758980904506, 100%: 1.0,
Jun 07, 2014 7:35:45 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: writing matrix to ./dat/sr/inlink/simple/feature.matrix
Jun 07, 2014 7:35:45 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: writing body to tmp file at ./.tmp/matrix1825832418235305019.tmp
Jun 07, 2014 7:35:45 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 10001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 20001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 30001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 40001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 50001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 60001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 70001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 80001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 90001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 100001 of 104075
Jun 07, 2014 7:35:46 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: wrote 29368576 bytes in body of matrix
Jun 07, 2014 7:35:46 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: generating header
Jun 07, 2014 7:35:46 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/feature.matrix: wrote 30545684 bytes to ./dat/sr/inlink/simple/feature.matrix
Jun 07, 2014 7:35:46 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/inlink/simple/feature.matrix: initializing sparse matrix with file length 30545684
Jun 07, 2014 7:35:46 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: writing matrix to ./dat/sr/inlink/simple/featureTranspose.matrix
Jun 07, 2014 7:35:46 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: writing body to tmp file at ./.tmp/matrix9179879541739114757.tmp
Jun 07, 2014 7:35:47 PM org.wikibrain.matrix.SparseMatrixTransposer countCellsPerColumn
INFO: found 221718 unique column ids in matrix
Jun 07, 2014 7:35:47 PM org.wikibrain.matrix.SparseMatrixTransposer accumulateBatch
INFO: processing 221718 columns in batch (total=221718 of 221718)
Jun 07, 2014 7:35:50 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: wrote 31453120 bytes in body of matrix
Jun 07, 2014 7:35:50 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: generating header
Jun 07, 2014 7:35:50 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/inlink/simple/featureTranspose.matrix: wrote 34113752 bytes to ./dat/sr/inlink/simple/featureTranspose.matrix
Jun 07, 2014 7:35:50 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/inlink/simple/featureTranspose.matrix: initializing sparse matrix with file length 34113752
Jun 07, 2014 7:35:50 PM org.wikibrain.sr.vector.GoogleSimilarity setMatrices
INFO: building cached matrix information
found 98091 and 221718
Jun 07, 2014 7:35:51 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:35:52 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:35:52 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:35:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:35:53 PM org.wikibrain.sr.utils.SrNormalizers trainMostSimilar
INFO: trained most similar normalizer: percentile normalizer: 0%: 0.3364241413430419, 5%: 0.4323502541717601, 10%: 0.4495926291070297, 15%: 0.45809359791838866, 20%: 0.4798782942618601, 25%: 0.5062332185796676, 30%: 0.5178307795948953, 35%: 0.5288921401344702, 40%: 0.5406302023667577, 45%: 0.5614463113354178, 50%: 0.5743620412183215, 55%: 0.5931769019295817, 60%: 0.6001656614724569, 65%: 0.622430456712793, 70%: 0.6500507002569338, 75%: 0.6578122411984024, 80%: 0.6864749400174084, 85%: 0.7317678171723871, 90%: 0.7889446627666747, 95%: 1.0, 100%: 1.0,
Jun 07, 2014 7:35:53 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric outlink
Jun 07, 2014 7:35:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:35:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:35:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:35:53 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:35:53 PM org.wikibrain.sr.utils.SrNormalizers trainSimilarity
INFO: trained similarity normalizer: percentile normalizer: 0%: 0.0, 5%: 0.0, 10%: 0.0, 15%: 0.0, 20%: 0.0, 25%: 0.0, 30%: 0.0, 35%: 0.0, 40%: 0.0, 45%: 0.0, 50%: 0.0, 55%: 0.0010634160873798635, 60%: 0.014066008354108853, 65%: 0.028227379351713443, 70%: 0.042185959642105725, 75%: 0.05433789210629009, 80%: 0.07424672494408206, 85%: 0.09719121719433076, 90%: 0.13814993163480477, 95%: 0.19457648600407124, 100%: 1.0,
Jun 07, 2014 7:35:54 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: writing matrix to ./dat/sr/outlink/simple/feature.matrix
Jun 07, 2014 7:35:54 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: writing body to tmp file at ./.tmp/matrix5575559796660841538.tmp
Jun 07, 2014 7:35:54 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 104075
Jun 07, 2014 7:35:59 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 10001 of 104075
Jun 07, 2014 7:36:00 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 20001 of 104075
Jun 07, 2014 7:36:00 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 30001 of 104075
Jun 07, 2014 7:36:00 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 40001 of 104075
Jun 07, 2014 7:36:00 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 50001 of 104075
Jun 07, 2014 7:36:01 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 60001 of 104075
Jun 07, 2014 7:36:01 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 70001 of 104075
Jun 07, 2014 7:36:01 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 80001 of 104075
Jun 07, 2014 7:36:01 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 90001 of 104075
Jun 07, 2014 7:36:02 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 100001 of 104075
Jun 07, 2014 7:36:02 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: wrote 27293400 bytes in body of matrix
Jun 07, 2014 7:36:02 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: generating header
Jun 07, 2014 7:36:02 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/feature.matrix: wrote 28537540 bytes to ./dat/sr/outlink/simple/feature.matrix
Jun 07, 2014 7:36:02 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/outlink/simple/feature.matrix: initializing sparse matrix with file length 28537540
Jun 07, 2014 7:36:02 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: writing matrix to ./dat/sr/outlink/simple/featureTranspose.matrix
Jun 07, 2014 7:36:02 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: writing body to tmp file at ./.tmp/matrix2942751139907455204.tmp
Jun 07, 2014 7:36:02 PM org.wikibrain.matrix.SparseMatrixTransposer countCellsPerColumn
INFO: found 93378 unique column ids in matrix
Jun 07, 2014 7:36:02 PM org.wikibrain.matrix.SparseMatrixTransposer accumulateBatch
INFO: processing 93378 columns in batch (total=93378 of 93378)
Jun 07, 2014 7:36:04 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: wrote 27166640 bytes in body of matrix
Jun 07, 2014 7:36:04 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: generating header
Jun 07, 2014 7:36:04 PM org.wikibrain.matrix.SparseMatrixWriter info
INFO: sparse matrix writer ./dat/sr/outlink/simple/featureTranspose.matrix: wrote 28287192 bytes to ./dat/sr/outlink/simple/featureTranspose.matrix
Jun 07, 2014 7:36:04 PM org.wikibrain.matrix.SparseMatrix info
INFO: sparse matrix ./dat/sr/outlink/simple/featureTranspose.matrix: initializing sparse matrix with file length 28287192
Jun 07, 2014 7:36:04 PM org.wikibrain.sr.vector.CosineSimilarity setMatrices
INFO: building cached matrix information
Jun 07, 2014 7:36:05 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:36:05 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:36:05 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:36:05 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:36:05 PM org.wikibrain.sr.utils.SrNormalizers trainMostSimilar
INFO: trained most similar normalizer: percentile normalizer: 0%: 0.030855840901534034, 5%: 0.04110007219482489, 10%: 0.04566202274144261, 15%: 0.050128325541606476, 20%: 0.05653285792509948, 25%: 0.06402750777568866, 30%: 0.07207769552276821, 35%: 0.07589431757171602, 40%: 0.08364461911338306, 45%: 0.09393981974771025, 50%: 0.09974933096984956, 55%: 0.1116174623953275, 60%: 0.12695458255487663, 65%: 0.13772847160295723, 70%: 0.14794665628706827, 75%: 0.16297589809609, 80%: 0.17699658365757923, 85%: 0.31464545884332495, 90%: 0.5625431451457016, 95%: 1.000000000297556, 100%: 1.0000000529764081,
Jun 07, 2014 7:36:05 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric category
Jun 07, 2014 7:36:06 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:36:06 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:36:07 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:36:07 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:36:08 PM org.wikibrain.sr.utils.SrNormalizers trainSimilarity
INFO: trained similarity normalizer: percentile normalizer: 0%: -0.30198376259386606, 5%: -0.12954166044106635, 10%: -0.09402034093692634, 15%: -0.05028864565865285, 20%: -0.02374797305117326, 25%: 0.0, 30%: 0.011578245863244758, 35%: 0.037628826027640554, 40%: 0.05445892883872089, 45%: 0.07985638686760554, 50%: 0.10158866415743976, 55%: 0.16064264250147148, 60%: 0.21956672026531207, 65%: 0.28256897707432455, 70%: 0.41550028905275715, 75%: 0.46870829471860054, 80%: 0.535348294419165, 85%: 0.655518886621537, 90%: 0.8821017938213499, 95%: 1.0, 100%: 1.0,
Jun 07, 2014 7:36:08 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:36:08 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:36:08 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:36:08 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Jun 07, 2014 7:36:08 PM org.wikibrain.sr.utils.SrNormalizers trainMostSimilar
INFO: trained most similar normalizer: percentile normalizer: 0%: 0.2015018403580326, 5%: 0.301016130011458, 10%: 0.4260765340986724, 15%: 0.43609264244178547, 20%: 0.4721265530690827, 25%: 0.484171269560965, 30%: 0.5224628271750432, 35%: 0.5438241326610264, 40%: 0.6205374252806691, 45%: 0.6424007485185562, 50%: 0.655518886621537, 55%: 0.6665553203681329, 60%: 0.7171321673306638, 65%: 0.8821017938213499, 70%: 1.0, 75%: 1.0, 80%: 1.0, 85%: 1.0, 90%: 1.0, 95%: 1.0, 100%: 1.0,
Jun 07, 2014 7:36:08 PM org.wikibrain.sr.SRBuilder buildMetric
INFO: building component metric milnewitten
Jun 07, 2014 7:36:09 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 1 of 381
Jun 07, 2014 7:36:09 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 101 of 381
Jun 07, 2014 7:36:09 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 201 of 381
Jun 07, 2014 7:36:09 PM org.wikibrain.utils.ParallelForEach$3 run
INFO: processing list element 301 of 381
Exception in thread "main" org.apache.commons.math3.exception.NonMonotonicSequenceException: points 0 and 1 are not strictly increasing (-∞ >= -∞)
at org.apache.commons.math3.util.MathArrays.checkOrder(MathArrays.java:271)
at org.apache.commons.math3.util.MathArrays.checkOrder(MathArrays.java:288)
at org.apache.commons.math3.util.MathArrays.checkOrder(MathArrays.java:299)
at org.apache.commons.math3.analysis.interpolation.LinearInterpolator.interpolate(LinearInterpolator.java:56)
at org.wikibrain.sr.normalize.PercentileNormalizer.makeInterpolater(PercentileNormalizer.java:46)
at org.wikibrain.sr.normalize.PercentileNormalizer.observationsFinished(PercentileNormalizer.java:33)
at org.wikibrain.sr.utils.SrNormalizers.trainSimilarity(SrNormalizers.java:127)
at org.wikibrain.sr.BaseMonolingualSRMetric.trainSimilarity(BaseMonolingualSRMetric.java:196)
at org.wikibrain.sr.vector.VectorBasedMonoSRMetric.trainSimilarity(VectorBasedMonoSRMetric.java:210)
at org.wikibrain.sr.milnewitten.MilneWittenMetric.trainSimilarity(MilneWittenMetric.java:61)
at org.wikibrain.sr.SRBuilder.buildMetric(SRBuilder.java:205)
at org.wikibrain.sr.SRBuilder.build(SRBuilder.java:124)
at org.wikibrain.sr.SRBuilder.main(SRBuilder.java:499)`

shilad commented

I think this was due to a version mismatch fixed on 0.5.1. I'm closing since I haven't heard anybody reproduce this recently.