ubleipzig/marctools

marcmap index out of range

Closed this issue · 4 comments

download

$ wget http://natlib.govt.nz/files/innz/innzmetadata.mrc.gz
$ gunzip innzmetadata.mrc.gz

count

$ marccount innzmetadata.mrc
804276

db

$ marcdb -o innz.db innzmetadata.mrc
panic: runtime error: index out of range

goroutine 16 [running]:
runtime.panic(0x424fbc0, 0x437549c)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/panic.c:279 +0xf5
github.com/miku/marctools.MarcMap(0x7fff5fbff9ea, 0x10, 0x4436528, 0xc2080289a0)
    /Users/void/devel/golang/src/github.com/miku/marctools/common.go:243 +0x40f
main.main()
    /Users/void/devel/golang/src/github.com/ubleipzig/marctools/cmd/marcdb/marcdb.go:55 +0x4de

goroutine 19 [finalizer wait]:
runtime.park(0x40c1030, 0x4385f78, 0x4377b49)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x4385f78, 0x4377b49)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/proc.c:1445

marcmap

$ marcmap innzmetadata.mrc
panic: runtime error: index out of range

goroutine 16 [running]:
runtime.panic(0x424f0c0, 0x437449c)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/panic.c:279 +0xf5
github.com/miku/marctools.MarcMap(0x7fff5fbff9e8, 0x10, 0x4435340, 0xc20804e008)
    /Users/void/devel/golang/src/github.com/miku/marctools/common.go:243 +0x40f
main.main()
    /Users/void/devel/golang/src/github.com/ubleipzig/marctools/cmd/marcmap/marcmap.go:50 +0x3fc

goroutine 19 [finalizer wait]:
runtime.park(0x40bfbe0, 0x4384f78, 0x4376b49)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x4384f78, 0x4376b49)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/proc.c:1445

i tested also splitting the whole file into several small file, same result

env

go env
GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/void/devel/golang"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.3.3/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.3.3/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"
miku commented

Tried to reproduce:

$ marccount innzmetadata.mrc
804276
$ marcdb -o innz.db innzmetadata.mrc
$ sqlite3 innz.db
SQLite version 3.7.12 2012-04-03 19:43:07
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .schema
CREATE TABLE store (id TEXT, secondary TEXT, record BLOB, PRIMARY KEY (id, secondary));
CREATE INDEX idx_store_id ON store (id);
sqlite> select count(*) from store;
804276

$ marcmap innzmetadata.mrc > innz.map
$ wc -l innz.map
804276 innz.map
$ head -2 innz.map
3   0   827
4   827 609

But innzmetadata.mrc.gz seems to work. Env seems to be the same, too:

GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/tir/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.3.3/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.3.3/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"

I used the latest marcdb, version 1.5.5 (did you?):

$ marcdb -v
1.5.5

marcdb 1.5.5 for me too
i'll try on another linux machine

miku commented

Tried it on Ubuntu 14.04 myself, but I could not reproduce with the above innzmetadata.mrc. Strange.

it works fine on a linux machine.
don't mind, thanks.