michaelklishin/monger

Freeze on `ensure-index`

pholz opened this issue · 1 comments

pholz commented

My application occasionally freezes when calling ensure-index on a db which seems to be connected. In the code below, I call connect and then initialize-indexes, and "initializing indexes" is the last output I see. The code never reaches the next info output in the function that called initialize-indexes. I cannot reproduce this in the REPL, or locally, it happens once every few weeks with the uberjar on the production machine. There is no error, the program does not exit, it just freezes completely and needs to be restarted.

monger 3.1.0, mongodb 5 in a docker container.

What is the best way to debug this?

(defn connect []
  (reset! gldb (let [conn (mg/connect-with-credentials
                           "127.0.0.1"
                           (mcr/create "mySrv" "admin" (.toCharArray "myPass")))]
                 (info "connecting to db: " conn)
                 (mg/get-db conn "myDb"))))

(defn initialize-indexes []
  (info "initalizing indexes")
  (mc/ensure-index @gldb "weather-monthly" (array-map :time 1 :station 1) {:unique true})
  (mc/ensure-index @gldb "weather-10min" (array-map :time 1 :station 1) {:unique true})
  (mc/ensure-index @gldb "airq" (array-map :time 1 :station 1) {:unique true})

  (mc/ensure-index @gldb "grid-wind-s" (array-map :time 1) {:unique true})
  (mc/ensure-index @gldb "grid-wind-m" (array-map :time 1) {:unique true})
  (mc/ensure-index @gldb "grid-wind-l" (array-map :time 1) {:unique true}))

Server logs or a traffic capture. An index can take a while to build.