ylongqi/openrec

Getting ResourceExhaustedError

akshitj1 opened this issue · 1 comments

Hi I am trying to run BPR on my test dataset.
max_users are: 326608
max_items are: 458334
I was referring to the tutorial "OpenRec Tutorial #1"
12 Gigs of GPU memory and 64 G RAM.

But while trying to build BPR model with batch size 100 it gives ResourceExhaustedError.

Here is the part of last stack trace:

File "/mnt/data/virtualenv/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
    op_def=op_def)
  File "/mnt/data/virtualenv/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1718, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

ResourceExhaustedError (see above for traceback): OOM when allocating tensor of shape [458334,20] and type float
	 [[Node: item/embedding/Adam/Initializer/zeros = Const[dtype=DT_FLOAT, value=Tensor<type: float shape: [458334,20] values: [0 0 0]...>, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Does this library dosen't yet supports this size of input or I am doing something wrong. Can you please help.

really sorry. some process was taking up memory. Currently trying on smaller dataset. Will reopen in case fails on original dataset.