Rangozhang/VideoCaption

Error using eval.lua on a video

Opened this issue · 9 comments

oxmah commented

Hi

I discovered your work on VideoCaption and neuraltalk2 while working on a documentary about respublica Tuva which is a small country, near to Mongolia, federated by Russia. The movie itself is about a brief history of Tuva colonisation and, thought this history, we're trying to explore the relationship between subject and object, in art, as authorship. How russian historians became authors of a Tuvanian history and how the process of making a documentary can be a process of writing a fiction. I'm trying to use neuraltalk2 as a translation process in the project and VideoCaption looks really insteresting in that way but I'm stucked with an error that I can't manage to solve alone. Could you help me a bit ?

When doing that :

th ./eval.lua -model ./model -image_folder ./videos -gpuid 0 -num_images 100 >> ./videos/reel.txt

I get that :

/home/workshop/torch/install/bin/luajit: ./layer/LanguageModel.lua:54: attempt to perform arithmetic on field 'frame_length' (a nil value)
stack traceback:
./layer/LanguageModel.lua:54: in function 'createClones'
./eval.lua:104: in main chunk
[C]: in function 'dofile'
...shop/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x00405d50

The model used is from neuraltalk2 doc. Also, the videofile is a .mp4/h264 file.

Regards.

M

Rangozhang commented

Hi

You guys are doing something interesting! Sorry for the messy code, my apologies.

As for the "frame_length" issue, I think it's probably due to the fact that the neuraltalk model is directly used. Our model is different from neuraltalk's model although ours is based on theirs in a sense that neuraltalk model doesn't have the variable "frame_length" with it. This is because neural talk tackles image while we're dealing with videos. The variable "frame_length" is used for indicating how many consecutive frames of the video are we going to feed to the neural network as input.

Best Regards,
Yu Zhang

oxmah commented

Hi ! Thanks for the fast answer ! Is there a way to get the model you use
somehow ?

M
Le 31 août 2016 21:04, "Yu Zhang" notifications@github.com a écrit :

Hi

You guys are doing something interesting! Sorry for the messy code, my
apologies.

As for the "frame_length" issue, I think it's probably due to the fact
that the neuraltalk model is directly used. Our model is different from
neuraltalk's model although ours is based on theirs in a sense that
neuraltalk model doesn't have the variable "frame_length" with it. This is
because neural talk tackles image while we're dealing with videos. The
variable "frame_length" is used for indicating how many consecutive frames
of the video are we going to feed to the neural network as input.

Best Regards,
Yu Zhang

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/APQ4WETuq3XYoLUPag0m9fuP5GvwxkmGks5qldA1gaJpZM4JxksP
.

Rangozhang commented

Hi TRGNN,
I'm sorry the model has already removed by reinstalling Ubuntu due to a security issue this summer. You could probably use the train.lua to train it with the help of MSVD dataset.
Yu

oxmah commented

It's okay. I'll launch it tonight and I'll upload it in a few days for
futur users. In the mean time, could you help me a bit with this dataset ?
Is there something to know about how to train it specifically ? Should I
just follow the neuraltalk2 docs here ?
Le 31 août 2016 21:32, "Yu Zhang" notifications@github.com a écrit :

Hi TRGNN,

I'm sorry the model has already removed by reinstalling Ubuntu due to a security issue this summer. You could probably use the train.lua to train it with the help of MSVD dataset.

Yu

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/APQ4WKkrG9uPD9UgjZn98QMOmVknQWR7ks5qldbigaJpZM4JxksP
.

oxmah commented

In the meantime, I downloaded pretrained models from here (https://www.cs.utexas.edu/~vsub/s2vt.html) end here (https://www.cs.utexas.edu/~vsub/naacl15_project.html) and I did some tests. But I get another error. I don't know if it come from the model or from some specific dependencies I didn't already installed.

th ./eval.lua -model ./model3 -image_folder ./videos -gpuid 0 -num_images 100 >> ./videos/reel.txt

/home/workshop/torch/install/bin/luajit: /home/workshop/torch/install/share/lua/5.1/torch/File.lua:375: unknown object
stack traceback:
    [C]: in function 'error'
    /home/workshop/torch/install/share/lua/5.1/torch/File.lua:375: in function 'readObject'
    /home/workshop/torch/install/share/lua/5.1/torch/File.lua:409: in function 'load'
    ./eval.lua:73: in main chunk
    [C]: in function 'dofile'
    ...shop/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
    [C]: at 0x00405d50
erichuang0771 commented

Hi TRGNN,

Sorry for the late reply, I wonder how things go right now? I implemented the data preprocessing and first part of project. Is there anything I can help you out?

oxmah commented

Well, I'm still completely lost with this and the training process. I took a look to MSVD dataset but it seems to be strangely small, isn't it ? It's a dataset that can be used for translation algorithm ?

https://www.microsoft.com/en-us/download/details.aspx?id=52422

If somebody can help me to setup a proper model for this, I'll be super grateful !

oxmah commented
Rangozhang commented

That sounds super cool! Looking forward to the web publishing! :)