tc/RMongo

RMONGO with streaming data

louisdv1 opened this issue · 6 comments

louisdv1 commented

Hi , I am pulling data out of an API with R and it seems that the insert to mongodb keeps failing?
This is the message I am having:
Error in .jcall(rmongo.object@javaMongo, "S", "dbInsertDocument", collection, :
com.mongodb.util.JSONParseException:

tc commented

What query command are you running? what does your mongo collection look like?

louisdv1 commented

Hi Tommy and thank you for the fast response.
I am pulling data out of stocktwits and pushing it to mongodb.
Sometimes I have a parsing problem but the most problematic is that the
json feed representing several messages seem to be bundled up together so
that mongo only recognizes one message per call .
Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url="
https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo =
"cacert.pem")},error=function(e){})

if (length(result)>0)  dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much
On 1 May 2013 17:26, "Tommy Chheng" notifications@github.com wrote:

What query command are you running? what does your mongo collection look
like?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/18#issuecomment-17290176
.

tc commented

What is z in dbInsertDocument(mongo, "stocktwits7", z)? It looks like an unreferenced variable.

tc commented

You can also use rjson to extract the messages array out of the url
response.

On Wed, May 1, 2013 at 11:01 AM, louisdv1 notifications@github.com wrote:

Hi Tommy and thank you for the fast response.
I am pulling data out of stocktwits and pushing it to mongodb.
Sometimes I have a parsing problem but the most problematic is that the
json feed representing several messages seem to be bundled up together so
that mongo only recognizes one message per call .
Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url="
https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo =
"cacert.pem")},error=function(e){})

if (length(result)>0) dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much
On 1 May 2013 17:26, "Tommy Chheng" notifications@github.com wrote:

What query command are you running? what does your mongo collection look
like?

—
Reply to this email directly or view it on GitHub<
https://github.com/tc/RMongo/issues/18#issuecomment-17290176>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/18#issuecomment-17296054
.

Tommy Chheng

louisdv1 commented

Stocktwits is just the name of tge collection and z is a json response
fetched by rcurl
On 1 May 2013 20:39, "Tommy Chheng" notifications@github.com wrote:

You can also use rjson to extract the messages array out of the url
response.

On Wed, May 1, 2013 at 11:01 AM, louisdv1 notifications@github.com
wrote:

Hi Tommy and thank you for the fast response.
I am pulling data out of stocktwits and pushing it to mongodb.
Sometimes I have a parsing problem but the most problematic is that the
json feed representing several messages seem to be bundled up together
so
that mongo only recognizes one message per call .
Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url="
https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo =
"cacert.pem")},error=function(e){})

if (length(result)>0) dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much
On 1 May 2013 17:26, "Tommy Chheng" notifications@github.com wrote:

What query command are you running? what does your mongo collection
look
like?

—
Reply to this email directly or view it on GitHub<
https://github.com/tc/RMongo/issues/18#issuecomment-17290176>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/tc/RMongo/issues/18#issuecomment-17296054>
.

Tommy Chheng

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/18#issuecomment-17301621
.

tc commented

Try printing out the z variable to see if it's malformed. That could be
the reason why it's not inserting correctly.

On Wed, May 1, 2013 at 12:42 PM, louisdv1 notifications@github.com wrote:

Stocktwits is just the name of tge collection and z is a json response
fetched by rcurl
On 1 May 2013 20:39, "Tommy Chheng" notifications@github.com wrote:

You can also use rjson to extract the messages array out of the url
response.

On Wed, May 1, 2013 at 11:01 AM, louisdv1 notifications@github.com
wrote:

Hi Tommy and thank you for the fast response.
I am pulling data out of stocktwits and pushing it to mongodb.
Sometimes I have a parsing problem but the most problematic is that
the
json feed representing several messages seem to be bundled up together
so
that mongo only recognizes one message per call .
Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url="
https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo =
"cacert.pem")},error=function(e){})

if (length(result)>0) dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much
On 1 May 2013 17:26, "Tommy Chheng" notifications@github.com wrote:

What query command are you running? what does your mongo collection
look
like?

—
Reply to this email directly or view it on GitHub<
https://github.com/tc/RMongo/issues/18#issuecomment-17290176>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/tc/RMongo/issues/18#issuecomment-17296054>
.

Tommy Chheng

—
Reply to this email directly or view it on GitHub<
https://github.com/tc/RMongo/issues/18#issuecomment-17301621>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/18#issuecomment-17301832
.

Tommy Chheng