geoffleyland/lua-csv

Large text comment in field causing parse error

lateefj opened this issue · 2 comments

I downloaded a big pile of reddit comments. Now I am trying to parse the csv file and I am getting an error ":5:91: unmatched quote". I think this is because the comment in the csv file is very large. I have pasted a section of the csv file below:

1430438438,1,t5_2qmg3,t3_34gjek,t1_cqug9su,0,broncos,Broncos,nfl,cqug9su,,0,0,0,tnick771,1,1432703089,"ALRIGHT ALRIGHT ALRIGHT ALRIGHT",,0,0,t3_34gjek
1430438438,2,t5_2qh1i,t3_34gktp,t1_cqug9sv,0,,,AskReddit,cqug9sv,,0,0,0,Toxic_Widow,2,1432703089,"You never forget your first. Her eyes were so beautiful I had to keep them.",,0,0,t3_34gktp
1430438438,3,t5_2qo4s,t3_34g8mx,t1_cqug9sw,0,Bucks2,Bucks,nba,cqug9sw,,0,0,0,yungtito,3,1432703089,"at least we have giannis",,0,0,t3_34g8mx
1430438438,1,t5_2qur2,t3_34cvek,t1_cqug9sx,0,,,excel,cqug9sx,,0,0,0,22Mugatu,1,1432703089,"I got everything to work except that Every Cell in the ranges is a hyperlink, but the hyperlinks don't ""come""with the new cell:


Sub IF_Else_IF_Function()
Dim num As Integer
Selection = Worksheets(""Source"").Range(""B2"")

If Selection = ""1"" Then
    Worksheets(""HomePage"").Range(""A10:A18"").Value = Worksheets(""Source"").Range(""F2:F10"").Value
ElseIf Selection = 2 Then
    Worksheets(""HomePage"").Range(""A10:A18"").Value = Worksheets(""Source"").Range(""C2:C10"").Value
ElseIf Selection = 3 Then
    Worksheets(""HomePage"").Range(""A10:A18"").Value = Worksheets(""Source"").Range(""D2:D10"").Value
ElseIf Selection = 4 Then
    Worksheets(""HomePage"").Range(""A10:A18"").Value = Worksheets(""Source"").Range(""E2:E10"").Value
ElseIf Selection = 5 Then
    Worksheets(""HomePage"").Range(""A10:A18"").Value = Worksheets(""Source"").Range(""F2:F10"").Value
Else
Cells(10, 10) = ""Fart""

End If
End Sub",,0,0,t1_cqtolo2
1430438438,1,t5_2sf6m,t3_34gj05,t1_cqug9sy,0,,,pathofexile,cqug9sy,,0,0,0,xEldora,1,1432703089,"Apart from all the in-game mechanics, I want to test out the new client, I'm running a horrible crappy toaster and I can manage to run the current PoE at around 20 - 30 FPS. From what I've heard, the new client seems to be really promising and I want to make sure that it can run on crappy toasters too so people like me, don't have problems running PoE and find the best optimization to do so.

Edit : Also, I forgot about removal of desync and the introduction of ping, I'm from India (which means horrible packet loss but I blame that on my ISP from sucking ass) so my ping is always 150 - 200 ms even while using Singapore gateway, this will, to some extent allow me to test out how lockstep mode feels with high ping (which maybe others won't be able to do?) and whether desync is still the more viable option for players with high ping. 
",,1430438802,0,t3_34gj05
1430438438,1,t5_2spu5,t3_34g2xu,t1_cqug9sz,0,85,"",HunterXHunter,cqug9sz,,0,0,0,Black_Goreinu,1,1432703089,"I thought it would be fun to add that for a player like Killua, who can run there in a flash and defeat the captain instead of wasting a card, it's the logical choice. I don't even know if t
hey had a leave card, and I think those are expensive.",,0,0,t3_34g2xu
1430438438,8,t5_2qiel,t3_34g9bg,t1_cqug9t0,0,VANStickInRinkNHL,VANStickInRinkNHL,hockey,cqug9t0,,0,0,0,gargamelol,8,1432703089,"gettin stoned, eatin gummy bears, posting to reddit

 #myplayoffsmoment",,0,0,t3_34g9bg
1430438438,1,t5_2qo4s,t3_34b832,t1_cqug9t1,0,,,nba,cqug9t1,,0,0,0,GheeGhee,1,1432703089,"Giggle fart!!!",,0,0,t1_cquazx6
1430438438,8,t5_2qstm,t3_34foq8,t1_cqug9t2,0,,,personalfinance,cqug9t2,,0,0,0,ajanitsunami,8,1432703089,"/r/budgetfood mod here. thanks for the plug!",,0,0,t1_cqu6vyn

The section of the file you sent works for me. Can you isolate a single line, or better, a single field that doesn't work?

Sorry for the delay I got distracted with other things: The section that is bad I put on this url: http://coding.lhj.me:9100/bad.csv

Code is:

local csv = require("csv")

local count = 0
local size = 0

for fields in csv.use(io.stdin, {}):lines() do 
--for line in io.stdin:lines() do
  for i, v in ipairs(fields) do 
    size = size + 1
    --print(i, v) 
    io.write(v, " , ")
  end
  io.write("\n")
  count = count + 1
  --io.write("count " , count, " size ", size, "\r")
end
io.write("count " , count, " size ", size, "\n")

There is a bit more on this url: http://coding.lhj.me:9100/small_reddit_comments.csv.bz2