Parsing asc file with both eyes tracked
Closed this issue · 5 comments
Hi Simon,
I'm currently using eyelinker to parse asc file. At one particular file, I forgot to track only one eye and during parsng, I've encoutered a bug (probably).
During parsing of this file 29.zip, it terminates with an error
Warning: 6 parsing failures.
row col expected actual
2519 X5 a double .
2519 X6 a double .
2520 X5 a double .
2520 X6 a double .
2521 X5 a double .
.... ... ........ ......
See problems(...) for more details.
Looking at the particular line at the file, the whole problem occurs, when two blink events (one for each eye separately) occurs and one finishes before the other one
In the asc file, those are the problematic lines
SBLINK L 19825744
SBLINK R 19825744
19825744 . . 0.0 . . 0.0 .
19825748 . . 0.0 . . 0.0 .
19825752 . . 0.0 . . 0.0 .
19825756 . . 0.0 . . 0.0 .
19825760 . . 0.0 . . 0.0 .
19825764 . . 0.0 . . 0.0 .
19825768 . . 0.0 . . 0.0 .
19825772 . . 0.0 . . 0.0 .
19825776 . . 0.0 . . 0.0 .
19825780 . . 0.0 . . 0.0 .
19825784 . . 0.0 . . 0.0 .
19825788 . . 0.0 . . 0.0 .
MSG 19825791 SYNCEND
19825792 . . 0.0 . . 0.0 .
19825796 . . 0.0 . . 0.0 .
19825800 . . 0.0 . . 0.0 .
19825804 . . 0.0 . . 0.0 .
19825808 . . 0.0 . . 0.0 .
19825812 . . 0.0 . . 0.0 .
19825816 . . 0.0 . . 0.0 .
19825820 . . 0.0 . . 0.0 .
19825824 . . 0.0 . . 0.0 .
EBLINK L 19825744 19825824 84
19825828 598.3 683.5 210.0 . . 0.0 .
19825832 598.3 683.5 210.0 . . 0.0 .
19825836 598.4 683.5 210.0 . . 0.0 .
EBLINK R 19825744 19825836 96
The final three lines caused the problem, removing them helped.
I've run into the same error, and found a solution. The process.block function in eyelink_parser.R uses str_detect() to determine whether a line is missing an eye sample or not, and discards lines where an eye is missing. The problem is, it's only checking if the item in the 3rd column is a period, meaning it only removes lines where the right eye is closed and leaves in lines where only the left is closed, causing the error. replacing this:
iscrap <- str_detect(raw,"^\\d+\\s+\\.")
with this:
iscrap <- str_detect(raw,"^\\d+\\s+\\.|^\\d+\\s+\\d+\\.\\d+\\s+\\d+\\.\\d+\\s+\\d+\\.\\d+\\s+\\.")
fixes the issue.
EDIT: Nevermind, this still misses lines where the left eye is missing and the either of the right eye values are negative. A much simpler and cleaner solution is
iscrap <- str_detect(raw, "\\s+\\.\\s+\\.\\s+")
which catches all lines where there are two periods separated by one or more spaces and bound on either side by one or more spaces (i.e. all lines where gaze is missing).
Apologies to you both for not getting to this issue sooner. a-hurst's solution appears to work, I've updated the package on github, will upload to CRAN soon. Thanks for the report and the fix!
Hi dahtah, just wanted to let you know before you pushed to CRAN that I've actually written a better fix since my last post, as I was still experiencing issues importing .asc files containing trials where one eye was missing the whole time (it would fail since all samples were considered 'iscrap', causing the rest of process.block to error out). It's a bit more involved, but makes read.asc more error-resistant and has the advantage of keeping samples where just one eye is missing (missing eye positions are replaced with NAs instead of being removed):
(Ignore the page column, that's something I added after importing for the purposes of my study.)
First, I figured out that the reason the dots for missing samples caused problems on some trials but not others was that the read_tsv function that's used to convert sample data into a data frame tries to automatically guess the type of each column based on its first 1000 items. This is a pretty sane assumption for most use cases, but since 1000 rows is only 4 seconds at 250hz recording, it means it often gueses that the xpl/ypl and/or xpr/ypr columns are all going to be numbers and then errors out spectacularly when it encounters several hundred rows containing periods instead of doubles later in the trial.
To fix this, the read_tsv function also allows you to manually specify column types with a string, which I edited the tsv2df function to pass to it as an argument:
tsv2df <- function(dat, coltypes=NULL)
{
if (length(dat)==1)
{
dat <- paste0(dat,"\n")
}
else
{
dat <- paste0(dat,collapse="\n")
}
out <- read_tsv(dat,col_names=FALSE,col_types=coltypes)
if (!(is.null(attr(suppressWarnings(out), "problems")))) browser()
out
}Then I rewrote part of the process.block function to check if the asc is binocular or monocular and specify column types to tsv2df accordingly (treating eye position columns as characters), then replace any "."s with NAs and convert the pupil position columns back to doubles:
process.block <- function(blk,info)
{
hd <- process.block.header(blk)
blk <- hd$the.rest
raw.colnames <- coln.raw(info)
#Get the raw data (lines beginning with a number)
which.raw <- str_detect(blk,'^\\d')
raw <- blk[which.raw] %>% str_select('^\\d')
# Treat values in columns 2/3 (and 5/6 if binocular) as characters instead
# of doubles, as they are periods "." instead of numbers if the eye is
# missing that sample and will cause read_tsv to spew thousands of warnings
if (info$mono) {
raw <- tsv2df(raw, "iccdc")
raw[raw=="."] <- NA # Set eye coordinates to NA if missing
for (col in c(2,3)) { raw[[col]] <- as.double(raw[[col]]) }
} else {
raw <- tsv2df(raw, "iccdccdc")
raw[raw=="."] <- NA # Set eye coordinates to NA if missing
for (col in c(2,3,5,6)) { raw[[col]] <- as.double(raw[[col]]) }
}
if (ncol(raw) == length(raw.colnames))
{
names(raw) <- raw.colnames
}
else
{
warning("Unknown columns in raw data. Making a guess, but please check the results")
names(raw)[1:length(raw.colnames)] <- raw.colnames
}
#The events (lines not beginning with a number)
evt <- blk[!which.raw]
res <- process.events(evt,hd$events)
res$raw <- raw
res$sampling.rate <- hd$events$srate
res$left.eye <- hd$events$left
res$right.eye <- hd$events$right
res
}Hope this helps! Thank you for developing and maintaining this package, by the way. I'd be much further behind on my thesis without it.
Thanks for your input! I followed your suggestion and added explicit column types in the call to read_tsv. The update is on github, give it a try if you have a minute, it'd be good to know that it works on your data.
Yes, the latest update works! However, the new method "NA"s all data for the other eye when one eye is missing for a given sample. Would it be possible to add a flag to read.asc that would allow for preserving the properly captured eye when the other eye is missing, as my code does? Thanks again!