Missing data

[zedawam]

Hello,

I was trying to handle missing data (inputs) with PALM and I'm facing an error. To my understanding, missing data are handled using the -imiss missing.csv, where missing.csv is a table of same size as Y, with 0 if the data is not missing and either -1, 1 or 2 if the data is missing. In my case, I set these missing instances to 1, although I don't know what does it change to set them to -1 or 2.

I run the following command line in octave:
palm -i Data.csv -d DesignMatrix.csv -t contrast.csv

It works fine. When including the -imiss option:
palm -i Data.csv -d DesignMatrix.csv -t contrast.csv -imiss missing.csv

It works fine if missing.csv is actually only 0s. However, if I set some elements to 1 (or -1 or 2), I have many warnings and an error. Here is the log:

-imiss missing.csv
warning: With missing data MAR/MNAR, the option "-zstat" is mandatory.
         Adding it automatically.
warning: called from
    palm_takeargs at line 1342 column 9
    palm_core at line 33 column 11
    palm at line 81 column 1
    GenerateFakeData at line 47 column 1
warning: With missing data, the option "-cmcx" is mandatory.
         Adding it automatically.
warning: called from
    palm_takeargs at line 1348 column 9
    palm_core at line 33 column 11
    palm at line 81 column 1
    GenerateFakeData at line 47 column 1
warning: With missing data, the partitioning must use the "Guttman".
         method. Adding automatically the options
         "-pmethodp Guttman" and "-pmethodr Guttman".
warning: called from
    palm_takeargs at line 1360 column 9
    palm_core at line 33 column 11
    palm at line 81 column 1
    GenerateFakeData at line 47 column 1
Reading input 1/1: Data.csv
Reading design matrix and contrasts.
Elapsed time parsing inputs: ~ 1.4873 seconds.
Preparing designs for missing data [Design: 1/1, Contrast 1/1]
error: 'idx' undefined near line 192 column 12
error: called from
    palm_misspart at line 192 column 4
    palm_core at line 362 column 42
    palm at line 81 column 1
    GenerateFakeData at line 47 column 1

Do you have an idea where it may come from ?

Besides, is it possible to investigate a correlation (-pearson option) in case of missing inputs ?

Thank you !

[andersonwinkler]

Hi, Thanks for the message. The missing data functionality has not been fully implemented (or even validated) yet. Please don't use it yet... All the best, Anderson

…

On Thu, 28 Jun 2018 at 15:26, zedawam ***@***.***> wrote: Hello, I was trying to handle missing data (inputs) with PALM and I'm facing an error. To my understanding, missing data are handled using the -imiss missing.csv, where missing.csv is a table of same size as Y, with 0 if the data is not missing and either -1, 1 or 2 if the data is missing. In my case, I set these missing instances to 1, although I don't know what does it change to set them to -1 or 2. I run the following command line in octave: palm -i Data.csv -d DesignMatrix.csv -t contrast.csv It works fine. When including the -imiss option: palm -i Data.csv -d DesignMatrix.csv -t contrast.csv -imiss missing.csv It works fine if missing.csv is actually only 0s. However, if I set some elements to 1 (or -1 or 2), I have many warnings and an error. Here is the log: -imiss missing.csv warning: With missing data MAR/MNAR, the option "-zstat" is mandatory. Adding it automatically. warning: called from palm_takeargs at line 1342 column 9 palm_core at line 33 column 11 palm at line 81 column 1 GenerateFakeData at line 47 column 1 warning: With missing data, the option "-cmcx" is mandatory. Adding it automatically. warning: called from palm_takeargs at line 1348 column 9 palm_core at line 33 column 11 palm at line 81 column 1 GenerateFakeData at line 47 column 1 warning: With missing data, the partitioning must use the "Guttman". method. Adding automatically the options "-pmethodp Guttman" and "-pmethodr Guttman". warning: called from palm_takeargs at line 1360 column 9 palm_core at line 33 column 11 palm at line 81 column 1 GenerateFakeData at line 47 column 1 Reading input 1/1: Data.csv Reading design matrix and contrasts. Elapsed time parsing inputs: ~ 1.4873 seconds. Preparing designs for missing data [Design: 1/1, Contrast 1/1] error: 'idx' undefined near line 192 column 12 error: called from palm_misspart at line 192 column 4 palm_core at line 362 column 42 palm at line 81 column 1 GenerateFakeData at line 47 column 1 Do you have an idea where it may come from ? Besides, is it possible to investigate a correlation (-pearson option) in case of missing inputs ? Thank you ! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBqNO_FyWIRgns3ov_kne8DhlX1kZoIks5uBS3kgaJpZM4U78kx> .

[jflournoy]

Hi Anderson,

I'm curious if the missing data functionality has progressed enough to be useful. Thanks for your help!