Calculating Reliability

[ea9872]

With two different duration keys, each time I calculate a reliability report between two data collectors, each key has the exact partial interval agreement (for example, if key F denotes one behavior and key L denotes another, Data Tracker is calculating 85.67 PIA for both keys although the behaviors for these two keys are not occurring at similar rates). In addition, after calculating reliability by hand I was not able to get the same agreement- can someone explain to me how Data Tracker is calculating reli?

[miyamot0]

The logic for scoring is indicated in the code: https://github.com/miyamot0/DataTracker3/blob/master/reliabilityscoring.h

Specifically, the coding you are referring to is included in the "GetDurationBins" method.

The binning of responses into intervals follows the conventions of BDataPro (i.e., 10s bins by default). This is by no means the best scenario for all cases, but it was done to support comparisons with BDataPro.

Per PI logic, the interval is coding as true if the target occurs in that 10s bin (briefly or in entirety).

That's the best I can say without seeing a log file (a sample json or excel log file would help).

[ea9872]

Hi there, Thanks for your response- I attached to this email the primary and reli (raw and excel) files for a session 11 as well as the calculated reliability and my own reliability calculations, even when I am using a 10 s. interval I am getting 80 percent and above agreement for both of the keys Thanks again for your help I really appreciate it.

…

On Tue, Jun 30, 2020 at 1:03 AM Shawn Gilroy ***@***.***> wrote: The logic for scoring is indicated in the code: https://github.com/miyamot0/DataTracker3/blob/master/reliabilityscoring.h Specifically, the coding you are referring to is included in the "GetDurationBins" method. The binning of responses into intervals follows the conventions of BDataPro (i.e., 10s bins by default). This is by no means the best scenario for all cases, but it was done to support comparisons with BDataPro. Per PI logic, the interval is coding as true if the target occurs in that 10s bin (briefly or in entirety). That's the best I can say without seeing a log file (a sample json or excel log file would help). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQD4VOZPXER5ZH2GP2CFACDRZFWZNANCNFSM4OLZ2IZQ> .

[miyamot0]

I do not see a file attached--I believe you must use the GitHub website to attach files (you cannot include them via email reply).

[ea9872]

011FacTraAnt_P 2.json
011FacTraAnt_P 2.xlsx
011FacTraAnt_R (Kaitlyn Hancock's conflicted copy 2020-06-29).json
011FacTraAnt_R (Kaitlyn Hancock's conflicted copy 2020-06-29).xlsx
Emma_Katie_11.xlsxEmma_Katie_11.xlsx
Personal IOA calculations Session 11.xlsx

[ea9872]

Let me know if those files worked! Thank you.

[miyamot0]

I don't that they attached? If it's easier, send to my faculty address: sgilroy1@lsu.edu

[miyamot0]

Closed (Fixed in .8)