Hello everyone,

I am not sure whether this belongs into the Millisecond Community at all and I have checked out the other post about IAT reliabilities but it is not helping me. 

I carried out an ST-IAT with a script from the download area (based on Blümke & Friese, 2008) and now I want to calculate its reliability.

Blümke & Friese calculate Cronbach's alpha for their ST-IAT: "We calculated Cronbach’s a based on a full 34-item scale of trial-wise latency-differences (with missing data being
replaced by participants’ mean latencies, i.e. a z-score of zero."

Schnabel, Asendorpf and Greenwald (2007) say about the IAT: "There exist various ways for calculating internal consistencies of IAT D measures. Some
compute difference scores for every single trial of the combined blocks and treat them as
separate items to calculate Cronbach’s internal consistency alpha, some employ difference
scores for blocks of 5, 10, 20 or more trials, some calculate split-half reliabilities over blocks
with identical number of trials, and some over blocks with different numbers of trials."

I am a Master's student and am doing this on my own and with no experience in reaction time research. I don't know if I am being really dense, but I really don't understand what exactly it is they are doing.

e.g. "trial-wise latency-differences" - do they mean calculate the latency difference between the 3rd trial of block 1 (motiv + positive) and the 3rd trial of block 2 (motive + negative)? Would I have to make sure it's the same kind of trial (trial order is not identical between blocks)? 

And for split-half: some use odd vs. even numbered trials and correlate the D-scores. Does this mean I calculate a D-score for all even numbered trials and a separate one for all odd numbered trials of one block pair? That would make sense to me, though I would still be a bit worried about whether roughly equal numbers of the same trial kind were included in both.

Can anyone walk me through it?

If a given article doesn't make clear what exactly the authors did and how, the best idea may be to contact the authors and ask them to fill in the missing details. They might even have analysis scripts available they'd be willing to share.

That's all I can contribute to this topic.

Regards,

~Dave

Thanks, Dave. I've done that as well just now.

Different papers are equally unclear about how exactly they did it, so I thought it may be obvious for people who work with (ST-)IATs a lot and I was just missing it. Maybe someone else here has a tip and if not I will hopefully get a reply from the author. (If that is the case, I will be happy to post the result on here in case someone in the future has the same question)

Sophia

Hi Sophia,

I have noticed the same thing in terms of reporting reliability. Perhaps my statistical knowledge isn't up to scratch, but I can't understand the logic in presenting a single value for internal consistency, when we are looking for disparities between the compatible and incompatible tests.

If anyone can shed some light on this it would be greatly appreciated!!!

Cheers,

John