I am a student running an IAT test for a Intro Psych class. In other words, I
have no experience in IAT tests. I just had a subject take a test and I have a
.dat file, but I have no idea how to read/interpret the data.



The columns of data are as follows from left to right:

date      

time     

subject

blockcode         

blocknum      

trialcode          

trialnum          

response     

correct

latency         

stimulusnumber1  

stimulusitem1

expressions.da

expressions.db

expressions.d

I have
taken basic stats so I do know how to run basic z tests.

Please
help! I'm in way over my head.

Note that the column "expressions.d" contains the Cohen's D score that indicates the direction and magnitude of the association. Inquisit keeps a running tally of this score for each trial. The last row of data for each participant therefore has the final score for that participant.

So, you could just take the last row of data for each particpant and get the value of expressions.d to get each participant's score.

-Sean

Hi,

Just one clarification. Isn't the D score the IAT score which is similar to Cohen's d but not the same thing? Most studies using the IAT report both the D score and Cohen's d. Or am I just confusing things :)?

Best,

Kaarinen

My understanding was that these are the same thing. Cohen's D is a general measure of effect size, and is computed as the difference between conditions divided by the standard deviation. That's the same way the IAT D score is computed.

However, there may be some subtle difference I'm not aware of.

-Sean

In fact there is a subtle difference between the two measures. Quoting Greenwald, Banaji and Nosek (2003, p.201):

"Division of a difference between means by a standard deviation is quite
similar to the well-known effect-size measure, d (Cohen, 1977). The
difference between the present D measure and the d measure of effect size
is that the standard deviation in the denominator of D is computed from the
scores in both conditions, ignoring the condition membership of each
score. By contrast, the standard deviation used in computing the effect size
d is a pooled within-treatment standard deviation. To acknowledge both
this measure’s similarity to d and its difference, the present measure is
identified with an italicized uppercase letter (D) rather than an italicized
lowercase letter."

Hope this helps,

~Dave

Awesome, Dave, thanks for setting things straight.

-Sean

No problem. Now here's a question for you, Sean: Do the IAT templates provide d or D? I'm just too lazy to find out myself right now. Besides, any further investigation on my part would severly conflict with my personal 'No (more) IAT' policy...;-)

~Dave

Hi,

Maybe this one's for Sean, then, if Dave has had it with the method :).

There's some huge difference between D and d, since many studies report both figures and these seem to differ a lot. It could be that the effect size d is calculated somewhat differently from your idea here, or there's some other differing thing to it, but the figures might be really far apart when reporting the study mean effects. For example, the first three studies in a methodological IAT paper* give the figures D=0.49/d=1.23; D=0.37/d=0.86; D=0.30/d=0.73. Cohen's d can also have a number greater than 1, whereas D cannot.

This is kinda giving me the impression that I should think about how to calculate d in addition to D for my results...

In the simplest version, Cohen's d is just the difference between two means divided by standard deviation. The IAT D, however, takes a lot more into account - at least when the improved scoring algorithm is used. This is what I was going after in a previous question concerning the need for making a SPSS syntax for my ST-IAT; is your ST-IAT algorithm simply comparing the absolute latencies in the two relevant tasks or does it, for example, also calculate the mean latencies for some practice trials to get the latency scale for an individual user. The improved scoring algo does this, among other things, but I think the ST-IAT template doesn't. Does it, however, take too long latencies (+10000ms) or great number of errors into account?

Best,

Kaarinen

* Lane et al. (2007). Understanding and Using
the Implicit Association Test: IV

Don't even get me started...;-)

Which is not true. While Cohen's d may trail off into infinity in theory, D actually varies between -2 and +2. See Sriram, Greenwald, & Nosek (2006), p. 57, footnote 2, or Nosek & Sriram (2007), p. 396, footnote 2.

This will be my last post on the topic of IAT for a while. I'll leave the rest to Sean.

~Dave

Whoops, just edited my post while you were answering :).

Has this been resolved off-forum?

Apologies for letting this drop.

The scoring algorithm used in the ST-IAT was taken directly from the IAT, which in turn was taken from the SPSS script, which in turn was based on Tony's IAT script, which in turn reflects the improved scoring algorithm reported in Greenwald, Nosek, and Banaji (2003).

Latencies above 10000 ms are indeed discarded by both the Inquisit and SPSS script. I may be misunderstanding something, but I'm not aware of any part of the improved algorithm that involves calculating "the mean latencies for some practice trials to get the latency scale for an individual user". Do you have a reference for this?

It's certainly possible that in adapting the IAT score to the ST-IAT that something isn't right, but I'm not aware of any mistakes. Is there anything in particular that seems fishy here?

-Sean

Hi,

I must have said that in the wrong way :). The mentioned article Greenwald, Nosek, and Banaji (2003, p. 42)  summarises the difference in this way:

The improved algorithm has three substantial changes from the conventional procedure: (a) use of practice-block data (Step 1 in Table 7), (b)
use of error penalties (computed in Steps 5 and 7), and (c) use of individual-respondent standard deviations to provide the measure’s scale unit (computed in Step 6 and applied in Step 11).

Now, as you might have already guessed, I'm not too knowledgeable in IAT scoring and I'm interested in this because I'd like to know if whtether I can report the running D scores as ones calculated by the improved algorithm or as something else. I'm also not even completely sure how the imporved alogrithm  is applied with a ST-IAT, since the article is obviously advising the scoring of conventional IATs, which has a different number of block etc.

Basically, I'm just trying to grasp what the template calculates from the user latencies, but I'm not trying to take this to any higher level or start a methodological discussion on different procedures which would be way beyond my skills :).

Best,

Kaarinen

Hi! I see there's been a lot of clarification of what the D score actually is and how it's calculated, but I have one more question.

Is there a formal way to put the D scores into categories of effect size, like "Small", "Medium" and "Large" effects as is commonly done with Cohen's d scores? I'm hesitant to just jump in and use the same system as is used with Cohen's d scores because IAT D scores go from -2 to +2. I'd be grateful for any advice! Katherine

Katherine,

you will find your question answered in the basic IAT literature such as Greenwald, McGhee & Schwartz (1998) or Greenwald, Nosek & Banaji (2003) ,which anyone doing IATs *must* read.

Regards,

~Dave

Hi Dave,

Thanks for the tips. I had read the 2003 paper, but not the 1998 one. But, I must confess, that now, even after reading both articles, I still do not have the answer to my question. In the 1998 article, I only see that they present the conventional effect sizes for the d measure and in the later article in which they discuss the IAT D measure, I don't see anything about conventional effect sizes or how to report effect sizes. 

Basically, what I need to know is how to report and discuss my D measures an how to understand when to expect that a difference between groups' D measures might be significant or worth looking into. For example, if one group has a D score of .30 and another has .17, how can I discuss this and think about it before doing a significance test? I wonder if it's possible to say that a D score of .30 represents a large association, while a score of .20 represents a small association. Or is this only possible after calculating a d score?

Thanks for any help! Katherine

Katherine,

as an example see e.g. https://implicit.harvard.edu/implicit/demo/background/raceinfo.html.

You can glean the same info by simply looking at any of the IAT scripts:

<trial summary>
[...]
/ ontrialbegin = [if( abs(expressions.d) > 0.15 ) values.magnitude = "a slight"]
/ ontrialbegin = [if( abs(expressions.d) > 0.35 ) values.magnitude = "a moderate"]
/ ontrialbegin = [if( abs(expressions.d) >= 0.65 ) values.magnitude = "a strong"]
[...]
</trial>

Regards,

~Dave

Aha, thank you!