Millisecond Forums » Millisecond Forums » Inquisit 3 » How to interpret IAT scores in Inquisit

How to interpret IAT scores in Inquisit

seandr — Wed, 14 Jun 2017 08:31:05 GMT

The question of how to interpret IAT scores comes up a lot, so I'd post an explanation here to help demystify this subject.

As a participant completes the IAT, Inquisit keeps a running trial-by-trial tally of d, which is the standard metric used to interpret IAT results. The final d score for a given participant thus appears on the last row of data for that participant. Inquisit reports 3 d scores, which appear in the last 3 columns labeled "expressions.da", "expressions.db", and "expression.d".

Expressions.da is the d score from just the practice (i.e., first) blocks for both pairings.

Expressions.db is the d score from just the test (i.e. second) blocks for both pairings.

Expressions.d is the d score for both practice and test blocks. This is the score that is typically reported as the measure of association.

D scores can be positive or negative. A positive score indicates an association of targetA with attributeA and targetB with attributeB. A negative score indicates an association of targetA with attributeB and targetB with attributeA. Translating the score into a preference or attitude thus depends on how you have assigned your real world categories to these 4 groups. You can determine the mappings by looking at the topmost section of your IAT script, where you'll see the <item> definitions for each category.

For example, let's say you are studying implicit attitudes towards neckties vs ascots, where neckties are targetA, ascots are targetB, pleasant words are attributeA, and unpleasant words are attributeB. A positive d score would indicate the participant is more of a necktie sort of person. A negative score would indicate an affinity for the classic and elegant ascot look. A score of zero would indicate no preference.

Inquisit also records all of the raw responses and response latencies for every trial so that you have the option of running post hoc analyses on the data (e.g., to experiment with different methods of handling outliers). We offer an SPSS command script that provides a good starting point for this, which you can download from our IAT page:

http://www.millisecond.com/download/samples/v3/IAT/default.aspx

If you run the script as is, you'll get exactly the same d scores as those described above, but the script can be easily edited to do custom analyses.

-Sean

RE: How to interpret IAT scores in Inquisit

Dave — Wed, 14 Jun 2017 08:31:05 GMT

[quote]

[b]AndresCorredor - Wednesday, June 14, 2017[/b]

Hi,

I am running an IAT and I am trying to get familiar with the data output. For some reason I´m only getting 1 output file instead of two (according to the user manual - https://www.millisecond.com/download/library/v5/iat/pictureiat/pictureiat.manual - I should be getting two different files, the raw data and the summary data.). Does anybody knows why is this happening?

Also in the RAW Data file I am getting 180 lines of d-scores (- the practice trials). Is there a summary score for the overall association? how can I know if the association is positive or negative between the targets and the attributes?

Thank you in advance for all your help,

Andres

[/quote]

Whether you get one or two files depends on the script and Inquisit version you're using. At any rate, the overall D-score in the raw data file is the value in the expressions.d column in the final row of data for a given participant. As for what the score means, see the first post in this thread.

AndresCorredor — Wed, 14 Jun 2017 05:12:52 GMT

RE: How to interpret IAT scores in Inquisit

Philipp-Werner@gmx.de — Thu, 12 Jan 2017 02:37:29 GMT

Is it possible to run a mediation with the d-scores (Greenwald, 2003, improved) as the results of the IAT for the implicit associations as a dependent variable? We are not sure about it because of the negative values and because we have a different scale (7-point lickert) which is also included in the mediation analysis. We also thought about a transformation of the d-score data or something similar before running the mediaion to make the results of the IAT comparable with our other scales or is this not necessary? The same question arises for running a regression analysis. Is this possible with the d-score results?

Maybe somebody of you has experience with something similar or can give some advice.

Thanks a lot in advance!

RE: How to interpret IAT scores in Inquisit

Dave — Mon, 13 Jun 2016 10:37:05 GMT

> So it is ok if I don't follow the recommendation in this case?

That seems like a subjective call to make, and it certainly isn't mine to make -- given that the question is neither about how scores are calculated (they are calculated using the "improved algorithm" regardless) nor to Inquisit more broadly. If you can put forth good arguments for not excluding subjects, I'd say it's fine. But I am neither in a position to find those arguments (I know nothing about your broader study) nor to judge their merit (my opinion matters little; what matters are the opinions of your co-authors, reviewers, thesis advisors, etc.).

Hope this helps.

RE: How to interpret IAT scores in Inquisit

EmilieN — Mon, 13 Jun 2016 07:39:44 GMT

Thanks a lot for your quick reply!
So it is ok if I don't follow the recommendation in this case? Could I justify this with arguing that I've used six IATs and I didn't want to loose any data or/and subjects?
Thanks again for helping me out!

RE: How to interpret IAT scores in Inquisit

Dave — Mon, 13 Jun 2016 06:53:24 GMT

The scores are calculated according to the improved scoring algorithm.

Whether a given subject has latencies < 300 ms in 10% or more of all trials changes nothing about the calculation of D.

Instead, the recommendation is that such subjects' data should be *discarded* entirely from any analysis. I.e., if you have a participant X who responded < 300 ms in more than 10% of all trials, you simply throw her/his entire data set away.

RE: How to interpret IAT scores in Inquisit

EmilieN — Mon, 13 Jun 2016 03:22:55 GMT

Hi Dave,
I hope it's not too late for this reply.
I have a few questions concerning 'expressions.d'. You wrote that the only difference between the calculation of Inquisit and the 'improved scoring algorithm' would be that subjects below 300ms on 10+% of the trials are not automatically discarded. I'm writing my bachelor thesis at the Moment and I used only the automatic generated d-scores for my calculations. now I'm asking myself if it's still ok to write, that the computation of IAT scores is based on the scoring algorithm provided by Greenwald et al. (2003) and how I could establish the fact that subjects below 300ms on 10+% of the trials were not automatically discarded? Even though I have the raw data collected it would cost me months to transfer them into a SPSS data set and compute them with the provided syntax for several reasons. So is there a convincing explanation which i could use for my argumentation why I haven't discarded subjects below 300ms? Next time i would handle it different because now I know much more about IATs and the functions of Inquisit. But for this time it would be so helpful if I could to it that way. Thanks in advance for helping me out!

RE: How to interpret IAT scores in Inquisit

Dave — Sun, 06 Mar 2016 15:57:19 GMT

D-scores are sometimes classified as indicating a "slight", "moderate" or "strong" association. The respective cutoff values are chosen to roughly correspond to typical effect size characterizations.

See https://implicit.harvard.edu/implicit/demo/background/raceinfo.html for an example.

An absolute value >= .65 would indicate a "strong" association, it is not that case that any value below that would indicate *no association*. Instead, absolute values >= .35 are usually taken to indicate a "moderate" association, while >= .15 would indicate a "slight" effect.
Anything between 0 and .15 would be taken as showing no association.

RE: How to interpret IAT scores in Inquisit

mparekh — Sun, 06 Mar 2016 10:48:18 GMT

Hello, I have received conflicting reviews on cutoff scores for the IAT. In one article it is said that any value above +0.65 or below -0.65 indicates a preference one way or another. Some other areas say that any value positive or negative indicates a preference whereas 0 does not. In my data the value I receive (mean value of 150 participants) was 0.46, standard deviation of 0.31. Does this indicate a preference? I would be happy to talk more if you should have the time.

Thank you for your help,

RE: How to interpret IAT scores in Inquisit

Dave — Thu, 22 Oct 2015 10:23:37 GMT

If you used any of the templates available from the millisecond.com library, the D-scores are also recorded to the raw data files. The last line for a given participant gives you the final scores.

Of course, you can also calculate D from the raw latency data. You can use the SPSS syntax files available from the library's IAT page as a starting point.

RE: How to interpret IAT scores in Inquisit

benmittman — Thu, 22 Oct 2015 09:04:51 GMT

Hello,

I've been running the IAT and I've lost the summaries for most of my files -- is there any way to re-generate the summaries from the full data files? This would just save me a lot of time from having to recalculate the values shown on the summaries.

Thank you,

Ben M.
Hamilton College

RE: How to interpret IAT scores in Inquisit

Dave — Tue, 09 Dec 2014 14:33:05 GMT

> I pasted my expressions script below. It looks like it calculated d1 and d2 correctly.

Yes, that looks correct.

> Is it possible that one should interpret traditionally-calculated IAT D scores (as they are calculated within the Inquisit script)
> differently than BIAT D scores calculated via the SPSS syntax?

No. There is no difference. At least in theory / leaving aside the sign mismatch under discussion, the script performs the exact same calculations as the SPSS syntax.

It's perfectly possible that the wrong order of terms is in the SPSS syntax, not the script.

RE: How to interpret IAT scores in Inquisit

Kala — Tue, 09 Dec 2014 13:09:22 GMT

Based on your last comment, I now wonder if the error is in the SPSS syntax file. Based on how you saidCOMPUTE DNumer = MA - MB. Inquisit should calculate the espressions, should "COMPUTE DNumer = MA - MB." in the SPSS syntax actually be "COMPUTE DNumer = MB - MA."?

****************************************************************************.
* These are the numerator components in millisecond units.
****************************************************************************.
IF(Test=1) MA1 = MA .
IF(Test=1) MB1 = MB .
IF(Test=2) MA2 = MA .
IF(Test=2) MB2 = MB .

IF(Test=1) SD1a = SD1 .
IF(Test=1) SD2a = SD2 .
IF(Test=2) SD1b = SD1 .
IF(Test=2) SD2b = SD2 .

IF(Test=1) NA1 = NA .
IF(Test=1) NB1 = NB .
IF(Test=2) NA2 = NA .
IF(Test=2) NB2 = NB .

COMPUTE DNumer = MA - MB.

****************************************************************************.
*Use SD based on all responses (StanDevX) as denominator for D .
****************************************************************************.
COMPUTE DDenom = SQRT( ( ((NA-1) * SD1**2 + (NB-1) * SD2**2)
+ ((NA+NB) * ((MA-MB)**2) / 4) ) / (NA + NB - 1) ) .

IF (Test=1) D1 = DNumer / DDenom .
IF (Test=2) D2 = DNumer / DDenom .
IF (Test=1) N1 = NA+NB .
IF (Test=2) N2 = NA+NB .

RE: How to interpret IAT scores in Inquisit

Kala — Tue, 09 Dec 2014 12:48:48 GMT

I pasted my expressions script below. It looks like it calculated d1 and d2 correctly. Is it possible that one should interpret traditionally-calculated IAT D scores (as they are calculated within the Inquisit script) differently than BIAT D scores calculated via the SPSS syntax?

<expressions>
/ A1m = values.A1sum / values.A1n
/ A2m = values.A2sum / values.A2n
/ B1m = values.B1sum / values.B1n
/ B2m = values.B2sum / values.B2n
/ A1sd = sqrt((values.A1ss - (values.A1n* (expressions.A1m * expressions.A1m))) / (values.A1n - 1))
/ A2sd = sqrt((values.A2ss - (values.A2n* (expressions.A2m * expressions.A2m))) / (values.A2n - 1))
/ B1sd = sqrt((values.B1ss - (values.B1n* (expressions.B1m * expressions.B1m))) / (values.B1n - 1))
/ B2sd = sqrt((values.B2ss - (values.B2n* (expressions.B2m * expressions.B2m))) / (values.B2n - 1))
/ sd1 = sqrt((((values.A1n - 1) * (expressions.A1sd * expressions.A1sd) + (values.B1n - 1) * (expressions.B1sd * expressions.B1sd)) + ((values.A1n + values.B1n) * ((expressions.A1m - expressions.B1m) * (expressions.A1m - expressions.B1m)) / 4) ) / (values.A1n + values.B1n - 1) )
/ sd2 = sqrt((((values.A2n - 1) * (expressions.A2sd * expressions.A2sd) + (values.B2n - 1) * (expressions.B2sd * expressions.B2sd)) + ((values.A2n + values.B2n) * ((expressions.A2m - expressions.B2m) * (expressions.A2m - expressions.B2m)) / 4) ) / (values.A2n + values.B2n - 1) )
/ d1 = (expressions.B1m - expressions.A1m) / expressions.sd1
/ d2 = (expressions.B2m - expressions.A2m) / expressions.sd2
/ d = if ( values.extended ) { (d1+d2) / 2 } else { d1 }
/ currentblocknumber = max(expt.1.currentblocknumber, expt.2.currentblocknumber)
/ preferred = "unknown"
/ notpreferred = "unknown"
/ totalblockcount = if (values.extended) {expt.1.blockcount} else {expt.1.blockcount-2}
/ percentcorrect = (values.n_correct/ (values.a1n + values.a2n + values.b1n + values.b2n)) * 100
</expressions>

RE: How to interpret IAT scores in Inquisit

Dave — Tue, 09 Dec 2014 10:23:59 GMT

I think the sing-mismatch may be due to a mistake in the script's relevant expressions:

<expressions>
...
/ d1 = (expressions.A1m - expressions.B1m) / expressions.sd1
/ d2 = (expressions.A2m - expressions.B2m) / expressions.sd2
...
</expressions>

gets the order of terms in the difference wrong in calculating A-B, which reverses the sign . It should be

<expressions>
...
/ d1 = (expressions.B1m - expressions.A1m) / expressions.sd1
/ d2 = (expressions.B2m - expressions.A2m) / expressions.sd2
...
</expressions>

i.e., B-A. You'll want to check if that's the case in your script.

RE: How to interpret IAT scores in Inquisit

Kala — Tue, 09 Dec 2014 09:57:53 GMT

Thank you. I am going through the syntax file and my Inquisit .exp file with a fine-tooth comb. Just in case you can offer some insight, I have posted how I assigned BIAT order, as well as the relevant section of SPSS script that decodes the order, pairing, and test conditions, below. I adapted the syntax file by putting in the correct blocknums for my BIAT blocks in the VALUE LABELS section and in the COMPUTE Test section; other than that I did not see where I needed to make additional adaptations.

<expt>
/ subjects = (1 of 2)
/ blocks = [1=BSRI;2=READ;3=ManipCheck;4=IATinstructions;5=short_a;6=short_b;7=A;8=B;9=A;10=B;11=ROMANTIC;12=CQ;13=opentext2;14=demographics]
/onexptend = [values.completed = 1]
</expt>

<expt>
/ subjects = (2 of 2)
/ blocks = [1=BSRI;2=READ;3=ManipCheck;4=IATinstructions;5=short_b;6=short_a;7=B;8=A;9=B;10=A;11=ROMANTIC;12=CQ;13=opentext2;14=demographics]
/onexptend = [values.completed = 1]
</expt>

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

GET TRANSLATE FILE = 'BriefIAT2.dat'
/TYPE=TAB /MAP /FIELDNAMES .

VALUE LABELS blocknum
5 'Attribute practice'
6 'Attribute practice'
7 'First pairing practice'
8 'Reverse pairing practice'
9 'First pairing test'
10 'Reverse pairing test'.

IF (MOD(subject,2) = 1) Order = 1 .
IF (MOD(subject,2) = 0) Order = 2 .

COMPUTE Pairing = 0.
IF (blockcode = 'A' ) Pairing = 1.
IF (blockcode = 'B' ) Pairing = 2.

COMPUTE Test = 0.
IF (blocknum=7|blocknum=8) Test = 1.
IF (blocknum=9|blocknum=10) Test = 2.

RE: How to interpret IAT scores in Inquisit

Dave — Tue, 09 Dec 2014 09:45:53 GMT

Make sure you use the syntax correctly and adapt it as needed. Particularly check how your *script* assigns between-subjects conditions (Based on subject id? Based on group id? Randomly?) and verify that the SPSS syntax reflects that.

RE: How to interpret IAT scores in Inquisit

Kala — Tue, 09 Dec 2014 09:29:13 GMT

Thank you for the quick reply! At this point I am trying to figure out why, when I run the SPSS syntax file, the D scores are negative by the DInquisit scores are positive. It seems that although they may not be precisely the same number (because of the 300 ms latency correction in the calculation of the D score), they should at least be the same direction most of the time.

RE: How to interpret IAT scores in Inquisit

Dave — Tue, 09 Dec 2014 08:30:19 GMT

As detailed in the 1st post of the thread, D-scores can be either positive or negative. The sign indicates the "direction" of the effect. It makes no difference in this regard whether you're looking at D-scores stemming from a Brief IAT or from a "standard" IAT.

> Does the running tally of D take the order of the trials into account?

The order of trials does not matter. D is a difference score, in essence (mean latency in incompatible condition - mean latency in compatible condition) / normalized by standard deviation. Each line in the raw data file can be read as "this is the D-score *given all the data available up to this point*.

> Also, does the "DInquisit" variable that is generated in the "Criteria" outfile step of the SPSS script a reliable metric for
> analysis?

That's simply the D-score *as calculated by the script and recorded to the data file*. The LAST function extracts the value from the last line, i.e. the final D-score (when all relevant data are in). For general guidance on SPSS syntax, please the documentation provided by IBM / SPSS.

RE: How to interpret IAT scores in Inquisit

Kala — Tue, 09 Dec 2014 08:11:04 GMT

Hello,

I have questions about two general topics. First, is the interpretation of the BIAT D score the same as the IAT score? Sriram & Greenwald (2009) explain how to interpret D scores above zero, but as far as I can see do not explicitly state how to interpret scores below zero (i.e., negative scores). I have negative score means in my data, so I want to be absolutely sure of my interpretation.

Second, I am curious how the "expressions" variables are calculated by the Inquisit software. Does the running tally of D take the order of the trials into account? Also, does the "DInquisit" variable that is generated in the "Criteria" outfile step of the SPSS script a reliable metric for analysis? I'm unsure what the "LAST" command in the syntax file refers to. I ask because when I use the D generated by the syntax file I get negative score means, but when I use the DInquisit I get positive score means.

Thank you for your assistance!

RE: How to interpret IAT scores in Inquisit

Ana_Oliva — Mon, 29 Sep 2014 16:17:47 GMT

I have already introduced the alterations and all is fine!!! Not one single error!

Thank you, you're a life saviour! :D

Ana

RE: How to interpret IAT scores in Inquisit

Ana_Oliva — Mon, 29 Sep 2014 15:17:53 GMT

No? :D What a relief! :p

So, I interpreted the results in the right directions and don't need to recode anything? :D I've never been so happy for not making an upgrade...! :p

I'll try the alterations regarding the errors then... I hope that it all goes well now.

Thank you! :)

RE: How to interpret IAT scores in Inquisit

Dave — Mon, 29 Sep 2014 15:11:11 GMT

No, you don't have to correct or modify the SPSS syntax -- Inquisit 3 IAT scripts assign block order based on subjectnumber just as the SPSS syntax assumes.

RE: How to interpret IAT scores in Inquisit

Ana_Oliva — Mon, 29 Sep 2014 15:08:28 GMT

Thank you again Dave.

I am using Inquisit 3... Here's what I found in my IAT script:

***********************************************************************
Experiment
***********************************************************************

<defaults>
/ fontstyle = ("Arial", 3.5%)
/ screencolor = (0,0,0)
/ txbgcolor = (0,0,0)
/ txcolor = (255, 255, 255)
/ minimumversion = "3.0.0.0"
</defaults>

<expt>
/ blocks = [1=block1; 2=attributepractice; 3=block3; 4=block4; 5=block5; 6=block6; 7=block7; 8=block8; 9=block9]
</expt>

<variables>
/ group = (1 of 2) (block1=targetcompatiblepractice; block3=compatibletest1; block4=compatibletestinstructions; block5=compatibletest2; block6=targetincompatiblepractice; block7=incompatibletest1; block8=incompatibletestinstructions; block9=incompatibletest2)
/ group = (2 of 2) (block1=targetincompatiblepractice; block3=incompatibletest1; block4=incompatibletestinstructions; block5=incompatibletest2; block6=targetcompatiblepractice; block7=compatibletest1; block8=compatibletestinstructions; block9=compatibletest2)
</variables>

So, I guess that this means that I am not using the correct syntax...

How (and where) can I correct the syntax in order for it to make the proper calculations then?

Thank you!

Ana