How to interpret IAT scores in Inquisit


Author
Message
seandr
seandr
Supreme Being (143K reputation)Supreme Being (143K reputation)Supreme Being (143K reputation)Supreme Being (143K reputation)Supreme Being (143K reputation)Supreme Being (143K reputation)Supreme Being (143K reputation)Supreme Being (143K reputation)Supreme Being (143K reputation)
Group: Administrators
Posts: 1.3K, Visits: 5.6K

The question of how to interpret IAT scores comes up a lot, so I'd post an explanation here to help demystify this subject.


As a participant completes the IAT, Inquisit keeps a running trial-by-trial tally of d, which is the standard metric used to interpret IAT results. The final d score for a given participant thus appears on the last row of data for that participant. Inquisit reports 3 d scores, which appear in the last 3 columns labeled "expressions.da", "expressions.db", and "expression.d".


Expressions.da is the d score from just the practice (i.e., first) blocks for both pairings.


Expressions.db is the d score from just the test (i.e. second) blocks for both pairings.


Expressions.d is the d score for both practice and test blocks. This is the score that is typically reported as the measure of association.


D scores can be positive or negative. A positive score indicates an association of targetA with attributeA  and targetB with attributeB.  A negative score indicates an association of targetA with attributeB and targetB with attributeA. Translating the score into a preference or attitude thus depends on how you have assigned your real world categories to these 4 groups. You can determine the mappings by looking at the topmost section of your IAT script, where you'll see the <item> definitions for each category.


For example, let's say you are studying implicit attitudes towards neckties vs ascots, where neckties are targetA, ascots are targetB, pleasant words are attributeA, and unpleasant words are attributeB. A positive d score would indicate the participant is more of a necktie sort of person. A negative score would indicate an affinity for the classic and elegant ascot look. A score of zero would indicate no preference.


Inquisit also records all of the raw responses and response latencies for every trial so that you have the option of running post hoc analyses on the data (e.g., to experiment with different methods of handling outliers). We offer an SPSS command script that provides a good starting point for this, which you can download from our IAT page:


http://www.millisecond.com/download/samples/v3/IAT/default.aspx


If you run the script as is, you'll get exactly the same d scores as those described above, but the script can be easily edited to do custom analyses.


-Sean



Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 103K

D scores can be positive or negative.


One addition on my part, if I may, since this also a common source of confusion: IAT D-scores may vary between -2 and +2, *not* -1 and +1 as is often erroneously stated (Sriram, Greenwald, & Nosek (2006), p. 57, footnote 2; Nosek & Sriram (2007), p. 396, footnote 2)


~Dave


Jamie
Jamie
Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)
Group: Forum Members
Posts: 10, Visits: 1

Hi,


When I look at my data, I do not see the rows labeled with "expressions" at all. I see all the individual latency scores for each trial, though. What does this mean? Can I run the SPSS script and get the D scores that way? I also have my survey data to deal with. What steps do you recommend so that I can get my D score and then correlate to data?


Thanks,


Jamie


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 103K

When I look at my data, I do not see the rows labeled with "expressions" at all. I see all the individual latency scores for each trial, though. What does this mean?


This probably means you did not use any of the templates available from http://www.millisecond.com/download/samples/v3/IAT/default.aspx (all of which compute D) but instead used an outdated script from a different source (see http://www.millisecond.com/forums/Topic3889.aspx#3889).


Can I run the SPSS script and get the D scores that way?


Once again #2 in http://www.millisecond.com/community/forums/p/1456/4893.aspx#4893.


I also have my survey data to deal with. What steps do you recommend so that I can get my D score and then correlate to data?


This sounds like a topic that should be discussed between you and your advisor.


Regards,


~Dave


Laura
Laura
Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)
Group: Forum Members
Posts: 8, Visits: 1

Hi Sean,


First I want to say that I really appreciate this overview!  It has been helpful to me as I have begun my data analysis.


One question I have regarding how the data are calculated is whether errors and latency were considered when calculating d scores.  In other words, were Greenwald et al.'s (2003) "improved scoring algorithm" recommendations for calculating d scores followed?  If not, would I need to modify and use the syntax that has been developed in order to examine errors and latencies?


Thanks for the help!


Laura


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 103K

One question I have regarding how the data are calculated is whether errors and latency were considered when calculating d scores.  In other words, were Greenwald et al.'s (2003) "improved scoring algorithm" recommendations for calculating d scores followed?  If not, would I need to modify and use the syntax that has been developed in order to examine errors and latencies?


Laura,


the IAT templates scripts available at http://www.millisecond.com/download/samples/v3/IAT compute "D with built in error penalty" (d_biep) according to Greenwald et al. (2003). This is the recommend measure for the specfic IAT procedure implemented by these scripts. The only difference to Greenwald et al.'s improved scoring algorithm is that the Inquisit scripts do not discard data or stop the IAT if subjects have < 300ms latencies on 10+% of the trials. You would have to do that later during data preparation. You can compute other variants of D based on the raw data with your preferred stats package.


Regards,


~Dave


Laura
Laura
Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)
Group: Forum Members
Posts: 8, Visits: 1

HI Dave,


As always, thanks for the quick response!  I should have been more specific.  Are the "expressions.d" scores calculated using Greenwald's improved scoring algorithm?  How do the "expressions.d" scores differ from "d_biep" scores?  Basically I am wondering if I can use the scores that were automatically calculated for me or if I need to run the template script/syntax in order to follow Greenwald's guidelines.  Thanks!


Laura


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 103K

As I already said in my previous response: 'expressions.d' is 'd_biep' which is computed according to the improved scoring algorithm, the only difference being that subjects with latencies below 300ms on 10+% of the trials are not automatically discarded.


~Dave


declan174
declan174
Respected Member (428 reputation)Respected Member (428 reputation)Respected Member (428 reputation)Respected Member (428 reputation)Respected Member (428 reputation)Respected Member (428 reputation)Respected Member (428 reputation)Respected Member (428 reputation)Respected Member (428 reputation)
Group: Forum Members
Posts: 2, Visits: 1

Hi Dave,


Thanks for being so helpful. I have downloaded a sample script from the millisec's site, but i couldn't find "d_biep" in the IAT script, nor is it in the .dat file. I'm wondering does that mean that the when an error is made, does the script simply compute the latency after a correct response is made, or does it add 600 ms to the block mean?


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 103K

See my previous reply to this thread. The IATs are set up to only move on after the correct key has been pressed, thus D includes a "built in error penalty" (biep).


Regards,


~Dave


GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search