Millisecond Forums

SC-IAT Scoring

https://forums.millisecond.com/Topic12559.aspx

By becgwin - 3/5/2014

Hi,

I ran an SC-IAT last year based on the Inquisit task library version dated 03-26-2013 and I am slightly confused by the summary output. While I get no expressions.da (it appears to leave out calculations of the practice trials), I do get an expressions.db (which according to previous posts is for the test trials - topics3444) and an expressions.d (which according to same post is meant to combine practice and test trials). The expressions.d seems to be the expressions.da divided by 2. The expressions.latdiffb and expressions.latdiff are identical which I think makes sense if the practice trials are not included. I have since seen the updated version of the SC-IAT (dated 11-21-2013) and the programmed output makes more sense to me, so I assume it may have been amended in part to address this issue.  I just want to confirm that expressions.db in my summary output is D for test trials, and what I should use assuming I am satisfied to exclude trials larger than 10000ms and error trials.  Also, can anyone explain what expressions.d is in this case when it appears the practice trials were not included in the analysis, and yet it differs from expressionsdb. The script is as follows:

* 1 is compatible, 2 is incompatible
* a is first block, b is second block

<expressions>
/ m1a = values.sum1a / values.n1a
/ m2a = values.sum2a / values.n2a
/ m1b = values.sum1b / values.n1b
/ m2b = values.sum2b / values.n2b
/ sd1a = sqrt((values.ss1a - (values.n1a * (expressions.m1a * expressions.m1a))) / (values.n1a - 1))
/ sd2a = sqrt((values.ss2a - (values.n2a * (expressions.m2a * expressions.m2a))) / (values.n2a - 1))
/ sd1b = sqrt((values.ss1b - (values.n1b * (expressions.m1b * expressions.m1b))) / (values.n1b - 1))
/ sd2b = sqrt((values.ss2b - (values.n2b * (expressions.m2b * expressions.m2b))) / (values.n2b - 1))
/ sda = sqrt((((values.n1a - 1) * (expressions.sd1a * expressions.sd1a) + (values.n2a - 1) * (expressions.sd2a * expressions.sd2a)) + ((values.n1a + values.n2a) * ((expressions.m1a - expressions.m2a) * (expressions.m1a - expressions.m2a)) / 4) ) / (values.n1a + values.n2a - 1) )
/ sdb = sqrt((((values.n1b - 1) * (expressions.sd1b * expressions.sd1b) + (values.n2b - 1) * (expressions.sd2b * expressions.sd2b)) + ((values.n1b + values.n2b) * ((expressions.m1b - expressions.m2b) * (expressions.m1b - expressions.m2b)) / 4) ) / (values.n1b + values.n2b - 1) )
/ da = (m2a - m1a) / expressions.sda
/ db = (m2b - m1b) / expressions.sdb
/ d = (expressions.da + expressions.db) / 2
/ latdiffa = m2a - m1a
/ latdiffb = m2b - m1b
/ latdiff = ((values.sum2a + values.sum2b) / (values.n2a + values.n2b)) - ((values.sum1a + values.sum1b) / (values.n1a + values.n1b))
/ attitude = "unknown"
</expressions>


Finally, can any-one confirm that the raw error trial latencies are uncorrected, or corrected for in calculations only by their exclusion?  That way I can easily run my own analysis if I want to change the parameters e.g. delete trails less than 350ms etc.

Thanks,

Rebecca 
By Dave - 3/5/2014

expressions.da, expressions.db and expressions.d represent what https://www.millisecond.com/forums/Topic3444.aspx details them to represent -- the kind of IAT (standard or single-category) does not matter in that regard.

If, for some reason (I don't know which and it it not possible to infer from the information you posted), expressions.da is not properly computed and thus remains at zero, your observations directly follow from the math in the respective expressions:

> The expressions.d seems to be the expressions.da divided by 2 [...]
(I assume you really mean expressions.db here, as previously you state "I get no expressions.da [...]")

d = (expressions.da + expressions.db) / 2 = (0 + expressions.db) = expressions.db / 2

> The expressions.latdiffb and expressions.latdiff are identical [...]

latdiff = ((values.sum2a + values.sum2b) / (values.n2a + values.n2b)) - ((values.sum1a + values.sum1b) / (values.n1a + values.n1b)) = ((0 + values.sum2b) / (0 + values.n2b)) - (0 + values.sum1b) / (0 + values.n1b)) = m2b - m1b = lattdiffb

> can anyone explain what expressions.d is in this case when it appears the practice trials were not included in the analysis, and yet it differs from expressionsdb

You have already perfectly answered that yourself: "The expressions.d seems to be the expressions.db divided by 2". It should not be used in this case.

> can any-one confirm that the raw error trial latencies are uncorrected

The RTs on error trials include a "built-in error penalty" (as is recommend; see Greenwald et al., 2003) -- if a person presses the wrong key in a given trial, she is not allowed to progress until she presses the correct key. The latency reflects when the latter happened, not when the first (wrong) response occurred.

Otherwise, raw RTs are what they appear to be -- no raw data is excluded or changed. You can thus re-compute the various d's from the raw data at any time.

Outlier RTs (latency > 10000ms) are excluded from automatic calculations only, which does not affect or alter the recorded raw latency in any way (you can see this for yourself by looking at the /ontrialend attributes in the script -- you will find that only latencies < 10000 are added to the respective summing variables).

Hope this helps.
By becgwin - 3/6/2014

Hi Dave,
Hi Dave,


That helps a lot.  Thanks again for answering my questions so quickly and thoroughly.  And yes I did mean expressions.db (not expressions.da) - sorry! Just one further question re the built in error penalty - I used the Karpinski and Steinman (2006) version (the SC-IAT as versus the ST-IAT) which allows a subject to move on without getting the correct answer (it just shows an X to indicate an incorrect answer).  It therefore doesn't have an automatically built in error penalty and it records both correct and incorrect latencies.  It doesn't however include any incorrect response latencies in the calculations of D as block instructions only relate to correct responses (e.g. / ontrialend = [if(values.correct == 1 && block.incompatibletest.latency  <= 10000) values.sum2 =  values.sum2 + block.incompatibletest.latency].  Am I understanding that correctly?  If I wanted to include error responses, I assume I would have to follow Karpinski and Steinman's recommendation of replacing error responses with the block mean plus 400ms error penalty. I have also worked out why the practice trials were not included - as Karpinski and Steinman excluded them from their analysis, the Inquisit script I adapted did not have any commands re generating calculations for the practice trials!


Thanks again,

Bec
By Dave - 3/6/2014

> It doesn't however include any incorrect response latencies in the calculations of D as block instructions only relate to correct
> responses [...].  Am I understanding that correctly?

I believe so. It's also in line with the task description at the top of the script:

Script Specifics:

(1) EXPERIMENTAL PROCEDURE:
a) Block Practice Compatible* (24 trials); responses not counted towards D-score
b) Block Compatible (72 trials)
c) Block Practice InCompatible (24 trials); responses not counted towards D-score
d) Block InCompatible (72 trials)

*order compatible - incompatible counterbalanced by groupnumber

=> one D-score is determined

(2) TRIALS
After a pretrialpause of 250ms(default, editable), stimuli are presented for 1500ms max (default, editable).
If no response occurs during that time, a reminder is presented to "respond more quickly."
In contrast to the general IAT procedure used in the Millisecond Software scripts,
participants don't have to correct their response to move on. They are given
error feedback but no response correction is necessary.

(3) STIMULI:
- 21 words for Attribute Categories Good/Bad
- 7 words for target category (here: Animals)

(4) SCORING:
- latencies of trials larger than 10000ms and error trials are excluded from D-score analyses
By becgwin - 3/6/2014

Thanks for confirming that Dave.  I'll move on now with more confidence!
By AC - 9/11/2015

Hello,

 

Does anyone have SPSS syntax for scoringthe SC-IAT? I am not syntax savvy and am tying myself in mental knots trying toamend the Brief IAT SPSS syntax to suit.

 

Hopeful thanks in advance

 

AC
By Dave - 9/11/2015

If you are going to use any existing syntax as the basis, you should *not* use the Brief IAT syntax, but the Standard IAT syntax. The Brief IAT is *very different* from a regular IAT, while the Standard IAT and the SC-IAT are a much closer match. Beyond that, I'm not aware of any existing scoring syntax and -- since the script itself already computes the usual dependent variables and logs them to the data file -- most people don't actually need any.

Hope this helps..
By AC - 9/11/2015

Hello, thank you so much for the speedy for the response. I cansee the d scores, but I need to exclude the responses below 350ms (as Karpinskiand Steinman guidelines) so d will have to be recalculated in spss and I was just hoping that someone might havesyntax so that I avoided the likelihood of me making a mistake as I do itmanually. Even if Ididn’t need to recalculate I can’t get my head around how to tell spss to pickup just the last d score for an individual in a particular block when restructuring‘selected cases to variables’.

Thanks again

By Dave - 9/11/2015

> Even if Ididn’t need to recalculate I can’t get my head around how to tell spss to pickup just the last d score for an individual in a > particular block when restructuring‘selected cases to variables’.

You use SPSS's last() function. https://www-01.ibm.com/support/docview.wss?uid=swg21476173

In general, aggregating data is not hard. Good introductions are among others:

- Lacroix & Giguère (2006) available at http://www.tqmp.org/Content/vol02-1/p020/p020.pdf

- https://computingforpsychologists.wordpress.com/2012/12/11/how-to-analyse-reaction-time-rt-data-part-1/

- http://ron.dotsch.org/importing-inquisit-data-files-into-spss-pasw/

- http://jeromyanglim.blogspot.com/2010/07/how-to-process-inquisit-raw-data-in.html

Once you've got the basics down, you should be able to (a) read and understand any existing SPSS syntax and (b) be able to modify it according to your needs.
By AC - 9/11/2015

Hi again, for what I think its the final time. Surprisingly I seem to have sorted it out. It's great to know about the last function. Thanks for all of your help!
By AC - 9/13/2015

Hi Dave,

sadly I am back with another question. My D scores were different to theInquisit calculated ones and I am trying to figure out why. 

 

My understanding of Karpinski andSteinman’s article is that the D score is calculated using the block means ofall latencies (ie errors with penalty included) divided by the pooled sd ofonly the correct responses from compatible and incompatible blocks.

 

I have been going through the SC-IATscript again and just want to confirm that in the scoring and<expressions> sections when ‘correct responses’ is mentioned it alwaysrefers to all responses (i.e. the initial correct and the ‘correct’ with thebuilt in error penalty), and that initial is referenced when only the initial correctresponses are required (as in
/ n_correct: countsall initial correct responses of all trials that count towards D score)

 

Thanks again in advance

AC
By Dave - 9/13/2015

Yes, that is correct. The differences in the scoring are also mentioned in the comments at the top of the script.

Differences btw. this script and Karpinski & Steinman (2006)
- error responses in this script need to be corrected before moving on
- no response window:
stimuli stay on screen until a correct response is given. A reminder to "respond more quickly" is
optional if latencies exceed a certain limit (see section "Editable Values" for more info)
- no error response correction necessary for d-score calculation as error responses needed to be corrected
before moving on (therefore an error penalty was added automatically to each error latency)
By AC - 9/14/2015

Hi Dave,

 

I did read the info in the script beforerunning the experiment and it was really helpful and clear. I think perhaps I wasn't clear in what I was asking though...

 

I am trying to figure out why my d-scoreends up being different to the inquisit calculated one.  So in my reading of it there are two optionsfor correct responses- initial correct responses and ‘all correct’ responses(which includes the initial correct and the corrected responses – those withbuilt in error penalty). The way that the scoring and expressions read there isonly one that uses initial correct only and it is not used in the d-algorithm(percent correct). So does the inquisit use initial correct for all of theD-scoring, or does it use ‘all correct’ for the D-scoring. Or perhaps it uses ‘allcorrect’ for everything except the SDs which are initial correct only (which Ithink is how it is meant to be calculated) but it is just not clearly indicatedin the script explanation? 

AC
By AC - 9/14/2015

Hi again,
I thought I would paste in the section of the SC-IAT script I am talking about for a quick reference.

I am trying to figure out why my d-scoreends up being different to the inquisit calculated one.  So in my reading of it there are two optionsfor correct responses- initial correct responses and ‘all correct’ responses(which includes the initial correct and the corrected responses – those withbuilt in error penalty). The way that the scoring and expressions read there isonly one that uses initial correct only and it is not used in the d-algorithm(percent correct). So does the inquisit use initial correct for all of theD-scoring, or does it use ‘all correct’ for the D-scoring. Or perhaps it uses‘all correct’ for everything except the SDs which are initial correct only(which I think is how it is meant to be calculated) but it is just not clearlyindicated in the script explanation?

 

/completed:   0= script was not completed; 1 = script was completed (all conditions run)

/correct:   1= correct response; 0 = incorrect response

/ sum1: tracks the sum of the latencies tocorrect responses (and latencies <= 10000ms) for the compatible block

/ sum2a: tracks the sum of the latencies tocorrect responses (and latencies <= 10000ms) for the incompatible block

/ n1:  countsthe number of correct trials in compatible block

/ n2:  countsthe number of correct trials in incompatible block

/ ss1: tracksthe sum of the squared latencies to correct responses (and latencies <=10000ms) in the compatible block

/ ss2  tracksthe sum of the squared latencies to correct responses (and latencies <=10000ms) in the incompatible block

/ magnitude:            stores the magnitude of the implicit preference:"little to no", "a slight", "a moderate", "astrong"

/ n_correct:   counts all initial correct responses of all trials that counttowards D score


/ m1:            meanlatencies of correct responses in compatible block

/ m2: meanlatencies of correct responses in incompatible block

/ sd1: standarddeviation of latencies of correct responses in compatible block

/ sd2: standarddeviation of latencies of correct responses in incompatible block

/ sd: standard deviation of latenciesoverall

/ d: overall D-score

/ latdiff: difference between meanlatencies in incompatible and compatible block

/ attitude: positive vs. negative

/ percentcorrect:       calculates the overall percent correctscore of initial responses of compatible and incompatible trials with specified latencies

 

<expressions>

/ m1 = values.sum1 / values.n1

/ m2 = values.sum2 / values.n2

/ sd1 = sqrt((values.ss1 - (values.n1 *(expressions.m1 * expressions.m1))) / (values.n1 - 1))

/ sd2 = sqrt((values.ss2 - (values.n2 *(expressions.m2 * expressions.m2))) / (values.n2 - 1))

/ sd = sqrt((((values.n1 - 1) *(expressions.sd1 * expressions.sd1) + (values.n2 - 1) * (expressions.sd2 *expressions.sd2)) + ((values.n1 + values.n2) * ((expressions.m1 -expressions.m2) * (expressions.m1 - expressions.m2)) / 4) ) / (values.n1 +values.n2 - 1) )

/ d = (m2 - m1) / expressions.sd

/ latdiff = m2 - m1

/ attitude = "unknown"

/ percentcorrect = (values.n_correct/(block.compatibletest.trialcount + block.incompatibletest.trialcount)) * 100

</expressions>
By Dave - 9/14/2015

*All trials* with latency <= 10000 are counted towards values.n1 and values.n2 respectively -- regardless of whether the *initial* response in a given trial was correct. You can see this by looking at the /ontrialend attributes in the respective <block> elements:

<block compatibletest>
...
/ ontrialend = [if(block.compatibletest.latency  <= 10000) values.n1 =  values.n1 + 1]
...
</block>

<block incompatibletest>
...
/ ontrialend = [if(block.incompatibletest.latency  <= 10000) values.n2 =  values.n2 + 1]
...
</block>

values.n1 and values.n2 are the counts involved in calculating D (that includes the means and associated SDs):

<expressions>
/ m1 = values.sum1 / values.n1
/ m2 = values.sum2 / values.n2
/ sd1 = sqrt((values.ss1 - (values.n1 * (expressions.m1 * expressions.m1))) / (values.n1 - 1))
/ sd2 = sqrt((values.ss2 - (values.n2 * (expressions.m2 * expressions.m2))) / (values.n2 - 1))
/ sd = sqrt((((values.n1 - 1) * (expressions.sd1 * expressions.sd1) + (values.n2 - 1) * (expressions.sd2 * expressions.sd2)) + ((values.n1 + values.n2) * ((expressions.m1 - expressions.m2) * (expressions.m1 - expressions.m2)) / 4) ) / (values.n1 + values.n2 - 1) )
/ d = (m2 - m1) / expressions.sd
...
</expressions>

values.n_correct (which reflects the number of *initial* correct responses across the two combined blocks), on the other hand, does *not* figure into calculating D. The only thing it's used for is calculating the percentage of (initial) correct responses:

<expressions>
...
/ percentcorrect = (values.n_correct/ (block.compatibletest.trialcount + block.incompatibletest.trialcount)) * 100
</expressions>

Hope this clarifies.
By AC - 9/14/2015

Thank you Dave, I am sorry I over lookedon trial end. I must confess to being a little confused still as I re-readGreenwald et al (2003) p.14 to mean that the pooled sd was to be calculatedfrom initial correct only as it says “Compute one pooled SD for all trials inB3 & B6; another for B4 & B7” before the step to replace errorlatencies and seek the means. So when Karpinski and Steinman wrote “The averageresponse times of Block 2 (e.g., Coke _ good) were subtracted from the averageresponse times of Block 4 (e.g., Coke _ bad). This quantity was divided by thestandard deviation of all correct response times within Blocks 2 and 4” itseemed like they only used the initial correct too.
So D is calculated usingall of the responses, which is what I was doing days ago and thought was wrong.

This is an embarrassing question, Inquisitcalculates two d scores(compatible and incompatible) – how do I take them tomake one overall d score?  Or is thefinal d score the last figure in the compatible block? If so then mycalculation is out by .03, which may be a rounding error as I believe youmentioned to someone else on the forum once.

By Dave - 9/14/2015

The script does not calculate two D-scores. There is a single one, expressions.d. In fact,

> [... ] two d scores(compatible and incompatible)

would not make sense: D is a *difference score*. It is -- in essence -- the mean response latency in *incompatible trials* minus the mean response latency in *compatible trials* (scaled by SD).

The final row of data in your data file represents the final D-score. That's when all the relevant data -- from compatible and incompatible blocks -- are in.
By Dave - 9/14/2015

Regarding Greenwald et al. (2003), p. 210: It depends on what D-"variant" you want to calculate. The script calculates D1 -- which is the recommended variant overall and the one applicable to procedures where participants have to correct wrong responses ("built-in error penalty").

The table on page 214, on the other hand, contrasts the "conventional" scoring procedure with a different D-variant (the text says D2).
By AC - 9/14/2015

No, I didn’t think it would make sense, butas I am not a great stats mind (not even feeling like an average one at themoment!)I could easily have missed something.

The last d score (at trial 72 ofincompatible) is 1.742, the last D score at trial 72 of compatible is .0219.For participant 2 compatible was presented first followed by incompatible,which would suggest that D is 1.742. Yet my calculation has D as .22, whichmatches the compatible last. At least it matches something...it is odd though. Especially as  the ‘last’ lat.diffs fromInquisit are both different to the lat diff calculated in SPSS
By AC - 9/14/2015

Hi Dave,
You may have realised, but the reason I was getting two D scores is because I was still using 'block' when aggregating using the last function. hhmm. Looking forward to getting rid of the L plates.
Thank you for all of your help!