Millisecond Forums

Hi Dave,

sadly I am back with another question. My D scores were different to theInquisit calculated ones and I am trying to figure out why. 

 

My understanding of Karpinski andSteinman’s article is that the D score is calculated using the block means ofall latencies (ie errors with penalty included) divided by the pooled sd ofonly the correct responses from compatible and incompatible blocks.

 

I have been going through the SC-IATscript again and just want to confirm that in the scoring and<expressions> sections when ‘correct responses’ is mentioned it alwaysrefers to all responses (i.e. the initial correct and the ‘correct’ with thebuilt in error penalty), and that initial is referenced when only the initial correctresponses are required (as in
/ n_correct: countsall initial correct responses of all trials that count towards D score)

 

Thanks again in advance

AC

Yes, that is correct. The differences in the scoring are also mentioned in the comments at the top of the script.

Differences btw. this script and Karpinski & Steinman (2006)
- error responses in this script need to be corrected before moving on
- no response window:
stimuli stay on screen until a correct response is given. A reminder to "respond more quickly" is
optional if latencies exceed a certain limit (see section "Editable Values" for more info)
- no error response correction necessary for d-score calculation as error responses needed to be corrected
before moving on (therefore an error penalty was added automatically to each error latency)

Hi Dave,

 

I did read the info in the script beforerunning the experiment and it was really helpful and clear. I think perhaps I wasn't clear in what I was asking though...

 

I am trying to figure out why my d-scoreends up being different to the inquisit calculated one.  So in my reading of it there are two optionsfor correct responses- initial correct responses and ‘all correct’ responses(which includes the initial correct and the corrected responses – those withbuilt in error penalty). The way that the scoring and expressions read there isonly one that uses initial correct only and it is not used in the d-algorithm(percent correct). So does the inquisit use initial correct for all of theD-scoring, or does it use ‘all correct’ for the D-scoring. Or perhaps it uses ‘allcorrect’ for everything except the SDs which are initial correct only (which Ithink is how it is meant to be calculated) but it is just not clearly indicatedin the script explanation? 

AC

Hi again,
I thought I would paste in the section of the SC-IAT script I am talking about for a quick reference.

I am trying to figure out why my d-scoreends up being different to the inquisit calculated one.  So in my reading of it there are two optionsfor correct responses- initial correct responses and ‘all correct’ responses(which includes the initial correct and the corrected responses – those withbuilt in error penalty). The way that the scoring and expressions read there isonly one that uses initial correct only and it is not used in the d-algorithm(percent correct). So does the inquisit use initial correct for all of theD-scoring, or does it use ‘all correct’ for the D-scoring. Or perhaps it uses‘all correct’ for everything except the SDs which are initial correct only(which I think is how it is meant to be calculated) but it is just not clearlyindicated in the script explanation?

 

/completed:   0= script was not completed; 1 = script was completed (all conditions run)

/correct:   1= correct response; 0 = incorrect response

/ sum1: tracks the sum of the latencies tocorrect responses (and latencies <= 10000ms) for the compatible block

/ sum2a: tracks the sum of the latencies tocorrect responses (and latencies <= 10000ms) for the incompatible block

/ n1:  countsthe number of correct trials in compatible block

/ n2:  countsthe number of correct trials in incompatible block

/ ss1: tracksthe sum of the squared latencies to correct responses (and latencies <=10000ms) in the compatible block

/ ss2  tracksthe sum of the squared latencies to correct responses (and latencies <=10000ms) in the incompatible block

/ magnitude:            stores the magnitude of the implicit preference:"little to no", "a slight", "a moderate", "astrong"

/ n_correct:   counts all initial correct responses of all trials that counttowards D score

/ m1:            meanlatencies of correct responses in compatible block

/ m2: meanlatencies of correct responses in incompatible block

/ sd1: standarddeviation of latencies of correct responses in compatible block

/ sd2: standarddeviation of latencies of correct responses in incompatible block

/ sd: standard deviation of latenciesoverall

/ d: overall D-score

/ latdiff: difference between meanlatencies in incompatible and compatible block

/ attitude: positive vs. negative

/ percentcorrect:       calculates the overall percent correctscore of initial responses of compatible and incompatible trials with specified latencies

 

<expressions>

/ m1 = values.sum1 / values.n1

/ m2 = values.sum2 / values.n2

/ sd1 = sqrt((values.ss1 - (values.n1 *(expressions.m1 * expressions.m1))) / (values.n1 - 1))

/ sd2 = sqrt((values.ss2 - (values.n2 *(expressions.m2 * expressions.m2))) / (values.n2 - 1))

/ sd = sqrt((((values.n1 - 1) *(expressions.sd1 * expressions.sd1) + (values.n2 - 1) * (expressions.sd2 *expressions.sd2)) + ((values.n1 + values.n2) * ((expressions.m1 -expressions.m2) * (expressions.m1 - expressions.m2)) / 4) ) / (values.n1 +values.n2 - 1) )

/ d = (m2 - m1) / expressions.sd

/ latdiff = m2 - m1

/ attitude = "unknown"

/ percentcorrect = (values.n_correct/(block.compatibletest.trialcount + block.incompatibletest.trialcount)) * 100

</expressions>

*All trials* with latency <= 10000 are counted towards values.n1 and values.n2 respectively -- regardless of whether the *initial* response in a given trial was correct. You can see this by looking at the /ontrialend attributes in the respective <block> elements:

<block compatibletest>
...
/ ontrialend = [if(block.compatibletest.latency  <= 10000) values.n1 =  values.n1 + 1]
...
</block>

<block incompatibletest>
...
/ ontrialend = [if(block.incompatibletest.latency  <= 10000) values.n2 =  values.n2 + 1]
...
</block>

values.n1 and values.n2 are the counts involved in calculating D (that includes the means and associated SDs):

<expressions>
/ m1 = values.sum1 / values.n1
/ m2 = values.sum2 / values.n2
/ sd1 = sqrt((values.ss1 - (values.n1 * (expressions.m1 * expressions.m1))) / (values.n1 - 1))
/ sd2 = sqrt((values.ss2 - (values.n2 * (expressions.m2 * expressions.m2))) / (values.n2 - 1))
/ sd = sqrt((((values.n1 - 1) * (expressions.sd1 * expressions.sd1) + (values.n2 - 1) * (expressions.sd2 * expressions.sd2)) + ((values.n1 + values.n2) * ((expressions.m1 - expressions.m2) * (expressions.m1 - expressions.m2)) / 4) ) / (values.n1 + values.n2 - 1) )
/ d = (m2 - m1) / expressions.sd
...
</expressions>

values.n_correct (which reflects the number of *initial* correct responses across the two combined blocks), on the other hand, does *not* figure into calculating D. The only thing it's used for is calculating the percentage of (initial) correct responses:

<expressions>
...
/ percentcorrect = (values.n_correct/ (block.compatibletest.trialcount + block.incompatibletest.trialcount)) * 100
</expressions>

Hope this clarifies.

Thank you Dave, I am sorry I over lookedon trial end. I must confess to being a little confused still as I re-readGreenwald et al (2003) p.14 to mean that the pooled sd was to be calculatedfrom initial correct only as it says “Compute one pooled SD for all trials inB3 & B6; another for B4 & B7” before the step to replace errorlatencies and seek the means. So when Karpinski and Steinman wrote “The averageresponse times of Block 2 (e.g., Coke _ good) were subtracted from the averageresponse times of Block 4 (e.g., Coke _ bad). This quantity was divided by thestandard deviation of all correct response times within Blocks 2 and 4” itseemed like they only used the initial correct too.
So D is calculated usingall of the responses, which is what I was doing days ago and thought was wrong.

This is an embarrassing question, Inquisitcalculates two d scores(compatible and incompatible) – how do I take them tomake one overall d score?  Or is thefinal d score the last figure in the compatible block? If so then mycalculation is out by .03, which may be a rounding error as I believe youmentioned to someone else on the forum once.

The script does not calculate two D-scores. There is a single one, expressions.d. In fact,

> [... ] two d scores(compatible and incompatible)

would not make sense: D is a *difference score*. It is -- in essence -- the mean response latency in *incompatible trials* minus the mean response latency in *compatible trials* (scaled by SD).

The final row of data in your data file represents the final D-score. That's when all the relevant data -- from compatible and incompatible blocks -- are in.

Regarding Greenwald et al. (2003), p. 210: It depends on what D-"variant" you want to calculate. The script calculates D1 -- which is the recommended variant overall and the one applicable to procedures where participants have to correct wrong responses ("built-in error penalty").

The table on page 214, on the other hand, contrasts the "conventional" scoring procedure with a different D-variant (the text says D2).

No, I didn’t think it would make sense, butas I am not a great stats mind (not even feeling like an average one at themoment!)I could easily have missed something.

The last d score (at trial 72 ofincompatible) is 1.742, the last D score at trial 72 of compatible is .0219.For participant 2 compatible was presented first followed by incompatible,which would suggest that D is 1.742. Yet my calculation has D as .22, whichmatches the compatible last. At least it matches something...it is odd though. Especially as  the ‘last’ lat.diffs fromInquisit are both different to the lat diff calculated in SPSS

Hi Dave,
You may have realised, but the reason I was getting two D scores is because I was still using 'block' when aggregating using the last function. hhmm. Looking forward to getting rid of the L plates.
Thank you for all of your help!