Millisecond Forums

+x

 

AnthonyFCollinsSussex - 5/12/2021

Hello. I have a remaining outstanding issue that I need to resolve before data collection, and I have been searching for the solution and would just like to double clarify if I am correct before I go ahead with it.

There are two conditions (Positive IBT training or Negative IBT training). In each of these conditions, there is a pre- and post- cognitive bias assessment that assesses cognitive bias before and after the IBT training type - RTA or RTB (Recognition Task). RTA and RTB are both the same test, only differing in material set used (e.g. set A or B). The order with which RTA and RTB are presented pre- and post- training are counterbalanced as can be seen below:

<batch>
/ subjects = (1 of 2)
/ groupassignment = groupnumber
/ file = "RTA.iqx"
/ file = "Positive IBT.iqx"
/ file = "RTB.iqx
</batch>

<batch>
/ subjects = (2 of 2)
/ groupassignment = groupnumber
/ file = "RTB.iqx"
/ file = "Positive IBT.iqx"
/ file = "RTA.iqx
</batch>

HOWEVER,  a random 50% of each training condition need to get RTA followed by RTB, and the other 50% the other way around. Hence material order is orthogonal to condition. In my data output, I would need information about which material set (e.g. RTA or RTB) were encountered first for each participant. I will eventually create in excel columns denoting something like  'material order' (0 = RTA first, 1 = RTB first), 'first RT score' (regardless of material type) 'second RT score' (regardless of material type). I can do these last parts independently, but the way I was going to approach this design set-up was to simply do the following:


// order RTA -> Positive IBT -> RTB
<batch>
/ subjects = (1 of 4)
/ groupassignment = groupnumber
/ file = "RTA.iqx"
/ file = "Positive IBT.iqx"
/ file = "RTB.iqx
</batch>

// order RTA -> Control IBT -> RTB
<batch>
/ subjects = (2 of 4)
/ groupassignment = groupnumber
/ file = "RTA.iqx"
/ file = "Positive IBT.iqx"
/ file = "RTB.iqx
</batch>

// order RTB -> Positive IBT -> RTA
<batch>
/ subjects = (3 of 4)
/ groupassignment = groupnumber
/ file = "RTB.iqx"
/ file = "Positive IBT.iqx"
/ file = "RTA.iqx
</batch>

// order RTB -> Control IBT -> RTA
<batch>
/ subjects = (4 of 4)
/ groupassignment = groupnumber
/ file = "RTB.iqx"
/ file = "Positive IBT.iqx"
/ file = "RTA.iqx
</batch>


As mentioned though, a random 50% of each training condition must get RTA followed by RTB, and the other 50% the other way around. I just need to check if this strategy is correct.

Yours Sincerely
Anthony

+x

 

AnthonyFCollinsSussex - 5/12/2021

One of those days, my bad :// order RTA -> Positive IBT -> RTB/ subjects = (1 of 4)/ groupassignment = groupnumber / file = "RTA.iqx"/ file = "Positive IBT.iqx"/ file = "RTB.iqx// order RTA -> Control IBT -> RTB/ subjects = (2 of 4)/ groupassignment = groupnumber/ file = "RTA.iqx"/ file = "Control IBT.iqx"/ file = "RTB.iqx// order RTB -> Positive IBT -> RTA/ subjects = (3 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Positive IBT.iqx"/ file = "RTA.iqx// order RTB -> Control IBT -> RTA/ subjects = (4 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Control IBT.iqx"/ file = "RTA.iqxThis is to ensure that we can identify that positive IBT does exert an effect that is not confounded by materials Set presentation (to rule out a material presentation effect, in this case A and B, we very much don’t want this , hence the above

+x

 

Dave - 5/12/2021

+x

 

AnthonyFCollinsSussex - 5/12/2021

One of those days, my bad :// order RTA -> Positive IBT -> RTB/ subjects = (1 of 4)/ groupassignment = groupnumber / file = "RTA.iqx"/ file = "Positive IBT.iqx"/ file = "RTB.iqx// order RTA -> Control IBT -> RTB/ subjects = (2 of 4)/ groupassignment = groupnumber/ file = "RTA.iqx"/ file = "Control IBT.iqx"/ file = "RTB.iqx// order RTB -> Positive IBT -> RTA/ subjects = (3 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Positive IBT.iqx"/ file = "RTA.iqx// order RTB -> Control IBT -> RTA/ subjects = (4 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Control IBT.iqx"/ file = "RTA.iqxThis is to ensure that we can identify that positive IBT does exert an effect that is not confounded by materials Set presentation (to rule out a material presentation effect, in this case A and B, we very much don’t want this , hence the above

Okay, that looks correct to me now. Group ID will be logged in the data files (unless you deliberatey excluded it), and you can derive order (RTA first vs RTB first) from that.
As for how condition assignment works generally, see https://www.millisecond.com/forums/Topic13856.aspx

+x

 

AnthonyFCollinsSussex - 5/14/2021

+x

 

Dave - 5/12/2021

+x

 

AnthonyFCollinsSussex - 5/12/2021

One of those days, my bad :// order RTA -> Positive IBT -> RTB/ subjects = (1 of 4)/ groupassignment = groupnumber / file = "RTA.iqx"/ file = "Positive IBT.iqx"/ file = "RTB.iqx// order RTA -> Control IBT -> RTB/ subjects = (2 of 4)/ groupassignment = groupnumber/ file = "RTA.iqx"/ file = "Control IBT.iqx"/ file = "RTB.iqx// order RTB -> Positive IBT -> RTA/ subjects = (3 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Positive IBT.iqx"/ file = "RTA.iqx// order RTB -> Control IBT -> RTA/ subjects = (4 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Control IBT.iqx"/ file = "RTA.iqxThis is to ensure that we can identify that positive IBT does exert an effect that is not confounded by materials Set presentation (to rule out a material presentation effect, in this case A and B, we very much don’t want this , hence the above

Okay, that looks correct to me now. Group ID will be logged in the data files (unless you deliberatey excluded it), and you can derive order (RTA first vs RTB first) from that.
As for how condition assignment works generally, see https://www.millisecond.com/forums/Topic13856.aspx

Hello Dave

Hope all is well. So it turns out this set-up is not quite right and may still impose an element of order effect (as it is sequential). I need to check whether the following is implementable within Inquisit, or whether this needs to be done in Qualtrics . Given there are 4 'conditions' in the above, we need to create the following array, where the length is determined by maximum number of participants (we hope this to be 150). I have gone for 20 below for sake of ease:

Condition = [ 1 3 2 4 4 3 1 2 2 3 4 1 1 2 3 4 4 2 1 3]

The condition for each participant would be based on the order in which they are run, such that the first participant would be allocated to condition 1 (the first number in the condition array), the second person would be allocate to condition 3 (the second number in the condition array) etc. the array is constructed such that the order of every 4 participants is random but includes one in each condition i.e. above you have 1 3 2 4 followed by 4 3 1 2 etc. In this way you know that provided you run a multiple of 4 participants you will always have an equal number in each of the 4 conditions. Is there a way I can achieve this?

+x

 

Dave - 5/14/2021

+x

 

AnthonyFCollinsSussex - 5/14/2021

+x

 

Dave - 5/12/2021

+x

 

AnthonyFCollinsSussex - 5/12/2021

One of those days, my bad :// order RTA -> Positive IBT -> RTB/ subjects = (1 of 4)/ groupassignment = groupnumber / file = "RTA.iqx"/ file = "Positive IBT.iqx"/ file = "RTB.iqx// order RTA -> Control IBT -> RTB/ subjects = (2 of 4)/ groupassignment = groupnumber/ file = "RTA.iqx"/ file = "Control IBT.iqx"/ file = "RTB.iqx// order RTB -> Positive IBT -> RTA/ subjects = (3 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Positive IBT.iqx"/ file = "RTA.iqx// order RTB -> Control IBT -> RTA/ subjects = (4 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Control IBT.iqx"/ file = "RTA.iqxThis is to ensure that we can identify that positive IBT does exert an effect that is not confounded by materials Set presentation (to rule out a material presentation effect, in this case A and B, we very much don’t want this , hence the above

Okay, that looks correct to me now. Group ID will be logged in the data files (unless you deliberatey excluded it), and you can derive order (RTA first vs RTB first) from that.
As for how condition assignment works generally, see https://www.millisecond.com/forums/Topic13856.aspx

Hello Dave

Hope all is well. So it turns out this set-up is not quite right and may still impose an element of order effect (as it is sequential). I need to check whether the following is implementable within Inquisit, or whether this needs to be done in Qualtrics . Given there are 4 'conditions' in the above, we need to create the following array, where the length is determined by maximum number of participants (we hope this to be 150). I have gone for 20 below for sake of ease:

Condition = [ 1 3 2 4 4 3 1 2 2 3 4 1 1 2 3 4 4 2 1 3]

The condition for each participant would be based on the order in which they are run, such that the first participant would be allocated to condition 1 (the first number in the condition array), the second person would be allocate to condition 3 (the second number in the condition array) etc. the array is constructed such that the order of every 4 participants is random but includes one in each condition i.e. above you have 1 3 2 4 followed by 4 3 1 2 etc. In this way you know that provided you run a multiple of 4 participants you will always have an equal number in each of the 4 conditions. Is there a way I can achieve this?

If you select random (without replacement) group number generation in the Inquisit Web experiment's settings and indicate the number of groups as 4, then for each set of four participants that visit the start page, one will be randomly assigned to condition 1, one will be randomly assigned to condition 2, one to condition 3, one to condition 4.

If this is not to your liking, you can have Qualtrics perform the condition assignment and have it send group ID over via URL parameter, just like subject ID.

+x

 

AnthonyFCollinsSussex - 5/14/2021

+x

 

Dave - 5/14/2021

+x

 

AnthonyFCollinsSussex - 5/14/2021

+x

 

Dave - 5/12/2021

+x

 

AnthonyFCollinsSussex - 5/12/2021

One of those days, my bad :// order RTA -> Positive IBT -> RTB/ subjects = (1 of 4)/ groupassignment = groupnumber / file = "RTA.iqx"/ file = "Positive IBT.iqx"/ file = "RTB.iqx// order RTA -> Control IBT -> RTB/ subjects = (2 of 4)/ groupassignment = groupnumber/ file = "RTA.iqx"/ file = "Control IBT.iqx"/ file = "RTB.iqx// order RTB -> Positive IBT -> RTA/ subjects = (3 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Positive IBT.iqx"/ file = "RTA.iqx// order RTB -> Control IBT -> RTA/ subjects = (4 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Control IBT.iqx"/ file = "RTA.iqxThis is to ensure that we can identify that positive IBT does exert an effect that is not confounded by materials Set presentation (to rule out a material presentation effect, in this case A and B, we very much don’t want this , hence the above

Okay, that looks correct to me now. Group ID will be logged in the data files (unless you deliberatey excluded it), and you can derive order (RTA first vs RTB first) from that.
As for how condition assignment works generally, see https://www.millisecond.com/forums/Topic13856.aspx

Hello Dave

Hope all is well. So it turns out this set-up is not quite right and may still impose an element of order effect (as it is sequential). I need to check whether the following is implementable within Inquisit, or whether this needs to be done in Qualtrics . Given there are 4 'conditions' in the above, we need to create the following array, where the length is determined by maximum number of participants (we hope this to be 150). I have gone for 20 below for sake of ease:

Condition = [ 1 3 2 4 4 3 1 2 2 3 4 1 1 2 3 4 4 2 1 3]

The condition for each participant would be based on the order in which they are run, such that the first participant would be allocated to condition 1 (the first number in the condition array), the second person would be allocate to condition 3 (the second number in the condition array) etc. the array is constructed such that the order of every 4 participants is random but includes one in each condition i.e. above you have 1 3 2 4 followed by 4 3 1 2 etc. In this way you know that provided you run a multiple of 4 participants you will always have an equal number in each of the 4 conditions. Is there a way I can achieve this?

If you select random (without replacement) group number generation in the Inquisit Web experiment's settings and indicate the number of groups as 4, then for each set of four participants that visit the start page, one will be randomly assigned to condition 1, one will be randomly assigned to condition 2, one to condition 3, one to condition 4.

If this is not to your liking, you can have Qualtrics perform the condition assignment and have it send group ID over via URL parameter, just like subject ID.

Sounds like this set-up achieves what we need it to, in other words participant 1 could be assigned to any one of the 4, let's say at random, 3rd condition, the second participant gets assigned to the 1st condition, and then the final two participants will be randomly allocated to one of the final remaining conditions (e.g. 2nd and 4th condition in this case). Then this randomizing process restarts for the next 4 participants (with a different randomised order/allocation) and so on. If I have understood this correctly, then we should not need Qualtrics, and this achieves the array above (random 50% of each training condition get RTA followed by RTB, and the other 50% the other way around)

Thank you Dave again, this is very much appreciated

+x

 

Dave - 5/14/2021

+x

 

AnthonyFCollinsSussex - 5/14/2021

+x

 

Dave - 5/14/2021

+x

 

AnthonyFCollinsSussex - 5/14/2021

+x

 

Dave - 5/12/2021

+x

 

AnthonyFCollinsSussex - 5/12/2021

One of those days, my bad :// order RTA -> Positive IBT -> RTB/ subjects = (1 of 4)/ groupassignment = groupnumber / file = "RTA.iqx"/ file = "Positive IBT.iqx"/ file = "RTB.iqx// order RTA -> Control IBT -> RTB/ subjects = (2 of 4)/ groupassignment = groupnumber/ file = "RTA.iqx"/ file = "Control IBT.iqx"/ file = "RTB.iqx// order RTB -> Positive IBT -> RTA/ subjects = (3 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Positive IBT.iqx"/ file = "RTA.iqx// order RTB -> Control IBT -> RTA/ subjects = (4 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Control IBT.iqx"/ file = "RTA.iqxThis is to ensure that we can identify that positive IBT does exert an effect that is not confounded by materials Set presentation (to rule out a material presentation effect, in this case A and B, we very much don’t want this , hence the above

Okay, that looks correct to me now. Group ID will be logged in the data files (unless you deliberatey excluded it), and you can derive order (RTA first vs RTB first) from that.
As for how condition assignment works generally, see https://www.millisecond.com/forums/Topic13856.aspx

Hello Dave

Hope all is well. So it turns out this set-up is not quite right and may still impose an element of order effect (as it is sequential). I need to check whether the following is implementable within Inquisit, or whether this needs to be done in Qualtrics . Given there are 4 'conditions' in the above, we need to create the following array, where the length is determined by maximum number of participants (we hope this to be 150). I have gone for 20 below for sake of ease:

Condition = [ 1 3 2 4 4 3 1 2 2 3 4 1 1 2 3 4 4 2 1 3]

The condition for each participant would be based on the order in which they are run, such that the first participant would be allocated to condition 1 (the first number in the condition array), the second person would be allocate to condition 3 (the second number in the condition array) etc. the array is constructed such that the order of every 4 participants is random but includes one in each condition i.e. above you have 1 3 2 4 followed by 4 3 1 2 etc. In this way you know that provided you run a multiple of 4 participants you will always have an equal number in each of the 4 conditions. Is there a way I can achieve this?

If you select random (without replacement) group number generation in the Inquisit Web experiment's settings and indicate the number of groups as 4, then for each set of four participants that visit the start page, one will be randomly assigned to condition 1, one will be randomly assigned to condition 2, one to condition 3, one to condition 4.

If this is not to your liking, you can have Qualtrics perform the condition assignment and have it send group ID over via URL parameter, just like subject ID.

Sounds like this set-up achieves what we need it to, in other words participant 1 could be assigned to any one of the 4, let's say at random, 3rd condition, the second participant gets assigned to the 1st condition, and then the final two participants will be randomly allocated to one of the final remaining conditions (e.g. 2nd and 4th condition in this case). Then this randomizing process restarts for the next 4 participants (with a different randomised order/allocation) and so on. If I have understood this correctly, then we should not need Qualtrics, and this achieves the array above (random 50% of each training condition get RTA followed by RTB, and the other 50% the other way around)

Thank you Dave again, this is very much appreciated

> in other words participant 1 could be assigned to any one of the 4, let's say at random, 3rd condition, the second participant gets assigned to the 1st condition, and then the final two participants will be randomly allocated to one of the final remaining conditions (e.g. 2nd and 4th condition in this case)

Yes, that's how it works. Note, though, that group ID generation happens when the participant visits the start page, regardless of whether that particpant ends up actually launching the study at that point or not. So you will see gaps due to dropouts, that's just a reality of online research