Millisecond Forums » Millisecond Forums » Inquisit 5 » Counterbalancing on MTurk

Counterbalancing on MTurk

darbyh — Fri, 03 Mar 2017 11:45:25 GMT

Hi!

I'm running a study using MTurk, and I'm wondering if there's a way to counterbalance subjects across four conditions on Inquisit. I'm using batches for each condition and have /groupassignment set to random for each batch, but I think this just randomizes, not counterbalances the subjects into conditions. I've read through other posts, but haven't been able to find a solution for this. Is this because it can't be done through Inquisit and must happen through MTurk?

Thank you!

RE: Counterbalancing on MTurk

darbyh — Fri, 03 Mar 2017 11:45:25 GMT

[quote]

[b]Dave - Friday, March 3, 2017[/b]

[quote]

[b]darbyh - Friday, March 3, 2017[/b]

[quote]

[b]Dave - Thursday, March 2, 2017[/b]

[quote]

[b]darbyh - Thursday, March 2, 2017[/b]

[/quote]

Hi Dave,

Thank you so much, as usual. In the first option you gave, would I program the Inquisit script to generate sequential group numbers 1-4?

[/quote]

> In the first option you gave, would I program the Inquisit script to generate sequential group numbers 1-4?

No, that's not something you do in the *script*. It's a simple setting in the web experiment's settings:

Every time a participant visits the launch page, the page will give her/him a sequential groupnumber i.e. 1 for the 1st visitor, 2 for the 2nd visitor, 3 for the 3rd, 4 for the 4th, 5 for the 5th and so forth.

Per https://www.millisecond.com/forums/Topic13856.aspx the person with id 1 gets the 1st condition, id 2 gets the 2nd condition, id 3 gets the 3rd condition, 4 gets the 4th condition, and then again 5 gets the 1st condition, id 6 gets the 2nd condition, and so forth. I.e. the condition assignment is groupnumber *modulo* the number of conditions.

[/quote]

Ohhh great. Thank you!

RE: Counterbalancing on MTurk

Dave — Fri, 03 Mar 2017 10:34:22 GMT

[quote]

[b]darbyh - Friday, March 3, 2017[/b]

[quote]

[b]Dave - Thursday, March 2, 2017[/b]

[quote]

[b]darbyh - Thursday, March 2, 2017[/b]

[/quote]

Hi Dave,

Thank you so much, as usual. In the first option you gave, would I program the Inquisit script to generate sequential group numbers 1-4?

[/quote]

RE: Counterbalancing on MTurk

darbyh — Fri, 03 Mar 2017 10:05:53 GMT

[quote]

[b]Dave - Thursday, March 2, 2017[/b]

[quote]

[b]darbyh - Thursday, March 2, 2017[/b]

[/quote]

Hi Dave,

Thank you so much, as usual. In the first option you gave, would I program the Inquisit script to generate sequential group numbers 1-4?

RE: Counterbalancing on MTurk

Dave — Thu, 02 Mar 2017 17:19:47 GMT

[quote]

[b]darbyh - Thursday, March 2, 2017[/b]

[/quote]

/groupassignment = random will indeed randomly (think: coin flip) determine to which condition a given participant is assigned. To get better balance across conditions, you can set

/ groupassignment = groupnumber

and either

(1) configure the web experiment to generate *sequential* group numbers, or
(2) for even more control read the group number from a query parameter supplied to the launch page via MTurk.

In the latter case, you would essentially give out four different links (one per condition), and distribute each link to a quarter of participants (how exactly you do that would be up to you -- I don't know whether MTurk offers an easy way to achieve this, some online survey platforms however do). The four links would only differ in the numerical value of the query parameter, i.e.

http://research.millisecond.com/yourusername/yourexperimentname.web?groupid=1 for condition #1,
http://research.millisecond.com/yourusername/yourexperimentname.web?groupid=2 for condition #2,
http://research.millisecond.com/yourusername/yourexperimentname.web?groupid=3 for condition #3,
and
http://research.millisecond.com/yourusername/yourexperimentname.web?groupid=4 for condition #4.

Also see https://www.millisecond.com/support/docs/v5/html/howto/interopsurveys.htm , which covers the use of query parameters for sharing subject identifiers, but it works the same for group numbers.