Millisecond Forums

Counterbalancing based on completed tasks

https://forums.millisecond.com/Topic17297.aspx

By luca7 - 9/29/2015

Dear Inquisit users,

my experiment counterbalances lists by subject number, for example (I don’t think the details of the experiment are important):

<variables>
/group=(1 of 3) (mwstimsList=mwstims1, questionList=question1)
/group=(2 of 3) (mwstimsList=mwstims2, questionList=question2)
/group=(3 of 3) (mwstimsList=mwstims3, questionList=question3)
</variables>

This experiment will be run on the internet and the problem is that I’ll have (I know it based on experience) many people starting the experiment and then aborting it. For that reason, the number of participants that will complete the experiment in the different lists will be unbalanced, that is, the dropping rate will be larger for one list than another.

How can fix that problem?
An example that could work (but I don’t know if it can be done and how) would be making Inquisit to write in a text file the number of participants that complete the experiment. For example, at the end of the experiment Inquisit opens that file, reads the number, adds one and replaces the number, then saves it. When a new participant starts, Inquisint opens that file, reads the number, and uses that number to assign the participant to the different lists. It’s just an idea.

Do you have any feedback, idea, or example to help me to fix that problem?

Thank you!
Luca

By Dave - 9/29/2015

> An example that could work (but I don’t know if it can be done and how) would be making Inquisit to write in a text file the
> number of participants that complete the experiment. For example, at the end of the experiment Inquisit opens that file, reads
> the number, adds one and replaces the number, then saves it. When a new participant starts, Inquisint opens that file, reads
> the number, and uses that number to assign the participant to the different lists. It’s just an idea.

That can't be done using Inquisit alone. You'd have to build web service infrastructure around it to make that work and take care of the actual quota management.

I guess the easiest, low-tech solution would be to (1) monitor the distribution across conditions during the initial phase of the study (by regularly reviewing the available data files) and at some point (2) disable those conditions that are "full" and only keep running those that need a bunch more participants.

Hope this helps.
By luca7 - 9/29/2015

Thank you for your answer, Dave.
Well, it's not ideal but it may work.
Can I modify a running experiment? I mean, what happens if I change the active experiment (say I activate my modified version) when somebody is doing it?

thank you again
Luca

By Dave - 9/29/2015

Yes, you can modify it. Since Inquisit web scripts aren't executed on the server, it does not matter whether someone is doing the experiment at the moment or not. When someone starts the experiment, Inquisit downloads the script to the participant's machine. Once downloaded, it is executed locally. Replacing the script on the server during this time has no effect on that participant.

Nonetheless, it is certainly a good idea to pick a time when the likelihood of many participants trying to access the experiment is low to switch the script.
By luca7 - 9/29/2015

thank you very much again, Dave.
I really appreciate your help.

Luca
By Dave - 9/29/2015

You're absolutely welcome. If any further questions arise, please don't hesitate to post them.