Millisecond Forums

Participants quitting- random assignment issue

https://forums.millisecond.com/Topic17375.aspx

By jennifer.perry - 10/12/2015

Hello,

We are currently hosting an experiment using a combination of Qualtrics and Inquisit and are stumped about a random assignment issue. We have 6 between subjects conditions. Right now, participants complete the consent form in Qualtrics, which then randomly (and evenly) sends them to the task with one of six links that have a URL query parameter that defines the condition. Inquisit then uses this parameter as the group number and loads the correct task. 

The problem we're having is that some of the participants (MTurkers) quit the study after Qualtrics sends them to the task. In this case, Qualtrics does not know that the experiment wasn't completed and continues to randomly assign to the conditions as though the cell counts are even. Some of our cell counts are therefore less than others, as by chance, participants have quit more within certain conditions than in others. 

We would like to re-run this study and are wondering if there a way to use random assignment methods in Inquisit that would update the condition counts only if the task was completed? Or any other way that would get around this issue?
By Dave - 10/12/2015

Since it is Qualtrics that tracks the condition counts, there's not much you could do on the Inquisit-side, especially if a participant never even starts Inquisit. The only way I can think of is to send participants back to Qualtrics after completing the Inquisit-portion and set up Qualtrics to only update condition counts if that redirection actually happens (i.e., if it "sees" the participant again at a specific URL). Participants who never start the Inquisit-portion would never get there and thus not inflate your condition counts artificially. Whether that's technically possible in Qualtrics, I don't know (but I would think it is).

Hope this helps.
By jennifer.perry - 10/13/2015

Great, thank you, I can definitely try that. 

I'm wondering, though, if I don't try to randomize with Qualtrics at all, is there a way to randomize only with Inquisit that can take into account if people quit?
By Dave - 10/13/2015

> I'm wondering, though, if I don't try to randomize with Qualtrics at all, is there a way to randomize only with Inquisit that can take > into account if people quit?

At present there isn't anything that would do that automatically, no.