Millisecond Forums » Millisecond Forums » Inquisit 5 » selecting survey questions randomly from a list

selecting survey questions randomly from a list

brendan — Thu, 15 Jun 2017 16:51:31 GMT

Hi,

The high level problem that I am trying to solve is that I have a scale with 20 items that I want to randomly distribute over 4 survey pages with 5 items per page.

I found this forum post from a while back that I thought provided an approach that would work:

https://www.millisecond.com/forums/Topic11044.aspx

That approach is to define a number of survey items (in this case, radiobuttons elements) and store them in a list.

Then, I would reference that list in the questions attribute of the four surveypages.

Near the end of that thread, it looks like they discover a bug that prevents Inquisit from saving responses.

After some experimentation, I think I got this approach to work in terms of user interaction, but I may be running into the same bug saving data.

Below is my test code which runs for me in version 5.0.7.0. For simplicity, there are only four questions presented over two pages.

I did verify that other, simpler surveys successfully save data for me.

Any idea about what is going on or suggestions to solve this problem some other way would be welcome.

Thanks!

[code]<survey myscale>
/ pages = [1 = scale1; 2 = scale2]
/ showpagenumbers = false
/ itemfontstyle = ("Times New Roman", 12pt, false, false, false, false, 5, 0)
/ responsefontstyle = ("Lucida Console", 7pt, false, false, false, false, 5, 0)
</survey>

<surveypage scale1>
/ questions = [
1 = list.scale;
2 = list.scale;
]
/ showquestionnumbers = false
/ showbackbutton = false
/ nextbuttonposition = (90%, 90%)
</surveypage>

<surveypage scale2>
/ questions = [
1 = list.scale;
2 = list.scale;
]
/ showquestionnumbers = false
/ showbackbutton = false
/ nextbuttonposition = (90%, 90%)
</surveypage>

<list scale>
/ items = (
radiobuttons.item01,
radiobuttons.item02,
radiobuttons.item03,
radiobuttons.item04)
/ selectionmode = random
/ selectionrate = always
</list>

<radiobuttons item01>
/ caption = "Item 01 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<radiobuttons item02>
/ caption = "Item 02 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<radiobuttons item03>
/ caption = "Item 03 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<radiobuttons item04>
/ caption = "Item 04 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>[/code]

RE: selecting survey questions randomly from a list

Dave — Thu, 15 Jun 2017 16:51:31 GMT

[quote]

[b]brendan - Thursday, June 15, 2017[/b]

Thanks! Of the options, I think I like the second best because it seems to require less code redundancy.

I tested option 2 and it seems to work well for the use case where 20 items are divided evenly over four pages.

In option 2, how exactly is the interpreter handling the noreplace list in the surveypage element?

I ask only because it looks like there may be some general principle operating there that I can apply to thinking about attributes for elements being accessed in plural/list contexts in general.

I did run into one minor complication when trying to use this method for another scale that has 13 items.

In the 13 item case, the last page should only have 3 items.

It seems like the implied list element (or whatever it is) that the noreplace is pulling from should be empty after the third item on the last page of a 13 item scale.

However, instead it seems to duplicate several items (as if it starts over, or something like that).

Do you have any suggestions about how to deal with a scale that a number of items that does not quite fill up the last page, but that I still want to present randomly as described above? Other than deleting a few items or adding filler items.

[/quote]

So the general principle of a noreplace() pool goes something like this. I'm using <trial> elements sampled by in <block> for the illustration:

<block myblock>
/ trials = [1-4 = noreplace(a,b,c,d)]
</block>

works out evenly, you'll get 1x trial "a", 1x trial "b", 1x trial "c", and 1x trial "d" in random order. Similarly with an exact multiple of 4:

/ trials = [1-8 = noreplace(a,b,c,d)]

here the pool effectively expands to noreplace(a,a,b,b,c,c,d,d)

i.e. you'll end up with 2x "a", 2x "b", 2x "c", and 2x "d" in random order.

You run into problems if you oversample, i.e. something like this:

/ trials = [1-5 = noreplace(a,b,c,d)]

Here, too, the pool effectively expands to noreplace(a,a,b,b,c,c,d,d) and then 5 out of the 8 elements contained are sampled at random. I.e. you might end up with something like

b -> a -> b -> d -> d

This would be the analogue to your 13-question / scale case.

Unfortunately, I can't see an elegant way to deal with that using noreplace(). Hence my advice would be to go for the 1st option outlined (<list> plus <summarydata>) in my previous reply, despite the slightly increased amount of code required.

Hope this helps.

RE: selecting survey questions randomly from a list

brendan — Thu, 15 Jun 2017 15:43:51 GMT

RE: selecting survey questions randomly from a list

Dave — Thu, 15 Jun 2017 14:04:50 GMT

[quote]

[b]brendan - Thursday, June 15, 2017[/b]

https://www.millisecond.com/forums/Topic11044.aspx

[/quote]

The <list> approach sadly won't work like that (for boring technical reasons), but here are two options that should work:

#1: A slight extension of the <list> approach that captures the responses via <summarydata>:

<survey myscale>
/ pages = [1 = scale1; 2 = scale2]
/ showpagenumbers = false
/ itemfontstyle = ("Times New Roman", 12pt, false, false, false, false, 5, 0)
/ responsefontstyle = ("Lucida Console", 7pt, false, false, false, false, 5, 0)
</survey>

<surveypage scale1>
/ questions = [
    1 = list.scale;
    2 = list.scale;
    ]
/ showquestionnumbers = false
/ showbackbutton = false
/ nextbuttonposition = (90%, 90%)
</surveypage>

<surveypage scale2>
/ questions = [
    1 = list.scale;
    2 = list.scale;
    ]
/ showquestionnumbers = false
/ showbackbutton = false
/ nextbuttonposition = (90%, 90%)
</surveypage>

<list scale>
/ items = (
    radiobuttons.item01,
    radiobuttons.item02,
    radiobuttons.item03,
    radiobuttons.item04)
/ selectionmode = random
/ selectionrate = always
</list>

<radiobuttons item01>
/ caption = "Item 01 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<radiobuttons item02>
/ caption = "Item 02 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<radiobuttons item03>
/ caption = "Item 03 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<radiobuttons item04>
/ caption = "Item 04 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<summarydata>
/ columns = (script.startdate script.starttime script.subjectid script.groupid
    radiobuttons.item01.response radiobuttons.item02.response radiobuttons.item03.response radiobuttons.item04.response)
/ separatefiles = true
</summarydata>

#2: Using a <block>, a single <surveypage> and a noreplace() selection pool for the four questions:

<block myscale>
/ trials = [1-2 = scale]
</block>

<surveypage scale>
/ questions = [1,2 = noreplace(item01,item02,item03,item04)]
/ showquestionnumbers = false
/ showbackbutton = false
/ nextbuttonposition = (90%, 90%)
/ showpagenumbers = false
/ itemfontstyle = ("Times New Roman", 12pt, false, false, false, false, 5, 0)
/ responsefontstyle = ("Lucida Console", 7pt, false, false, false, false, 5, 0)
</surveypage>

<radiobuttons item01>
/ caption = "Item 01 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<radiobuttons item02>
/ caption = "Item 02 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<radiobuttons item03>
/ caption = "Item 03 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>

<radiobuttons item04>
/ caption = "Item 04 text"
/ options = ("Strongly disagree", " ", " ", " ", " ", " ", "Strongly agree")
/ optionvalues = ("1", "2", "3", "4", "5", "6", "7")
/ orientation = horizontalequal
/ required = false
</radiobuttons>