Millisecond Forums

Getting all permutations of possible list selections

https://forums.millisecond.com/Topic29472.aspx

By AKrishna - 7/21/2020

Hi all,

this one's more of a general programming question. I'm planning an Inquisit Web experiment where I want to show people videos. The videos vary according to the following factors:
2 (gender of target individual in video) x 2 (presence vs. absence of nonverbal expression type A) x 2 (presence vs. absence of nonverbal expression type B) x 4 (specific individual in video).
I have a full set of 32 videos covering all combinations. For reasons of experiment length, I'm showing participants a subset of 8 videos such that for every combination of gender x expression A x expression B, participants see exactly one video with one randomly-drawn unique individual (so a specific target individual is only shown once in the set, for one combination of factors). Ultimately, this video selection is shown twice, as every participant first sees a short version of the 8 videos first (for prospective judgments) and the longer version later on in the experiment (for post-hoc judgments).

So far so good, and I have code that does this for me in a lab experiment. However, my video files are quite large (~30 MB per file for short videos and ~80 MB per file for large ones), and in a Web experiment, Inquisit would download ALL of the potential files at the beginning (over 1.3 GB!) in order to draw a subset from them. I want to avoid that, so I thought I'd set up a batch file that randomly selects a version of the experiment with a given stimulus selection set and runs that. That should mean that participants are only downloading exactly what they need.

So... essentially I need to write 576 versions of the same file that differ only in two item lists (short and long video filenames). Obviously, I don't want to do this by hand, both due to the labour involved and the potential for human error in going through all possible permutations. My current solution is to create the stimulus lists via script as separate text files, then use a cmd batch to copy my basic Inquisit experiment lots of times and append an <include> element with the individual text file names. So it would look like this:

First I use a simple Inquisit script with the monkey to get me my item lists:

<expt Ladeda>
/ blocks = [1 = ElementStart1]
</expt>

<block ElementStart1>
/ trials = [1 = ElementStart1]
/ branch = [block.Creator]
</block>

<trial ElementStart1>
/ ontrialbegin = [values.Output = "<item VideoList1>"]
/ trialduration = 0
</trial>

<block ElementEnd1>
/ trials = [1 = ElementEnd1]
/ branch = [block.ElementStart2]
</block>

<trial ElementEnd1>
/ ontrialbegin = [values.Output = "</item>"]
/ trialduration = 0
</trial>

<block ElementEnd2>
/ trials = [1 = ElementEnd1]
</block>

<block ElementStart2>
/ trials = [1 = ElementStart2]
/ branch = [block.Creator2]
</block>

<trial ElementStart2>
/ ontrialbegin = [values.Output = "<item VideoList2>"]
/ trialduration = 0
</trial>

<block Creator>
/ onblockbegin = [values.CurrentTrial = 0]
/ trials = [1-8 = Creator]
/ branch = [block.ElementEnd1]
</block>

<trial Creator>
/ ontrialbegin = [values.CurrentTrial += 1]
/ ontrialbegin = [if (values.CurrentTrial >= 5) values.MF = "F" else values.MF = "M"]
/ ontrialbegin = [if (values.CurrentTrial == 1) values.1 = list.ABCD.nextvalue
else if (values.CurrentTrial == 2) values.2 = list.ABCD.nextvalue
else if (values.CurrentTrial == 3) values.3 = list.ABCD.nextvalue
else if (values.CurrentTrial == 4) values.4 = list.ABCD.nextvalue
else if (values.CurrentTrial == 5) values.5 = list.ABCD.nextvalue
else if (values.CurrentTrial == 6) values.6 = list.ABCD.nextvalue
else if (values.CurrentTrial == 7) values.7 = list.ABCD.nextvalue
else if (values.CurrentTrial == 8) values.8 = list.ABCD.nextvalue]
/ ontrialend = [values.Output = concat("/",concat(values.CurrentTrial,concat("=~"",concat(values.MF,concat("/Bis Eskalation/",concat(evaluate(
concat("values.",values.CurrentTrial)),concat("_1st_head",concat(expressions.dom,concat(expressions.pain,"_bE.mp4~"")))))))))]
/ trialduration = 0
</trial>

<list ABCD>
/ items = ("A","B","C","D")
/ selectionmode = random
/ selectionrate = trial
</list>

<block Creator2>
/ onblockbegin = [values.CurrentTrial = 0]
/ trials = [1-8 = Creator2]
/ branch = [block.ElementEnd2]
</block>

<trial Creator2>
/ ontrialbegin = [values.CurrentTrial += 1]
/ ontrialbegin = [if (values.CurrentTrial >= 5) values.MF = "F" else values.MF = "M"]
/ ontrialend = [values.Output = concat("/",concat(values.CurrentTrial,concat("=~"",concat(values.MF,concat("/Ganz/",concat(evaluate(
concat("values.",values.CurrentTrial)),concat("_1st_head",concat(expressions.dom,concat(expressions.pain,".mp4~"")))))))))]
/ trialduration = 0
</trial>


<expressions>
/pain = if (mod(values.CurrentTrial,2) == 1) "kL" else "L"
/dom = if (mod(values.CurrentTrial-1,4) <= 1) "_dom_" else "_sub_"
</expressions>

<values>
/CurrentTrial = 0
/MF = "F"
/1 = ""
/2 = ""
/3 = ""
/4 = ""
/5 = ""
/6 = ""
/7 = ""
/8 = ""
/Output = ""
</values>

<data>
/ file = "Lists/List.iqdat"
/ columns = (values.Output)
/ separatefiles = true
</data>


This code outputs a data file called "List.iqdat" that contains my different <item> elements (and is the source of one of my major questions below). I run this script 100 times using the monkey to create lots of lists. Then I use a cmd batch to rename those files for easier use:

cd "DIRECTORY WHERE LISTS ARE"
del *.txt
ren *.iqdat *.txt
set /a count=1
setlocal ENABLEDELAYEDEXPANSION
for /f %%G IN ('dir /b') DO (
echo "%%G"
ren %%G*.* !count!.*
set /a count+=1
echo !count!)

move /y *.txt "STIM DIRECTORY FOR MY EXPERIMENT"
pause


Then, I use another cmd batch to grab my original experiment file, copy it 100 times and append the <include> element (as well as make an Inquisit batch file).

xcopy "PATH TO MY EXPERIMENT\MyExperiment.iqx" "TEMPORARY DIRECTORY"
ren "MyExperiment.iqx" "MyExperiment1.iqx"
for /l %%G IN (2,1,100) DO (
>> "MyExperiment"%%G".iqx" rem/
xcopy /y "MyExperiment1.iqx" "MyExperiment"%%G".iqx")

for /l %%G IN (1,1,100) DO (
echo. >> "MyExperiment"%%G".iqx"
echo. >> "MyExperiment"%%G".iqx"
echo ^<include^> >> "MyExperiment"%%G".iqx"
echo /file = ^"Lists/%%G.txt^" >> "MyExperiment"%%G".iqx"
echo ^</include^> >> "MyExperiment"%%G".iqx")

for /l %%G IN (1,1,100) DO (
move "MyExperiment"%%G".iqx" "PATH TO MY EXPERIMENT")


>> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx" rem/
echo ^<batch^> >> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx"
echo / selectionmode=random >> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx"
for /l %%G IN (1,1,100) DO (
echo / file = ^"MyExperiment%%G.iqx^" >> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx")
echo ^</batch^> >> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx"
pause


This all seems to work, for now.

So, here are my questions (sorry for the wall of text):

1) Is there a better approach to this that prevents participants from having to download unnecessary files?
2) If not, you might have noticed that my Inquisit file that generates the <item> elements in the text files basically just randomly draws combinations. This means I can't be sure that any particular stimulus set configuration of the 576 possibilities is roughly equally likely to be drawn as the others in the final experiment unless I output an arbitrarily high number of possible configurations, many of which will be redundant. Is there a way to force Inquisit to permutate the combinations it draws? That way, I could just generate exactly the 576 possibilities and not have any one configuration be more likely than any other.

Many thanks if anyone has the patience to actually work through this. :-/


By Dave - 7/21/2020

AKrishna - 7/21/2020
Hi all,

this one's more of a general programming question. I'm planning an Inquisit Web experiment where I want to show people videos. The videos vary according to the following factors:
2 (gender of target individual in video) x 2 (presence vs. absence of nonverbal expression type A) x 2 (presence vs. absence of nonverbal expression type B) x 4 (specific individual in video).
I have a full set of 32 videos covering all combinations. For reasons of experiment length, I'm showing participants a subset of 8 videos such that for every combination of gender x expression A x expression B, participants see exactly one video with one randomly-drawn unique individual (so a specific target individual is only shown once in the set, for one combination of factors). Ultimately, this video selection is shown twice, as every participant first sees a short version of the 8 videos first (for prospective judgments) and the longer version later on in the experiment (for post-hoc judgments).

So far so good, and I have code that does this for me in a lab experiment. However, my video files are quite large (~30 MB per file for short videos and ~80 MB per file for large ones), and in a Web experiment, Inquisit would download ALL of the potential files at the beginning (over 1.3 GB!) in order to draw a subset from them. I want to avoid that, so I thought I'd set up a batch file that randomly selects a version of the experiment with a given stimulus selection set and runs that. That should mean that participants are only downloading exactly what they need.

So... essentially I need to write 576 versions of the same file that differ only in two item lists (short and long video filenames). Obviously, I don't want to do this by hand, both due to the labour involved and the potential for human error in going through all possible permutations. My current solution is to create the stimulus lists via script as separate text files, then use a cmd batch to copy my basic Inquisit experiment lots of times and append an <include> element with the individual text file names. So it would look like this:

First I use a simple Inquisit script with the monkey to get me my item lists:

<expt Ladeda>
/ blocks = [1 = ElementStart1]
</expt>

<block ElementStart1>
/ trials = [1 = ElementStart1]
/ branch = [block.Creator]
</block>

<trial ElementStart1>
/ ontrialbegin = [values.Output = "<item VideoList1>"]
/ trialduration = 0
</trial>

<block ElementEnd1>
/ trials = [1 = ElementEnd1]
/ branch = [block.ElementStart2]
</block>

<trial ElementEnd1>
/ ontrialbegin = [values.Output = "</item>"]
/ trialduration = 0
</trial>

<block ElementEnd2>
/ trials = [1 = ElementEnd1]
</block>

<block ElementStart2>
/ trials = [1 = ElementStart2]
/ branch = [block.Creator2]
</block>

<trial ElementStart2>
/ ontrialbegin = [values.Output = "<item VideoList2>"]
/ trialduration = 0
</trial>

<block Creator>
/ onblockbegin = [values.CurrentTrial = 0]
/ trials = [1-8 = Creator]
/ branch = [block.ElementEnd1]
</block>

<trial Creator>
/ ontrialbegin = [values.CurrentTrial += 1]
/ ontrialbegin = [if (values.CurrentTrial >= 5) values.MF = "F" else values.MF = "M"]
/ ontrialbegin = [if (values.CurrentTrial == 1) values.1 = list.ABCD.nextvalue
else if (values.CurrentTrial == 2) values.2 = list.ABCD.nextvalue
else if (values.CurrentTrial == 3) values.3 = list.ABCD.nextvalue
else if (values.CurrentTrial == 4) values.4 = list.ABCD.nextvalue
else if (values.CurrentTrial == 5) values.5 = list.ABCD.nextvalue
else if (values.CurrentTrial == 6) values.6 = list.ABCD.nextvalue
else if (values.CurrentTrial == 7) values.7 = list.ABCD.nextvalue
else if (values.CurrentTrial == 8) values.8 = list.ABCD.nextvalue]
/ ontrialend = [values.Output = concat("/",concat(values.CurrentTrial,concat("=~"",concat(values.MF,concat("/Bis Eskalation/",concat(evaluate(
concat("values.",values.CurrentTrial)),concat("_1st_head",concat(expressions.dom,concat(expressions.pain,"_bE.mp4~"")))))))))]
/ trialduration = 0
</trial>

<list ABCD>
/ items = ("A","B","C","D")
/ selectionmode = random
/ selectionrate = trial
</list>

<block Creator2>
/ onblockbegin = [values.CurrentTrial = 0]
/ trials = [1-8 = Creator2]
/ branch = [block.ElementEnd2]
</block>

<trial Creator2>
/ ontrialbegin = [values.CurrentTrial += 1]
/ ontrialbegin = [if (values.CurrentTrial >= 5) values.MF = "F" else values.MF = "M"]
/ ontrialend = [values.Output = concat("/",concat(values.CurrentTrial,concat("=~"",concat(values.MF,concat("/Ganz/",concat(evaluate(
concat("values.",values.CurrentTrial)),concat("_1st_head",concat(expressions.dom,concat(expressions.pain,".mp4~"")))))))))]
/ trialduration = 0
</trial>


<expressions>
/pain = if (mod(values.CurrentTrial,2) == 1) "kL" else "L"
/dom = if (mod(values.CurrentTrial-1,4) <= 1) "_dom_" else "_sub_"
</expressions>

<values>
/CurrentTrial = 0
/MF = "F"
/1 = ""
/2 = ""
/3 = ""
/4 = ""
/5 = ""
/6 = ""
/7 = ""
/8 = ""
/Output = ""
</values>

<data>
/ file = "Lists/List.iqdat"
/ columns = (values.Output)
/ separatefiles = true
</data>


This code outputs a data file called "List.iqdat" that contains my different <item> elements (and is the source of one of my major questions below). I run this script 100 times using the monkey to create lots of lists. Then I use a cmd batch to rename those files for easier use:

cd "DIRECTORY WHERE LISTS ARE"
del *.txt
ren *.iqdat *.txt
set /a count=1
setlocal ENABLEDELAYEDEXPANSION
for /f %%G IN ('dir /b') DO (
echo "%%G"
ren %%G*.* !count!.*
set /a count+=1
echo !count!)

move /y *.txt "STIM DIRECTORY FOR MY EXPERIMENT"
pause


Then, I use another cmd batch to grab my original experiment file, copy it 100 times and append the <include> element (as well as make an Inquisit batch file).

xcopy "PATH TO MY EXPERIMENT\MyExperiment.iqx" "TEMPORARY DIRECTORY"
ren "MyExperiment.iqx" "MyExperiment1.iqx"
for /l %%G IN (2,1,100) DO (
>> "MyExperiment"%%G".iqx" rem/
xcopy /y "MyExperiment1.iqx" "MyExperiment"%%G".iqx")

for /l %%G IN (1,1,100) DO (
echo. >> "MyExperiment"%%G".iqx"
echo. >> "MyExperiment"%%G".iqx"
echo ^<include^> >> "MyExperiment"%%G".iqx"
echo /file = ^"Lists/%%G.txt^" >> "MyExperiment"%%G".iqx"
echo ^</include^> >> "MyExperiment"%%G".iqx")

for /l %%G IN (1,1,100) DO (
move "MyExperiment"%%G".iqx" "PATH TO MY EXPERIMENT")


>> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx" rem/
echo ^<batch^> >> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx"
echo / selectionmode=random >> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx"
for /l %%G IN (1,1,100) DO (
echo / file = ^"MyExperiment%%G.iqx^" >> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx")
echo ^</batch^> >> "PATH TO MY EXPERIMENT\MyInquisitBatch.iqx"
pause


This all seems to work, for now.

So, here are my questions (sorry for the wall of text):

1) Is there a better approach to this that prevents participants from having to download unnecessary files?
2) If not, you might have noticed that my Inquisit file that generates the <item> elements in the text files basically just randomly draws combinations. This means I can't be sure that any particular stimulus set configuration of the 576 possibilities is roughly equally likely to be drawn as the others in the final experiment unless I output an arbitrarily high number of possible configurations, many of which will be redundant. Is there a way to force Inquisit to permutate the combinations it draws? That way, I could just generate exactly the 576 possibilities and not have any one configuration be more likely than any other.

Many thanks if anyone has the patience to actually work through this. :-/



> 1) Is there a better approach to this that prevents participants from having to download unnecessary files?

If you're not overly concerned with the timing of the videos, use the /stream attribute:
https://www.millisecond.com/support/docs/v5/html/language/attributes/stream.htm
That way, no videos will downloaded upfront and only those a participant actually has to see will be streamed from the web at runtime.
By AKrishna - 7/22/2020

Wow. My search on this issue only turned up old Inquisit 3 threads and I never thought to check whether the current version might be able to do this. Now I feel silly... :D Thanks so much, that should solve the problem.