Randomization error, while script appears correct. Help !


Author
Message
evas
evas
Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)
Group: Forum Members
Posts: 5, Visits: 10
So in my script, I ask people to rate some pictures. There are 16 stimuli (and by a programming error 18 trials). I specified noreplace way of drawing (see full code at the end of the post).
This way it should show all 16 stimuli and ask for a rating (then cause of the programming error it ask for a rating for 2 random stimuli again that people have seen before).
Dealing with the programming error was easy, just exclude trail 17 and 18 and then move on with your life happily and blissfully.
However, something very strange seems to be happening. Here is an example of how data (that is generated by me playing the participant).

trialnum:      stimulusnumber1:
1                        10
2                         6
3                         9
4                         8
5                         7
6                         3
7                         5
8                         4
9                        11
10                       2
11                       14
12                       1
13                       16
14                       13
15                       13
16                       12
17                        7
18                        3

As you can see trial 17&18 behave according to the programming error and show a stimulus that has been shown before. However, as you can also see trial 14 and 15 ALSO show this error. Despite the noreplace command.
Even weirder is that there seems to be no system to this error. In this case trial 15 is where it "went wrong" but for other people its another trial.
Also basically for every person it differs which stimulus is shown twice and also which stimulus is not shown. There are even cases were 2 stimuli are shown twice and 2 stimuli are omitted.
To top it off, i have 2 different rating scales, one works perfect and the other has the problem. In the code below, likertpos works fine, likertlight shows this problem. As you can also see in the script they are programmed the same way.
Any idea what is going wrong here ????????????????




SCRIPT:


<block likertlight>
/ preinstructions=(lightness)
/ trials = [1-18 = lightness]
</block>

<item pictures>


/1 = "Black_10%_R000.png"
/2 = "Black_10%_R090.png"
/3 = "Black_10%_R180.png"
/4 = "Black_10%_R270.png"
/5 = "Black_20%_R000.png"
/6 = "Black_20%_R090.png"
/7 = "Black_20%_R180.png"
/8 = "Black_20%_R270.png"
/9 = "Black_80%_R000.png"
/10 = "Black_80%_R090.png"
/11 = "Black_80%_R180.png"
/12 = "Black_80%_R270.png"
/13 = "Black_90%_R000.png"
/14 = "Black_90%_R090.png"
/15 = "Black_90%_R180.png"
/16 = "Black_90%_R270.png"
</item>



<picture patches>
/ size = (55%, 55%)
/ items = pictures
/ position = (50, 45)
/ select = noreplace
</picture>

<text questionlight>
/ items = ("Wie hell ist dieses Farbfeld?")
/ position = (50,80)
/ fontstyle = ("Arial", 2.04%, false, false, false, false, 5, 1)
</text>



<surveypage lightness>
/ stimulusframes = [1=patches, questionlight]
/ nextbuttonposition = (48,92)
/ navigationbuttonsize = (5,3)
/ questions = [1=box]
/ showquestionnumbers = false
/ showpagenumbers = false
</surveypage>



<radiobuttons box>
/ options = ("1=dunkel", "2", "3", "4", "5", "6", "7=hell")
/ required = true
/ orientation = horizontal
/ position = (39,87)
/ fontstyle = ("Arial", 1.48%, false, false, false, false, 5, 1)

</radiobuttons>




<page lightness>
Wie "hell" schätzen Sie die folgenden 16 Bilder auf einer Skala von 1 (sehr dunkel) bis 7 (sehr hell) ein?
Bitte wählen Sie eine Option mit der linken Maustaste aus und klicken Sie dann auf den "Continue-Button".
</page>

<instruct >
/ fontstyle = ("Arial", 3.18%, false, false, false, false, 5, 1)
/ screencolor = #808080
</instruct>





++++++++++++++++++++++++++++++++++++

<block likertpos>
/ preinstructions=(positivity)
/ trials = [1-18 = positivity]

</block>



<surveypage positivity>
/ stimulusframes = [1=patches, questionpos]
/ nextbuttonposition = (48,92)
/ navigationbuttonsize = (5,3)
/ showquestionnumbers = false
/ showpagenumbers = false
/ questions = [1=box2]
</surveypage>




<radiobuttons box2>
/ options = ("1=negativ", "2", "3", "4", "5", "6", "7=positiv")
/ required = true
/ orientation = horizontal
/ position = (39,87)
/ fontstyle = ("Arial", 1.48%, false, false, false, false, 5, 1)
</radiobuttons>



<page positivity>
Wie "positiv" schätzen Sie die folgenden 16 Bilder auf einer Skala von 1 (sehr negativ) bis 7 (sehr positiv) ein?
Bitte wählen Sie eine Option mit der linken Maustaste aus und klicken Sie dann auf den "Continue-Button".
</page>


<text questionpos>
/ items = ("Wie positiv ist dieses Farbfeld?")
/ position = (50,80)
/ fontstyle = ("Arial", 2.04%, false, false, false, false, 5, 1)
</text>




Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 105K
evas - Friday, January 27, 2017
So in my script, I ask people to rate some pictures. There are 16 stimuli (and by a programming error 18 trials). I specified noreplace way of drawing (see full code at the end of the post).
This way it should show all 16 stimuli and ask for a rating (then cause of the programming error it ask for a rating for 2 random stimuli again that people have seen before).
Dealing with the programming error was easy, just exclude trail 17 and 18 and then move on with your life happily and blissfully.
However, something very strange seems to be happening. Here is an example of how data (that is generated by me playing the participant).

trialnum:      stimulusnumber1:
1                        10
2                         6
3                         9
4                         8
5                         7
6                         3
7                         5
8                         4
9                        11
10                       2
11                       14
12                       1
13                       16
14                       13
15                       13
16                       12
17                        7
18                        3

As you can see trial 17&18 behave according to the programming error and show a stimulus that has been shown before. However, as you can also see trial 14 and 15 ALSO show this error. Despite the noreplace command.
Even weirder is that there seems to be no system to this error. In this case trial 15 is where it "went wrong" but for other people its another trial.
Also basically for every person it differs which stimulus is shown twice and also which stimulus is not shown. There are even cases were 2 stimuli are shown twice and 2 stimuli are omitted.
To top it off, i have 2 different rating scales, one works perfect and the other has the problem. In the code below, likertpos works fine, likertlight shows this problem. As you can also see in the script they are programmed the same way.
Any idea what is going wrong here ????????????????




SCRIPT:


<block likertlight>
/ preinstructions=(lightness)
/ trials = [1-18 = lightness]
</block>

<item pictures>


/1 = "Black_10%_R000.png"
/2 = "Black_10%_R090.png"
/3 = "Black_10%_R180.png"
/4 = "Black_10%_R270.png"
/5 = "Black_20%_R000.png"
/6 = "Black_20%_R090.png"
/7 = "Black_20%_R180.png"
/8 = "Black_20%_R270.png"
/9 = "Black_80%_R000.png"
/10 = "Black_80%_R090.png"
/11 = "Black_80%_R180.png"
/12 = "Black_80%_R270.png"
/13 = "Black_90%_R000.png"
/14 = "Black_90%_R090.png"
/15 = "Black_90%_R180.png"
/16 = "Black_90%_R270.png"
</item>



<picture patches>
/ size = (55%, 55%)
/ items = pictures
/ position = (50, 45)
/ select = noreplace
</picture>

<text questionlight>
/ items = ("Wie hell ist dieses Farbfeld?")
/ position = (50,80)
/ fontstyle = ("Arial", 2.04%, false, false, false, false, 5, 1)
</text>



<surveypage lightness>
/ stimulusframes = [1=patches, questionlight]
/ nextbuttonposition = (48,92)
/ navigationbuttonsize = (5,3)
/ questions = [1=box]
/ showquestionnumbers = false
/ showpagenumbers = false
</surveypage>



<radiobuttons box>
/ options = ("1=dunkel", "2", "3", "4", "5", "6", "7=hell")
/ required = true
/ orientation = horizontal
/ position = (39,87)
/ fontstyle = ("Arial", 1.48%, false, false, false, false, 5, 1)

</radiobuttons>




<page lightness>
Wie "hell" schätzen Sie die folgenden 16 Bilder auf einer Skala von 1 (sehr dunkel) bis 7 (sehr hell) ein?
Bitte wählen Sie eine Option mit der linken Maustaste aus und klicken Sie dann auf den "Continue-Button".
</page>

<instruct >
/ fontstyle = ("Arial", 3.18%, false, false, false, false, 5, 1)
/ screencolor = #808080
</instruct>





++++++++++++++++++++++++++++++++++++

<block likertpos>
/ preinstructions=(positivity)
/ trials = [1-18 = positivity]

</block>



<surveypage positivity>
/ stimulusframes = [1=patches, questionpos]
/ nextbuttonposition = (48,92)
/ navigationbuttonsize = (5,3)
/ showquestionnumbers = false
/ showpagenumbers = false
/ questions = [1=box2]
</surveypage>




<radiobuttons box2>
/ options = ("1=negativ", "2", "3", "4", "5", "6", "7=positiv")
/ required = true
/ orientation = horizontal
/ position = (39,87)
/ fontstyle = ("Arial", 1.48%, false, false, false, false, 5, 1)
</radiobuttons>



<page positivity>
Wie "positiv" schätzen Sie die folgenden 16 Bilder auf einer Skala von 1 (sehr negativ) bis 7 (sehr positiv) ein?
Bitte wählen Sie eine Option mit der linken Maustaste aus und klicken Sie dann auf den "Continue-Button".
</page>


<text questionpos>
/ items = ("Wie positiv ist dieses Farbfeld?")
/ position = (50,80)
/ fontstyle = ("Arial", 2.04%, false, false, false, false, 5, 1)
</text>




Not sure what is going wrong, but try explicitly forcing a reset of the <picture> element's selection pool at the start of the <block>:

<block likertpos>
/ onblockbegin = [picture.patches.resetselection();]
/ preinstructions=(positivity)
/ trials = [1-18 = positivity]
</block>
evas
evas
Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)
Group: Forum Members
Posts: 5, Visits: 10
Dave - Friday, January 27, 2017
evas - Friday, January 27, 2017
So in my script, I ask people to rate some pictures. There are 16 stimuli (and by a programming error 18 trials). I specified noreplace way of drawing (see full code at the end of the post).
This way it should show all 16 stimuli and ask for a rating (then cause of the programming error it ask for a rating for 2 random stimuli again that people have seen before).
Dealing with the programming error was easy, just exclude trail 17 and 18 and then move on with your life happily and blissfully.
However, something very strange seems to be happening. Here is an example of how data (that is generated by me playing the participant).

trialnum:      stimulusnumber1:
1                        10
2                         6
3                         9
4                         8
5                         7
6                         3
7                         5
8                         4
9                        11
10                       2
11                       14
12                       1
13                       16
14                       13
15                       13
16                       12
17                        7
18                        3

As you can see trial 17&18 behave according to the programming error and show a stimulus that has been shown before. However, as you can also see trial 14 and 15 ALSO show this error. Despite the noreplace command.
Even weirder is that there seems to be no system to this error. In this case trial 15 is where it "went wrong" but for other people its another trial.
Also basically for every person it differs which stimulus is shown twice and also which stimulus is not shown. There are even cases were 2 stimuli are shown twice and 2 stimuli are omitted.
To top it off, i have 2 different rating scales, one works perfect and the other has the problem. In the code below, likertpos works fine, likertlight shows this problem. As you can also see in the script they are programmed the same way.
Any idea what is going wrong here ????????????????




SCRIPT:


<block likertlight>
/ preinstructions=(lightness)
/ trials = [1-18 = lightness]
</block>

<item pictures>


/1 = "Black_10%_R000.png"
/2 = "Black_10%_R090.png"
/3 = "Black_10%_R180.png"
/4 = "Black_10%_R270.png"
/5 = "Black_20%_R000.png"
/6 = "Black_20%_R090.png"
/7 = "Black_20%_R180.png"
/8 = "Black_20%_R270.png"
/9 = "Black_80%_R000.png"
/10 = "Black_80%_R090.png"
/11 = "Black_80%_R180.png"
/12 = "Black_80%_R270.png"
/13 = "Black_90%_R000.png"
/14 = "Black_90%_R090.png"
/15 = "Black_90%_R180.png"
/16 = "Black_90%_R270.png"
</item>



<picture patches>
/ size = (55%, 55%)
/ items = pictures
/ position = (50, 45)
/ select = noreplace
</picture>

<text questionlight>
/ items = ("Wie hell ist dieses Farbfeld?")
/ position = (50,80)
/ fontstyle = ("Arial", 2.04%, false, false, false, false, 5, 1)
</text>



<surveypage lightness>
/ stimulusframes = [1=patches, questionlight]
/ nextbuttonposition = (48,92)
/ navigationbuttonsize = (5,3)
/ questions = [1=box]
/ showquestionnumbers = false
/ showpagenumbers = false
</surveypage>



<radiobuttons box>
/ options = ("1=dunkel", "2", "3", "4", "5", "6", "7=hell")
/ required = true
/ orientation = horizontal
/ position = (39,87)
/ fontstyle = ("Arial", 1.48%, false, false, false, false, 5, 1)

</radiobuttons>




<page lightness>
Wie "hell" schätzen Sie die folgenden 16 Bilder auf einer Skala von 1 (sehr dunkel) bis 7 (sehr hell) ein?
Bitte wählen Sie eine Option mit der linken Maustaste aus und klicken Sie dann auf den "Continue-Button".
</page>

<instruct >
/ fontstyle = ("Arial", 3.18%, false, false, false, false, 5, 1)
/ screencolor = #808080
</instruct>





++++++++++++++++++++++++++++++++++++

<block likertpos>
/ preinstructions=(positivity)
/ trials = [1-18 = positivity]

</block>



<surveypage positivity>
/ stimulusframes = [1=patches, questionpos]
/ nextbuttonposition = (48,92)
/ navigationbuttonsize = (5,3)
/ showquestionnumbers = false
/ showpagenumbers = false
/ questions = [1=box2]
</surveypage>




<radiobuttons box2>
/ options = ("1=negativ", "2", "3", "4", "5", "6", "7=positiv")
/ required = true
/ orientation = horizontal
/ position = (39,87)
/ fontstyle = ("Arial", 1.48%, false, false, false, false, 5, 1)
</radiobuttons>



<page positivity>
Wie "positiv" schätzen Sie die folgenden 16 Bilder auf einer Skala von 1 (sehr negativ) bis 7 (sehr positiv) ein?
Bitte wählen Sie eine Option mit der linken Maustaste aus und klicken Sie dann auf den "Continue-Button".
</page>


<text questionpos>
/ items = ("Wie positiv ist dieses Farbfeld?")
/ position = (50,80)
/ fontstyle = ("Arial", 2.04%, false, false, false, false, 5, 1)
</text>




Not sure what is going wrong, but try explicitly forcing a reset of the <picture> element's selection pool at the start of the <block>:

<block likertpos>
/ onblockbegin = [picture.patches.resetselection();]
/ preinstructions=(positivity)
/ trials = [1-18 = positivity]
</block>

Thanks! I will try to see if that changes it. Since the problem occurs in 40% of the cases it didn't show when I test ran the study before data collection. Since data collection is now finished and I'm stuck with this issue in my data it would be great to find out what made it happen so I can make sure I wont have this issue in future studies.

evas
evas
Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)Respected Member (471 reputation)
Group: Forum Members
Posts: 5, Visits: 10
Dave - Friday, January 27, 2017
evas - Friday, January 27, 2017
So in my script, I ask people to rate some pictures. There are 16 stimuli (and by a programming error 18 trials). I specified noreplace way of drawing (see full code at the end of the post).
This way it should show all 16 stimuli and ask for a rating (then cause of the programming error it ask for a rating for 2 random stimuli again that people have seen before).
Dealing with the programming error was easy, just exclude trail 17 and 18 and then move on with your life happily and blissfully.
However, something very strange seems to be happening. Here is an example of how data (that is generated by me playing the participant).

trialnum:      stimulusnumber1:
1                        10
2                         6
3                         9
4                         8
5                         7
6                         3
7                         5
8                         4
9                        11
10                       2
11                       14
12                       1
13                       16
14                       13
15                       13
16                       12
17                        7
18                        3

As you can see trial 17&18 behave according to the programming error and show a stimulus that has been shown before. However, as you can also see trial 14 and 15 ALSO show this error. Despite the noreplace command.
Even weirder is that there seems to be no system to this error. In this case trial 15 is where it "went wrong" but for other people its another trial.
Also basically for every person it differs which stimulus is shown twice and also which stimulus is not shown. There are even cases were 2 stimuli are shown twice and 2 stimuli are omitted.
To top it off, i have 2 different rating scales, one works perfect and the other has the problem. In the code below, likertpos works fine, likertlight shows this problem. As you can also see in the script they are programmed the same way.
Any idea what is going wrong here ????????????????




SCRIPT:


<block likertlight>
/ preinstructions=(lightness)
/ trials = [1-18 = lightness]
</block>

<item pictures>


/1 = "Black_10%_R000.png"
/2 = "Black_10%_R090.png"
/3 = "Black_10%_R180.png"
/4 = "Black_10%_R270.png"
/5 = "Black_20%_R000.png"
/6 = "Black_20%_R090.png"
/7 = "Black_20%_R180.png"
/8 = "Black_20%_R270.png"
/9 = "Black_80%_R000.png"
/10 = "Black_80%_R090.png"
/11 = "Black_80%_R180.png"
/12 = "Black_80%_R270.png"
/13 = "Black_90%_R000.png"
/14 = "Black_90%_R090.png"
/15 = "Black_90%_R180.png"
/16 = "Black_90%_R270.png"
</item>



<picture patches>
/ size = (55%, 55%)
/ items = pictures
/ position = (50, 45)
/ select = noreplace
</picture>

<text questionlight>
/ items = ("Wie hell ist dieses Farbfeld?")
/ position = (50,80)
/ fontstyle = ("Arial", 2.04%, false, false, false, false, 5, 1)
</text>



<surveypage lightness>
/ stimulusframes = [1=patches, questionlight]
/ nextbuttonposition = (48,92)
/ navigationbuttonsize = (5,3)
/ questions = [1=box]
/ showquestionnumbers = false
/ showpagenumbers = false
</surveypage>



<radiobuttons box>
/ options = ("1=dunkel", "2", "3", "4", "5", "6", "7=hell")
/ required = true
/ orientation = horizontal
/ position = (39,87)
/ fontstyle = ("Arial", 1.48%, false, false, false, false, 5, 1)

</radiobuttons>




<page lightness>
Wie "hell" schätzen Sie die folgenden 16 Bilder auf einer Skala von 1 (sehr dunkel) bis 7 (sehr hell) ein?
Bitte wählen Sie eine Option mit der linken Maustaste aus und klicken Sie dann auf den "Continue-Button".
</page>

<instruct >
/ fontstyle = ("Arial", 3.18%, false, false, false, false, 5, 1)
/ screencolor = #808080
</instruct>





++++++++++++++++++++++++++++++++++++

<block likertpos>
/ preinstructions=(positivity)
/ trials = [1-18 = positivity]

</block>



<surveypage positivity>
/ stimulusframes = [1=patches, questionpos]
/ nextbuttonposition = (48,92)
/ navigationbuttonsize = (5,3)
/ showquestionnumbers = false
/ showpagenumbers = false
/ questions = [1=box2]
</surveypage>




<radiobuttons box2>
/ options = ("1=negativ", "2", "3", "4", "5", "6", "7=positiv")
/ required = true
/ orientation = horizontal
/ position = (39,87)
/ fontstyle = ("Arial", 1.48%, false, false, false, false, 5, 1)
</radiobuttons>



<page positivity>
Wie "positiv" schätzen Sie die folgenden 16 Bilder auf einer Skala von 1 (sehr negativ) bis 7 (sehr positiv) ein?
Bitte wählen Sie eine Option mit der linken Maustaste aus und klicken Sie dann auf den "Continue-Button".
</page>


<text questionpos>
/ items = ("Wie positiv ist dieses Farbfeld?")
/ position = (50,80)
/ fontstyle = ("Arial", 2.04%, false, false, false, false, 5, 1)
</text>




Not sure what is going wrong, but try explicitly forcing a reset of the <picture> element's selection pool at the start of the <block>:

<block likertpos>
/ onblockbegin = [picture.patches.resetselection();]
/ preinstructions=(positivity)
/ trials = [1-18 = positivity]
</block>

Hi Dave,

Thanks again for your answer, through this solution I also found what was going wrong.
Because its drawing from the same item pool (and they werent reset as your solution does) it just keeps drawing causing the error.
Shame for this study but good to know what made it happen.



GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search