Problems presenting videos for an emotion recognition task


Author
Message
Emma@UNSW
Emma@UNSW
Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)
Group: Forum Members
Posts: 2, Visits: 179
Hi all,
I am trying to write a script for a simple emotion recognition task, where participants are shown a video of a facial movement (smile, frown, etc) and asked to indicate which emotion they thought the face made using a scale presented on the screen and numbers on the keyboard (1 = happy, 2 = surprised and so on). Originally I had written the script to present images rather than videos, and it worked perfectly, but I decided to use videos instead as I am using the same videos in another task. I modified the original script to instead play the videos, and while the script will play them it does not present the emotion scale after the video finishes. For each trial a video of a facial movement should play followed by the emotion scale, which is when participants indicate the face's emotion. At the moment the video will play through but then a black screen, rather than the scale, replaces the video. If no button is pressed then the screen stays black, but if I press one of the numbers to indicate an emotion, eg 1, then the next video plays. I've copied the script that I've tried (modified from presenting pictures to presenting videos), and I would really appreciate any help as I have no idea how to fix this problem. 
Thank you!




<defaults>
/ inputdevice = keyboard
/ screencolor = black

</defaults>


****************************************
Stimuli
****************************************

<item instructions>
/1 = "Instructions.jpg"
</item>

<picture instructions>
/items = instructions
/size = (100%, 100%)
/valign = center
/position = (50%, 50%)
</picture>


<item happy>
/1 = "F01 - HAPPY (video).mp4"
/2 = "F02 - HAPPY (video).mp4"
/3 = "F03 - HAPPY (video).mp4"
/4 = "M01 - HAPPY (video).mp4"
/5 = "M02 - HAPPY (video).mp4"
/6 = "M03 - HAPPY (video).mp4"
</item>

<video happy>
/ items = happy
/ size = (100%, 100%)
/ position = (50%, 50%)
/ playthrough = true
</video>



<item angry>
/1 = "F01 - ANGRY (video).mp4"
/2 = "F02 - ANGRY (video).mp4"
/3 = "F03 - ANGRY (video).mp4"
/4 = "M01 - ANGRY (video).mp4"
/5 = "M02 - ANGRY (video).mp4"
/6 = "M03 - ANGRY (video).mp4"
</item>

<video angry>
/ items = angry
/ size = (100%, 100%)
/ position = (50%, 50%)
/ playthrough = true
</video>



<item neutral>
/1 = "F01 - NEUTRAL (video).mp4"
/2 = "F02 - NEUTRAL (video).mp4"
/3 = "F03 - NEUTRAL (video).mp4"
/4 = "M01 - NEUTRAL (video).mp4"
/5 = "M02 - NEUTRAL (video).mp4"
/6 = "M03 - NEUTRAL (video).mp4"
</item>

<video neutral>
/ items = neutral
/ size = (100%, 100%)
/ position = (50%, 50%)
/ playthrough = true
</video>



<item emoscale>
/1= "Emotion - Scale.jpg"
</item>

<picture emoscale>
/ items = emoscale
/ size = (130%, 130%)
/ position = (50%, 50%)
</picture>


*****************************
TRIALS
*****************************

<trial instructionstrial>
/stimulustimes = [0 = instructions]
/validresponse = (" ")
/inputdevice = keyboard
</trial>


<trial emo_happy>
/stimulustimes = [0=happy; 3000 = emoscale]
/validresponse = ("1","2","3","4","5","6","7")
/inputdevice = keyboard
/timeout = 6000
</trial>


<trial emo_angry>
/stimulustimes = [0=angry; 3000 = emoscale]
/validresponse = ("1","2","3","4","5","6","7")
/inputdevice = keyboard
/timeout = 6000
</trial>


<trial emo_neutral>
/stimulustimes = [0=neutral; 3000 = emoscale]
/validresponse = ("1","2","3","4","5","6","7")
/inputdevice = keyboard
/timeout = 6000
</trial>



*****************************
BLOCKS
*****************************

<block ERT>
/trials = [1 = instructionstrial; 2-20 = noreplacenorepeat(emo_happy, emo_angry, emo_neutral)]
</block>




******************************************************************************
Data Specifications
******************************************************************************
<data>
/file = "MNS ERT.iqdat"
/columns = [date, time, subject, trialnum, trialcode, stimulusitem, response, latency]
/separatefiles = true
</data>


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 105K
Emma@UNSW - Tuesday, July 11, 2017
Hi all,
I am trying to write a script for a simple emotion recognition task, where participants are shown a video of a facial movement (smile, frown, etc) and asked to indicate which emotion they thought the face made using a scale presented on the screen and numbers on the keyboard (1 = happy, 2 = surprised and so on). Originally I had written the script to present images rather than videos, and it worked perfectly, but I decided to use videos instead as I am using the same videos in another task. I modified the original script to instead play the videos, and while the script will play them it does not present the emotion scale after the video finishes. For each trial a video of a facial movement should play followed by the emotion scale, which is when participants indicate the face's emotion. At the moment the video will play through but then a black screen, rather than the scale, replaces the video. If no button is pressed then the screen stays black, but if I press one of the numbers to indicate an emotion, eg 1, then the next video plays. I've copied the script that I've tried (modified from presenting pictures to presenting videos), and I would really appreciate any help as I have no idea how to fix this problem. 
Thank you!




<defaults>
/ inputdevice = keyboard
/ screencolor = black

</defaults>


****************************************
Stimuli
****************************************

<item instructions>
/1 = "Instructions.jpg"
</item>

<picture instructions>
/items = instructions
/size = (100%, 100%)
/valign = center
/position = (50%, 50%)
</picture>


<item happy>
/1 = "F01 - HAPPY (video).mp4"
/2 = "F02 - HAPPY (video).mp4"
/3 = "F03 - HAPPY (video).mp4"
/4 = "M01 - HAPPY (video).mp4"
/5 = "M02 - HAPPY (video).mp4"
/6 = "M03 - HAPPY (video).mp4"
</item>

<video happy>
/ items = happy
/ size = (100%, 100%)
/ position = (50%, 50%)
/ playthrough = true
</video>



<item angry>
/1 = "F01 - ANGRY (video).mp4"
/2 = "F02 - ANGRY (video).mp4"
/3 = "F03 - ANGRY (video).mp4"
/4 = "M01 - ANGRY (video).mp4"
/5 = "M02 - ANGRY (video).mp4"
/6 = "M03 - ANGRY (video).mp4"
</item>

<video angry>
/ items = angry
/ size = (100%, 100%)
/ position = (50%, 50%)
/ playthrough = true
</video>



<item neutral>
/1 = "F01 - NEUTRAL (video).mp4"
/2 = "F02 - NEUTRAL (video).mp4"
/3 = "F03 - NEUTRAL (video).mp4"
/4 = "M01 - NEUTRAL (video).mp4"
/5 = "M02 - NEUTRAL (video).mp4"
/6 = "M03 - NEUTRAL (video).mp4"
</item>

<video neutral>
/ items = neutral
/ size = (100%, 100%)
/ position = (50%, 50%)
/ playthrough = true
</video>



<item emoscale>
/1= "Emotion - Scale.jpg"
</item>

<picture emoscale>
/ items = emoscale
/ size = (130%, 130%)
/ position = (50%, 50%)
</picture>


*****************************
TRIALS
*****************************

<trial instructionstrial>
/stimulustimes = [0 = instructions]
/validresponse = (" ")
/inputdevice = keyboard
</trial>


<trial emo_happy>
/stimulustimes = [0=happy; 3000 = emoscale]
/validresponse = ("1","2","3","4","5","6","7")
/inputdevice = keyboard
/timeout = 6000
</trial>


<trial emo_angry>
/stimulustimes = [0=angry; 3000 = emoscale]
/validresponse = ("1","2","3","4","5","6","7")
/inputdevice = keyboard
/timeout = 6000
</trial>


<trial emo_neutral>
/stimulustimes = [0=neutral; 3000 = emoscale]
/validresponse = ("1","2","3","4","5","6","7")
/inputdevice = keyboard
/timeout = 6000
</trial>



*****************************
BLOCKS
*****************************

<block ERT>
/trials = [1 = instructionstrial; 2-20 = noreplacenorepeat(emo_happy, emo_angry, emo_neutral)]
</block>




******************************************************************************
Data Specifications
******************************************************************************
<data>
/file = "MNS ERT.iqdat"
/columns = [date, time, subject, trialnum, trialcode, stimulusitem, response, latency]
/separatefiles = true
</data>


Two things to try:
#1: Clear the screen after the presentation of the video / before the display of the scale, i.e something along the lines of:

<trial emo_happy>
/stimulustimes = [0=happy; 3000=clearscreen; 3100 = emoscale]
/validresponse = ("1","2","3","4","5","6","7")
/inputdevice = keyboard
/timeout = 6100
</trial>

(Toy around with the timings if your videos are longer than exactly 3 seconds. FWIW, my guess would be that your videos actually last a little longer than 3 seconds and thus end up obscuring the scale stimulus displayed a 3000ms per your /stimulustimes.)

#2: If that doesn't work, separating the video display and the scale / response collection into two separate trials should to the trick. I.e.

<trial emo_happy>
/stimulustimes = [0=happy]
/validresponse = (0)
/inputdevice = keyboard
/timeout = 3000
/ branch = [trial.emo_happy_response]
</trial>

<trial emo_happy_response>
/stimulustimes = [0 = emoscale]
/validresponse = ("1","2","3","4","5","6","7")
/inputdevice = keyboard
/timeout = 3000
</trial>
Edited 8 Years Ago by Dave
Emma@UNSW
Emma@UNSW
Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)Associate Member (177 reputation)
Group: Forum Members
Posts: 2, Visits: 179
Thanks Dave, I separated the trials as per your second suggestion and it now works exactly as I want it to. Thank you so much for your help! 
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search