Verbal version of the Automated Operation Span task (Automated OSPAN)


Author
Message
s.laborde
s.laborde
Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)
Group: Forum Members
Posts: 8, Visits: 7

Hi all, 



I'm trying to adapt the Automated OSPAN task of the Inquisit library to a verbal response mode, and I've been only partially successful. My main concern is whether the speech recognition engine is able to recognise letters.


Here are the different steps I followed:


First, changing the response mode to voice input:


<defaults>


/ canvassize = (100%,100%)

/ canvasaspectratio = (4,3)

/ screencolor = (white)

/ txcolor = (black)

/ txbgcolor = (transparent)

/ fontstyle = ("Verdana", 4.0%, true)

/ inputdevice = voice

/ minimumversion = "4.0.0.0"

/ halign = center

/ valign = center

</defaults>


Then, changing the way to progress after each instruction screen from "left mouse button" to the vocal command "Next"


<trial instructions>


/ pretrialpause = 250

/ posttrialpause = 250

/ stimulustimes = [1=instructions]

/ validresponse = ("next")

/ responsetime = 2000

/ recorddata = false

</trial>


So far, everything is working, I can go through the 3 first instructions screens saying "Next", meaning that the voice input is correctly recognised by Inquisit.


The problems starts when I start the practice, and I need to name the letters. Then Inquisit don't seem to recognise them, nothing happens, and even the words "Blank", "Clear" and "Exit" seem not to be recognised:


<trial recall_letter>


/ ontrialbegin = [values.recallcount+=1]

/ ontrialbegin = [if(values.recallcount==1)values.recalldelay=500 else values.recalldelay=0]

/ pretrialpause = values.recalldelay

/ stimulusframes = [1=WhiteScreen, F, H, J, K, L, N, P, Q, R, S, T, Y, _, clear, exit, recalledletters, recallprompt, letterstrings]

/ validresponse = (F, H, J, K, L, N, P, Q, R, S, T, Y, blank, clear, exit)

/ monkeyresponse = ("F", "H", "J", "K", "exit")

/ ontrialend = [if(trial.recall_letter.response!="exit" && trial.recall_letter.response!="clear")

     {item.RecalledLetters.item=trial.recall_letter.response; values.recalledletters=concat(values.recalledletters, trial.recall_letter.response)}]

/ ontrialend = [if(trial.recall_letter.response=="clear")

     {clear(item.RecalledLetters); values.recalledletters=""}]

/ responsemessage = (F, clickF, 150)

/ responsemessage = (H, clickH, 150)

/ responsemessage = (J, clickJ, 150)

/ responsemessage = (K, clickK, 150)

/ responsemessage = (L, clickL, 150)

/ responsemessage = (N, clickN, 150)

/ responsemessage = (P, clickP, 150)

/ responsemessage = (Q, clickQ, 150)

/ responsemessage = (R, clickR, 150)

/ responsemessage = (S, clickS, 150)

/ responsemessage = (T, clickT, 150)

/ responsemessage = (Y, clickY, 150)

/ responsemessage = (clear, clickclear, 150)

/ responsemessage = (exit, clickexit, 150)

/ responsemessage = (_, click_, 150)

/ branch = [if(trial.recall_letter.response=="exit")trial.letter_feedback else trial.recall_letter]

/ trialdata = [recalledletters]

/ recorddata = true

</trial>



Any ideas how I could make Inquisit to recognise the letters when a person say them? I even tried to write them "phonetically", like I thought the speech recognition engine would recognise them, but nothing changed. 


Many thanks in advance for your help, 


Sylvain 

Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 108K

/ validresponse = (F, H, J, K, L, N, P, Q, R, S, T, Y, blank, clear, exit)


Try using double-quotes as in your other <trial>. Overall, though, I think it's a bad idea to try to use speech input for this particular task / paradigm. And unless you first properly train the recognition engine for *every individual speaker* I highly doubt this can be made to work (anywhere near) reliably.


s.laborde
s.laborde
Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)
Group: Forum Members
Posts: 8, Visits: 7

Hi Dave, 


thanks for the quick reply, I tried it with double quotes and unfortunately it does not work...


if it needs individual recognition, is it possible to imagine a practice where participants will first pronounce each letter, and then get Inquisit compare what participants say to what was recorded during practice?


I'm afraid the answer is no as I understood Inquisit is based on Microsoft Speech Engine...


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 108K

Inquisit is based on Microsoft Speech Engine...


Not based on, to be precise, it merely uses MS Speech Engine (under Windows). As such, you are correct, you can't train the engine via an Inquisit script and/or compare recordings to recognition results on the fly / at runtime.


Question: What exactly is the reason for your desire to switch to voice input?


s.laborde
s.laborde
Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)
Group: Forum Members
Posts: 8, Visits: 7

Valid question Dave, indeed we wanted to use this script when participants won't be in a proper position to use a mouse (for example exercising at the same time), therefore we thought about using voice input and a headset to address this issue.


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 108K

Thanks. Low-tech solution: Have the attending research assistant take verbal commands from the (exercising, etc.) participant and enter them for her/him via traditional input (mouse). Several advantages: Close to 100% reliable (human beings are incredibly competent recognizers of speech as opposed to computers), maximally error resistant especially under noisy conditions such as exercise which *will* impair speech (heavy breathing etc.) and consequently hamper any automated recognition efforts, etc.


Thoughts?


s.laborde
s.laborde
Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)Expert (1.1K reputation)
Group: Forum Members
Posts: 8, Visits: 7

Good one Dave! Indeed as response time is not an issue in the OSPAN to enter the letters, it seems totally ok if the research assistant does it for the participant, 


thanks a lot for the input!



GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search