Millisecond Forums

Playing a sound during onscreen button press

https://forums.millisecond.com/Topic35489.aspx

By Nejc900 - 6/27/2023

Hi,

First of all, thank you for actively running this forum! It has been an incredible help in getting to grips with Inquisit and trying to solve issues that have arisen. I am still a complete novice when it comes to any kind of coding so apologies if any of the bellow does not make sense or has a very simple solution.

tl;dr
I want to have a sound play while someone is holding down an onscreen button, with the sound starting when they press down and ending when they release the button.

Longer explanation:
For context, this is a redeveloped version of the BART script that is available through the library. What I have done is added a third button named the valve button which the participants need to press and hold for a specific duration.
The first issue that I needed to resolve (which I managed to do with significant help from this forum) was to actually record someone pressing and holding down the button. I did so by having validresponses be lbuttondown and lbuttonup. I added shapes behind actual buttons and defined clickable areas based on them. i.e.
<expressions>
/ isovervalvebutton = (mouse.x > shape.valve.left && mouse.x < shape.valve.right && mouse.y > shape.valve.top && mouse.y < shape.valve.bottom)
</expressions>

I then added isvalidresponse as seen below with a time being recorded for lbutton down and then deducted for lbuttonup, giving is the duration of the button hold.
/ isvalidresponse = [
if(trial.choiceaftervalve.response == "lbuttondown" && expressions.isovervalvebutton) {
  values.time_of_button_down = trial.choiceaftervalve.latency;
  values.n_clicks += 1;
 return false;
};
  else if (trial.choice.response == "lbuttonup" && expressions.isovervalvebutton) {
  values.time_of_button_up = trial.choice.latency;
 if (values.time_of_button_down > 0) {
    values.diff = values.time_of_button_up - values.time_of_button_down;
 values.n_clicks += 1
   };
 return true; 
  };
else if(trial.choiceaftervalve.response == "lbuttondown" && expressions.isoverpumpbutton) {
 return true;
};
else if (trial.choiceaftervalve.response == "lbuttondown" && expressions.isovercollectbutton){
 return true;
};
]

What I want to add now is that a sound (valvesound) should play for the duration of the button being pressed. There's a few ways in which I have tried to accomplish this, but none have yielded any useful results. Below is the full trial within which this would be implemented, with the full script also attached if useful.

Thank you!

<trial choice>
/ ontrialbegin = [
  values.countchoice += 1;
  values.pumpresult = list.pumpresult.nextvalue;
  if (values.pumpsound == 1)
   trial.choice.insertstimulustime(sound.inflatesound,0);
  values.pumpsound = 0;
  values.time_of_button_down = -1;
values.time_of_button_up = -1;
values.n_clicks = 0;
values.diff = 0;
  clock.stopwatch.start();
]
/ stimulustimes = [1=balloon, pump, collect, collectbutton, valve, pumpbutton, totalearnings, pumpcount, potentialearnings, ballooncount, redsquare, whitesquare, valvebutton, valveinstruct, stopwatch]
/ inputdevice = mouse
/ validresponse = (lbuttondown, lbuttonup)
/ isvalidresponse = [
if(trial.choice.response == "lbuttondown" && expressions.isovervalvebutton) {
  values.time_of_button_down = trial.choice.latency;
  values.n_clicks += 1;
 return false;
};
  else if (trial.choice.response == "lbuttonup" && expressions.isovervalvebutton) {
  values.time_of_button_up = trial.choice.latency;
 if (values.time_of_button_down > 0) {
    values.diff = values.time_of_button_up - values.time_of_button_down;
 values.n_clicks += 1;
   };
 return true; 
  };
else if(trial.choice.response == "lbuttondown" && expressions.isoverpumpbutton) {
 return true;
};
else if (trial.choice.response == "lbuttondown" && expressions.isovercollectbutton){
 return true;
};
]
/ ontrialend = [
  trial.choice.resetstimulusframes();
  if (trial.choice.response == "lbuttondown" && expressions.isoverpumpbutton) {
   values.scalingFactor += parameters.balloonsizeincrement;
   values.pumpcount += 1;
   if (values.pumpcount > 1) {
    values.mean_timebtwpumps = values.sum_timebtwpumps / (values.pumpcount-1);
   };
   values.totalpumpcount += 1
   if (values.countchoice == 1) {
    values.timebefore1stpump = trial.choice.latency;
   } else {
    values.timebtwpumps = trial.choice.latency;
    values.sum_timebtwpumps += values.timebtwpumps;
   };
  } else if (trial.choice.response == "lbuttondown" && expressions.isovercollectbutton) {
   if (values.countchoice == 1) {
    values.timebeforecollectwithoutpump = trial.choice.latency;
   } else {
    values.timebtwlastpumpandcollect = trial.choice.latency;
   };
  };

/ branch = [
  if (trial.choice.response == "lbuttondown" && expressions.isoverpumpbutton) {
   if (values.pumpresult == 1) {
    return trial.pop; // If you select pump button and the value that is picked out is 1 then go to the pop trial.
   } else {
    values.pumpsound = 1;
    return trial.choice; // Otherwise continue with the trial
   }
  }
  else if (trial.choice.response == "lbuttondown" && expressions.isovercollectbutton) {
   return trial.collect; // If you select the collect button, go to the collect trial.
  };
  else if (trial.choice.response == "lbuttonup" && expressions.isovervalvebutton && values.n_clicks == 1) {
   return trial.choiceaftervalve;
  };

]
/ recorddata = true
</trial>
By Dave - 6/27/2023

Nejc900 - 6/27/2023
Hi,

First of all, thank you for actively running this forum! It has been an incredible help in getting to grips with Inquisit and trying to solve issues that have arisen. I am still a complete novice when it comes to any kind of coding so apologies if any of the bellow does not make sense or has a very simple solution.

tl;dr
I want to have a sound play while someone is holding down an onscreen button, with the sound starting when they press down and ending when they release the button.

Longer explanation:
For context, this is a redeveloped version of the BART script that is available through the library. What I have done is added a third button named the valve button which the participants need to press and hold for a specific duration.
The first issue that I needed to resolve (which I managed to do with significant help from this forum) was to actually record someone pressing and holding down the button. I did so by having validresponses be lbuttondown and lbuttonup. I added shapes behind actual buttons and defined clickable areas based on them. i.e.
<expressions>
/ isovervalvebutton = (mouse.x > shape.valve.left && mouse.x < shape.valve.right && mouse.y > shape.valve.top && mouse.y < shape.valve.bottom)
</expressions>

I then added isvalidresponse as seen below with a time being recorded for lbutton down and then deducted for lbuttonup, giving is the duration of the button hold.
/ isvalidresponse = [
if(trial.choiceaftervalve.response == "lbuttondown" && expressions.isovervalvebutton) {
  values.time_of_button_down = trial.choiceaftervalve.latency;
  values.n_clicks += 1;
 return false;
};
  else if (trial.choice.response == "lbuttonup" && expressions.isovervalvebutton) {
  values.time_of_button_up = trial.choice.latency;
 if (values.time_of_button_down > 0) {
    values.diff = values.time_of_button_up - values.time_of_button_down;
 values.n_clicks += 1
   };
 return true; 
  };
else if(trial.choiceaftervalve.response == "lbuttondown" && expressions.isoverpumpbutton) {
 return true;
};
else if (trial.choiceaftervalve.response == "lbuttondown" && expressions.isovercollectbutton){
 return true;
};
]

What I want to add now is that a sound (valvesound) should play for the duration of the button being pressed. There's a few ways in which I have tried to accomplish this, but none have yielded any useful results. Below is the full trial within which this would be implemented, with the full script also attached if useful.

Thank you!

<trial choice>
/ ontrialbegin = [
  values.countchoice += 1;
  values.pumpresult = list.pumpresult.nextvalue;
  if (values.pumpsound == 1)
   trial.choice.insertstimulustime(sound.inflatesound,0);
  values.pumpsound = 0;
  values.time_of_button_down = -1;
values.time_of_button_up = -1;
values.n_clicks = 0;
values.diff = 0;
  clock.stopwatch.start();
]
/ stimulustimes = [1=balloon, pump, collect, collectbutton, valve, pumpbutton, totalearnings, pumpcount, potentialearnings, ballooncount, redsquare, whitesquare, valvebutton, valveinstruct, stopwatch]
/ inputdevice = mouse
/ validresponse = (lbuttondown, lbuttonup)
/ isvalidresponse = [
if(trial.choice.response == "lbuttondown" && expressions.isovervalvebutton) {
  values.time_of_button_down = trial.choice.latency;
  values.n_clicks += 1;
 return false;
};
  else if (trial.choice.response == "lbuttonup" && expressions.isovervalvebutton) {
  values.time_of_button_up = trial.choice.latency;
 if (values.time_of_button_down > 0) {
    values.diff = values.time_of_button_up - values.time_of_button_down;
 values.n_clicks += 1;
   };
 return true; 
  };
else if(trial.choice.response == "lbuttondown" && expressions.isoverpumpbutton) {
 return true;
};
else if (trial.choice.response == "lbuttondown" && expressions.isovercollectbutton){
 return true;
};
]
/ ontrialend = [
  trial.choice.resetstimulusframes();
  if (trial.choice.response == "lbuttondown" && expressions.isoverpumpbutton) {
   values.scalingFactor += parameters.balloonsizeincrement;
   values.pumpcount += 1;
   if (values.pumpcount > 1) {
    values.mean_timebtwpumps = values.sum_timebtwpumps / (values.pumpcount-1);
   };
   values.totalpumpcount += 1
   if (values.countchoice == 1) {
    values.timebefore1stpump = trial.choice.latency;
   } else {
    values.timebtwpumps = trial.choice.latency;
    values.sum_timebtwpumps += values.timebtwpumps;
   };
  } else if (trial.choice.response == "lbuttondown" && expressions.isovercollectbutton) {
   if (values.countchoice == 1) {
    values.timebeforecollectwithoutpump = trial.choice.latency;
   } else {
    values.timebtwlastpumpandcollect = trial.choice.latency;
   };
  };

/ branch = [
  if (trial.choice.response == "lbuttondown" && expressions.isoverpumpbutton) {
   if (values.pumpresult == 1) {
    return trial.pop; // If you select pump button and the value that is picked out is 1 then go to the pop trial.
   } else {
    values.pumpsound = 1;
    return trial.choice; // Otherwise continue with the trial
   }
  }
  else if (trial.choice.response == "lbuttondown" && expressions.isovercollectbutton) {
   return trial.collect; // If you select the collect button, go to the collect trial.
  };
  else if (trial.choice.response == "lbuttonup" && expressions.isovervalvebutton && values.n_clicks == 1) {
   return trial.choiceaftervalve;
  };

]
/ recorddata = true
</trial>

(1) If you share code that requires numerous external files to run, please always provide those files along with the code. Code that cannot be run is of limited use.
(2) Please comment your code, i.e. make clear what any given part you added is intended to do (which may be different from what it actually does).
(3) As a quick take, I don't believe what you want is doable wihtin a single <trial>. You'll likely have to set up multiple: One trial that collects the button press, /branch to another trial that plays back the sound and waits for the release, and then whatever is supposed to come next. Similar to https://forums.millisecond.com/Topic35478.aspx
By Nejc900 - 6/27/2023

Hi Dave,

I have attached a zip file with all the files inside.

I have included two scripts. The "original" one is the one attached and discussed above, with the button press duration being processed within a single trial.

Based on your suggestion I have also added a "developed" version where I have split the choice trial into two. It seems to work great, apart from registering the lbuttonup from the continuous hold, meaning that the trial does not end on the first button release. If I then press it again, and then release it, it has no issue recognising the input.

Is there a way for the trial to "know"/assume that the lbutton is already pressed at the start, which would then enable lbuttonup to be registered (I'm assuming that might be the cause, but could be way off)?

Thank you again for your help,
Nejc
By Dave - 6/27/2023

Nejc900 - 6/27/2023
Hi Dave,

I have attached a zip file with all the files inside.

I have included two scripts. The "original" one is the one attached and discussed above, with the button press duration being processed within a single trial.

Based on your suggestion I have also added a "developed" version where I have split the choice trial into two. It seems to work great, apart from registering the lbuttonup from the continuous hold, meaning that the trial does not end on the first button release. If I then press it again, and then release it, it has no issue recognising the input.

Is there a way for the trial to "know"/assume that the lbutton is already pressed at the start, which would then enable lbuttonup to be registered (I'm assuming that might be the cause, but could be way off)?

Thank you again for your help,
Nejc

Change <trial choiceaftervalve> to

<trial choiceaftervalve>
/ stimulustimes = [0=sound.valvesound, clock.stopwatch]
/ inputdevice = mouse
/ validresponse = (lbuttonup)
/ beginresponsetime = -1
/ branch = [
if (trial.choiceaftervalve.response == "lbuttonup") {
        return trial.choice; // If you release the valve button, go to choiceaftervalve
    }
]
/ recorddata = true
/ ontrialend = [
    values.aftervalve = 1; // Identifying when valve was pressed to differentiate for choice trial
]
</trial>


and things should work like you want them to.
By Nejc900 - 6/27/2023

Thank you so much!