Inquisit timing accuracy.


Author
Message
matt2004
matt2004
Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)
Group: Forum Members
Posts: 11, Visits: 1

Dear Inquisit-forum-denizens,


I have been struggling with a timing issue with Inquisit for some time now, in fact I posted a question about it some time ago, related to Inquisit 2: http://www.millisecond.com/forums/Topic2348.aspx. I'm now testing out Inquisit 3 with a view to upgrading and am finding the same issue.


The issue is a consistent timing error, such that the total length of each trial is slightly longer than it should be. Sean's response to my original post was to suggest inserting pre and post-trial pauses to give Inquisit time to wrap the trial up, however this doesn't seem to help. 


I've been coping with the issue by deliberately setting the trial duration lower than it 'should' be, so if I want a trial to last for 1000 ms, I actually set the trial duration to, say, 9900 ms. However, fixing the issue is not as simple as having a simple rule of "knock off 100ms from the duration of every trial" - the error seems to be somewhat capricious depending on what kind of stimuli (and how many) are being presented, although it's generally consistent across trials of the same type. Fixing the error and getting the timing I want therefore requires a fair bit of tedious data-logging with external hardware, or sitting there with a stopwatch timing the total length of the program and iteratively knocking off or adding a few milliseconds to the trial durations in order to get the total length as accurate as possible.


To illustrate this I've written a very simple example program. This presents shape stimuli (a green circle, and a black circle to 'erase' the green one) on every trial, and the trial duration has been set at 1000ms. There are 60 trials, so the total length of the program should be 1 minute. In fact, the total length of the program is approximately 1 minute and 3 seconds. Inquisit is therefore adding on approximately 50ms (or three frames) to each trial. Changing the pre and post trial pauses from 100 ms to 0 ms has no effect. Inquisit logs the trial duration in the data file as whatever is specified in the script, which is unhelpful.


Apologies for the length of this post, but I now have several queries:


1. Why does this happen? As I understand it, Inquisit uses the system timer, which should have sub-microsecond resolution. Given the timer's resolution it seems very odd that such a large error occurs.


2. How can I ensure that the timings I enter into the script are what I actually get when the script executes? Is there some little trick I'm missing here? I've tried controlling the length of trials using the timeout feature as well, but that doesn't seem to help. As noted before, Sean's original suggestion of inserting a post-trial pause doesn't seem to help.


3. Are the values for trialduration which are logged in the data file actually measured as the program executes, or are they just copied from the relevant parts of the script? I'm guessing the latter, since they seem to be inaccurate.


4. Has anyone else seen behaviour like this?


5. Can some kind person help me out by trying to reproduce the error using the attached script? The magnitude of the error does seem to be somewhat hardware dependent, but there's always some kind of lag. I generally use up-to-date hardware (dual core processer, lots of RAM, Windows XP, things like anti-virus disabled) for running my programs.


Many thanks for any help you can give me with this. I've been using Inquisit for years and genuinely adore it, but if I can't solve this problem I'll have to start using something else. Plus, I've just embarrassed myself in front of my supervisor by running a load of fMRI scans with crappy timing. :o(


Thanks,


M.


Attachments
timing_example.exp (973 views, 1,004 bytes)
Tags
Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 12K, Visits: 98K

Hi Matt,


Inquisit's timing is inevitably tied to the computer's display refresh rate. A decent CRT monitor usually refreshes at 100Hz (approx. every 10.00 ms), most LCD or TFT monitors refresh at 60Hz (approx. every 16.67 ms). Inquisit's stimulus display depends on that figure. When using '/stimulustimes', Inquisit will try display your stimulus in the display frame closest to the specified time in ms, i.e. your stimulus might show up either a tad earlier or later than specified (i.e. when the timing entered is not an exact multiple of the refresh rate). If you need your timing to be exact, use the '/stimulusframes' command which let's you specifiy timing in terms of ordinal frame position in a given trial.


As far as I know, the timings entered in '/pretrialpause' and '/posttrialpause' do not directly depend on the refresh rate, however Inquisit will still (have to) wait for the next refresh cycle to start before displaying any stimulus. I believe the same is true for the '/trialduration' settings. That may be another source of the systematic timing error you're seeing.


Another often overlooked issue is Inquisit's stimulus erasing behavior. All stimuli will be erased *by default* at the end of the trial unless specified otherwise (by setting '/erase = false'). This may add (at least one) additional, rogue frame between trials (see here for a discussion: http://www.millisecond.com/forums/Topic4488.aspx).


I suggest you adjust your trial timings, pretrial- and posttrialpauses to be multiples of refresh rate and review the '/erase' setttings. Let me know what you find after doing so.


Best wishes from a fellow Inquisit user,


~Dave



matt2004
matt2004
Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)
Group: Forum Members
Posts: 11, Visits: 1

Hi Dave,


Many thanks for the very swift, and very useful reply!


I've already compared the use of /stimulusframes and /stimulustimes and have found little difference between the two in terms of the timing lag. However, as a matter of course I've been using /stimulusframes because it's (in theory) more accurate.


>As far as I know, the timings entered in '/pretrialpause' and '/posttrialpause' do not directly depend on the refresh rate, however Inquisit will still (have to) >wait for the next refresh cycle to start before displaying any stimulus. I believe the same is true for the '/trialduration' settings. That may be another source of >the systematic timing error you're seeing.


This is very interesting and it's something I hadn't considered, although it seems totally obvious now that you've pointed it out! However, in my test program (running at 60Hz) the pre and post-trial pauses are set at 100 ms (i.e. 6 frames, near as dammit) and /trialduration is set at 1000 ms (i.e. 60 frames) so I don't see how that could be optimised any further, and also don't think it could be responsible for the relatively large errors I'm seeing. Good tip though - thanks.


>Another often overlooked issue is Inquisit's stimulus erasing behavior. All stimuli will be erased *by default* at the end of the trial unless specified >otherwise (by setting '/erase = false'). This may add (at least one) additional, rogue frame between trials (see here for a >discussion:http://www.millisecond.com/forums/Topic4488.aspx).


Also very interesting, and in fact, by adding /erase = false to the stimulus elements in my test program I've reduced the error from approximately 3 seconds, to 2 seconds - looks like the erasure process was indeed adding one frame on to the end of each trial. Unfortunately it still seems like there are two more extra frames per trial.


Any other bright ideas about where the error might be creeping in?


Many thanks for your help.


M.


PS. Updated version of the test program (with /erase = false added) attached.


Attachments
timing_example.exp (980 views, 1.00 KB)
Tags
Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 12K, Visits: 98K

Matt,


I've been giving this some thought and have been conducting some tests as well. While I'm seeing some minor timing drifts on my system (which btw is not tuned for running experiments), I can't seem to replicate the huge second-range deviations you've been reporting.


I've attached a modified version of your timing/testing script which might be helpful in narrowing this down. Keep me posted on your observations.


~Dave


Attachments
timing_testscript.exp (967 views, 2.00 KB)
seandr
seandr
Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)
Group: Administrators
Posts: 1.3K, Visits: 5.6K

Dave -  thanks for the test script, this really simplifies timing the script.


I've run it a couple of times, and each of the "show_circle" trials starts exactly 1000ms after the previous one started. Note that there are trials at the beginning and end that add some more time, but the 60 show_circle trials last exactly 60 seconds as expected. I've confirmed this by adding the


<data>
/ audit = true
</audit>


command, and the audit times show exactly the same thing. Matt - can you try Dave's script and see if you get similar results?


-Sean




Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 12K, Visits: 98K

<data>
/ audit = true
</audit>


Wonderful feature, too bad this isn't mentioned anywhere in the docs...;-)


~Dave


matt2004
matt2004
Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)
Group: Forum Members
Posts: 11, Visits: 1

Dear Dave/Sean,


Many, many thanks for all your help with this - I'm immensely grateful for the time you've put into helping me out.


I've run Dave's modified script (which is fantastic, by the way!) and attached the data, as well as the log file which results when I include Sean's data auditing element as well. As you can see, I'm still having issues, and it appears that exactly two frames (33ms) are being added to the length of each trial - this very neatly adds up to the roughly two-second lag I'm seeing, over the 60 trials.


Since you guys can't reproduce the problem, I guess this is something hardware-specific - seems too regular to be an interrupt from a background process or something like that. I just searched the forum for any threads containing advice about how to optimise hardware and Windows for running experiments (I already turn off things like virus scanners etc.) but couldn't find much.


Curiouser and curiouser...


Many thanks again,


Matt.


Attachments
timing_testscript.dat (964 views, 16.00 KB)
matt2004
matt2004
Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)
Group: Forum Members
Posts: 11, Visits: 1

Oh - it appears I can't attach two files at once. Here is the log file (renamed to .txt for uploading purposes).



M.


Attachments
timing_testscript.txt (739 views, 4.00 KB)
Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 12K, Visits: 98K

Matt,


definitely sounds like an issue with your specific hardware to me. Some monitors introduce consistent lags by one frame (see Plant & Turner (2009) for an example; http://brm.psychonomic-journals.org/content/41/3/598.abstract). Or it may be a graphics card / driver issue. Anyhoo, here's what you can do to troubleshoot furher:


(1) Check and update your graphics card drivers.


(2) Connect a different monitor to your system.


(3) Run the timing tests on a completely different system for control purposes.


~Dave


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 12K, Visits: 98K

Hmm, looking at the data file you posted there's something fishy going on with respect to the posttrialpause column. Only the first trial reports a value of 100 (~6 frames), the remaining trials report 0. According to my tests, the posttrialpause matters as Inquisit apparently needs some refractory period to do some memory cleanup at the end of a given trial. I believe that's where your additional frames creep in. Which exact build / version of Inquisit are you running?


~Dave


EDIT: I see you're running 3.0.2.0. You should definitely update to 3.0.4.0 or 3.0.5.0 (beta). You're probably hitting a bug that was fixed ages ago.


GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search