Inquisit timing accuracy.


Author
Message
Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 108K

Interestingly, the above is only true on the computer I use for testing - when I run the programs on the computer in my office (similar specifications) I still see timing lags.


Request for clarification, if I may: This is to say that on your office PC


(a) the debugging script does not report timing lags although there are such lags (as measured via external equipment / TTL)?


or


(b) the debugging script does accurately report timing lags despite the various optimizations (postrialpause, erase=false, etc)?


Thanks,


~Dave


matt2004
matt2004
Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)Esteemed Member (2K reputation)
Group: Forum Members
Posts: 11, Visits: 1

Dear Dave/Sean,


Many thanks once again for the extensive testing and many helpful comments you guys have provided on this topic - really, really helpful. I'm very happy to report that including a post-trial pause, setting erase=false for all my stimuli, and updating to the latest build of Inquisit have pretty much solved the problem I was having - Dave's debugging timing code reports trial durations which are pretty much bang-on, with an occasional deviation by a millisecond or two. Testing with external hardware (getting Inquisit to output a TTL pulse on the parallel port with each trial and recording it with a analogue-digital converter with 1ms resolution) confirms that the timing is accurate as well. This is a massive improvement over the situation I was in before and I'm re-commencing my experiments with renewed confidence in what I'm doing. 


Interestingly, the above is only true on the computer I use for testing - when I run the programs on the computer in my office (similar specifications) I still see timing lags. Nevertheless, at least with Dave's debugging code I can be certain about what's going on whatever hardware I use in the future.


I have a suggestion actually - it would have been immensely helpful to me if the timing code had been easily available somewhere. I've been using Inquisit for a while but am in no way a 'proper' programmer and had no idea Inquisit could be used in this way, and even less idea how to implement those functions. Would it perhaps be possible to include an example script similar to Dave's in the Inquisit Task Library on the main website?


Many, many thanks again,


Matt.


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 108K

More weird - did you update the test script? I just downloaded it again, and the calculations were different than the script I previous had. The version I have now does indeed correct for the start_trial.


I might have initially uploaded the wrong version -- however said version can't have been online for more than a couple of minutes... Sorry for the confusion.


~Dave


seandr
seandr
Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)
Group: Administrators
Posts: 1.3K, Visits: 5.9K

More weird - did you update the test script? I just downloaded it again, and the calculations were different than the script I previous had. The version I have now does indeed correct for the start_trial.


I'm still not seeing any delays larger than 1ms, however. I'll try on some more machines...


-Sean


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 108K

Ah, that might explain the discrepancy. The mean difference includes "start_trial", which does not have a fixed length (it's length depends on how quickly you respond). The performance in question is the duration of the fixed length trials, so the mean difference isn't relevant.


No. My script corrects for that.


seandr
seandr
Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)
Group: Administrators
Posts: 1.3K, Visits: 5.9K

Ah, that might explain the discrepancy. The mean difference includes "start_trial", which does not have a fixed length (it's length depends on how quickly you respond). The performance in question is the duration of the fixed length trials, so the mean difference isn't relevant.


I've been opening the data file and looking at the reported difference for each fixed length trial, and all the durations have fallen within the range of 1000-1001. Are you seeing any single trial durations outside this range?


-Sean


Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 108K

Thanks for the reply, Sean. Really weird. I can reproduce these delays reliably, though.


Just to make sure we are looking at the same numbers, I'm looking at the "Difference" value as a measure of the trial duration - is that what you are looking at?


Yes, that's what I'm talking about. The mean difference to be precise. In the test script, "Difference" reflects the duration of the last trial (measured as difference between the previous and current trial's start time), "Mean" reflects the mean duration across all trials.


Thanks,


~Dave


seandr
seandr
Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)
Group: Administrators
Posts: 1.3K, Visits: 5.9K

P.S. If you are still seeing the delays, it would be interesting to know whether any of the following steps cumulatively have an impact:


1) Removing all data and audit recording (you've already done this)
2) Remove the ontrialend logic and the text stimuli showing the debug info from the
3) Remove the shape stimuli from the presentation


seandr
seandr
Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)Supreme Being (144K reputation)
Group: Administrators
Posts: 1.3K, Visits: 5.9K

Weird. I've run the test several more times with both data recording and auditing turned on, and I'm seeing trial durations all in the range of 1000-1001 ms. I've run the test on a fast and slow machine with the same results. I can't even reproduce the delays I initially saw, and I'm actually thinking those resulted from me running the tests using a debug build with the debugger attached.


Just to make sure we are looking at the same numbers, I'm looking at the "Difference" value as a measure of the trial duration - is that what you are looking at?


-Sean




Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 108K

A couple of more observations from tests conducted today:


- Even setting recorddata=false to prevent disc access does not resolve the issue for me. While the delay is somewhat reduced, there are still approx. 33ms added to each trial.


- Unrelated to the timing issue: recorddata doesn't seem to work when set at the <expt> level.


~Dave


GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search