By AKrishna - 5/23/2018
Hi all,
I just finished final testing on my next web experiment, which I typically do from a participant point of view (i.e. from the Launch page of the experiment). I made a few changes to one of my batched .iqx files on my local PC, deleted the old one from the millisecond servers (under "Upload Script" in the "Register Web Script" menu) and uploaded the new script from my local PC. However, when I then launched the experiment from the Launch page, my modifications did not show up - the experiment ran using the old script files, even though I had deleted them.
I've tried the following:
- clicking "Save" after each step
- deleting all my uploaded files and reuploading them, then clicking "Save"
- deleting the target .iqx file and running the experiment without uploading a new version, then clicking "Save"
- hard reloading (Ctrl+F5) the Launch page after taking these steps
In all of these cases, the web script ran as though I had never made any changes to the files. Does anyone have any idea what might be causing this and how to actually change the web script? I ask for future reference, as this time, I'm likely just going to unregister the web script and re-register it from scratch - annoying, but not too bad a workaround.
Best,
AK
|
By AKrishna - 5/23/2018
Quick update:
I just unregistered the script, then reregistered it and followed all the steps individually with the new script. I'm still getting the old version of the experiment. I'm going to try changing some of the names now.
|
By AKrishna - 5/23/2018
+xQuick update: I just unregistered the script, then reregistered it and followed all the steps individually with the new script. I'm still getting the old version of the experiment. I'm going to try changing some of the names now.
This worked. I suspect the web version of Inquisit doesn't re-download files it already has, which is a bit of an issue if people modify their own files (or, even worse, commonly used pre-made test scripts) without renaming them and incorporate them into their studies. If, for example, I used the standard Inquisit IAT file, but changed its stimuli, but then a participant did another experiment which also used this file with their own specific stimuli, then the participant might see my stimuli in the other experiment! Is there any clarification as to whether this might be the cause of the problem and how it might be addressed if so?
Thanks,
AK
|
By Dave - 5/24/2018
+x+xQuick update: I just unregistered the script, then reregistered it and followed all the steps individually with the new script. I'm still getting the old version of the experiment. I'm going to try changing some of the names now. This worked. I suspect the web version of Inquisit doesn't re-download files it already has, which is a bit of an issue if people modify their own files (or, even worse, commonly used pre-made test scripts) without renaming them and incorporate them into their studies. If, for example, I used the standard Inquisit IAT file, but changed its stimuli, but then a participant did another experiment which also used this file with their own specific stimuli, then the participant might see my stimuli in the other experiment! Is there any clarification as to whether this might be the cause of the problem and how it might be addressed if so? Thanks, AK
It can happen that either (1) an old version of a script is still cached briefly on your device, or (2) an edge cache server near your location is still serving the old script (it takes a few moments for the old cache object to be invalidated and the edge cache beginning to serve the new one).
In both cases, the issue should be temporary and short, although in case (1), you may want to clear your system's temporary files / temporary internet files to speed things up. Confusion with files from other experiments, even if those carry the same name, should not be possible, these would end up cached elsewhere and separately both locally as well as on the edge cache servers involved.
|
By AKrishna - 5/24/2018
Excellent, that's reassuring. Thanks for the prompt answer!
|
By AKrishna - 7/5/2018
Well, it looks like I got burned on this.
I ran an online study on Prolific yesterday. We did a pilot run with a small sample which turned up a few bugs, so I corrected them and uploaded a new script. As I am situated in Germany and the Prolific participants are pretty much all in England/Scotland, I assumed there would be no trouble with the old script hanging around as a ghost (my personal cache certainly wouldn't impact Prolific users and I would have thought that users in the UK wouldn't be affected by edge cache servers "near my location"). Turns out I was wrong.
I started data collection a good twenty minutes after uploading the latest version of the script (modification time in the script shows as 4:33 pm, first participant after change logged at 4:52 pm). Now based on messages I've been receiving, it seems as though a noticeable proportion of the 200 participants I will have to pay for experienced (at least) one of the very bugs I had solved. I caught the messages about four and a half hours later and checked to see whether I had gone mad and just failed to upload the corrected script, but all was working correctly at that time.
Now I have a dataset of participants, some of who experienced the bug and others who did not, and limited options to find out who is who, which means I can't be certain of a noticeable proportion of my data. Any readers - consider this a cautionary tale.
|
By seandr - 7/6/2018
Hello,
Apologies for this issue, and thanks for sharing the details.
As Dave mentioned, we use AWS "edge caching" to speed up download times for scripts, and it can take some minutes for changes to your script file to propagate to all of the caches. AWS doesn't publish any official numbers on how long this takes (I've seen 5-10 minutes mentioned by tech bloggers), likely because it's indeterminate and varies depending on location. The fact that it took at least 20 minutes in your case is quite a surprise, but it's clearly a possibility we need to take into account. We'll consider options for preventing others from accidentally collecting data with stale scripts, but the most obvious solution is to include a warning in the script wizard to allow for at 30 minutes for changes to propagate to all locations.
-Sean
|
By matt_bic - 7/12/2020
+xHello, Apologies for this issue, and thanks for sharing the details. As Dave mentioned, we use AWS "edge caching" to speed up download times for scripts, and it can take some minutes for changes to your script file to propagate to all of the caches. AWS doesn't publish any official numbers on how long this takes (I've seen 5-10 minutes mentioned by tech bloggers), likely because it's indeterminate and varies depending on location. The fact that it took at least 20 minutes in your case is quite a surprise, but it's clearly a possibility we need to take into account. We'll consider options for preventing others from accidentally collecting data with stale scripts, but the most obvious solution is to include a warning in the script wizard to allow for at 30 minutes for changes to propagate to all locations. -Sean Hi everyone,
I know it is an old post but I was experiencing the same issue today and I found it very useful. I just want to inform Inquisit Web version 4 users that the warning message of which Sean wrote about now appears on the top of the Upload Script page and says:
"Changes to ***.iqx may take 5-10 minutes (occasionally more) to propogate across the network. "
Thanks!
Matteo
|
|