Data uploads failing - any countermeasures?

AKrishna — Tue, 23 Jul 2024 12:04:31 GMT

Dear all,

in our current online study (aatsrbr2), I'm observing relatively high rates of participants who complete the study, but whose data doesn't seem to arrive. In some cases, no data at all is uploaded, while in others, the data from our first scripts (informed consent and demographics) is uploaded, but the data from the main experiment isn't. Now obviously, some losses are to be expected (participants dropping out or encountering technical issues after the demographics), but after exchanging messages with quite a few participants on Prolific, there are 9 cases in which I could verify the participant took part (they were able to describe details of the experiment that are otherwise inaccessible), but no data arrived, plus three likely potentials. Given that Inquisit currently lists 254 sessions, including several test sessions we did ourselves before data collection started and including some early drop-outs, it seems like this is around 3.5 - 5% of otherwise valid datasets that may not be arriving.

I asked the affected participants to restart the InquisitPlayer app (insofar as they hadn't already uninstalled it), which seems to have belatedly uploaded the data in two cases. However, in the remaining cases, it either didn't work or people simply never responded. Now, it's clear that environmental issues will sometimes prevent data from being uploaded, but I was wondering whether there are any known measures or tips anyone can recommend for reducing the likelihood of this happening? It's starting to become a cost factor (we have to pay these people, as the technical problem isn't their fault) and it's particularly frustrating in cases like this experiment, where we're two datasets short of our preregistered sample, but the last three participants on Prolific have been affected by this problem, causing me to keep on having to add individual slots and write messages to affected participants.

I'd appreciate any tips!

RE: Data uploads failing - any countermeasures?

Dave — Tue, 23 Jul 2024 12:04:31 GMT

+x[quote]

[b]AKrishna - 7/23/2024[/b]
Dear all,

in our current online study (aatsrbr2), I'm observing relatively high rates of participants who complete the study, but whose data doesn't seem to arrive. In some cases, no data at all is uploaded, while in others, the data from our first scripts (informed consent and demographics) is uploaded, but the data from the main experiment isn't. Now obviously, some losses are to be expected (participants dropping out or encountering technical issues after the demographics), but after exchanging messages with quite a few participants on Prolific, there are 9 cases in which I could verify the participant took part (they were able to describe details of the experiment that are otherwise inaccessible), but no data arrived, plus three likely potentials. Given that Inquisit currently lists 254 sessions, including several test sessions we did ourselves before data collection started and including some early drop-outs, it seems like this is around 3.5 - 5% of otherwise valid datasets that may not be arriving.

I asked the affected participants to restart the InquisitPlayer app (insofar as they hadn't already uninstalled it), which seems to have belatedly uploaded the data in two cases. However, in the remaining cases, it either didn't work or people simply never responded. Now, it's clear that environmental issues will sometimes prevent data from being uploaded, but I was wondering whether there are any known measures or tips anyone can recommend for reducing the likelihood of this happening? It's starting to become a cost factor (we have to pay these people, as the technical problem isn't their fault) and it's particularly frustrating in cases like this experiment, where we're two datasets short of our preregistered sample, but the last three participants on Prolific have been affected by this problem, causing me to keep on having to add individual slots and write messages to affected participants.

I'd appreciate any tips!
[/quote]

The participant logs for your study are full of timeout and connection closed errors, which typically indicate that the participants aren't on a stable internet connection. Obviously, if connectivity is bad or gone entirely, then no data can be uploaded and the files will remain on the participant's device until they can be uploaded (app running, internet connection available).

If I were you, I'd reinforce to participants:
Make sure your internet connection is stable. Don't use a spotty WiFi, don't do the study from any network that automatically kicks you out after some time (e.g. public WiFi or school / corporate networks with captive portals). When using a laptop, make sure it's plugged in and not running on battery alone, otherwise the device may turn off the network interface after some idle period to save power.

There'd be a few other things to mention and consider if mobile devices (phones, tablets) were involved, but as far as I can see you already disallow these for this particular study.

Millisecond Forums » Millisecond Forums » Inquisit 6 » Data uploads failing - any countermeasures?

Data uploads failing - any countermeasures?

RE: Data uploads failing - any countermeasures?