|
dng36
|
|
|
Group: Forum Members
Posts: 7,
Visits: 21
|
Hi,
We are collecting data from participants on Inquisit6, with the experiment on version 6.6.3, using SONA systems to recruit participants. When looking at the raw data files, there are 20, 17 excluding a couple test runs. At least 2 participants, however, claim to have completed the study and report lengthy details about it, but we are not able to see any data associated with their SONA ID or the time it would have been collected at. There might be up to 4 more that ended up in a similar situation.
We have a different participant that was not able to complete the study, and their data still shows up in a raw data file, just incomplete. This is what we’ve typically seen in the past, so not seeing data files at all when allegedly they did complete at least some, if not all, of the study is a little perplexing. Could you guide us as to where we might be able to find those data files and/or how to prevent this data loss in the future?
Thank you,
Daniela Graffeo
Person and Context Lab Manager
|
|
|
|
|
Dave
|
|
|
Group: Administrators
Posts: 13K,
Visits: 110K
|
+xHi, We are collecting data from participants on Inquisit6, with the experiment on version 6.6.3, using SONA systems to recruit participants. When looking at the raw data files, there are 20, 17 excluding a couple test runs. At least 2 participants, however, claim to have completed the study and report lengthy details about it, but we are not able to see any data associated with their SONA ID or the time it would have been collected at. There might be up to 4 more that ended up in a similar situation. We have a different participant that was not able to complete the study, and their data still shows up in a raw data file, just incomplete. This is what we’ve typically seen in the past, so not seeing data files at all when allegedly they did complete at least some, if not all, of the study is a little perplexing. Could you guide us as to where we might be able to find those data files and/or how to prevent this data loss in the future? Thank you, Daniela Graffeo Person and Context Lab Manager Data files remain on the participant's device until such time they can be uploaded. This is the expected and intended behavior to guard against complete data loss. There are various reasons why data files might not be uploaded right away. - The device lost connectivity or the connection was unreliable and timed out (e.g. due to congestion). - A firewall or similar on the network blocked the upload. - Instead of actually exiting the task as instructed, the participant simply closed the app by swiping it off screen. - Automatic data upload was, perhaps mistakenly, disabled in the app (it is enabled by default, so this requires user action). The primary cause for missing data uploads is participants switching to a different app before uploads complete. On mobile devices and Chromebooks, this stops the upload process entirely, and on Windows/Mac it slows it down. To get any remaining data uploaded, make sure the given iPad has a stable internet connection, then open the Inquisit app on the device and go to the Data tab. Any not yet uploaded data files should be listed there, and should be uploaded automatically so long as the Automatic Upload option is checked. If not, upload can be triggered manually by tapping the Cloud Upload button on the Data tab.
|
|
|
|
|
dng36
|
|
|
Group: Forum Members
Posts: 7,
Visits: 21
|
+x+xHi, We are collecting data from participants on Inquisit6, with the experiment on version 6.6.3, using SONA systems to recruit participants. When looking at the raw data files, there are 20, 17 excluding a couple test runs. At least 2 participants, however, claim to have completed the study and report lengthy details about it, but we are not able to see any data associated with their SONA ID or the time it would have been collected at. There might be up to 4 more that ended up in a similar situation. We have a different participant that was not able to complete the study, and their data still shows up in a raw data file, just incomplete. This is what we’ve typically seen in the past, so not seeing data files at all when allegedly they did complete at least some, if not all, of the study is a little perplexing. Could you guide us as to where we might be able to find those data files and/or how to prevent this data loss in the future? Thank you, Daniela Graffeo Person and Context Lab Manager Data files remain on the participant's device until such time they can be uploaded. This is the expected and intended behavior to guard against complete data loss. There are various reasons why data files might not be uploaded right away. - The device lost connectivity or the connection was unreliable and timed out (e.g. due to congestion). - A firewall or similar on the network blocked the upload. - Instead of actually exiting the task as instructed, the participant simply closed the app by swiping it off screen. - Automatic data upload was, perhaps mistakenly, disabled in the app (it is enabled by default, so this requires user action). The primary cause for missing data uploads is participants switching to a different app before uploads complete. On mobile devices and Chromebooks, this stops the upload process entirely, and on Windows/Mac it slows it down. To get any remaining data uploaded, make sure the given iPad has a stable internet connection, then open the Inquisit app on the device and go to the Data tab. Any not yet uploaded data files should be listed there, and should be uploaded automatically so long as the Automatic Upload option is checked. If not, upload can be triggered manually by tapping the Cloud Upload button on the Data tab. Hi Dave, Thanks for the info! After reaching out to one of the participants, she went into her Inquisit app and the data was sent to us, which is great! What we are wondering is that is there anything that could be in our script that would make it crash/fail this frequently? We’ve run studies like this in the lab with many many more participants and not run into issues like this before; 2 participants crashing and 2 participants’ data failing to upload (one overlaps in both groups) over around 20 participants. Could there be a data file that is difficult to load or something along those lines? We really want to prevent this as we’re having to pay participants for their time even if we didn’t get full data, so if there’s anything you can think of, I’m more than willing to check it. The one participants that helped us with our troubleshooting efforts did seem to be using a MacBook, but would it be helpful to make participants complete it on a computer (rather than an iPad or something similar)? If so, is there a setting in Inquisit that can force this? Thanks so much for your troubleshooting help. Best, Daniela
|
|
|
|
|
Dave
|
|
|
Group: Administrators
Posts: 13K,
Visits: 110K
|
+x+x+xHi, We are collecting data from participants on Inquisit6, with the experiment on version 6.6.3, using SONA systems to recruit participants. When looking at the raw data files, there are 20, 17 excluding a couple test runs. At least 2 participants, however, claim to have completed the study and report lengthy details about it, but we are not able to see any data associated with their SONA ID or the time it would have been collected at. There might be up to 4 more that ended up in a similar situation. We have a different participant that was not able to complete the study, and their data still shows up in a raw data file, just incomplete. This is what we’ve typically seen in the past, so not seeing data files at all when allegedly they did complete at least some, if not all, of the study is a little perplexing. Could you guide us as to where we might be able to find those data files and/or how to prevent this data loss in the future? Thank you, Daniela Graffeo Person and Context Lab Manager Data files remain on the participant's device until such time they can be uploaded. This is the expected and intended behavior to guard against complete data loss. There are various reasons why data files might not be uploaded right away. - The device lost connectivity or the connection was unreliable and timed out (e.g. due to congestion). - A firewall or similar on the network blocked the upload. - Instead of actually exiting the task as instructed, the participant simply closed the app by swiping it off screen. - Automatic data upload was, perhaps mistakenly, disabled in the app (it is enabled by default, so this requires user action). The primary cause for missing data uploads is participants switching to a different app before uploads complete. On mobile devices and Chromebooks, this stops the upload process entirely, and on Windows/Mac it slows it down. To get any remaining data uploaded, make sure the given iPad has a stable internet connection, then open the Inquisit app on the device and go to the Data tab. Any not yet uploaded data files should be listed there, and should be uploaded automatically so long as the Automatic Upload option is checked. If not, upload can be triggered manually by tapping the Cloud Upload button on the Data tab. Hi Dave, Thanks for the info! After reaching out to one of the participants, she went into her Inquisit app and the data was sent to us, which is great! What we are wondering is that is there anything that could be in our script that would make it crash/fail this frequently? We’ve run studies like this in the lab with many many more participants and not run into issues like this before; 2 participants crashing and 2 participants’ data failing to upload (one overlaps in both groups) over around 20 participants. Could there be a data file that is difficult to load or something along those lines? We really want to prevent this as we’re having to pay participants for their time even if we didn’t get full data, so if there’s anything you can think of, I’m more than willing to check it. The one participants that helped us with our troubleshooting efforts did seem to be using a MacBook, but would it be helpful to make participants complete it on a computer (rather than an iPad or something similar)? If so, is there a setting in Inquisit that can force this? Thanks so much for your troubleshooting help. Best, Daniela Data files are generally small, so there is no reason why any particular one should cause an issue. The typical reasons why uploads fail or are delayed, I have already enumerated, so I would recommend you check to what degree those apply and revise your script and/or general participant instructions accordingly. For example: If you have some end page in your script that says something like "You have completed the task, thank you for participating," that will cause many mobile users to simply close the app, thinking they're done, instead of actually exiting the script normall (by pressing the next button). This will prevent uploads. You can either get rid of that end page entirely so that the task exits without any participant action or you can make the instructions clearer, so that participants actually exit the task instead of closing the app. Note that unuploaded data doesn't mean the script crashed or failed, so I'm not sure why you think it is. Are there any reports from participants complainng about crashes?
|
|
|
|
|
dng36
|
|
|
Group: Forum Members
Posts: 7,
Visits: 21
|
+x+x+x+xHi, We are collecting data from participants on Inquisit6, with the experiment on version 6.6.3, using SONA systems to recruit participants. When looking at the raw data files, there are 20, 17 excluding a couple test runs. At least 2 participants, however, claim to have completed the study and report lengthy details about it, but we are not able to see any data associated with their SONA ID or the time it would have been collected at. There might be up to 4 more that ended up in a similar situation. We have a different participant that was not able to complete the study, and their data still shows up in a raw data file, just incomplete. This is what we’ve typically seen in the past, so not seeing data files at all when allegedly they did complete at least some, if not all, of the study is a little perplexing. Could you guide us as to where we might be able to find those data files and/or how to prevent this data loss in the future? Thank you, Daniela Graffeo Person and Context Lab Manager Data files remain on the participant's device until such time they can be uploaded. This is the expected and intended behavior to guard against complete data loss. There are various reasons why data files might not be uploaded right away. - The device lost connectivity or the connection was unreliable and timed out (e.g. due to congestion). - A firewall or similar on the network blocked the upload. - Instead of actually exiting the task as instructed, the participant simply closed the app by swiping it off screen. - Automatic data upload was, perhaps mistakenly, disabled in the app (it is enabled by default, so this requires user action). The primary cause for missing data uploads is participants switching to a different app before uploads complete. On mobile devices and Chromebooks, this stops the upload process entirely, and on Windows/Mac it slows it down. To get any remaining data uploaded, make sure the given iPad has a stable internet connection, then open the Inquisit app on the device and go to the Data tab. Any not yet uploaded data files should be listed there, and should be uploaded automatically so long as the Automatic Upload option is checked. If not, upload can be triggered manually by tapping the Cloud Upload button on the Data tab. Hi Dave, Thanks for the info! After reaching out to one of the participants, she went into her Inquisit app and the data was sent to us, which is great! What we are wondering is that is there anything that could be in our script that would make it crash/fail this frequently? We’ve run studies like this in the lab with many many more participants and not run into issues like this before; 2 participants crashing and 2 participants’ data failing to upload (one overlaps in both groups) over around 20 participants. Could there be a data file that is difficult to load or something along those lines? We really want to prevent this as we’re having to pay participants for their time even if we didn’t get full data, so if there’s anything you can think of, I’m more than willing to check it. The one participants that helped us with our troubleshooting efforts did seem to be using a MacBook, but would it be helpful to make participants complete it on a computer (rather than an iPad or something similar)? If so, is there a setting in Inquisit that can force this? Thanks so much for your troubleshooting help. Best, Daniela Data files are generally small, so there is no reason why any particular one should cause an issue. The typical reasons why uploads fail or are delayed, I have already enumerated, so I would recommend you check to what degree those apply and revise your script and/or general participant instructions accordingly. For example: If you have some end page in your script that says something like "You have completed the task, thank you for participating," that will cause many mobile users to simply close the app, thinking they're done, instead of actually exiting the script normall (by pressing the next button). This will prevent uploads. You can either get rid of that end page entirely so that the task exits without any participant action or you can make the instructions clearer, so that participants actually exit the task instead of closing the app. Note that unuploaded data doesn't mean the script crashed or failed, so I'm not sure why you think it is. Are there any reports from participants complainng about crashes? Hi Dave, Okay, sounds good; yes two participants emailed us saying that the script would not progress about 30 minutes apart from each other. It was not in the same spot, and one participant’s partial data uploaded, the other one quit out and was retrieved by them opening the app later on. We’ve run more participants since then who didn’t experience issues. All other participants have experienced no crashes, including one whose data didn’t upload, but theoretically completed it (I’m thinking they quit out without realizing it). Usually when we’ve had participants experiencing crashes, it’s attributable to something in the script, but there didn’t seem to be something wrong with the script, and both of the participants who experienced crashes claim their internet was stable (though it’s possible it wasn’t). It was surprising encountering this many issues while only having run 20 participants, and I just want to do everything I can to make sure no issues occur in the future. If you think these are likely idiosyncratic and not due to anything on our end, then we will hope for the best. Thanks so much for your help, Daniela
|
|
|
|
|
Dave
|
|
|
Group: Administrators
Posts: 13K,
Visits: 110K
|
+x+x+x+x+xHi, We are collecting data from participants on Inquisit6, with the experiment on version 6.6.3, using SONA systems to recruit participants. When looking at the raw data files, there are 20, 17 excluding a couple test runs. At least 2 participants, however, claim to have completed the study and report lengthy details about it, but we are not able to see any data associated with their SONA ID or the time it would have been collected at. There might be up to 4 more that ended up in a similar situation. We have a different participant that was not able to complete the study, and their data still shows up in a raw data file, just incomplete. This is what we’ve typically seen in the past, so not seeing data files at all when allegedly they did complete at least some, if not all, of the study is a little perplexing. Could you guide us as to where we might be able to find those data files and/or how to prevent this data loss in the future? Thank you, Daniela Graffeo Person and Context Lab Manager Data files remain on the participant's device until such time they can be uploaded. This is the expected and intended behavior to guard against complete data loss. There are various reasons why data files might not be uploaded right away. - The device lost connectivity or the connection was unreliable and timed out (e.g. due to congestion). - A firewall or similar on the network blocked the upload. - Instead of actually exiting the task as instructed, the participant simply closed the app by swiping it off screen. - Automatic data upload was, perhaps mistakenly, disabled in the app (it is enabled by default, so this requires user action). The primary cause for missing data uploads is participants switching to a different app before uploads complete. On mobile devices and Chromebooks, this stops the upload process entirely, and on Windows/Mac it slows it down. To get any remaining data uploaded, make sure the given iPad has a stable internet connection, then open the Inquisit app on the device and go to the Data tab. Any not yet uploaded data files should be listed there, and should be uploaded automatically so long as the Automatic Upload option is checked. If not, upload can be triggered manually by tapping the Cloud Upload button on the Data tab. Hi Dave, Thanks for the info! After reaching out to one of the participants, she went into her Inquisit app and the data was sent to us, which is great! What we are wondering is that is there anything that could be in our script that would make it crash/fail this frequently? We’ve run studies like this in the lab with many many more participants and not run into issues like this before; 2 participants crashing and 2 participants’ data failing to upload (one overlaps in both groups) over around 20 participants. Could there be a data file that is difficult to load or something along those lines? We really want to prevent this as we’re having to pay participants for their time even if we didn’t get full data, so if there’s anything you can think of, I’m more than willing to check it. The one participants that helped us with our troubleshooting efforts did seem to be using a MacBook, but would it be helpful to make participants complete it on a computer (rather than an iPad or something similar)? If so, is there a setting in Inquisit that can force this? Thanks so much for your troubleshooting help. Best, Daniela Data files are generally small, so there is no reason why any particular one should cause an issue. The typical reasons why uploads fail or are delayed, I have already enumerated, so I would recommend you check to what degree those apply and revise your script and/or general participant instructions accordingly. For example: If you have some end page in your script that says something like "You have completed the task, thank you for participating," that will cause many mobile users to simply close the app, thinking they're done, instead of actually exiting the script normall (by pressing the next button). This will prevent uploads. You can either get rid of that end page entirely so that the task exits without any participant action or you can make the instructions clearer, so that participants actually exit the task instead of closing the app. Note that unuploaded data doesn't mean the script crashed or failed, so I'm not sure why you think it is. Are there any reports from participants complainng about crashes? Hi Dave, Okay, sounds good; yes two participants emailed us saying that the script would not progress about 30 minutes apart from each other. It was not in the same spot, and one participant’s partial data uploaded, the other one quit out and was retrieved by them opening the app later on. We’ve run more participants since then who didn’t experience issues. All other participants have experienced no crashes, including one whose data didn’t upload, but theoretically completed it (I’m thinking they quit out without realizing it). Usually when we’ve had participants experiencing crashes, it’s attributable to something in the script, but there didn’t seem to be something wrong with the script, and both of the participants who experienced crashes claim their internet was stable (though it’s possible it wasn’t). It was surprising encountering this many issues while only having run 20 participants, and I just want to do everything I can to make sure no issues occur in the future. If you think these are likely idiosyncratic and not due to anything on our end, then we will hope for the best. Thanks so much for your help, Daniela If you can get the application logs from those participants (past and future), who say they experienced a crash, that might provide some leads. I can't say anything about the script, since you haven't shared it. Two participants encountering what sounds like different issues, however, doesn't strke me as pointing to a deeper or more general problem, that said.
|
|
|
|