Millisecond Forums

Output isn't correctly saved

https://forums.millisecond.com/Topic22200.aspx

By Lea.Elle - 8/1/2017

Hi,

we have a problem: we used the AMP (the script millisecond provides) and the raw data output file is not right. All the columns we defined in the script are there but in some rows the data set of a paricipant isn't where it is supoosed to be, like you can see in the screen shot. 



Basically all the data is there, but it's not in the right place for all participants. For most it is, but not for all. For some the data continues in columns which aren't even defined. 
In some cases it might not even be possible for us to find out which data set belongs to which participant because sometimes the data row is interrupted in one row and continues in an other somewhere else.

Do you have an idea or can help us? 

Thank you!
Lea
By Dave - 8/2/2017

Lea.Elle - Wednesday, August 2, 2017
Hi,

we have a problem: we used the AMP (the script millisecond provides) and the raw data output file is not right. All the columns we defined in the script are there but in some rows the data set of a paricipant isn't where it is supoosed to be, like you can see in the screen shot. 



Basically all the data is there, but it's not in the right place for all participants. For most it is, but not for all. For some the data continues in columns which aren't even defined. 
In some cases it might not even be possible for us to find out which data set belongs to which participant because sometimes the data row is interrupted in one row and continues in an other somewhere else.

Do you have an idea or can help us? 

Thank you!
Lea

This would typically happen when you have data generated by different revisions of the script, some recording more columns than others. When you merge the data without taking that into account, you'll get misaligned columns. I'd recommend figuring out which files were recorded with which revision and then merging those sets separately.
By Lea.Elle - 8/2/2017

Dave - Wednesday, August 2, 2017
Lea.Elle - Wednesday, August 2, 2017
Hi,

we have a problem: we used the AMP (the script millisecond provides) and the raw data output file is not right. All the columns we defined in the script are there but in some rows the data set of a paricipant isn't where it is supoosed to be, like you can see in the screen shot. 



Basically all the data is there, but it's not in the right place for all participants. For most it is, but not for all. For some the data continues in columns which aren't even defined. 
In some cases it might not even be possible for us to find out which data set belongs to which participant because sometimes the data row is interrupted in one row and continues in an other somewhere else.

Do you have an idea or can help us? 

Thank you!
Lea

This would typically happen when you have data generated by different revisions of the script, some recording more columns than others. When you merge the data without taking that into account, you'll get misaligned columns. I'd recommend figuring out which files were recorded with which revision and then merging those sets separately.

Thanks for your fast answer! It's not the case for our study though. There is just one AMP all of the participants do. In the summary file everything looks perfect. It is just the raw data that's showing this seemingly unsystematic pattern. The only difference between the conditions of the experiment is when the AMP is done. For some it's earlier and for some it's later. Do you maybe have another idea for that problem?
By Dave - 8/3/2017

Lea.Elle - Wednesday, August 2, 2017
Dave - Wednesday, August 2, 2017
Lea.Elle - Wednesday, August 2, 2017
Hi,

we have a problem: we used the AMP (the script millisecond provides) and the raw data output file is not right. All the columns we defined in the script are there but in some rows the data set of a paricipant isn't where it is supoosed to be, like you can see in the screen shot. 



Basically all the data is there, but it's not in the right place for all participants. For most it is, but not for all. For some the data continues in columns which aren't even defined. 
In some cases it might not even be possible for us to find out which data set belongs to which participant because sometimes the data row is interrupted in one row and continues in an other somewhere else.

Do you have an idea or can help us? 

Thank you!
Lea

This would typically happen when you have data generated by different revisions of the script, some recording more columns than others. When you merge the data without taking that into account, you'll get misaligned columns. I'd recommend figuring out which files were recorded with which revision and then merging those sets separately.

Thanks for your fast answer! It's not the case for our study though. There is just one AMP all of the participants do. In the summary file everything looks perfect. It is just the raw data that's showing this seemingly unsystematic pattern. The only difference between the conditions of the experiment is when the AMP is done. For some it's earlier and for some it's later. Do you maybe have another idea for that problem?

No, offhand I do not have another idea. Looking at the screenshot in your original post, it appears that line breaks are missing or stripped out. How exactly were those raw data files merged?
By Lea.Elle - 8/3/2017

Dave - Thursday, August 3, 2017
Lea.Elle - Wednesday, August 2, 2017
Dave - Wednesday, August 2, 2017
Lea.Elle - Wednesday, August 2, 2017
Hi,

we have a problem: we used the AMP (the script millisecond provides) and the raw data output file is not right. All the columns we defined in the script are there but in some rows the data set of a paricipant isn't where it is supoosed to be, like you can see in the screen shot. 



Basically all the data is there, but it's not in the right place for all participants. For most it is, but not for all. For some the data continues in columns which aren't even defined. 
In some cases it might not even be possible for us to find out which data set belongs to which participant because sometimes the data row is interrupted in one row and continues in an other somewhere else.

Do you have an idea or can help us? 

Thank you!
Lea

This would typically happen when you have data generated by different revisions of the script, some recording more columns than others. When you merge the data without taking that into account, you'll get misaligned columns. I'd recommend figuring out which files were recorded with which revision and then merging those sets separately.

Thanks for your fast answer! It's not the case for our study though. There is just one AMP all of the participants do. In the summary file everything looks perfect. It is just the raw data that's showing this seemingly unsystematic pattern. The only difference between the conditions of the experiment is when the AMP is done. For some it's earlier and for some it's later. Do you maybe have another idea for that problem?

No, offhand I do not have another idea. Looking at the screenshot in your original post, it appears that line breaks are missing or stripped out. How exactly were those raw data files merged?

We used this:

<data>
/columns = (build, date, time, subject, blockcode, blocknum, trialcode, trialnum, values.primestim, values.targetstim,
response, correct, latency,
trial.prac1.totalpercentcorrect, trial.prac2.totalpercentcorrect, trial.prac3.totalpercentcorrect,
trial.Xa.totalpercentcorrect)
/separatefiles = false
</data>

I also took a closer look at the data and it appears that there are sometimes even two sets of variable saved in one column.

By Dave - 8/4/2017

Lea.Elle - Thursday, August 3, 2017
Dave - Thursday, August 3, 2017
Lea.Elle - Wednesday, August 2, 2017
Dave - Wednesday, August 2, 2017
Lea.Elle - Wednesday, August 2, 2017
Hi,

we have a problem: we used the AMP (the script millisecond provides) and the raw data output file is not right. All the columns we defined in the script are there but in some rows the data set of a paricipant isn't where it is supoosed to be, like you can see in the screen shot. 



Basically all the data is there, but it's not in the right place for all participants. For most it is, but not for all. For some the data continues in columns which aren't even defined. 
In some cases it might not even be possible for us to find out which data set belongs to which participant because sometimes the data row is interrupted in one row and continues in an other somewhere else.

Do you have an idea or can help us? 

Thank you!
Lea

This would typically happen when you have data generated by different revisions of the script, some recording more columns than others. When you merge the data without taking that into account, you'll get misaligned columns. I'd recommend figuring out which files were recorded with which revision and then merging those sets separately.

Thanks for your fast answer! It's not the case for our study though. There is just one AMP all of the participants do. In the summary file everything looks perfect. It is just the raw data that's showing this seemingly unsystematic pattern. The only difference between the conditions of the experiment is when the AMP is done. For some it's earlier and for some it's later. Do you maybe have another idea for that problem?

No, offhand I do not have another idea. Looking at the screenshot in your original post, it appears that line breaks are missing or stripped out. How exactly were those raw data files merged?

We used this:

<data>
/columns = (build, date, time, subject, blockcode, blocknum, trialcode, trialnum, values.primestim, values.targetstim,
response, correct, latency,
trial.prac1.totalpercentcorrect, trial.prac2.totalpercentcorrect, trial.prac3.totalpercentcorrect,
trial.Xa.totalpercentcorrect)
/separatefiles = false
</data>

I also took a closer look at the data and it appears that there are sometimes even two sets of variable saved in one column.


Can you attach the data file (or files if there are multiple)? To attach files to a post, click +Insert and then Add File.