Merged data in unfortunate format


Author
Message
Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 105K
SandraSchwark - Wednesday, November 16, 2016
I'm attaching two pictures, hoping to better explain what the issue is. On the left is the Inquisit data output. As you can see, we get the data from subject (participant) 1 in about 120 seperate columns (you can only see 9 on the screenshot). What we want is what's shown on the right: get one(!) row per subject with all the subject's data in it. 
 https://www.millisecond.com/forums/Uploads/Images/a01dd7ad-e324-4f40-8ae6-8cc8.JPGhttps://www.millisecond.com/forums/Uploads/Images/40751014-f31c-47bc-a426-bb71.JPG

> As you can see, we get the data from subject (participant) 1 in about 120 seperate columns
Those are rows, not columns. That is how raw, long-format data files always look like. They must. It is the trial-by-trial record of the participant's session.

> What we want is what's shown on the right: get one(!) row per subject with all the subject's data in it.
This is a format completely *unsuitable* for analysis. What you ultimately want is a single row of *summary* statistics per participant. If you used any of the Inquisit 4 or Inquisit 5 IAT scripts from the library, they create *summary* data files in *addition* to the trial-by-trial raw data files that provide exactly that: A single row with the respective D-scores. If you only have *raw* data files, you need to *process* them, i.e. calculate the aggregate statistics *from the long-format* raw data.

Please read the following resources:
- Lacroix & Giguère (2006) available athttp://www.tqmp.org/Content/vol02-1/p020/p020.pdf
- https://computingforpsychologists.wordpress.com/2012/12/11/how-to-analyse-reaction-time-rt-data-part-1/
- http://ron.dotsch.org/importing-inquisit-data-files-into-spss-pasw/
- http://jeromyanglim.blogspot.de/2010/07/how-to-process-inquisit-raw-data-in.html






SandraSchwark
SandraSchwark
Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)
Group: Forum Members
Posts: 2, Visits: 11
I'm attaching two pictures, hoping to better explain what the issue is. On the left is the Inquisit data output. As you can see, we get the data from subject (participant) 1 in about 120 seperate columns (you can only see 9 on the screenshot). What we want is what's shown on the right: get one(!) row per subject with all the subject's data in it. 
 https://www.millisecond.com/forums/Uploads/Images/a01dd7ad-e324-4f40-8ae6-8cc8.JPGhttps://www.millisecond.com/forums/Uploads/Images/40751014-f31c-47bc-a426-bb71.JPG

Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 105K
SandraSchwark - Tuesday, November 15, 2016
Dave - Tuesday, November 15, 2016
mongrel - Tuesday, November 15, 2016
Dave - Thursday, October 27, 2016
<data> records in *long* format. You need to sort and restructure long format files to prepare them for analysis. This is not hard once you have done it a few times. See e.g www.tqmp.org/RegularArticles/vol02-1/p020/p020.pdf .

You can make use of <summarydata>, which allows you to capture a single line of summary metrics per subject.

Hi Dave,

thanks!
Could you please explain how to use <summarydata> to capture a single line for each subject? There is no attribute, such as /rows in which I could put the subject-id. 

Best 
M.

You specify a <summarydata> element as detailed in the documentation: https://www.millisecond.com/support/docs/v5/html/language/elements/summarydata.htm

The /columns attribute is where you specify the variables you want it to capture. For the subject id that's

/ columns = [script.subjectid ...]

Hi Dave,

I'm mongrel's colleague and we're working on the same project. We have already specified all the variables and the data that is written looks fine - that's not the problem. The problem is the data's format, as described in the initial post under this topic. Rather than having separate columns for each attribute, we'd like to have one column per participant:

participant   Test1   Test2   Test3
1                      5          7         1
2                      2          2         2
3                      6          6         6
4                      4          4         7

This is what it looks like in any SPSS spreadsheet which is why we would much prefer this format for our output if possible. 

Thanks,

Sandra



#1: So far neither you nor your colleague has explained how exactly the data format described in the original post was generated. What do test1 to test 3 represent? Are they blocks run in the same script? Are they separate surveys? Are they results originating from various, separate scripts? Also, what exactly are you merging? Raw data files? Summary files? All of this is relevant.
#2: Depending on what the answers to the questions in #1 are, you will have to restructure your data to prepare it for analysis. The <data> and <summarydata> elements determine what a script records. In and of themselves, they have nothing to do with merging data files and preparing them for further analysis.

SandraSchwark
SandraSchwark
Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)Respected Member (308 reputation)
Group: Forum Members
Posts: 2, Visits: 11
Dave - Tuesday, November 15, 2016
mongrel - Tuesday, November 15, 2016
Dave - Thursday, October 27, 2016
<data> records in *long* format. You need to sort and restructure long format files to prepare them for analysis. This is not hard once you have done it a few times. See e.g www.tqmp.org/RegularArticles/vol02-1/p020/p020.pdf .

You can make use of <summarydata>, which allows you to capture a single line of summary metrics per subject.

Hi Dave,

thanks!
Could you please explain how to use <summarydata> to capture a single line for each subject? There is no attribute, such as /rows in which I could put the subject-id. 

Best 
M.

You specify a <summarydata> element as detailed in the documentation: https://www.millisecond.com/support/docs/v5/html/language/elements/summarydata.htm

The /columns attribute is where you specify the variables you want it to capture. For the subject id that's

/ columns = [script.subjectid ...]

Hi Dave,

I'm mongrel's colleague and we're working on the same project. We have already specified all the variables and the data that is written looks fine - that's not the problem. The problem is the data's format, as described in the initial post under this topic. Rather than having separate columns for each attribute, we'd like to have one column per participant:

participant   Test1   Test2   Test3
1                      5          7         1
2                      2          2         2
3                      6          6         6
4                      4          4         7

This is what it looks like in any SPSS spreadsheet which is why we would much prefer this format for our output if possible. 

Thanks,

Sandra



Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 105K
mongrel - Tuesday, November 15, 2016
Dave - Thursday, October 27, 2016
<data> records in *long* format. You need to sort and restructure long format files to prepare them for analysis. This is not hard once you have done it a few times. See e.g www.tqmp.org/RegularArticles/vol02-1/p020/p020.pdf .

You can make use of <summarydata>, which allows you to capture a single line of summary metrics per subject.

Hi Dave,

thanks!
Could you please explain how to use <summarydata> to capture a single line for each subject? There is no attribute, such as /rows in which I could put the subject-id. 

Best 
M.

You specify a <summarydata> element as detailed in the documentation: https://www.millisecond.com/support/docs/v5/html/language/elements/summarydata.htm

The /columns attribute is where you specify the variables you want it to capture. For the subject id that's

/ columns = [script.subjectid ...]

mongrel
mongrel
Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)
Group: Forum Members
Posts: 53, Visits: 147
Dave - Thursday, October 27, 2016
<data> records in *long* format. You need to sort and restructure long format files to prepare them for analysis. This is not hard once you have done it a few times. See e.g www.tqmp.org/RegularArticles/vol02-1/p020/p020.pdf .

You can make use of <summarydata>, which allows you to capture a single line of summary metrics per subject.

Hi Dave,

thanks!
Could you please explain how to use <summarydata> to capture a single line for each subject? There is no attribute, such as /rows in which I could put the subject-id. 

Best 
M.
Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 13K, Visits: 105K
<data> records in *long* format. You need to sort and restructure long format files to prepare them for analysis. This is not hard once you have done it a few times. See e.g www.tqmp.org/RegularArticles/vol02-1/p020/p020.pdf .

You can make use of <summarydata>, which allows you to capture a single line of summary metrics per subject.

mongrel
mongrel
Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)Guru (5.7K reputation)
Group: Forum Members
Posts: 53, Visits: 147
Hi Dave, hi everyone,

I just took a look at our merged dataset and what confuses me is that it is not ordered according to subject number. Instead we have the subject number in one column repeatedly and the variables of all tests in the rows next to that, however their values in the rows below. I give you a hypothetical example below:

subject Test1   Test2   Test3
1             5
2             2
3             6
4             4
1                       7
2                       2
3                       6 
4                       4         
1                                 1                                                
2                                 2
3                                 6
4                                 7

What I would like to have is the usual organisation in: one row per subjects. I know, we could restructure the data set with SPSS, however at this point, that does not seem like an easy option. Can I change that in Inquisit, so it looks like this:

subject   Test1   Test2   Test3
1              5          7         1
2              2          2         2
3              6          6         6
4              4          4         7

Thanks in Advance
M.


GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search