Millisecond Forums » Millisecond Forums » Inquisit 6 » random no replace vs random replace for groupid

random no replace vs random replace for groupid

caithow — Thu, 09 Dec 2021 01:26:23 GMT

Hi there,

I'm wanting to counterbalance the order in which my tests within my survey are presented to participants. Dave previously informed me that to do this I should assign by group number (in the batch script), select random group number generation in the Inquisit Web's settings and indicate the number of groups as 2 (in the batch script and in the settings).

If I want to achieve the above, that is, counterbalancing, would it be best to select "random replace" or "random no replace" for determining the groupid? I'm not quite sure of the difference between the two options and the consequences that they present for what I'd like to achieve.

Any guidance would be great.

Thanks,
Caitlin

RE: random no replace vs random replace for groupid

caithow — Thu, 09 Dec 2021 01:26:23 GMT

+x[quote]

[b]Dave - 12/9/2021[/b]
+x[quote]

[b]caithow - 12/9/2021[/b]
+x[quote]

[b]Dave - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
+x[quote]

[b]Dave - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
Hi there,

I'm wanting to counterbalance the order in which my tests within my survey are presented to participants. Dave previously informed me that to do this I should assign by group number (in the batch script), select random group number generation in the Inquisit Web's settings and indicate the number of groups as 2 (in the batch script and in the settings).

If I want to achieve the above, that is, counterbalancing, would it be best to select "random replace" or "random no replace" for determining the groupid? I'm not quite sure of the difference between the two options and the consequences that they present for what I'd like to achieve.

Any guidance would be great.

Thanks,
Caitlin
[/quote]

With random replace, i.e. random selection with replacement, it'd basically a coin toss. Every visitor that hits the start page has a 50% chance of getting assigned to group 1 and a 50% chance of getting assigned to group 2. However, toss a coin 10 times, and there's no guarantee that it'll come up 5 times heads and 5 times tails.

You'll want to go with random no replace, i.e. random selection *without* replacement. For every set of two visitors hitting the experiment's start page, one will be randomly assigned to the 1st group, the other one to the 2nd group. (Going for sequential selection is another option that would give you much the same result.)

Even so, you're unlikely to get an *exactly* equal distribution for the two groups. The group number is generated when a person visits the start page, and some people will visit, but then decide, for whatever reason, not to actually participate.
[/quote]
Thanks for the great explanation Dave. Very appreciated.

I have another question (I'm currently in the midst of setting up the experiment in the settings page) - I have uploaded all the files and scripts for my experiment on the Upload Experiment page and I have selected demographicssurvey.iqx as my start script. However, when I run the entire survey through, it's only presenting the demographics survey and none of my other scripts (i.e., CFI, WCST etc). I have uploaded a batch script as a file, but it seems that Inquisit is not reading it. Do you have any ideas on why this might be happening and how to go about fixing this?

Thanks,
Caitlin
[/quote]
This is the error message I'm getting...

https://scripts.millisecond.com/caitlinhowlett/cfandpain/demographicssurvey.iqx:Data file write validation failure! Expected 131 bytes but found 130 bytes in file 'C:/Users/Caitlin/AppData/Local/Millisecond Software/Inquisit Player/pending/files/summary_demographicssurvey_summary_TEST_2021-12-08-04-36-10-675.iqdat'
[/quote]

> I have uploaded a batch script as a file, but it seems that Inquisit is not reading it.

The file you uploaded has a .txt extension. For it to be recognized as an Inquisit script file, it needs to have either an .iqx or .exp extension.

The batch script needs to be set as the experiment's start script.
[/quote]

Thanks very much Dave - it worked!

Now that I've managed to get the script to run, I've tested out the survey and taken a look over the data. Is there any way to produce a raw and summary data file that has every participant in the one spreadsheet? I'm thinking that it might be easier if to export all the data into a single excel file rather than having to do this for each participant one by one. Since this is my first time using Inquisit though, I'm pretty naïve to what would be the best way to go for data the purposes of making data analysis easier. I'm very open to suggestions and would really appreciate your input on this.

Thanks,
Caitlin
[/quote]

You don't need to do it one by one. Select multiple files or even an entire folder in your account's Inquisit Web -> Data section, select the format you want, and you'll get a file containing the data from all participants you selected.

[/quote]

Wonderful, thanks Dave that's super helpful.

RE: random no replace vs random replace for groupid

Dave — Thu, 09 Dec 2021 01:24:05 GMT

+x[quote]

[b]caithow - 12/9/2021[/b]
+x[quote]

[b]Dave - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
+x[quote]

[b]Dave - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
Hi there,

I'm wanting to counterbalance the order in which my tests within my survey are presented to participants. Dave previously informed me that to do this I should assign by group number (in the batch script), select random group number generation in the Inquisit Web's settings and indicate the number of groups as 2 (in the batch script and in the settings).

If I want to achieve the above, that is, counterbalancing, would it be best to select "random replace" or "random no replace" for determining the groupid? I'm not quite sure of the difference between the two options and the consequences that they present for what I'd like to achieve.

Any guidance would be great.

Thanks,
Caitlin
[/quote]

With random replace, i.e. random selection with replacement, it'd basically a coin toss. Every visitor that hits the start page has a 50% chance of getting assigned to group 1 and a 50% chance of getting assigned to group 2. However, toss a coin 10 times, and there's no guarantee that it'll come up 5 times heads and 5 times tails.

You'll want to go with random no replace, i.e. random selection *without* replacement. For every set of two visitors hitting the experiment's start page, one will be randomly assigned to the 1st group, the other one to the 2nd group. (Going for sequential selection is another option that would give you much the same result.)

Even so, you're unlikely to get an *exactly* equal distribution for the two groups. The group number is generated when a person visits the start page, and some people will visit, but then decide, for whatever reason, not to actually participate.
[/quote]
Thanks for the great explanation Dave. Very appreciated.

I have another question (I'm currently in the midst of setting up the experiment in the settings page) - I have uploaded all the files and scripts for my experiment on the Upload Experiment page and I have selected demographicssurvey.iqx as my start script. However, when I run the entire survey through, it's only presenting the demographics survey and none of my other scripts (i.e., CFI, WCST etc). I have uploaded a batch script as a file, but it seems that Inquisit is not reading it. Do you have any ideas on why this might be happening and how to go about fixing this?

Thanks,
Caitlin
[/quote]
This is the error message I'm getting...

https://scripts.millisecond.com/caitlinhowlett/cfandpain/demographicssurvey.iqx:Data file write validation failure! Expected 131 bytes but found 130 bytes in file 'C:/Users/Caitlin/AppData/Local/Millisecond Software/Inquisit Player/pending/files/summary_demographicssurvey_summary_TEST_2021-12-08-04-36-10-675.iqdat'
[/quote]

> I have uploaded a batch script as a file, but it seems that Inquisit is not reading it.

The file you uploaded has a .txt extension. For it to be recognized as an Inquisit script file, it needs to have either an .iqx or .exp extension.

The batch script needs to be set as the experiment's start script.
[/quote]

Thanks very much Dave - it worked!

Now that I've managed to get the script to run, I've tested out the survey and taken a look over the data. Is there any way to produce a raw and summary data file that has every participant in the one spreadsheet? I'm thinking that it might be easier if to export all the data into a single excel file rather than having to do this for each participant one by one. Since this is my first time using Inquisit though, I'm pretty naïve to what would be the best way to go for data the purposes of making data analysis easier. I'm very open to suggestions and would really appreciate your input on this.

Thanks,
Caitlin
[/quote]

You don't need to do it one by one. Select multiple files or even an entire folder in your account's Inquisit Web -> Data section, select the format you want, and you'll get a file containing the data from all participants you selected.

RE: random no replace vs random replace for groupid

caithow — Thu, 09 Dec 2021 00:17:32 GMT

+x[quote]

[b]Dave - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
+x[quote]

[b]Dave - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
Hi there,

I'm wanting to counterbalance the order in which my tests within my survey are presented to participants. Dave previously informed me that to do this I should assign by group number (in the batch script), select random group number generation in the Inquisit Web's settings and indicate the number of groups as 2 (in the batch script and in the settings).

If I want to achieve the above, that is, counterbalancing, would it be best to select "random replace" or "random no replace" for determining the groupid? I'm not quite sure of the difference between the two options and the consequences that they present for what I'd like to achieve.

Any guidance would be great.

Thanks,
Caitlin
[/quote]

With random replace, i.e. random selection with replacement, it'd basically a coin toss. Every visitor that hits the start page has a 50% chance of getting assigned to group 1 and a 50% chance of getting assigned to group 2. However, toss a coin 10 times, and there's no guarantee that it'll come up 5 times heads and 5 times tails.

You'll want to go with random no replace, i.e. random selection *without* replacement. For every set of two visitors hitting the experiment's start page, one will be randomly assigned to the 1st group, the other one to the 2nd group. (Going for sequential selection is another option that would give you much the same result.)

Even so, you're unlikely to get an *exactly* equal distribution for the two groups. The group number is generated when a person visits the start page, and some people will visit, but then decide, for whatever reason, not to actually participate.
[/quote]
Thanks for the great explanation Dave. Very appreciated.

I have another question (I'm currently in the midst of setting up the experiment in the settings page) - I have uploaded all the files and scripts for my experiment on the Upload Experiment page and I have selected demographicssurvey.iqx as my start script. However, when I run the entire survey through, it's only presenting the demographics survey and none of my other scripts (i.e., CFI, WCST etc). I have uploaded a batch script as a file, but it seems that Inquisit is not reading it. Do you have any ideas on why this might be happening and how to go about fixing this?

Thanks,
Caitlin
[/quote]
This is the error message I'm getting...

https://scripts.millisecond.com/caitlinhowlett/cfandpain/demographicssurvey.iqx:Data file write validation failure! Expected 131 bytes but found 130 bytes in file 'C:/Users/Caitlin/AppData/Local/Millisecond Software/Inquisit Player/pending/files/summary_demographicssurvey_summary_TEST_2021-12-08-04-36-10-675.iqdat'
[/quote]

> I have uploaded a batch script as a file, but it seems that Inquisit is not reading it.

The file you uploaded has a .txt extension. For it to be recognized as an Inquisit script file, it needs to have either an .iqx or .exp extension.

The batch script needs to be set as the experiment's start script.
[/quote]

Thanks very much Dave - it worked!

Now that I've managed to get the script to run, I've tested out the survey and taken a look over the data. Is there any way to produce a raw and summary data file that has every participant in the one spreadsheet? I'm thinking that it might be easier if to export all the data into a single excel file rather than having to do this for each participant one by one. Since this is my first time using Inquisit though, I'm pretty naïve to what would be the best way to go for data the purposes of making data analysis easier. I'm very open to suggestions and would really appreciate your input on this.

Thanks,
Caitlin

RE: random no replace vs random replace for groupid

Dave — Wed, 08 Dec 2021 12:32:09 GMT

+x[quote]

[b]caithow - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
+x[quote]

[b]Dave - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
Hi there,

I'm wanting to counterbalance the order in which my tests within my survey are presented to participants. Dave previously informed me that to do this I should assign by group number (in the batch script), select random group number generation in the Inquisit Web's settings and indicate the number of groups as 2 (in the batch script and in the settings).

If I want to achieve the above, that is, counterbalancing, would it be best to select "random replace" or "random no replace" for determining the groupid? I'm not quite sure of the difference between the two options and the consequences that they present for what I'd like to achieve.

Any guidance would be great.

Thanks,
Caitlin
[/quote]

With random replace, i.e. random selection with replacement, it'd basically a coin toss. Every visitor that hits the start page has a 50% chance of getting assigned to group 1 and a 50% chance of getting assigned to group 2. However, toss a coin 10 times, and there's no guarantee that it'll come up 5 times heads and 5 times tails.

You'll want to go with random no replace, i.e. random selection *without* replacement. For every set of two visitors hitting the experiment's start page, one will be randomly assigned to the 1st group, the other one to the 2nd group. (Going for sequential selection is another option that would give you much the same result.)

Even so, you're unlikely to get an *exactly* equal distribution for the two groups. The group number is generated when a person visits the start page, and some people will visit, but then decide, for whatever reason, not to actually participate.
[/quote]
Thanks for the great explanation Dave. Very appreciated.

I have another question (I'm currently in the midst of setting up the experiment in the settings page) - I have uploaded all the files and scripts for my experiment on the Upload Experiment page and I have selected demographicssurvey.iqx as my start script. However, when I run the entire survey through, it's only presenting the demographics survey and none of my other scripts (i.e., CFI, WCST etc). I have uploaded a batch script as a file, but it seems that Inquisit is not reading it. Do you have any ideas on why this might be happening and how to go about fixing this?

Thanks,
Caitlin
[/quote]
This is the error message I'm getting...

https://scripts.millisecond.com/caitlinhowlett/cfandpain/demographicssurvey.iqx:Data file write validation failure! Expected 131 bytes but found 130 bytes in file 'C:/Users/Caitlin/AppData/Local/Millisecond Software/Inquisit Player/pending/files/summary_demographicssurvey_summary_TEST_2021-12-08-04-36-10-675.iqdat'
[/quote]

> I have uploaded a batch script as a file, but it seems that Inquisit is not reading it.

The file you uploaded has a .txt extension. For it to be recognized as an Inquisit script file, it needs to have either an .iqx or .exp extension.

The batch script needs to be set as the experiment's start script.

RE: random no replace vs random replace for groupid

caithow — Wed, 08 Dec 2021 04:47:17 GMT

+x[quote]

[b]caithow - 12/8/2021[/b]
+x[quote]

[b]Dave - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
Hi there,

I'm wanting to counterbalance the order in which my tests within my survey are presented to participants. Dave previously informed me that to do this I should assign by group number (in the batch script), select random group number generation in the Inquisit Web's settings and indicate the number of groups as 2 (in the batch script and in the settings).

If I want to achieve the above, that is, counterbalancing, would it be best to select "random replace" or "random no replace" for determining the groupid? I'm not quite sure of the difference between the two options and the consequences that they present for what I'd like to achieve.

Any guidance would be great.

Thanks,
Caitlin
[/quote]

With random replace, i.e. random selection with replacement, it'd basically a coin toss. Every visitor that hits the start page has a 50% chance of getting assigned to group 1 and a 50% chance of getting assigned to group 2. However, toss a coin 10 times, and there's no guarantee that it'll come up 5 times heads and 5 times tails.

You'll want to go with random no replace, i.e. random selection *without* replacement. For every set of two visitors hitting the experiment's start page, one will be randomly assigned to the 1st group, the other one to the 2nd group. (Going for sequential selection is another option that would give you much the same result.)

Even so, you're unlikely to get an *exactly* equal distribution for the two groups. The group number is generated when a person visits the start page, and some people will visit, but then decide, for whatever reason, not to actually participate.
[/quote]
Thanks for the great explanation Dave. Very appreciated.

I have another question (I'm currently in the midst of setting up the experiment in the settings page) - I have uploaded all the files and scripts for my experiment on the Upload Experiment page and I have selected demographicssurvey.iqx as my start script. However, when I run the entire survey through, it's only presenting the demographics survey and none of my other scripts (i.e., CFI, WCST etc). I have uploaded a batch script as a file, but it seems that Inquisit is not reading it. Do you have any ideas on why this might be happening and how to go about fixing this?

Thanks,
Caitlin
[/quote]

This is the error message I'm getting...

https://scripts.millisecond.com/caitlinhowlett/cfandpain/demographicssurvey.iqx:Data file write validation failure! Expected 131 bytes but found 130 bytes in file 'C:/Users/Caitlin/AppData/Local/Millisecond Software/Inquisit Player/pending/files/summary_demographicssurvey_summary_TEST_2021-12-08-04-36-10-675.iqdat'

RE: random no replace vs random replace for groupid

caithow — Wed, 08 Dec 2021 02:36:57 GMT

+x[quote]

[b]Dave - 12/8/2021[/b]
+x[quote]

[b]caithow - 12/8/2021[/b]
Hi there,

I'm wanting to counterbalance the order in which my tests within my survey are presented to participants. Dave previously informed me that to do this I should assign by group number (in the batch script), select random group number generation in the Inquisit Web's settings and indicate the number of groups as 2 (in the batch script and in the settings).

If I want to achieve the above, that is, counterbalancing, would it be best to select "random replace" or "random no replace" for determining the groupid? I'm not quite sure of the difference between the two options and the consequences that they present for what I'd like to achieve.

Any guidance would be great.

Thanks,
Caitlin
[/quote]

With random replace, i.e. random selection with replacement, it'd basically a coin toss. Every visitor that hits the start page has a 50% chance of getting assigned to group 1 and a 50% chance of getting assigned to group 2. However, toss a coin 10 times, and there's no guarantee that it'll come up 5 times heads and 5 times tails.

You'll want to go with random no replace, i.e. random selection *without* replacement. For every set of two visitors hitting the experiment's start page, one will be randomly assigned to the 1st group, the other one to the 2nd group. (Going for sequential selection is another option that would give you much the same result.)

Even so, you're unlikely to get an *exactly* equal distribution for the two groups. The group number is generated when a person visits the start page, and some people will visit, but then decide, for whatever reason, not to actually participate.
[/quote]

Thanks for the great explanation Dave. Very appreciated.

I have another question (I'm currently in the midst of setting up the experiment in the settings page) - I have uploaded all the files and scripts for my experiment on the Upload Experiment page and I have selected demographicssurvey.iqx as my start script. However, when I run the entire survey through, it's only presenting the demographics survey and none of my other scripts (i.e., CFI, WCST etc). I have uploaded a batch script as a file, but it seems that Inquisit is not reading it. Do you have any ideas on why this might be happening and how to go about fixing this?

Thanks,
Caitlin

RE: random no replace vs random replace for groupid

Dave — Wed, 08 Dec 2021 02:19:59 GMT

+x[quote]

[b]caithow - 12/8/2021[/b]
Hi there,

I'm wanting to counterbalance the order in which my tests within my survey are presented to participants. Dave previously informed me that to do this I should assign by group number (in the batch script), select random group number generation in the Inquisit Web's settings and indicate the number of groups as 2 (in the batch script and in the settings).

If I want to achieve the above, that is, counterbalancing, would it be best to select "random replace" or "random no replace" for determining the groupid? I'm not quite sure of the difference between the two options and the consequences that they present for what I'd like to achieve.

Any guidance would be great.

Thanks,
Caitlin
[/quote]

With random replace, i.e. random selection with replacement, it'd basically a coin toss. Every visitor that hits the start page has a 50% chance of getting assigned to group 1 and a 50% chance of getting assigned to group 2. However, toss a coin 10 times, and there's no guarantee that it'll come up 5 times heads and 5 times tails.

You'll want to go with random no replace, i.e. random selection *without* replacement. For every set of two visitors hitting the experiment's start page, one will be randomly assigned to the 1st group, the other one to the 2nd group. (Going for sequential selection is another option that would give you much the same result.)

Even so, you're unlikely to get an *exactly* equal distribution for the two groups. The group number is generated when a person visits the start page, and some people will visit, but then decide, for whatever reason, not to actually participate.