Hi All,

consider the attached script. It's objective is very simple: Display the four letters "A", "B", "C", and "D" at once in consecutive order. I know you could do that much easier but this is for testing purposes only. The counters were originally set up as follows:

<counter COUNTER_text1>
/ items = (1, 2, 3, 4)
/ select = 1
</counter>

<counter COUNTER_text2>
/ items = (1, 2, 3, 4)
/ select = 2   
</counter>

<counter COUNTER_text3>
/ items = (1, 2, 3, 4)
/ select = 3
</counter>

<counter COUNTER_text4>
/ items = (1, 2, 3, 4)
/ select = 4
</counter>

What I'd expect to happen is that the first counter selects element 1 (which would be "A" in the letters list), the second counter selects element 2 (letter "B") and so forth. The actual result is quite different, i.e. all letters are "A" in all trials, hence, the counters obviously ignore the "/ select" integer.

Then I tried to use "/ select = sequence(2,2,2,2)" for the second counter so that it reads

<counter COUNTER_text2>

/ items = (1, 2, 3, 4)

/ select = sequence(2,2,2,2)

</counter>

To my understanding of counters, this should have the counter select its second element four times in a row. Again, something very different happens. The counter selects its first item on the first trial (corresponding to letter "A"), then the second item (letter "B") on the second trial and so forth, so that the first letter iterates sequentially from "A" to "D" across trials. It is as if I hadn't specified the "(2,2,2,2)".

Finally, I used "/ select = constant(3)" for the third counter which in my conception should yield the same behavior as "/ select = 3". Well, it does, but just as ill-mannered. Inquisit disregards the integer setting but always selects the first element (letter "A" again).

I can't believe that this is a faulty implementation. So I must be doing something wrong. Any hints?

Best wishes,
Malte

Just a quick reply at this point: AFAIK, the '/ select' attribute behaves differently for <counter> as opposed to stimulus elements (<text> et al.), because -- my best guess -- <counter> elements are primarily intended for item selection. For example, having <counter a> select its items based on a value returned from <counter b> won't really work either (i.e. doesn't work at all or leads to unintended results). In my view, <counter>s currently are the end of Inquisit's selection hierarchy. Whether there are any bugs involved in what you've been observing or whether what you're seeing is 'by design' is beyond me. However, I really don't 'get' what the code you posted is supposed to achieve -- I can't see why one would ever try to implement something like this (no offense intended).

~Dave

Still not sure whether we're seeing bugs at work here, but I'll add a few further remarks. Basically, you're creating ambiguities in the <counter> definitions you posted. For example, '/ select = sequence(2,2,2,2)' is nothing but a shorthand notation.

<counter mycounter>
/ items = (2,2,2,2)
/ select = sequence
</counter>

is equivalent to

<counter mycounter>
/ select = sequence(2,2,2,2)
</counter>

Now, if you instead define

<counter mycounter>
/ items = (1,2,3,4)
/ select = sequence(2,2,2,2)
</counter>

you've essentially created an ambiguity. Should Inquisit select (1,2,3,4) in sequence or rather (2,2,2,2)? As you've observed, the expressions parser simply opts to ignore (2,2,2,2) and go for sequential selection of (1,2,3,4) as in

<counter mycounter>

/ items = (1,2,3,4)

/ select = sequence

</counter>

I'd prefer the parser to throw an error in case of such ambiguities. However, I'm not sure about the '/ select = integer' issue you observed. I've built similar counter-logics in the past a la

<counter mycounter>

/ items = (1,2,3,4)

/ select = values.number

</counter>

where values.number is an integer value dynamically determined by the script's logic. These have been working fine (check out this script for a working example: http://www.millisecond.com/download/samples/v3/KarpickeRoediger2008/default.aspx).

So, this one really appears to be a bug.

~Dave

Sorry, Dave, I'm completely not with you on this one. To my understanding, which derives only from the help examples on counters, the following should hold:

/ items = (11, 12, 13, 14)

are the contents of the counter, i.e. the values one of which will be returned when the counter is accessed. Let's call them "elements" for now.

Very different from that

/ select = sequence(1, 2, 3, 4)

gives indices to the elements, meaning that if "1" is selected from the sequence, the element "11" ist returned. I've always conceived the numbers in a sequence as an array of indices to another array of elements, with the latter being the elements of the counter. To me, the second example from the help is quite elucidating in this regard.

<text presidents>
/ items = ("George Washington",
"John Adams", "Thomas Jefferson")
/ select = sequence(3, 2,
1)
</text>

The sequence defines that the president names will be returned in inverse order. No ambiguity emerges whether Inquisit shall return the numbers 3, 2, 1 or the president names because 3, 2, and 1 are always just indices to the elements of the items list. Why should this suddenly change when the items list consists of numbers which incidently also appear in the sequence? If it did change, I'll file that as a bug because elements and indices to elements are two very different things.

The same holds for "/ select = 4". It should be clear that you don't tell Inquisit to select the number 4 as the next return value but the 4th element from the items list.

Oh, and for the real-world value of my code. Suppose you wanted to implement specific presentation sequences but on different classes of items. You could write

<counter A>
/ items = ("snappy", "hairy", "cold")
/ select = (1, 1, 1, 2, 2, 2, 3, 3, 3, 1, 2, 3, 2, 1)
</counter>

<counter B>

/ items = ("velvet", "silk", "hot")

/ select = (1, 1, 1, 2, 2, 2, 3, 3, 3, 1, 2, 3, 2, 1)

</counter>

<counter C>
/ items = ("chicken", "horse", "dog")


/ select = (1, 1, 1, 2, 2, 2, 3, 3, 3, 1, 2, 3, 2, 1)


</counter>

It saves you a lot of time when you don't have to convert each of the item lists into the desired sequence but just write down the element index sequence once and copy it to all counters. And, yes, I know there is another very obvious and similarly easy way to do such a thing but at least this should work, shouldn't it?

Bye, Malte

Granted, the help example for the "select" property
involves a text stimulus but it is referred to from the "counter"
help page, so I'd expect select to operate identically in both
contexts.

You need not be sorry about disagreeing with me. That's perfectly fine. Actually, I don't even think we have a disagreement here... Also note that I'm not advocating the current <counter> behavior as being "ideal" -- I'm just stating my observations (which for the most part are accurate). As I already said, whether the described aspects (different behavior of  /select for <text> vs <counter> elements, etc.) are due to bugs, implementation inconsistencies or intended is beyond me. I guess I've just learned to live with the way it is (which is not necessarily a good thing)...

~Dave

In a nutshell: as of today, one may completely forget

a) the "/ select = integer"

b) the sequence list in "/ select = sequence"

c) the number in "/ select = constant"

when dealing with counters because Inquisit is indecisive about the interpretation of the number(s) provided in any of these cases (are they indices, are they return values, or are they just to be ignored?).

Does it sound sensible when I say that you made something very unclear by perfectly clarifying it?

Malte.

Hi Malte,

I agree the behavior of counters are confusing in this regard. The original syntax assumed they would always be used as indexes into stimulus items sets rather than position their own pool of items to select from. Thus, the item pool and indexes are somewhat conflated.

As for the examples you posted, I agree that they are broken. However, the following syntax should accomplish exactly the same thing:

<counter COUNTER_text1>
/ select = 1
</counter>

<counter COUNTER_text2>
/ select = 2   
</counter>

<counter COUNTER_text3>
/ select = 3
</counter>

<counter COUNTER_text4>
/ select = 4
</counter>

This seems acceptable to me as a workaround for now, because I'm not sure how common or necessary it is to specify a big item pool if you plan to select the same item each time. Still, I agree this is confusing and the indexes and values clearly need to be separated out more clearly in the counter object.

-Sean

Dear Sean,

thanks for responding! The example was a stripped down version of what I intended to do, I like to keep things as simple as possible in the demo scripts I post here. To be honest, the actual script was conceived as nothing more than a test version, too. Yesterday, I stumbled upon the "/ not" property of counters and - contrary to the help - found that you can pass a list to it.

That seemed like digging up gold since it, if functional, would have solved a problem I was struggling with. I have four sets of stimuli, each of which contributes one element to every trial. The stimulus pairing scheme requires that in any given trial, each stimulus chooses an element with a unique index. Hence, when stimulus 1 drew item element 4, no other stimulus must draw element 4 from its associated item list in that particular trial.

So I thought, maybe this will work:

<counter COUNTER_text1>
/ items = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
/ select = noreplace
/ not = (COUNTER_text2,COUNTER_text3,COUNTER_text4)
</counter>

<counter COUNTER_text2>
/ items = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
/ select = noreplace
/ not = (COUNTER_text1,COUNTER_text3,COUNTER_text4)
</counter>

<counter COUNTER_text3>
/ items = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
/ select = noreplace
/ not = (COUNTER_text1,COUNTER_text2,COUNTER_text4)
</counter>

<counter COUNTER_text4>
/ items = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
/ select = noreplace
/ not = (COUNTER_text1,COUNTER_text2,COUNTER_text3)
</counter>

Of course it doesn't, which is why I tried to strip down the counter logic until I arrived at some configuration where "/ not" would actually evaluate multiple counters simultaneously. After some serious downstripping I arrived at the above script which had me understand that my conception of how counters work was far from correct.

Best wishes,
  Malte.

Malte,

multiple /not attributes will get you what you're looking for. If you download the SOPT sample from the Task Library and set all the contained counter's selectionmode to 'noreplace' you can even see it in action (the script's comments have more info on this).

~Dave

Dammit again -- I even have a demo script for this around (conceived for a recent Inquisit programming workshop). See attached.

~D.

Wait, you mean this

/ not = (counter1, counter2, counter3)

is different from this

/ not = (counter1)
/ not = (counter2)
/ not = (counter3)

D'oh! My bad... but, you know, this exactly resembles what I referred to in my rant about people's usage of Inquisit (see the wishlist posting). I'd rather opt for fixing and straightening out the current pitfalls (includes overhauling the help) instead of ever adding sophisticated functionality that entails inconsistencies. Please do not take me wrong, I am not at all complaining. Inquisit is a great piece of software that reduces the idea-to-lab time significantly for everybody using it but I just like well documented software that behaves consistently under all circumstances more than those feature packed, yet unpredictable monstrosities.

That said, thanks a lot for that great hint, Dave!

Agreed!

Again, I cannot tell you whether these should lead to different results by design (IMO they should not), all I can tell you is that (2) will work whereas (1) will not.

~Dave

That brings back an idea I had a while ago when we hired four new
student staff members who had to learn Inquisit. I then thought about
setting up a Wiki from the help (only for internal purposes) where
everybody could add examples, give tips, point out caveats and so forth.

Question
to Sean: Would turning the help into a Wiki or at least supplementing
the help with a Wiki be an option for you? I can envision Dave and
myself taking part in this. All those little notes we users have flying
around about what to do and what better not do do would be a valuable
addition to the help, I suppose. Some of the more obscure options
like"constant" (not explained in the help), the new keyboard scan codes
(not contained in the help), the unit specifiers (cannot find them in
the help), and much more could all be covered more optimally and
thoroughly with distributed resources. I as a spare time software
developer know how tedious writing the help is. That's the one thing
that always falls off the shelf but which is most important to users.

Bye, Malte.

Your point is taken, and I certainly agree. Personally, I think a "sophisticated" feature is justified only if it simplifies the programming required to achieve a particular kind of task or paradigm. In other words, if a feature allows the programmer to write less complicated scripts, then it's a good feature.

I also consider removing ambiguities from the language to be a priority, and I don't think it's incompatible with adding new functionality.

And yes, the two examples of multiple "not" should be equivalent. I'm guessing the case with multiple /not commands simply overwrite each other as this is the default behavior when multiple attributes are listed. I'll file a bug on that.

Thanks,
Sean

I need to clear that up before something goes wrong: It's exactly the other way round: Multiple /not attributes work *fine*

/ not = (counter1)
/ not = (counter2)
/ not = (counter3)

The problematic case is

/ not = (counter1, counter2, counter3)

~Dave

Ok, I think the only bug is in the documentation, because I've checked both cases and they be have identically as we all would expect. Are either of you seeing different behaviors?

-Sean

Okay, here we go: I had initially discovered the inconsistency between '/ not = (a, b, c, d)' and '/ not = a, / not = b, / not = c, / not  = d' when writing and testing the SOPT script. The Inquisit versions used at the time were 3.0.3.2 and various beta builds of 3.0.4.0. To recap the then-finding once again: Multiple '/ not ' attributes worked fine while the same logic implemented via a single '/ not = (a, b, c, d)' would lead to faulty results (double selections, etc.).

I've now revisited this scenario with version 3.0.5.0 and found that indeed -- as Sean noted -- both options behave equally in this version. However they behave equally *wrong*, i.e. there'll now be undesired, faulty selections in both cases. Bummer. A set of example scripts showcasing the problem for both not-options is attached to this message.

~Dave

Yep, same here. The reason for my disbelieving response to Dave's posting was that I had tested both variants and was pretty sure they yielded the same undesired results. However, I tend to believe the things Dave states, so I tried both options on one of our lab computers which indeed delivered. But: my notebook - Inquisit 3.0.5.0, lab computer - 3.0.3.2. So Dave's correct, version 3.0.3.2 respects multiple "/ not" statements but not a single "/ not" statement, whereas version 3.0.5.0 disregards both cases.

Malte.

Some more details about the code attached to my previous post to help in troubleshooting the issue. The code in the examples is essentially supposed to generate a random, n-th order Latin square (over the course of n trials, although this is not essential). Here's an example and outline of the underlying reasoning for the case of n=4 elements:

<counter a>
/ items = (1,2,3,4)
/ select = noreplace
[...]
</counter>

<counter b>
/ items = (1,2,3,4)
/ select = noreplace
/ not = (a)
[...]
</counter>

<counter c>
/ items = (1,2,3,4)
/ select = noreplace
/ not = (a, b)
[...]
</counter>

<counter d>
/ items = (1,2,3,4)
/ select = noreplace
/ not = (a, b, c)
[...]
</counter>

Over the course of 4 trials, this is what's supposed to happen:

(1) Trial 1

• A random value (e.g. 3) is drawn from <counter a>, leaving three values in its selection pool for the subsequent trial (1,2,4).


• A random value is drawn from <counter b>, whereas it shall not be the same value previously drawn from <counter a>. The value drawn might be 1, also leaving three values in the counter's selection pool for the subsequent trial (2,3,4).


• A random value is drawn from <counter c>, whereas it shall not be identical to the value drawn from either <counter a> and <counter b>. Thus the value determined might be 4, with again three values remaining in <counter c>'s selection pool (1,2,3).


• A value is drawn from <counter d>. Given the imposed selection constraints, this value must be 2, with values (1,3,4) remaining in <counter d>'s selection pool.

(2) Trial 2

• A value is selected from <counter a>. Since 3 has already been selected on the previous trial, this value might be 1, with (2,4) remaining in the selection pool for the third trial.


• A random value is drawn from <counter b>. 1 has been selected previously and the value shall not be identical to <counter a>'s current selection (1). So let's say the returned value from <counter b> is 3, leaving its selection pool at (2,4).


• <counter c> now has little options left: 4 has been selected on the previous trial, and due to the constraints 1 (<counter a>) and 3 (<counter b>) are also out of the picture. The returned value thus must be 2, with (1,3) remaining for subsequent trials.


• Again, the selection for <counter d> is fully determined at this point: 2 has been selected previously, and due to selection constraints 1,2, and 3 aren't available either. The value drawn can only be 4, leaving <counter d>'s selection pool at (1,3).

You get the idea... Summing up, over the course of 4 trials, here's the desired result (along the lines of the above example) in cross-tabulated form, with trial numbers as rows and counter selections as columns:

counter
a b c d

t1  3 1 4 2
t2  1 3 2 4
t3  4 2 1 3
t4  2 4 3 1

As we can see, we (should) have ended up with a nice, randomly generated Latin square. Now, if only this would (still) work. I've found the outlined counter-logic to function properly for smaller values of n (e.g. up to 4) even in 3.0.5.0 and the release 3.0.4.0 build, but according to my tests it fails miserably for larger values in both. As Malte confirmed (thanks!) the issue is not present in 3.0.3.2. However I am *very* sure that multiple '/ not' statements vs a single '/ not = (a,b,c,d,...)' did make a difference in 3.0.3.2 -- note that this might hold true for larger values of n only.

Thanks for enduring this tedious-to-read post!

~Dave

Thanks for the test cases, Dave, they definitely clarify things. I've run through a few of them, and I think the problem is more of a design issue than a bug per se.

The issue is that counters select values on the fly, which might lead to situations where it is impossible for a counter to satisfy both the NOREPLACE and NOT constraints. Consider the following example:

       a    b   c    d
t1   2    4    1    3
t2   3    1    2    4
t3   1    2    4    2
t4   4    3    3    1

Things go along nicely until it is time for d to select a value on t3. D has two items, 1 and 2, left in the pool. However, both of those have already been selected on t3 by a and b, so what should it do? Currently, the NOREPLACE rule is given precedent, so it selects from the remaining pool. However, even if NOT were given precedent and it selected 3, you still don't get a nice neat factorial because d already selected 3 on t1.

I'm fairly certain there is no way to always satisfy both NOT and NOREPLACE constraints with "on the fly" selection. Depending on the size of the matrix, there will be some probability that a counter will be painted into a corner. Perhaps we could preselect various matrices until we find one that works, but I'm not convinced that will work either since we have no way of knowing exactly a) how many trials there will be, and b) on which of the trials a given counter will be used, because trials and selections may be inserted or skipped based on data that isn't known at the start of an experiment. 

I think the solution must allow you to define selection matrices. This could take the form of a multi-dimensional counter such as the following:

<counter all>
/ items = ( (1,2,3,4), (1 3 2 4), (4 2 1 3), (2 4 3 1) )
/ select = noreplace
</counter>

where subitems can be referenced with an index:

<text one>
/ items = ("a" "b" "c" "d")
/ select = counter.all.selectedvalue.1
</text>

<text two>
/ items = ("a" "b" "c" "d")
/ select = counter.all.selectedvalue.2
</text>

Or it could be accomplished by a new /select option:

<counter group>
/ items = (1,2,3,4)
/ select = noreplace
[...]
</counter>

<counter subgroup>
/ items = (1,2,3,4)
/ select = factorial(group)
</counter>

<text one>
/ items = ("a" "b" "c" "d")
/ select = counter.subgroup.selectedvalue.1
</text>

<text two>
/ items = ("a" "b" "c" "d")
/ select = counter.subgroup.selectedvalue.2
</text>

I think the first option is more flexible, explicit, and intelligible, although things get a little cumbersome as the matrix grows (a 20x20 matrix would involve a lot of typing). Possibly there are some better approaches than either of these - ideas are certainly welcome.

As a historical note, support for factorial selection has been on the feature list since it was first requested way way back in version 1.

-Sean

Well, for a start, I'd suggest

<counter COUNTER_latin>
/ items = (1, 2, 3, 4)
/ items = (2, 3, 4, 1)
/ items = (3, 4, 1, 2)
/ items = (4, 1, 2, 3)
</counter>

which would be a tad less error-prone and, to me, appears more readable then the matrix based version. Plus, with the matrix based version, there'll definitely be people to request matrices of higher dimension than 2.

Further, I could imagine something like

<counter COUNTER_latin>

/ items = sequence(1, 2, 3, 4)

/ items = sequence(1, 2, 3)

/ items = sequence(1, 2, 3, 4, 5)

/ items = replace(1, 2)
/ itemselect = replace|noreplace|permute|...

</counter>

For each draw, the counter would select a new "/ items" list, commit the current draw on this list and return the value. If "permute" is selected as the item select mode, Inquisit would iterate through the "/ items" lists and select a new element from the first list on every draw, from the second list only every 4th draw, from the third list every 4*3th draw, from the fourth list every 4*3*5th draw.

Just a thought,
Malte

Of course you are absolutely right, Sean. And obviously I didn't think things through properly, thus causing confusion and dismay with my nonsensical post. Clearly, we're neither dealing with a bug nor a design issue -- the only thing that's wrong is my flawed logic (yes, my existence is undeniable proof of Unintelligent Design!). Sincere apologies to everyone for that.

That said, great suggestions presented by you and Malte to possibly make factorial selection easier to implement in a future release!

However, to make up for this utter f*ck-up on my part, and because Latin squares / factorials are a useful thing to have around, I've attached a script that should indeed produce a random, Latin square order of arbitrary size using the method described by Byers (1991).

Thanks,

~Dave

Nice!

What? Quite the contrary!

You brought this whole long thread into focus (at least for me), and my ideas for new counter syntax were stolen straight from your explanation how latin square selection should work. So, thank you!

P.S. Looking forward to seeing how you solved this.

-Sean

Ooh, the mean-spirited lawyer I keep locked up in my basement (just in case) will a have a field day with this comment!!! :-)

~Dave