How do you like this for a compiler optimization...

[SunnyD]

update: apparently delphi is supposed to do it this way, which imho is one of the stupidest things ever thought of

Borland has a nasty compiler optimization in Delphi 2006 which caused a nice little bug in my app today.

Apparently the conditional of a for loop gets cached which can be a bad thing, such as in the following case using a TObjectList:

for i := 0 to SomeObjectList.Count - 1 do
begin

// Some conditional causes a given object i to be removed which
// should update SomeObjectList.Count, but the original value of
// SomeObjectList.Count seems to be cached (probably in a register)
// meaning that further iterations through the loop test against the
// original value of SomeObjectList.Count instead of being updated
// every iteration of the loop. Bam, instant array index out of bound.

end;

Funny thing, in the debugger, inspecting the variables "appear" to work correctly. It's funny (or not) to see the loop actually test, then execute when i = SomeObjectList.Count which it should NEVER do.

Ironically, the only way around this "feature" was for me to use a while loop and manually emulate a for loop with it.

(PS - TallBill, exactly like what you did originally, except in my case the compiler screwed up!)

 

[EagleKeeper]

The .Count function is obtained at entry into the block of code.
It is not called everytime the loop is processed not is it intended to be.
MS compilers do the same thing.

for loop and do/while control variables from a function call are extracted/determined by the compiler before the block of code is entered.

When using the debugger, the debugger does extra data retrieval and initialization which is why such items are not detected.

 

[Cogman]

Umm, This is actually pretty common practice. Try something similar with a cpp compiler. Im pretty sure that it happens with GCC as well.

*edit* I stand corrected, looks like it does update every loop. Who knew My test script for the interested.

#include <iostream>

using namespace std;

int main()
{
for (int i = 21; i >= i / 7; --i)
cout << "I could hit that: " << i << endl;
return 0;
}

 

[SunnyD]

Originally posted by: Common Courtesy
The .Count function is obtained at entry into the block of code.
It is not called everytime the loop is processed not is it intended to be.
MS compilers do the same thing.

for loop and do/while control variables from a function call are extracted/determined by the compiler before the block of code is entered.

When using the debugger, the debugger does extra data retrieval and initialization which is why such items are not detected.

If that's the case why does it work correctly with a while loop?

I understand it's a nice optimization - but only for a variable that doesn't change! It's fairly trivial to see if within the body of a loop whether a control variable will change.

And... to debunk your theory:

----
vector<int> v;
int i = 0;
int LoopCount = 0;

for(i = 0; i < 10; i++)
{
v.push_back(rand());

printf("v[%d] = %d\n", i, v[.i.]);
LoopCount++;
}

printf("Vector initialized with %d entries.\n", LoopCount);

LoopCount = 0;

for(i = 0; i < v.size(); i++)
{
if(i == 5)
v.erase(v.begin() + 5);

printf("v[%d] = %d\n", i, v[.i.]);
LoopCount++;
}

printf("The second loop executed %d times.\n", LoopCount);

---- (Ignore the periods around the array operator, it's to keep FuseTalk from thinking it's italics)
The output from the above code:
v[0] = 41
v[1] = 18467
v[2] = 6334
v[3] = 26500
v[4] = 19169
v[5] = 15724
v[6] = 11478
v[7] = 29358
v[8] = 26962
v[9] = 24464
Vector initialized with 10 entries.
v[0] = 41
v[1] = 18467
v[2] = 6334
v[3] = 26500
v[4] = 19169
v[5] = 11478
v[6] = 29358
v[7] = 26962
v[8] = 24464
The second loop executed 9 times.
----

The second loop only executes 9 times in Visual Studio 2008. Hence, VC2008 is actually checking v.size() every iteration. It makes absolutely no sense for a loop conditional control that is a function to be optimized in the way you speak of. This is a pure optimization that in Delphi's case was done blindly. A compiler can tell much more easily when a variable isn't going to change within a given scope, but not nearly as easily with a function. This is not desirable behavior by any means.

 

[esun]

You're comparing what is essentially a foreach to a traditional for loop. Looking at it from that perspective, it makes sense that it doesn't re-evaluate the set you're iterating through.

I mean, if I were writing perl, for example:

$j = 5;

for $i (0..$j) {
print $i;
$j = 10;
}

I wouldn't expect the loop to print 0-10, but rather 0-5. This behavior is quite typical of foreach loops.

 

[SunnyD]

Originally posted by: esun
You're comparing what is essentially a foreach to a traditional for loop. Looking at it from that perspective, it makes sense that it doesn't re-evaluate the set you're iterating through.

I mean, if I were writing perl, for example:

$j = 5;

for $i (0..$j) {
print $i;
$j = 10;
}

I wouldn't expect the loop to print 0-10, but rather 0-5. This behavior is quite typical of foreach loops.

I would expect it to print 1 through 10, as that is literally what that code fragment should be doing. The conditional must be tested every iteration, and the conditional control must be updated every iteration.

Yes and no. The end result appears the same as a foreach loop, but this is a for loop with a control variable and a conditional (otherwise in Delphi I would have used the for each construct). The conditional MUST be checked each iteration. Optimizing the conditional especially when the conditional involves function call is very bad. As I said... this is NOT under ANY circumstances desirable behavior (obviously, as it leads to unexpected results such as my example).

Finally, as my example illustrates - if it's so typical why does the benchmark compiler (Visual C++ 2008) test the conditional properly every time?

I guess the long and the short of it, which people seem to be ignoring would be if this were the desired behavior of a loop, how on earth would anyone ever do things such as my example (removing arbitrary items iteratively from a container)?

 

[Crusty]

In C# you can not change the container while iterating through it using a foreach loop.

 

[SunnyD]

Originally posted by: Crusty
In C# you can not change the container while iterating through it using a foreach loop.

The key here is that this is a for loop, not a foreach loop. I could definitely see foreach having this restriction, because foreach first extracts a reference to a particular member of a container meaning the container is intimately tied to the loop.

A for loop is testing a counter/index against a test value, there's no intimate tying of a container aside from your explicit implementation (in this case).

For all intents... given the following code:

int ConditionalValue = 10;

for(int i = 0; i < ConditionalValue; i++)
{
ConditionalValue = 1;
}

What people like Common Courtesy are saying is that the loop itself should iterate 10 times, not once. What I'm saying is the loop should iterate once, not 10 times. What I'm describing is that Delphi is actually iterating the loop 10 times... (or whatever arbitrary amount in my actual code).

 

[SunnyD]

Alright alright - I give up here. After reading a bit about Delphi's for loop construct I get the following:

For purposes of controlling execution of the loop, the expressions initialValue and finalValue are evaluated only once, before the loop begins. Hence the for...to statement is almost, but not quite, equivalent to this while construction:

So Delphi is working as intended.

 

[EagleKeeper]

Originally posted by: SunnyD
Alright alright - I give up here. After reading a bit about Delphi's for loop construct I get the following:

For purposes of controlling execution of the loop, the expressions initialValue and finalValue are evaluated only once, before the loop begins. Hence the for...to statement is almost, but not quite, equivalent to this while construction:

So Delphi is working as intended.

The programmer apparently did not use the function/construct as intended.

Everyone will do that at least once. Most do not shout it to the heavens though.

 

[EagleKeeper]

Originally posted by: SunnyD

Originally posted by: Common Courtesy
The .Count function is obtained at entry into the block of code.
It is not called everytime the loop is processed not is it intended to be.
MS compilers do the same thing.

for loop and do/while control variables from a function call are extracted/determined by the compiler before the block of code is entered.

When using the debugger, the debugger does extra data retrieval and initialization which is why such items are not detected.

If that's the case why does it work correctly with a while loop?

I understand it's a nice optimization - but only for a variable that doesn't change! It's fairly trivial to see if within the body of a loop whether a control variable will change.

And... to debunk your theory:

----
vector<int> v;
int i = 0;
int LoopCount = 0;

for(i = 0; i < 10; i++)
{
v.push_back(rand());

printf("v[%d] = %d\n", i, v[.i.]);
LoopCount++;
}

printf("Vector initialized with %d entries.\n", LoopCount);

LoopCount = 0;

for(i = 0; i < v.size(); i++)
{
if(i == 5)
v.erase(v.begin() + 5);

printf("v[%d] = %d\n", i, v[.i.]);
LoopCount++;
}

printf("The second loop executed %d times.\n", LoopCount);

---- (Ignore the periods around the array operator, it's to keep FuseTalk from thinking it's italics)
The output from the above code:
v[0] = 41
v[1] = 18467
v[2] = 6334
v[3] = 26500
v[4] = 19169
v[5] = 15724
v[6] = 11478
v[7] = 29358
v[8] = 26962
v[9] = 24464
Vector initialized with 10 entries.
v[0] = 41
v[1] = 18467
v[2] = 6334
v[3] = 26500
v[4] = 19169
v[5] = 11478
v[6] = 29358
v[7] = 26962
v[8] = 24464
The second loop executed 9 times.
----

The second loop only executes 9 times in Visual Studio 2008. Hence, VC2008 is actually checking v.size() every iteration. It makes absolutely no sense for a loop conditional control that is a function to be optimized in the way you speak of. This is a pure optimization that in Delphi's case was done blindly. A compiler can tell much more easily when a variable isn't going to change within a given scope, but not nearly as easily with a function. This is not desirable behavior by any means.

A while loop is different than a do/while loop

 

[SunnyD]

Originally posted by: Common Courtesy

Originally posted by: SunnyD
Alright alright - I give up here. After reading a bit about Delphi's for loop construct I get the following:

For purposes of controlling execution of the loop, the expressions initialValue and finalValue are evaluated only once, before the loop begins. Hence the for...to statement is almost, but not quite, equivalent to this while construction:

So Delphi is working as intended.

The programmer apparently did not use the function/construct as intended.

Everyone will do that at least once. Most do not shout it to the heavens though.

Even so, it boggles my mind to think that someone would consider a construct with a test conditional that iterates to not evaluate the conditional every time. Again, other than aggressive optimization, this is probably one of the stupidest things I've seen as it makes the construct itself less usable and ... less intelligible.

I fell into the trap of "Language A should work like Language B".

 

[esun]

I just want to mention that even if the actual command is "for", it actually acts like a foreach, which is why I gave the comparison. It actually is acting like other languages (those that have a foreach command, or something that acts like one, e.g. perl and python), except that instead of calling its foreach a foreach, it calls it a for. Note that perl allows you to use for instead of foreach as syntactic sugar when writing foreach loops, which is why I wrote it that way.