How to get around the "collection has changed" BS in C#?

[Red Squirrel]

In C# whenever you loop through items, and make any changes to these items, or add a new one, or remove one, it will crap out. This is a really bad limitation of the language. There HAS to be a way around this. Anyone know how?

The only way I can think of is to break out of the loop, do the change, keep track of where you were, then start a whole new loop while skipping the items that were already done. This seems like a pretty counter productive way of doing things though. There has to be a better way.

 

[Crusty]

You can avoid that by not using iterators... but chances are you are using the wrong data structures if you are running into this problem often. Can you give some more details about what you are trying to accomplish?

 

[Red Squirrel]

Basically, say I have a list of Items on a game server. These can be weapons, etc...

So I have these in a Dictionary which is how they are kept track of in the world.

Now say I decide I want every item that is a specific sword to be deleted because it's overpowered, I will then loop through, and then delete them. This causes a crash.

I've tried foreach, for, does not really matter what I use, it will always crash.

 

[Crusty]

Are you using multiple threads?

 

[Leros]

If you're having multiple threads accessing the collection at once, you need to be careful about accessing it. If one thread is modifying the collection while another thread is reading the collection, you're bound to have problems.

There are several methods. Tell us more about what you're doing.

 

[RocksteadyDotNet]

You need to synchronize access to the data structure. I know in Java, you can use a synchronize block and synchronize to the collection object itself.

If that doesn't exist, you'll need to use a lock. Lock before you use the collection and unlock when you're done. That way, only a single thread can modify the collection at once.

If you're doing more reading that writing, you can use a read/write lock. This will allow multiple threads to read concurrently, but will only one thread to have access while the collection is being modified.

Also, look into the .NET concurrent data structure libraries.

It's not a threading problem.

He talking about this

For Each item as MyClass in myCollection
             myCollection.Remove(item) '<-------- That will fail because you're iterating throught the collection
Next

Hey OP. in this situtation I usually just add the ones you want removed to a second collection, then loop through that collection removing the items from the original one.

I know it's not pretty. I'm not sure if they're a better way to do it.

 

[MrChad]

Use the RemoveAll method.

 

[Red Squirrel]

It's not a threading problem.

He talking about this

For Each item as MyClass in myCollection
             myCollection.Remove(item) '<-------- That will fail because you're iterating throught the collection
Next

Hey OP. in this situtation I usually just add the ones you want removed to a second collection, then loop through that collection removing the items from the original one.

I know it's not pretty. I'm not sure if they're a better way to do it.

Yeah that's exactly it. If I remove or add (which may sometimes happen in another function call, not in the loop directly) then it craps out.

What I've reverted to doing is copying the collection to another collection, looping but when I add/remove/change it does it to the original. This just seems really dirty though.

The RemoveAll seems interesting, though in my case it may not always work. This is a large complex program, and sometimes the delete is actually triggered more deeply in the code, but it's in the loop.

At startup of my game server, all the objects are loaded, then there is a loop that calls the virtual function OnAfterLoad, certain objects may have an override for this which could cause it to be deleted, or maybe add a new object to the world, or maybe just change an existing object.

Though I do have other situations where I could use RemoveAll, so think I'll keep that one in mind.

 

[EvilManagedCare]

It looks like the RemoveAll method is for Lists, would it still work with a Dictionary (I presume you were using the one provided by the .NET collections)?

edit: I suppose it could if the Dictionary is converted to a list, then back again.

 

[Red Squirrel]

Most of the time I deal with Lists so it would work ok for me. There's some instances where it's dictionaries though. but think the .Values returns a list.

 

[brandonb]

I do it like Rock mentioned.

I create a System.Collections.Generic.List(of keytype of the dictionary), then as I loop through the dictionary and find ones I want to delete, I add an entry to the List.

Then I for each through the list and delete from the dictionary based on key.

Then clear the list.

 

[Crusty]

It's not a threading problem.

He talking about this

For Each item as MyClass in myCollection
             myCollection.Remove(item) '<-------- That will fail because you're iterating throught the collection
Next

Hey OP. in this situtation I usually just add the ones you want removed to a second collection, then loop through that collection removing the items from the original one.

I know it's not pretty. I'm not sure if they're a better way to do it.

It certainly could have been a threading problem, if one thread modifies a collection while you are iterating over it an another thread you'll get this exception too.

 

[EvilManagedCare]

I've only fairly recently started using .NET, and that does seem a bit of a limitation as the OP stated (i.e. cannot change a collection while iterating though it without causing problems). Does anyone know what causes this?

 

[KLin]

http://stackoverflow.com/questions/...n-enumerable-collection-while-iterating-throu

 

[Aikouka]

Hmm this is interesting given how much more Java work I've been doing recently. It seems C# and Java have a completely different definition on what an iterator is. In Java, you would actually use an iterator to solve your problem as it gets around concurrent modification exceptions, but it seems in C#, the iterator is simply what allows a for each statement to work.

I've only fairly recently started using .NET, and that does seem a bit of a limitation as the OP stated (i.e. cannot change a collection while iterating though it without causing problems). Does anyone know what causes this?

Say you have the following list: 1, 2, 3, 4, 5

In C#, your iterator is set to go from item 0 to the last item (based on the bounds). Let's say you're currently looking at '2', and you decide to remove it. The collection now looks like: 1,3,4,5. Given your for loop is still using the bounds of the collection, it won't get an index out of bounds exception, but the problem that will arise is what are you going to do about '3'? It essentially took 2's spot, but you just processed that spot. The next value that will be processed is '4'.

I mentioned Java above with its iterators, the difference there is an iterator is like having access to the link capability from a linked list. So even though you deleted '2', calling next() will still point to '3'.

 

[EvilManagedCare]

Thanks KLin & Aikouka

 

[Red Squirrel]

http://stackoverflow.com/questions/...n-enumerable-collection-while-iterating-throu

I don't think that would work either, since you're still modifying the collection midloop. This works perfectly fine in C++ though and that's usually what I do, though I also decrease the i after doing a delete so I get the next item.

I guess the best work around is what brandonb said. I've been doing something similar, sometimes that exactly (depends on the situation).

Seems dirty, but guess that's what I get for using C#, it's not meant for extremely high speed optimized applications, but faster deployment. The speed difference in this case is not affecting my program so I'll just stick with it.

Also in this case there are no threads involved.

 

[Aikouka]

I don't think that would work either, since you're still modifying the collection midloop. This works perfectly fine in C++ though and that's usually what I do, though I also decrease the i after doing a delete so I get the next item.

It should work fine since you're performing the iteration control. You can also go backwards through the collection and not worry about doing an "i--;" to correct your position.

 

[Kirby]

Hmm this is interesting given how much more Java work I've been doing recently. It seems C# and Java have a completely different definition on what an iterator is. In Java, you would actually use an iterator to solve your problem as it gets around concurrent modification exceptions, but it seems in C#, the iterator is simply what allows a for each statement to work.



Say you have the following list: 1, 2, 3, 4, 5

In C#, your iterator is set to go from item 0 to the last item (based on the bounds). Let's say you're currently looking at '2', and you decide to remove it. The collection now looks like: 1,3,4,5. Given your for loop is still using the bounds of the collection, it won't get an index out of bounds exception, but the problem that will arise is what are you going to do about '3'? It essentially took 2's spot, but you just processed that spot. The next value that will be processed is '4'.

I mentioned Java above with its iterators, the difference there is an iterator is like having access to the link capability from a linked list. So even though you deleted '2', calling next() will still point to '3'.

Just off the top of my head, could you do something like this?

int i = 0;
while(i < list.length)
{
  if(list.wannaRemove(i))
  {
    list.remove(i);
  }
  else
    i++;
}

I'm no expert on C#, but I'd think it'd recalculate length when it goes back to the top of the while loop.

 

[Aikouka]

I'm no expert on C#, but I'd think it'd recalculate length when it goes back to the top of the while loop.

Yeah, that should work fine as it will access the length property each time it processes that boolean statement.

 

[LokutusofBorg]

foreach (MyClass member in myList.ToList()) // <-- creates a new list to be iterated over
  if (member.NeedsToBeRemoved)
    myList.Remove(member);

 

[Kirby]

foreach (MyClass member in myList.ToList()) // <-- creates a new list to be iterated over
  if (member.NeedsToBeRemoved)
    myList.Remove(member);

I have no idea how memory allocation works in C#, but that seems like it would be a bad idea if the the list was really large. Although I wouldn't be surprised if there some magic optimizations behind it.

Speaking of gripes, I really wish C# had union and bitfield functionality like C. I use C# mainly for GUIs for embedded systems, and it'd be nice to grab an array from my serial port and use a simple memcpy to copy it to the correct bits in the structure, instead of using shifts and masks. I'm sure one in a million people would actually use it, but it'd be handy for what I do.

union
{
  byte full;
  struct
  {
    byte a : 1;
    byte b : 2:
    byte c : 3:
    byte d : 2:
  }fields;
}bacon;

 

[LokutusofBorg]

Yeah, it can be inefficient, but it gets the job done. Optimizing for developer effort is most often more valuable than optimizing for performance. Optimize for performance when you know you have a performance problem.

You can also use LINQ to accomplish what others have gone over above, but LINQ makes it pretty straight-forward (and elegant, in my mind):

List<MyClass> itemsToRemove = myList.Select(i => i.NeedsToBeRemoved).ToList();
myList.RemoveAll(itemsToRemove);

 

[BigDH01]

Yeah, it can be inefficient, but it gets the job done. Optimizing for developer effort is most often more valuable than optimizing for performance. Optimize for performance when you know you have a performance problem.

You can also use LINQ to accomplish what others have gone over above, but LINQ makes it pretty straight-forward (and elegant, in my mind):

List<MyClass> itemsToRemove = myList.Select(i => i.NeedsToBeRemoved).ToList();
myList.RemoveAll(itemsToRemove);

RemoveAll takes a predicate.

myList.RemoveAll(i => i.needstoberemoved);

Very easy. It comes with the added benefit of leaving performance optimization up to the people maintaining the collections libraries, who are almost certainly better than me at it anyway.

 

[LokutusofBorg]

There you go. Overloads FTW!