The Collections framework introduced iterators for traversing a list or other collection, which optimizes the process of iterating through the elements in a collection. However, the iterators implemented in the java.util Collections classes are fail-fast, which means that if one thread changes a collection while another thread is traversing it through an Iterator, the next Iterator.hasNext() or Iterator.next() call will throw ConcurrentModificationException.

Look at this,
[Only good for single thread]

   List list = new ArrayList();
   Iterator iterator = list.iterator();
   while(iterator.hasNext()) {
     Item item = iterator.next();
     if(isOld(item)) {
   	 iterator.remove();
     }
   }

if this is single-threaded code and need to modify a collection while iterating through it. That’s fine.
But you run in multiple-threaded, how to avoid ConcurrentModificationException?
First you may want to use synchronizedList like this,
[Scalability problems]

  List originalList = new ArrayList();
  List list = Collections.synchronizedList(originalList);
  synchronized(list) {
    Iterator iterator = list.iterator(); // Must be in synchronized block
    while (iterator.hasNext()) {
       Item item = iterator.next();
       if(isOld(item)) {
          iterator.remove();
       }
    }
  }

The bigger problem with the synchronized Collections wrappers(synchronizedList, synchronizedMap), and the earlier Hashtable and Vector classes, is that they synchronize on a single lock. This means that only one thread may access the collection at once, and if one thread is in the process of reading from a List/Map, all other threads that want to either read from it or write to it must wait.
So what’s perfect way to do?
[Concurrent Collections]

List originalList = new ArrayList();
List list = new CopyOnWriteArrayList(originalList);
Iterator iterator = list.iterator();
while (iterator.hasNext()) {
     Item item = iterator.next();
     if(isOld(item)) {
       iterator.remove();
     }
}

The same story for Map, you should use ConcurrentHashMap instead synchronizedMap.

Reference:
1. Concurrent collections classes
2. Building a better HashMap
3. Java Concurrency Bugs #4: ConcurrentModificationException
4. Taming Tiger: Concurrent collections
5. Introduction to nonblocking algorithms
6. Concurrency: Blocking Queues and You

Update: ConcurrentHashMap example
1. Correct use of ConcurrentHashMap
2. Usage and performance of ConcurrentHashMap
3. Performance Optimization in ConcurrentHashMap

Update: ConcurrentSkipListSet example
1. Java Ordered Collections: Trees and Skip Lists
2. What is new in Java 6.0 Collections API?