The key thing to remember here is that nested wildcards don't capture.
What this means is that the "normal" behavior you expect from top-level wildcards (i.e. wildcards stand for one specific something) doesn't apply to nested wildcards. Instead, nested wildcards stand for any type.
For example, take this declaration:
List<?> l;
This means l is a List of one specific type. Easy.
But what about this?
Collection<List<?>> c;
This is not a Collection of Lists of one specific type. This is a Collection of Lists, each of which is one specific type.
For example, you were expecting something like this to happen:
Collection<List<?>> c = new ArrayList<List<Long>>(); // Not valid, but pretend it is
c.add(new ArrayList<Long>()); // Valid
c.add(new ArrayList<Integer>()); // Invalid, because c is a Collection of Lists of Long
But consider this:
List<?> l = new ArrayList<String>();
c.add(l); // Should this compile?
The type of l exactly matches the type parameter for c, right? So shouldn't you be able to add l to c, even though l isn't a List<Long>?
Also consider this:
c.iterator().next(); // Assume there is an element to return
What type should this return? iterator() returns an Iterator<E>, and next() returns E, which means... c.iterator().next() returns a List<?>. Which is not the List<Long> you were expecting. Why is that?
Because nested wildcards don't capture. And that is the key distinction here. The wildcard in List<?> doesn't capture a single type "overall". It captures a single type for each of the elements in the Collection.
Thus, this is perfectly valid code:
Collection<List<?>> odd = new ArrayList<List<?>>();
odd.add(new ArrayList<String>());
odd.add(new ArrayList<Long>());
List<?> l = odd.iterator().next();
// returns the ArrayList<String>, but because odd is parameterized with
// List<?> we can technically end up with a list of anything
Keeping this in mind, let's look at your examples.
Collection<Pair<String,Long>> c1 = new ArrayList<Pair<String,Long>>();
Collection<Pair<String,Long>> c2 = c1;
That's intuitively OK. The types exactly match, so c1 is assignable to c2.
Collection<Pair<String,Long>> c1 = new ArrayList<Pair<String,Long>>();
Collection<Pair<String,?>> c3 = c1;
Now, let's look back. A Collection<Pair<String,?>> isn't a Collection of Pairs of Strings and a single unknown type. It's a Collection of Pairs, each of which is a pair of a String and some unknown type, which may or may not be the same type as another pair in the collection. So this is valid:
// Assume an appropriate object was assigned to c3
Pair<String, ?> p1 = new Pair<String, String>("Hello", "World");
Pair<String, ?> p2 = new Pair<String, List<String>>("Lorem", new ArrayList<>());
Pair<String, ?> p3 = new Pair<String, Map<String, Integer>>("Ispum", new HashMap<>());
c3.add(p1);
c3.add(p2);
c3.add(p3);
And because this is valid for c3, but shouldn't be valid for c1, assigning c1 to c3 isn't allowed, because it would allow you to put stuff into an ArrayList<Pair<String, Long>> that isn't a Pair<String, Long>.
Collection<Pair<String,Long>> c1 = new ArrayList<Pair<String,Long>>();
Collection<? extends Pair<String,?>> c4 = c1;
Now, this is a bit more tricky. The top-level captures one specific type that extends Pair<String, ?>. Because wildcards are supertypes of specific types (e.g. List<?> is a supertype of List<Integer>), Pair<String, Long> can be captured by ? extends Pair<String, ?>, because the former extends the latter. Thus, because Pair<String, Long> is assignment-compatible with ? extends Pair<String, ?>, the assignment is valid.
As you can tell from the variety of answers here, there are multiple ways of explaining the behavior of nested wildcards. I was going for a bit more of an intuitive explanation, which I hope I achieved.