Sheesh, I was pretty much thinking about this exact thing when I went to bed last night.
I'd like to add my observations to this (and probably muddy the water even more).
First, yes, I understand that from any one Galaxy (used as a reference) that all the others would be moving away from each other - use either the Balloon or the Rubber band with knots, and this effectively demonstrates this. Actually the Balloon thing concerns me somewhat, but I won't address that at this time.
So as the OP has asked, why can't a "center" be determined? Yes, everything is moving away from each other, but how about the Red shift?
Won't Galaxies that are pretty much moving in the same direction have a lower Red shift, and those moving at a more angular direction (to ours) have a greater Red shift? It seems by taking these values into account Vectors could be derived that would produce a "center", even if nothing is there (as I would expect).
To my knowledge a Blue shift Galaxy hasn't been observed. Logically, I wouldn't expect one, though I'm no expert on this.
My personal thoughts are that we haven't yet detected a Galaxy that is 100% directly moving away from us (or nearly 100%) - that is to say, on the exact opposite side of the Balloon. I think the Universe is actually bigger than anyone thinks . . . .