There are a couple types of catholic guilt, I believe. The first is guilt that you're a catholic and that your religion has done so much evil throughout the ages, what with the inquisition and religious wars and killing dissidents and popes having orgies (for more information check out "a world lit only by fire" or any other book on medieval/renaissance politics, particularly in italy. There's one by a guy named Bonney; I think it's "European Dynastic States" and a number more). That's why, post Vatican II, Catholics are always half-heartedly apologizing for all sorts of stuff. See Catholics have a martyr complex wherein they think they're always getting screwed, and they have a guilt complex where they realize that the church is responsible for much wrongdoing. This is why they're really really sorry about the holocaust, but they can't admit that the pope didn't speak out against it.
The other sort of Catholic guilt is that you're a Catholic, and you're supposed to do all this stuff (salvation through faith and through works) and you can never do all of it, and the relgion sort of shames you because you haven't done what you're supposed to do.