I think it does matter greatly. Accepting that the other side does indeed have an ethical code is the only way reasonable conversation becomes possible. It is only by accepting that atheists have one at all that you can then engage questions about the relative
natures of religious vs. non-religious ethical codes, which you get into below.
As someone who actively participates in an organized religion (Judaism) despite being an atheist, I partially agree with you. Ritual and tradition create a sense of unity that can be an important support system for a community. It's especially valuable that such communities don't necessarily have much in common with each other other than their religious background. If not for that community, for example, I would not have much connection with people outside my generation that are not family.
I will disagree with you on two points, though. The first is that such a sense of community and ritual has to be inherently religious in nature. Certainly that is the case now (although you could make a case for things like rotary clubs and fraternities), but that is in part because public atheism is still in its infancy. Over the next few generations, as larger numbers of people begin to crave the sense of community but struggle to believe in God, you'll see communities form that are not connected to worship, and that have their own sense of ritual and tradition. The left wings of
Judaism and
Christianity are already starting to take baby steps in this direction.
The second place I'll disagree with you is your dismissal of atheists as unreliable and as troublemakers. The black sheep are usually the ones who create the most change in society. If you look at our history of philosophical, scientific, technological, and even theological progress, those responsible for change are generally outcasts in their society. It's not, as some of those outcasts would like you to believe, because they're especially smarter than everyone else, but because they're willing to let go of some of the rules placed before them by those before them. Subversiveness is thus in and of itself a useful societal trait.