The Establishment Clause as spelled out by Justice Hugo Black in effect is a separation of religion from state.
The court has expanded the language at times a bit beyond what I think it says; other times, not so much. Unfortunately, the very sort of corruption we'd like not to see seems to have had the door opened with Bush's 'faith-based programs office'. Fundamentalist sects were important to Republican politics; in return, taxpayer dollars were made available to pay to these sects, as determined by the White House office and political appointees, for 'providing services' rather than the government doing so.
In principle, that could add up to a fortune of money, quite corruptly administered, greatly strengthening the finances of the churches selected.
Probably the most relevant founding father on the issue was James Madison. While the language was limited IMO, he had a strong preference for the 'separation'.
However, it's interesting to note that some of that seemed to be based on the policy increasing the strength of religion, rather than simply protecting people from coercion.
He commented that 'it can't be denied that religion in Virginia is now more fervently followed after the end of state sponsorship'; elsewhere, he wrote:
Prior to the Revolution, the Episcopal Church was established by law in this State. On the Declaration of Independence it was left, with all other sects, to a self-support. And no doubt exists that there is much more of religion among us now than there ever was before the change, and particularly in the sect which enjoyed the legal patronage. This proves rather more than that the law is not necessary to the support of religion
So, the argument is, 'separation of church and state is good, proven by the fact that the religion in particular that was official, is even stronger when not state sponsored.'
When the first amendment was adopted, some states even still had 'official religions', though times have changed culturally and legally on that.
This is yet another of those issues that might seem to have a clear answer - 'keep those hicks from mixing government and church' - but has two sides.
What if a non-religious setting were offered as well as church settings?
If I were voting on the issue as a citizen, I'd lean towards 'better to keep them separate and not do this, despite potential benefits of helping to reduce crime at less taxpayer expense and needless suffering and harm of people in jail.' I'm all for those goals, but despite what many would see as the 'common sense' of using already existing institutions, I'd rather another approach were used - like government counselors or classes. But as to whether this violates the establishment clause, it's not so clear to me that it does, though that case could be argued from precedents like the Black quote you quoted.