I work for a publisher of academic textbooks and I used to teach in a university and had to select books for students. I know both sides of this issue very well.
First thing to note: Don't necessarily blame the textbook publisher for the price you have to pay. Industry standard for most university book stores is a mark-up of 20% by the store. Textbook vendors generally sell their books at a "short discount" or "academic discount" so the book stores can take their cut.
This means that if I have a book with an MSRP of $100, I sell it to the book store at a discount, so that when they mark it back up, it will cost $100. The publisher eats the difference. The publisher also eats the returns. So, 50% of students decided not to buy the book, or the class had a large attrition rate? We take the books back for a full refund, and we may have to mark them down heavily next time we sell them (that obviously depends on the text). The price of a book on the Oxford website (I DO NOT work for Oxford, they are just my example), for instance, is not the price that your book store paid for that book.
Promotion and distribution also cost quite a bit. Do you want professors to adopt your textbook? You need to pay someone to visit the professors and tell them about the books. You also need to provide those professors with free books. This is generally called "comping". All the major publishers have a person living in your area who travels, to a number of different schools to hawk their books and provide free examination copies to professors. Those publishers also send out "blind comps" to all relevant professors for new texts - in many cases, this will mean shipping out thousands of free books, with potentially $0 revenue in return.
Permissions are also extremely expensive. If you are trying to publish a large anthology with a bunch of different papers or excerpts, you may be looking at anywhere from $10,000 to $100,000, just for North American distribution.
As a publisher, you will also have to pay for a booth at all the major conferences to display your books. Do you think those booths are free? The cost of a small booth can range from a couple hundred to a couple thousand dollars, depending on the conference. That does not include the cost of carpet. At larger conferences, you MUST rent a carpet - that will usually cost another few hundred (and no, you cannot bring your own). Tables, chairs, etc. cannot be brought, they must also be rented at exorbitant prices. WiFi access can generally be rented by book vendors. A single WiFi account for a conference is often around $200. A decent-sized conference, all costs included (travelling fees and wages for your representative, rental fees, shipping fees for your books, etc.) can hit around $10,000, depending on how much booth space you want, and how many books and persons you send. Remember, that is just one conference. You will have to pay to visit many conferences, every year. Every year, the price will go up.
Don't forget that your author(s)/editor(s) also get paid a percentage of sales as well. The royalty amount paid to authors/editors will be more contextual, depending on the title, the series, the stature of the authors/editors, etc.
For the last fiscal year, I know many academic publishers posted losses. Many professors have be unwilling to change texts for the last couple years, so adoption rates for new texts have been hit in the industry. The prices may seem high, but that doesn't mean the publishers are making money. This is an expensive business.
Finally. How much do you make? Do you make more than $50,000 a year? If so, you're doing better than almost everyone I know in the textbook industry.