Yeah, there's rackmount cases so you can build a system that goes in the rack. There's rackmount monitor holders, rackmount keyboard/mouse trays, I've seen a rackmount keyboard/mouse/lcd foldout that took 1U. And everything is measured in U's, the smallest rackmount systems are 1U, and need short heatsinks and angled memory (that's why the memory on the dual Tyan AMD board is angled).
The point is packing lots of power in a little space, or to make effective use of existing space. Before computers, audio equipment was rack mounted so there's another use. You can also build your own rack out of 2x4's.