Will someone please think of the titties, THINK OF THE TITTIES! Strippers and booze is usually a good night.
In all seriousness, you don't have to go. I obviously have no clue about your relationship with your brother or how the rest of the family would react but assuming it's normal it would sort of be a dick move. I personally have zero problems with being a dick though. If you don't want to be a dick I'd take one of the two suggestions above, go and leave early or come up with some last minute excuse as to why you can't go, just make sure it is believable.