there would be a chance for you to survive since you're much bigger than the baby...you could take more damage then it can
That wasn't the question. It is implied that you would be pretty darn sure beforehand that you would die.
In answer to the question, probably not. I would be leaving my wife and children to fend for themselves and I know for a fact that it would be extremely rough for them.
This question is something like another: If two children were drowning in a swollen, raging river, one of them was your child and the other was unknown to you, and you could only, possibly, save one, which one would you try to save?