Of fvcking course.
Being a castrate means you will never reach puberty, so many of the masculine features will not appear (limited testosterone production, by supplementing it with testosterone you can have the same results as having a set, it is actually quite possible for a 40 year old woman to reach male puberty by supplementing testosterone and part of the reason for pregnant womens mood (i have three kids, this is real) as they produce nandrolone (nor-testosterone, brand name deca-durabolin)) but the nerve endings are there before puberty, so your question is stupid.
Can a pre pubescent boy have an erection? There is your answer.