My work takes me all over San Jose, and especially to the worse neighborhoods. It's not a great area, and Santa Clara Street runs through some really rough neighborhoods on the other side of Downtown, which is only a mile or so wide. By "rough" I mean gangs, shootings, you watch what color you wear, that kind of thing. There aren't any neighborhoods in San Jose where I feel my immediate safety is in jeopardy, but I am a white male driving a reasonably nice car, and no one will mistake me for someone who is banging. But I would never choose to live in many of those neighborhoods, regardless of the practicality of the location. As for your particular spot, I think there's a lot of industrial and government stuff over there. My guess is if you're male and you don't look like someone that a gang member would care about, you're probably fine to walk downtown from that spot.