Yeah, it's fine as long as you're a normal healthy person. My folks used to leave stuff out to defrost like that all the time. Just leave it in the sink and go to work.
I mean, I wouldn't take a chance on food safety w/ small children or the elderly, but for myself, yeah all the time.