For the most part, there isn't anything like what you suggest. "Invisible" watermarks have consisted of slight shifts in the picture that can then be later read, a feature that is useful for identification and not detterance.
What you are suggesting is a proprietary picture format (and good luck getting widespread support for that...) that can't be edited legally, which is probably not what the problem is. Illegal picture usage will spawn illegal software, so the malicous users will still be able to do as they wish. Finally, the so called analog hole will exist for a while, you aren't going to protect your pictures from someone dedicated.
If i may suggest, if you have a picture portfolio or something you want to show, have a low/medium rez watermarked pictures in a gallery, and if anyone needs a high resolution sample or copy, email them it directly after indicating they have no redistribution rights (and be prepared to back that claim up legally). As an artist, your pictures are your livlihood and you must protect what is rightfully yours. Currently technology doesn't provide a dream protection package, but in fact has a myriad of means of getting around any sort of protection scheme.
PS I'm not really into digital art/photography, I've recapped the major stories and observations I've come across in the last few years