Yep. Mobile devices are usually setup to use MP3, but not always for Ogg, unfortunately.
There are a few that do though (and these tend to be higher quality and support many different formats, anyways.)
For some background the orginization that developed Ogg Vorbis also developed some other codecs...
Also 'Ogg' is a container format. Like .avi or quicktime formats. For instance can have AVI files that are Mpeg4 video + AC3 audio. Or you can have Mpeg4 + Mp3 inside avi files.
Well Ogg is like this. Ogg Vorbis is the actuall technical name for thesse audio files. But the container format 'Ogg' can contain more then just Vorbis sound.
Xiph also developed Flac, which is a lossless audio codec. (mp3's, Ogg Vorbis, and such are 'lossy'). They developed Speex which is a codec for voice only, not music. Speex is used in TeamSpeak, and some SIP or H.323-style VoIP setups. This makes clear speech over spotty and low bandwidth connections (as much as possible).
Ogg Vorbis is used in some games, too. Many games. A couple are Ut2004 and Doom3. See, it supports surround sound.
People have created OGM files by sticking Vorbis audio and Mpeg4 video into Ogg container format. Not the nicest thing though.
For video, last december, Theora came out of beta. This is a competator with mpeg4, but it should have nicer video quality and it doesn't have the 'blockiness' that comes with most high compressed video files.
So you now create Ogg Theora/Vorbis files, with full surround sound and high quality video and stream them over the internet, or in a local lan.
One thing that can do this is newer versions of Icecast.
http://www.icecast.org/
Icecast is a free version of the Shoutcast audio media streamer...
With version 2.0 they added support for Ogg Vorbis (became the major focus of it, I think) and since it supported Ogg Vorbis, it was a short hop to support Ogg Vorbis Theora video files...
So you can use it to stream video stuff, too. Makes a nice dedicated streamer.
However Theora hasn't been around for very long, so getting support for it is tough. However I do know that VLC supports it.
http://www.videolan.org/vlc/
VLC supports most formats, and can actually stream many formats itself. If your using Linux it's possible to even stream TV that you captured with TV capture cards (assuming that they are supported by Linux), and stuff like that. DVD's, audio, video, avis, theora. All that stuff. But Icecast would make a better dedicated Theora server.
VLC has OS X versions, Linux (was originally Linux app) and has good Windows versions. (and even PocketPC/WinCE/Zope, and BeOS versions)
http://www.videolan.org/vlc/
Just some stuff to play around with.

Theora is new, and therefore a bit diffucult to muck around with, but that will change as it matures.