Wow, now this is a hard question. There is no practical answer to this. Every encryption algorithm uses its own keying methods. Let's look at a simple hashing method (MD5 and SHA0 (note, this has been replaced by SHA1, just using it as an example)):
Speed Paper
As you can see, on identical equiptment the SHA (35,504 key/s) is much slower then MD5 (50,002 key/s) (64 octet message). This can be considered to analgous to the keying of an encryption algorithm.
If you assume a -true- brute force (not one that simply uses leters/numbers in a hybrid attack), the brute force time is analgous to the length of the keying stream. A 10 character password is just as safe a a 100 character password. Rijendael, for instance, uses keys of up to 128, 192, or 256 bits. As you get higher key lengths, the processing time of finding a collision (Two different passwords that satisfy the same method) increases.
If you are attacking the password itself using a shortest-first method, it's still hard to say. You no longer have to worry about collisions. However there is still a limit of the effective length of your password. At some point it becomes more effective to attack either the user, the computer, or the algorithm. Exactly how long this is depends on the methods and skills of the attacker.
Now that I gave you that completely useless answer, I would suggest using at least an 8 character password with mixed letters, numbers, and special characters for something you don't want breached by casual attackers. 12 is a good limit for determined attackers, and I would use a long phrase for something you want secure at all costs.
AnthraX101
EDIT: A note about the post above. A teraflop is
10^12, not 2^12. That changes your numbers to require about a 12 character password, assuming a 10 operation attempt. For practical concerns, at this point in time a single desktop can bruteforce about 5-6 million keys per second of MD5 (at about 1.5-2ghz)
All that said, brute force is tending to go the way of the dodo. Most encryption works off of some sort of hashing function for keys, and at this point rainbow tables are becoming popular. This allows you to brute force all the keys once, and then store the answer. That way you only look up the answer later, and already know the question.