If you are the type of person that loves tweaking with everything on Android, I would recommend a custom rom such as
cyanogen. The performance is noticeably better then default ROM's such as HTC's Sense and Motorola's Blur.
From there you will still see some performance degradation depending on what sort of apps you install and what they are doing in the background at times, but it will not be as noticeable since you started out with a smoother running OS at the beginning. Speaking of which, load Cyanogen fresh - meaning before you add a bunch of apps. Then add the apps afterward.
Other then that I would say that besides normal performance, you can also see a degradation in battery life depending on what you do. I'd suggest a task manager to make sure things aren't running when you may not want them to be.
Lastly, and this is a personal issue of mine with Android phones as of today - its not the OS which is fantastic - its the manufacturers that make android phones. HTC makes incredibly cheap feeling phones. I am unsure on longevity, but they just feel cheap. I cannot wait until Sony Ericsson gets into the fold making Android devices, and I would pray for other manufacturers as well. Nokia would rock but probably won't happen.
I know I will get lambasted for this last part, but I wish someone like ASUS would get involved. They make fantastic everything else why not phones? One of the reasons I got addicted to iPhones after owning the first one was its incredible resilience. I had it since June of 2007 and had absolutely zero issues with it. The battery started to go bad and wouldn't last like it used to, 3 years later and that was it. It also feels solid, sturdy and simply top notch - even more so then my iPhone 4 which I love. If a manufacturer could capture the same type of "feel" I would love it.
The so called Dell Thunder is a prime example, that thing looks like it might be exactly what the Dr. ordered. I'm sure with time, more companies will get involved and my wish will be fulfilled. I cannot wait for the new SE running gingerbread and PS games with that slide out controller. Looks like a killer phone to me as long as Sony follows through with great games on it. Will make the PSP obsolete, won't need to get an N3DS because I love my NDS so much right now. My iphone cannot replace my NDS, its about the only device it can't replace. This SE has a chance to replace everything since the camera should also be awesome being a Sony device.