I'm looknig at the Garmin eTrex right now, and it looks like it does what I need.
I have a garmin eTrex and it is a simple base-level unit. It does do what you want but with a few caveats.
The base model eTrex only has very limited memory - as a result it compresses tracks before it saves them. It will record a position in its log, only when its position deviates significantly from a prediction based on your heading and velocity, or a couple of minutes have gone by.
The eTrex only stores about a thousand points - this works out at about 4 hours of travelling, before it starts erasing the beginning of the trip.
The eTrex doesn't have an SBAS receiver so cannot use WAAS/EGNOS precision boosting signals - positions can easily deviate by 100-300 feet depending on reception. The eTrex does have a fully functioning receiver capable of locking 12 satellite signals simultaneously (subject to orbit geometry and line of sight).
The eTrex doesn't have a barometric altimeter - so altitude data is very imprecise and fluctuates wildly - virtually unusable for elevation changes of less than 200 feet. By that I mean that you can go out on a 2 hour trip, and come back and there is a 150 feet difference in the recorded altitude at the beginning and end of the trip.
The eTrex is fully compatible with the garmin Mapsource software (supplied as Metroguide or routes&recreation). Of course, you can't upload maps onto the unit, but you can download the track data into the mapping software.
There are other 3rd party and shareware/freeware track analysers that you can use. In case you are wondering, each point in the log consists of a time and position coordinate (including altitude). Velocity vectors are not stored.
It may be that a higher end model like an eTrex Summit or eTrex venture may be better. The summit and vista have barometric altimeters so the altitude recording should be better - but I've not tested these models so don't really know how much better. Models with larger memory can be set to record track points at predefined intervals rather than whenever the unit wants to. It is likely, that an SBAS capable receiver may have better altitude precision (certainly horizontal preceision is much better).