The best way to think of it, in my opinion, is to picture each cell in your brain as a very slow CPU and RAM stuck together (e.g. a single bit of memory combined with a CPU that can flip the switch). When I say slow, I mean in comparison to a computer that can flip calculations billions of times a second. The neural pathways transmit at roughly 80 meters a second, which means that the impulses in your brain are much slower than the electrical impulses in a computer, so this is normal. Because your brain has so many cells, however, it is incredibly powerful in terms of multiprocessing.
Each neuron can be connected to multiple other neurons, so the way that the brain thinks, stores knowledge, and operates is associative; that is to say, extremely long-chained branch logic. In order to recall an event stored in some area of your brain, you normally think of one thing associated with something else that is associated with something connected with that event.
When you recall that event, you re-write it in your brain simply be recalling it. The emotions and feelings you are thinking while you recall the memory can thus color the memory itself, so in time memories of the past can change emotional tone from what you may have been feeling at the time your brain recorded the event.
The brain is fully capable of remembering details from every day in your life, but doing so could be detrimental; it's most likely that we evolved a forgetting mechanism on purpose in order to not dwell in the past. The cannaboid receptors that are used when you smoke the mj are actually used for forgetful purposes as well; in other words, were you to remember every painful event, your brain would quickly become overwhelmed.
Therefore for a healthy and functioning brain, forgetting itself becomes purposeful. Over time, your brain tends to forget the painful events, and remember favorable events. As you travel in the present, your brain slowly categories and catalogues the events of each day via REM state / sleeping, and shuffles things from the forefront of your mind into storage. That storage is recalled by remembering events associated with the other events you are trying to remember, thus forming a logic branch that reaches the destination in your brain where the memory has been stored.
To sum up, your brain is constantly changing, re-writing your old memories and shuffling in new memories, but has to suppress memories at least somewhat otherwise people would become too distracted to survive in a world where constant dangers are present - moving cars, wild animals, etc.. If one had perfect memory and constantly dwelt in the past, one might miss things going on in the present and this would potentially result in injury / death, thus "perfect" memory mechanisms were bred out of humans over time.
Over the past several thousand years, with the advent of agriculture, etc., these memory mechanisms have slowly been re-introduced into the species by no longer requiring people to constantly attentive of the present, and also selectively breeding back in this characteristic by making those with exceptional memories become rich, thus having a greater likelyhood of passing those genetic traits on to their young.