The premise is not quite correct. The brain does a very good job of recording/storing memories during the first few years of life. The catch is that around six or seven a process called childhood amnesia kicks in and a lot of those early memories are lost. Testing of children before that process begins firmly demonstrates that the children do have effective memories.
It is a specific type of long term memory that is lost – episodic memory.
The networks of our mind ties all of these together such that to recall an event, all of the individual components are pulled out of long term memory and reassembled. Any of these memories being strong and well established can make it easier to find the others. This is why you remember the seventh grade teachers you liked and the ones you hated, but not the ones in between – the emotional memories are strong.
So, while people usually lose much of their episodic memories from the first six years of life, they do retain episodes with strong ties to other types of memory. For example, I can describe the day my grandfather died, when I was two-years and four-months old, with great detail.
Now, as to why this happens, there isn’t a strong consensus. Theories range from Freud believing that children are repressing memories of seeing penises to the development of the hippocampus not being complete in early childhood.
Original Author: Robert Frost