Here is a simple back-of-the envelope calculation based on Hinton’s talk on deep learning:
You live about 102 years. A year is π x 107 seconds, so your life is ~109 seconds. Let’s say you receive 10-100 “experiences” (impressions) per second (brain activity is between few tens to few hundreds of Hz).
Thus, your life is about 1010-11 experiences. Blink an eye and it’s gone.
Your brain has 1011 neurons, with average connectivity of almost 104. That is a total of 1015 synapses.
You have thus about ~104-5 synapses per experience. There is no way the brain could fit a proper model (in the statistical sense), since Nparameters >> Ndata. Instead it has to strongly rely on regularization and sparsity.
Experiences are exceedingly expansive, a synapse is very cheap.
This puts all the tired blank slate arguments to grave – brains have to come somewhat pre-trained/regularized via genetics.
Also relevant – the argument is actually just a reformulation and generalization of the poverty of the stimulus argument by Noam Chomsky in the field of linguistics (a field now incidentally totally dominated by deep learning).