This article relies largely or entirely on a single source. (September 2011)
Regularization is a linguistic phenomenon observed in language acquisition, language development, and language change typified by the replacement of irregular forms in morphology or syntax by regular ones. Examples are "gooses" instead of "geese" in child speech and replacement of the Middle English plural form for "cow", "kine", with "cows".  Regularization is a common process in natural languages; regularized forms can replace loanword forms (such as with "cows" and "kine") or coexist with them (such as with "formulae" and "formulas" or "hepatitides" and "hepatitises").
Erroneous regularization is also called overregularization. In overregularization the regular ways of modifying or connecting words are mistakenly applied to words that require irregular modifications or connections. It is a normal effect observed in the language of beginner and intermediate language-learners, whether native-speaker children or foreign-speaker adults. Because most natural languages have some irregular forms, moving beyond overregularization is a part of mastering them. Usually learners' brains move beyond overregularization naturally, as a consequence of being immersed in the language.
The same person may sometimes overregularize and sometimes say the correct form. Native-speaker adults can overregularize, but this does not happen often.
Models to explain overregularization
In these models, two mechanisms are at work. For example, the rule in English for forming a past-tense form is to add -ed. In cases where the recall from memory instantly retrieves the form, the brain does not need to execute the rule, and skips execution. In cases where retrieval doesn't happen fast enough, the rule is executed. The more practice the speaker gets with the language, the more reliable is the retrieval.
These models can explain why the same person may sometimes overregularize and sometimes say the correct form. A child that already knows the word bought may occasionally say buyed when retrieval fails to work well. The next time, retrieval works well, and she says bought correctly.
These models also explain why native-speaker adults rarely overregularize. They are so thoroughly practiced in the language that retrieval rarely fails.
This section needs expansion. You can help by adding to it. (June 2008)
Comparison and contrast with phonetic overcompensation
Phonetic overcompensation, one form of hypercorrection, can be compared and contrasted with overregularization.
In both cases, a learner must master the automatic overriding of a rule to the point that it happens unconsciously and instantly—one case being phonetic, the other being morphologic. (The neurologic mechanisms of how that happens are still being investigated. Perhaps the brain needs practice in sidestepping the rule entirely as the needed objects [e.g., phonetic strings or past-tense inflected verb forms] are called directly instead of being derived on-the-fly via the rule.)
Native-speaker children do not make phonetic overcompensation errors in the same manner or degree that foreign-speaker adults do, because they do not carry the baggage of an earlier language's differences. But it does not seem correct to say that overcompensation cannot happen at all to a monolingual speaker, because some minor tongue-twisting and some minor Freudian slips could possibly involve neurologic processes that are analogous to phonetic overcompensation.