It has been demonstrated that Genetic Algorithms (GAs) can perform extremely efficient searches of “solution space” in order to find optimal solutions to complex problems. John Holland (1975) details and David Goldberg (1989) further develops) a theory of schemata to show how even poor solutions hold implicit information about the targeted good solutions. This accounts in part for how GAs can do what they do.
However, it may be that the principles of Schema Theorem extend beyond GAs. One of the arguments against statistical approaches to language acquisition, for example, is that the amount of input required to learn complex linguistic structures would greatly outweigh that which is exhibited in reality. Schema Theorem may hold a key to understanding why this argument is invalid.
Thus, the purpose of this paper is to explain Schema Theorem and explore its potential as a more general principle of information processing by examining language acquisition in particular. An effort is made to restrict the discussion as much as possible; however, inevitably, the implications and arguments presented here will apply to and draw on areas of cognition other than language. This should be interpreted as the possible pervasiveness and usefulness of Schema Theorem.
Should the generalization of the principles of schemata to language acquisition prove valid, the consequence will be that dynamic, statistical models of language and language learning may be more efficient than is currently believed, lending support to the growing argument against assumed innateness.
Section 2. discusses Schema Theorem in depth and how it is applied to genetic algorithms.
Section 3. describes how the same fundamentals can be applied to language acquisition, and Section
4. summarizes and suggests areas for further study.
Chalmers, David. (1990). Syntactic Transformations on Distributed Representations. Connection Science, Vol. 2, Nos 1 & 2, 53-62.
Damasio, A. and H. Damasio. (1994). Cortical systems for retrieval of concrete knowledge: The convergence zone framework. In: Koch, C. and L. Davis (Eds.). Large-Scale Neuronal Theories of the Brain. Cambridge, MA: MIT Press.
Elman, Jeffery. (1995). Language as a dynamical system. In: Port et al.
Goldberg, David. (1989). Genetic Algorithms in Search, Optimization, and Machine Learning. New York, NY: Addison-Wesley Publishing Company, Inc.
Hebb, Donald. (1949). The Organization of Behavior. New York, NY: Wiley.
Holland, John. (1975). Adaption in Natural and Artificial Systems. Ann Arbor, MI: University of Michigan Press.
Johnson, J. and E. L. Newport. (1989). Critical period effects in second language learning: The influence of mturational state on the acquisition of English as a second language. Cognitive Psychology, 21, 60-99.
Miller, G.A. and Noam Chomsky. (1963). Finitary models of language users. In: R.D. Luce, R.R. Bush, and E. Galanter (Eds.), Handbook of Mathematical Psychology, Vol. 2. New York, NY: Wiley.
Mitchell, Melanie. (1996). An Introduction to Genetic Algorithms. Cambridge, MA: MIT Press.
Newport, E. L. (1990). Maturational constraints on language learning. Cognitive Science, 14, 11-29.
Port, Robert and Timothy van Gelder. (1995). Mind as Motion: Explorations in the Dynamics of Cognition. Cambridge, MA: MIT Press.
Rovee-Collier, C. (1991). The “memory system” of prelinguistic infants. Annals of the New York Academy of Sciences, 608, 517-536.
Rumelhart, David. (1997). The architecture of mind: A connectionist approach. In: Haugeland, John (Ed.) Mind Design II. Cambridge, MA: MIT Press.
Rumelhart, David and D. Zipser. (1985). Feature discovery by competitive learning. In: Rumelhart, D. and J. McClelland (Eds.). Parallel Distributed Processes: Explorations in the Microstructure of Cognition, Vol 1. Cambridge, MA: MIT Press.
Thelen, Ester and Linda Smith. (1994). A Dynamic Systems Approach to the Development of Cognition and Action. Cambridge, MA: MIT Press.
Thelen, Ester. (1995). Time-scale dynamics and the development of an embodied cognition. In: Port et al.