AI and learning: A personal journey

Large Language Models (LLMs) and most modern AI systems are based on iterative error correction using the backpropagation algorithm, popularised in 1986 by David Rumelhart, Geoffrey Hinton, and Ronald Williams. The system learns by adjusting its internal connection weights so that its output increasingly matches a desired target or "teaching input". In effect, it learns through imitation.

This approach is known as supervised learning because the correct answer is provided during training. The difference between the network's output and the desired output generates an error signal. That error is propagated backwards through the network, modifying potentially billions of connection weights so that future outputs more closely resemble the teacher's example.

Biological learning is fundamentally different. Humans and other organisms learn through interaction with the real world. We receive feedback from our actions, but there is usually no teaching input specifying the correct answer. Instead, we experience outcomes that are beneficial, harmful, or simply different from what we expected, and our nervous systems adapt accordingly.

In biological learning there is no external teacher calculating precise error gradients. The organism must solve the credit assignment problem itself: determining which actions, perceptions, or internal processes contributed to success or failure. This form of learning is more closely related to reinforcement learning and unsupervised learning than to the supervised learning used in contemporary AI LLM systems.

A Personal Journey

I began a PhD in machine learning and neural networks in 1985 after completing a Master's degree in Artificial Intelligence. My doctoral research focused on connectionist models and backpropagation networks using the methods pioneered by Geoffrey Hinton, now widely regarded as one of the founders of modern AI.

At the University of Essex, under the supervision of Professor Noel Sharkey, one of the UK's AI pioneers, I immersed myself in the influential two-volume Parallel Distributed Processing books edited by David Rumelhart and James McClelland. In particular, I studied the famous chapter by Rumelhart, Hinton, and Williams describing the backpropagation algorithm. I implemented these networks from scratch in POP-11, a programming language that is similar to Python today. I therefore developed a deep understanding of the inner workings of backpropagation neural networks and their learning dynamics.

My own PhD research sought to understand perception within a simulated ecological environment using the reinforcement learning approaches that were being championed by researchers such as Richard Sutton and Andy Barto. However, I came to the conclusion that it was impossible to simulate the richness of real ecological information using purely digital systems. Living organisms do not inhabit a digital simulation. They inhabit a continuously evolving ecological world. Indeed, the nature of information in the real world is fundamentally different from the information that can be represented in a digital simulation.

Around this time I had discoverd that there is an approach to understanding information that is not based on digital computation or symbolic representation. This approach is known as ecological psychology, pioneered by James Gibson, considered by many as the most radical and insightful perceptual psychologist of the 20th century. His approach emphasised the direct perception of the environment based on ecological information and the continuous interaction and reciprocity of organism and environment. Perception is based on highly structured, meaningful information in the environment, not on meaningless physical sensations or "cues" that must be interpreted in order to create meaning. I realised that this approach offered a radically different perspective on perception and action, and one that was more aligned with how humans and all organisms learn in the real world.

This realisation created a crisis for my PhD research as I realised all AI models depend on "features" and simple inputs. These features are defined by the human so it's all a bit of a hoax: the intelligence is already pre-defined in the inputs by a human! How can this be how learning occurs if the organism is supposed to be learning from the environment? I realised that the AI models I was working on were not really learning at all. They were just imitating patterns in the training data, and they were not discovering new affordances in the real world.

After making contact with key figures in ecological psychology in the UK and the US, and after a year of self-study in the field, I was offered the incredible opportunity of a scholarship at the University of Connecticut in the United States to continue my PhD studies at the Center for the Ecological Study of Perception and Action (CESPA), the leading centre for ecological psychology and human experimental studies of Gibsonian theory. There I worked with Professor Michael Turvey and became immersed in ecological psychology and the Gibsonian theory of perception. I worked on the problem of how animals acheieve coordinated rhythmic movement.

My PhD thesis was on human handedness and the asymmetric dynamics of bimanual coordination. This was not just another neural explanation of handedness, but a dynamical systems explanation based on recent discoveries of mathematical constraints - attractor dynamics - equations of motion that govern all rythmic movement in the terrestrial environment. Think pendulums and coupled oscillators. The human body is a complex system of coupled oscillators and the environment is a complex system of coupled processes. The dynamics of the human body and the environment are coupled and this coupling leads to emergent patterns of behaviour that are not reducible to the properties of the individual components. This is the essence of ecological psychology: the organism and the environment are inseparable and must be studied as a whole. This is what is meant by a "systems approach" to understanding behaviour of any phenomenon.

Dynamical systems theory draws extensively upon modern complex systems theory and new mathematical techniques of time series data analysis. It emphasises the continuous interaction between organism and environment and how this leads to new emergent patterns and properties such as stability and instability.

Following my PhD I was fortunate to receive a postdoctoral scholarship with another pioneer of applying dynamical systems theory to human movement, Professor Scott Kelso, at his Centre for Complex Systems and Brain Sciences at Florida Atlantic University. There I was able to apply the tools of complex systems theory to the study of stability and instability in human balance, using new methods for analysing "fractal" time series data from human movement experiments. This was about the physical property of "long memory" that many physical complex systems exhibit, and how this property can be used to understand the surprising "memory" (temporal correlation) within the structure of spatio-temporal events and human behaviour. All human behaviour is an event at some scale of analysis; Gibson's approach was all about "event perception" under real, ecological conditions. Similarly, the property of long memory is a property of the ecological system, not of the brain. It is a property of the organism-environment system, not of the individual components. Long memory is not due to the brain but it is in the dynamical system structure of many animate (and inanimate) events. Again, this is a property of the ecological dynamical system; it is not a representations or "memory" inside the brain. Cognition is not in the brain, it is in the organism-environment system. This is a radical departure from the traditional cognitive science view of cognition as a computational process inside the brain based on internal representations and symbolic manipulation. Here I describe long memory and its relationship to consciousness.

You can read more about these studies in my published papers and also see videos of the balancing experiments under Publications.

As part of complex systems science, ecological psychology provides a radically different perspective on perception, action, and learning. The tools of dynamical systems theory allow us to model the continuous, time-dependent interactions between organisms and their environments, capturing the richness and complexity of real-world learning experiences. Ecological psychology is not just a theoretical framework; it has practical implications for designing learning environments, educational technologies, and AI systems that are more aligned with how humans and other organisms learn in the real world. In many ways it has left cognitive science and AI behind, as it emphasises the importance of real-world interaction and the continuous flow of information between organism and environment, rather than relying solely on internal representations or symbolic processing.

Ecological psychology provided me with a radically different perspective on learning. Rather than viewing perception as the internal reconstruction of a world represented symbolically inside the brain, it treated perception as the direct detection of meaningful information in the environment. This information is fundamentally biological, ecological, and relational rather than merely computational.

Affordances and the Future of Learning

The future of human learning lies not in imitation but in interaction. Organisms learn through perceiving and engaging with the opportunities for action available in their environments. James Gibson called these perceivable opportunities for action affordances.

Learning emerges through active exploration, discovery, and adaptation. Meaning arises from the ongoing relationship between organism and environment rather than from matching outputs to a predefined set of correct answers. Human understanding is grounded in perception and action within a real ecological world. Perception is of affordances that are directly meaningful to the organism, and learning is a process of attuning to information about these affordances through experience.

Modern AI and LLMs provide a new kind of affordance. They can be perceived and acted upon but they are often digital entities, not physical objects, and so they can not be acted on or grasped in the usual way of grasping an object with a hand. Interestingly, they can be grasped in the other sense of the word grasp, meaning to understand or comprehend with the mind.

These things are what I have called metaffordances.

The same principle may ultimately apply to artificial intelligence. Current LLMs excel at symbolic language manipulation and statistical imitation. They can reproduce patterns found in their training data with remarkable sophistication. However, imitation is not the same as understanding. Genuine intelligence may require direct engagement with the environment and the ability to discover affordances and metaffordances through experience rather than merely reproducing patterns learned from examples.

For this reason, interaction with a world of metaffordances, rather than developing increasingly sophisticated forms of symbolic prediction (e.g., LLMs), may prove to be the next major step in the development of both artificial and biological theories of intelligence.

The Future of AI

Returning to my first PhD studies on neural nets and reinforcment learning, Richard Sutton, one of the pioneers of reinforcement learning, has argued that scaling language models alone is unlikely to lead to genuine intelligence. I concur. His perspective emphasises learning through interaction, exploration, and experience rather than imitation from static datasets.

On reflection, now after 40 years, I do not believe that AI can replicate, let alone exceed, human intelligence without incorporating the principles of ecological psychology and the direct perception of affordances. Indeed, it is not clear to me that AI can ever be truly intelligent without being embodied in a real organism-environment system. And even then, it may be that the intelligence of such a system is fundamentally different from human or animal intelligence, as it would be grounded in a different set of affordances and metaffordances.

The issue of consciousness and self-awareness in AI is a separate but important question. However, I believe that the principles of ecological psychology and the direct perception of affordances provide a valuable framework for understanding consciousness, and the question of whether an AI system can ever have consciousness. Currently, AI systems do not have consciousness in any sense of the term and it is unlikely that they will in the foreseeable future. I write about consciousness under Ideas.

Richard Sutton explains why he believes LLMs are not the path forward for artificial intelligence:

Reinforcement Learning Needs Gibson

Richard Sutton has argued convincingly that intelligence arises through learning from interaction with the environment rather than from human-crafted rules. Reinforcement learning replaces explicit instruction with trial-and-error learning guided by reward. However, this leaves a profound unanswered question.

Where does the reward function come from?

In artificial reinforcement learning systems, the reward function is almost always supplied by the programmer or the environment. It is assumed rather than explained. This is no different in principle from supervised learning, where the desired output is externally specified. The source of value remains external to the learning process.

Natural organisms are fundamentally different. An animal does not possess an arbitrary reward function. Its goals are the product of millions of years of evolution. Hunger, thirst, pain, fear, curiosity, attachment, and pleasure are not independent reward signals. They are manifestations of evolutionary constraints that exist to maintain the organism's viability.

This is where James Gibson's ecological psychology becomes essential. According to Gibson, organisms do not construct an internal model and then evaluate actions against an externally defined reward. They directly pick up ecological information specifying what the environment affords. Perception is the pickup of information specifying opportunities and dangers relative to the organism.

From this perspective, reward is not a primitive concept at all. The organism is continually engaged in maintaining itself within an ecological niche. Survival is not a distant objective but an ongoing process occurring simultaneously across multiple spatial and temporal scales: maintaining posture, avoiding obstacles, acquiring food, regulating body temperature, navigating the environment, reproducing, and countless other nested constraints.

What reinforcement learning calls "reward" is simply an abstraction of successful ecological engagement.

The reward function is ecological teleology.

Intelligence therefore begins not with reward maximization but with the lawful pickup of ecological information. Evolution determines what matters, ecological information specifies what is possible, and learning progressively attunes the organism to act more effectively within those constraints. This attunement is not through some poetic process. Attunement through resonance is the fundamental coupling principle of the universe. I did research on it as part of my PhD - my first PhD publication, of several. The paper was called "Resonance constraints on rhythmic movement" and it applied well known priciples from physics and dynamical systems (going back to Poincare over a century ago) to the understanding of human behaviour: RESONANCE COUPLING.

Sutton's reinforcement learning explains how behavior can improve through interaction. Gibson explains why there is anything worth learning in the first place. His theory of ecological information and affordances is the only way to understand information and the kind of intelligence we animals have. Anything else is just a simulation, just an artefact or a tool, and fake intelligence. Humans and all organisms are far more than that.

Paul Treffner
metaffordance.com