On Autonomy: A Long Road Ahead

In recent years, discussions about autonomous systems have gained significant traction, primarily driven by advancements in self-driving cars and the envisioning of AI agents powered by large language models (LLMs). Despite the growing excitement surrounding these technologies, I argue that we have not yet reached a level of technological sophistication that would allow for the widespread deployment of truly autonomous systems in the near future.

The Current State of Autonomous Systems

Our achievements in autonomy thus far have been limited to very simple, controlled environments:

1. Non-interactive systems: Devices like clocks operate autonomously but do not need to interact with their environment.
2. Simple environment interactions: Appliances such as robotic vacuum cleaners function autonomously with simple functionalities in highly constrained environments.

These examples demonstrate our ability to create autonomous systems, but only within narrow, often man-made environments that lack the complexity of the real world.

To date, we have not successfully created any autonomous system capable of functioning reliably in an open, unpredictable environment. The real world presents an infinite array of variables and scenarios that current AI and robotics technologies struggle to navigate consistently and safely. This explains why self-driving cars, promised more than a decade ago, remain largely an unfulfilled vision. The same holds true for AI agents powered by large language models.

The Limitations of Current AI Technology

Advances in AI technologies, especially generative AI, have ignited both hope and fear regarding the potential for building fully autonomous systems that can replace human labor. However, I contend that although generative AI can improve productivity, it has not yet reached the level of functionality required for true autonomy.

On the surface, chatbots like ChatGPT can converse like humans and often possess more extensive knowledge than a single person. However, the way they function reveals a fundamental flaw that distinguishes them from human cognition. Due to the limitations of current deep learning technology, LLMs lack self-reflection — they don't understand what they're saying and have no confidence in the information they produce. This phenomenon is often referred to as "hallucination."

As long as the hallucination problem persists, AI agents won't be reliable. While they may be used for non-critical tasks, they should never be trusted with important responsibilities. The hallucination issue is so intrinsic to LLM technology that some claim LLMs are essentially hallucination machines. Asking an LLM to stop hallucinating would render it non-functional. In this sense, the hallucination problem is not easily fixable for LLMs unless a new machine learning paradigm is invented — a significant unknown.

Following the natural progression of current technology, the hallucination problem is unlikely to be resolved in the near future. It may be mitigated by engineering solutions such as Retrieval Augmented Generation (RAG), which directs LLMs to use more strongly anchored real data in their prompts. However, complete elimination of hallucinations remains elusive. Without addressing this issue, autonomous systems such as AI agents cannot be considered reliable.

To build truly autonomous systems, further AI breakthroughs are necessary. Given that this flaw is central to the current AI technology wave, it is not a problem that can be addressed in the immediate future.

Conclusion

While the concept of autonomous systems captures our imagination and drives technological innovation, the reality is that we are still far from achieving widespread, reliable autonomy. The complexity of the real world, both in physical and digital domains, continues to outpace our technological capabilities.

As we continue to advance in fields such as AI, robotics, and sensor technology, it's crucial to maintain a realistic perspective on the challenges that lie ahead. True autonomy requires breakthroughs in AI technology, probably new paradigms of machine learning that go beyond curve fitting. Until we make these breakthroughs, the dream of ubiquitous autonomous systems remains just that – a dream.