2024 | Edward Hughes, Michael Dennis, Jack Parker-Holder, Feryal Behbahani, Aditi Mavalankar, Yuge Shi, Tom Schaul, Tim Rocktäschel
Open-endedness is essential for artificial superhuman intelligence (ASI). This paper argues that open-endedness is a critical property of ASI, enabling it to generate novel, learnable artifacts that are both creative and beneficial for humanity. Open-ended systems, built on top of foundation models, can continuously produce new knowledge and adapt to evolving environments. However, current foundation models are not yet open-ended, as they rely on fixed datasets and lack the ability to generate novel, learnable artifacts on their own.
The paper provides a formal definition of open-endedness as a system that produces artifacts that are both novel and learnable from the perspective of an observer. This definition is based on the ability of the system to generate increasingly unpredictable artifacts while still being learnable by the observer. Open-endedness is not only important for ASI but also for the safety and usefulness of AI systems. Open-ended systems must be guided towards knowledge that is understandable and beneficial for humanity.
The paper discusses various research directions for achieving open-endedness with foundation models, including the use of reinforcement learning, self-improvement, task generation, and evolutionary algorithms. These approaches aim to create systems that can generate novel, learnable artifacts while remaining safe and useful. The paper also highlights the importance of safety in open-ended systems, emphasizing the need for mechanisms that ensure the system remains open-ended and beneficial for humanity.
In conclusion, the paper argues that open-endedness is a critical property of ASI, and that combining open-endedness with foundation models is a promising path towards achieving ASI. The paper emphasizes the importance of safety and the need for responsible development of open-ended systems to ensure they are beneficial for humanity.Open-endedness is essential for artificial superhuman intelligence (ASI). This paper argues that open-endedness is a critical property of ASI, enabling it to generate novel, learnable artifacts that are both creative and beneficial for humanity. Open-ended systems, built on top of foundation models, can continuously produce new knowledge and adapt to evolving environments. However, current foundation models are not yet open-ended, as they rely on fixed datasets and lack the ability to generate novel, learnable artifacts on their own.
The paper provides a formal definition of open-endedness as a system that produces artifacts that are both novel and learnable from the perspective of an observer. This definition is based on the ability of the system to generate increasingly unpredictable artifacts while still being learnable by the observer. Open-endedness is not only important for ASI but also for the safety and usefulness of AI systems. Open-ended systems must be guided towards knowledge that is understandable and beneficial for humanity.
The paper discusses various research directions for achieving open-endedness with foundation models, including the use of reinforcement learning, self-improvement, task generation, and evolutionary algorithms. These approaches aim to create systems that can generate novel, learnable artifacts while remaining safe and useful. The paper also highlights the importance of safety in open-ended systems, emphasizing the need for mechanisms that ensure the system remains open-ended and beneficial for humanity.
In conclusion, the paper argues that open-endedness is a critical property of ASI, and that combining open-endedness with foundation models is a promising path towards achieving ASI. The paper emphasizes the importance of safety and the need for responsible development of open-ended systems to ensure they are beneficial for humanity.