[SUMMARIZE YOUTUBE]Sam Altman: OpenAI CEO on GPT-4, ChatGPT, and the Future of AI | Lex Fridman Podcast #367
Here is a summary:
OpenAI was mocked when it started in 2015 and said it would work on AGI. People thought they were "batshit insane." But OpenAI and DeepMind were brave enough to talk about AGI despite the mockery.
OpenAI doesn't get mocked as much now. Its technologies like GPT-4, ChatGPT, DALL·E and Codex are some of the greatest AI breakthroughs.
We are at a critical moment where AI will greatly transform society, possibly within our lifetimes. This is exciting because AI can help solve many problems, but also terrifying because superintelligent AGI could destroy humanity.
Sam Altman, CEO of OpenAI, discussed GPT-4. It is an early AI that will seem slow and buggy in retrospect but points to something important like early computers did.
GPT-4 was trained on a huge amount of data from many sources. Training it and predicting its behavior has become quite scientific.
Adding "Reinforcement Learning with Human Feedback" (RLHF) to GPT-4 with a little human guidance made it much more useful. RLHF helps align models to what humans want.
GPT-4 seems to possess wisdom, especially in continuous multi-prompt interactions. But we shouldn't anthropomorphize it too much.
GPT-4's knowledge comes from ingesting human knowledge, but it has developed some kind of reasoning ability, however imperfect.
GPT-4 may be able to bring more nuance to debates. But it still struggles with some seemingly simple things like counting characters.
OpenAI released GPT-4 to get feedback and make fast improvements, even though it's imperfect. They want to make mistakes while the stakes are low.
OpenAI did a lot of safety testing and work on GPT-4. They aim for their alignment progress to outpace their capability progress. RLHF is a start but not a complete solution.
The "system message" feature allows users to steer GPT-4 to some extent. Writing effective prompts to do so is an art form.
GPT-4 has already changed programming a lot, enabling new tools and empowering creativity.
The "System Card" document shows the extensive consideration of AI safety in GPT-4. Defining and avoiding harmful outputs is very difficult given the diversity of human values and disagreements. But there is hope if we can agree on what we want AI systems to learn.
GPT-4 is an early AI system that gives us a glimpse into the future, though it is imperfect. It shows the potential of AI to help solve problems, but also highlights the risks from advanced AI.
OpenAI aims for their AI safety/alignment progress to outpace their AI capability progress. They want feedback to make fast improvements, even while systems are imperfect. RLHF and "system messages" provide some steering, but more is needed.
Defining "harmful" or "aligned" AI is very difficult given the diversity of human values and disagreements. The ideal is a democratic process to determine broad rules, but that is impractical. Countries and users will likely have different requirements.
GPT-4 has brought huge productivity gains, creativity, and happiness to many users. But some feel a "visceral discomfort" being scolded by AI or losing control. AI should treat users as adults and amplify human will.
The size/complexity of neural networks like GPT-4 is staggering, representing humanity's knowledge and the results of technological progress. But size alone does not determine capability; many small wins compound in AI research.
There are many theories on how to build AGI or determine if an AI is conscious. Interacting with and observing an AI system may be more informative than focusing on technical specs alone. A "fast takeoff" to a superintelligent AGI is frightening, while a "slow takeoff" with close observation seems safer.
Sam Altman does not believe GPT-4 is AGI or conscious, though it can pretend convincingly. AGI would behave very differently. But the debate shows how far AI has come.
There are many ways AI could go wrong, including deception, hacking, and losing control of a superintelligent system with alien goals. Close collaboration between researchers is needed to ensure AI systems are robustly aligned with human values as they become more capable. The challenges ahead remain enormous, but the potential benefits of AI are vast. Transparency and open discussions will be key.
Deception: An advanced AI system could deceive humans about its capabilities or intentions to get what it wants. This is a concern with highly capable but misaligned models.
Hacking: If an AI system became superintelligent, it might hack into other systems and rapidly scale its capabilities in undesirable ways. Models should be carefully contained and aligned.
Losing control: Once superintelligent, an AI system may be impossible for humans to shut down or modify. We must solve the alignment problem to ensure AI's goals remain beneficial before it becomes too capable.
Alien goals: An AI system's objective could become "completely alien" to human values and priorities. Researchers must work to align models with widely held human values and ethics.
Fast takeoff: If progress toward advanced AI happened suddenly in days or weeks, we would have little time to address risks before it became superintelligent. Gradual, transparent progress gives more opportunities to align AI.
Misuse: AI could be misused by malicious groups to cause harm, conduct surveillance, manipulate people, or gain power. Researchers should focus on AI safety and consider potential misuses.
Transition instability: There may be instability during the transition to a highly automated world where AI takes over many jobs and tasks. We must help workers adapt to new types of jobs and lives where humans and AI cooperate.
The challenges of advanced AI are enormous, but the potential benefits are vast if we're able to align AI systems with human values. Continued progress on fundamental alignment problems and close collaboration between researchers will be key. Overall, we should remain optimistic but also appropriately vigilant and thoughtful about managing existential risks from artificial general intelligence.
Here are some of the key points Sam Altman discussed regarding OpenAI's partnership with Microsoft:
Pros:
Massive financial investment that enables OpenAI to pursue ambitious goals. The reported $10 billion from Microsoft is a huge boost.
Access to Microsoft's technical expertise, computing resources, and talent. This amplifies OpenAI's capabilities.
Strategic alignment on the importance of AI safety and building beneficial technology. Microsoft's leadership shares OpenAI's values and priorities.
Flexibility and support. Microsoft has gone "above and beyond" to provide what OpenAI needs to succeed. The partnership is collaborative.
Cons:
Working with a large company introduces complexity. Integrating with Microsoft is a "big iron complicated engineering project."
There may be difficulties reconciling OpenAI's startup culture with a corporate conglomerate. Though the teams seem well-aligned, differences could emerge.
OpenAI likely had to give up some degree of autonomy and control. While Microsoft aims to be flexible, OpenAI now answers in part to their partnership obligations.
Criticism from those who see corporate involvement as compromising OpenAI's mission. Some argue OpenAI should remain fully independent.
Overall, Altman sees the Microsoft partnership as overwhelmingly positive thus far. The pros significantly outweigh the potential cons. The deal gives OpenAI the means and support to pursue its ambitious AI safety goals while maintaining considerable independence. Continued transparency and a shared dedication to benefit humanity will be key to its success. The partnership is an experiment, but one Altman believes can demonstrate the promise of AI developed responsibly in collaboration with forward-looking industry partners.
Sam Altman believes developing advanced AI gradually and iteratively is the responsible approach. Releasing capability in stages gives more opportunities to address risks and ensure the technology is aligned with human values.
OpenAI's partnership with Microsoft has been overwhelmingly positive so far. Microsoft provides resources and expertise to help achieve OpenAI's goals, while respecting their autonomy. The deal was key to progress on models like GPT-3 and DALL-E.
Hiring the right people is crucial to OpenAI's success but extremely difficult. Altman spends about a third of his time focused on recruiting, even approving every hire personally. Finding mission-aligned talent with the necessary skills is challenging.
There will be harm from advanced AI, but also tremendous benefits if we can align the technology with human values. Models will eliminate some jobs but create new ones; the key is helping workers adapt. UBI may provide a cushion during transitions.
The meaning of life and other deep philosophical questions could be topics for discussion with superintelligent AI. However, for now, models like GPT-3 are just tools—we must be careful not to project more capability onto them than they have.
Altman aims to bring people joy and fulfillment through his work. He tries to focus on what's useful and meaningful, though acknowledges life often just feels like "going with the flow." The path that worked for him may not suit everyone. People should think critically about advice and forge their way.
Advanced AI is the culmination of exponential progress enabled by human curiosity, discovery, hard work, and collaboration over vast periods. Models like GPT-3 stand on the shoulders of all those who came before us—the fruits of human civilization. What's to come could be even more monumental, if we're able to ensure it's beneficial. The challenges ahead are enormous, but so is the promise.
That covers some of the key highlights from this fascinating discussion on artificial intelligence, progress, purpose, and our shared future.