The next grand challenge for AI | Jim Fan

你正在收听《TED每日演讲》。

You're listening to TED Talks Daily.

我是你的主持人，伊莉丝·胡。

I'm your host, Elise Hu.

我们能让AI代理像《星球大战》里的机器人那样工作，或者像《瓦力》那样贴心吗？

Could we get AI agents to work in the same way as the robots in Star Wars or be as helpful as Wall E?

吉姆·范正在通过在《我的世界》中训练AI，来拓展它的能力。

Jim Fan is expanding the capabilities of AI by training it on Minecraft.

在2023年TED AI大会上，范向我们介绍了他的工作，以及这项研究如何在广告后成为AI领域的下一次重大突破。

In his 2023 talk from Ted AI, Fan takes us through his work and how it's part of the next big advancement in AI after the break.

2016年春天，我坐在哥伦比亚大学的教室里，但根本没听讲。

In the spring of twenty sixteen, I was sitting in a classroom at Columbia University but wasn't paying attention to the lecture.

我当时正用笔记本电脑看一场棋类比赛。

Instead, I was watching a board game tournament on my laptop.

这可不是普通的比赛，而是一场非常特别的比赛。

It wasn't just any tournament, but a very, very special one.

那场比赛是Arthur Go对Lee Sedol。

The match was between Arthur Go and Lee Sedol.

AI已经赢了五局中的三局，成为历史上首个击败人类围棋冠军的程序。

The AI has just won three out of five games and became the first ever to beat a human champion at a game of Go.

我至今还记得那天见证历史时刻的激动心情——AI代理终于进入主流视野的辉煌瞬间。

I still remember the adrenaline of seeing history unfold that day, the glory moment when AI agents finally entered the mainstream.

但当兴奋劲过去后，我意识到，尽管AlphaGo很强，它却只能做一件事，而且只做这一件事。

But when the excitement fades, I realized that as mighty as AlphaGo was, it could only do one thing and one thing alone.

它没法玩其他游戏，比如《超级马里奥》或《我的世界》，更别提帮你洗衣服或今晚做顿美味的晚餐了。

It isn't able to play any other games, like Super Mario or Minecraft, and it certainly cannot do a dirty laundry or cook a nice dinner for you tonight.

但我们真正想要的，是像瓦力一样全能、像《星球大战》里各种机器人形态一样多样，并能在无限现实中自由穿梭——无论是虚拟还是现实，就像《头号玩家》那样。

But what we truly want are AI agents as versatile as WALL E, as diverse as all the robot body forms or embodiments in Star Wars, and works across infinite realities, virtual or physical, as in Ready Player One.

那么，我们该如何在不久的将来实现这些科幻场景呢？

So, how can we achieve these science fictions in possibly the near future?

这是一份关于通用型AI代理的实践指南。

This is a practitioner's guide towards generally capable AI agents.

目前大部分研究工作都可以从三个维度来梳理：智能体能掌握的技能数量、它能控制的身体形态，以及它能适应的现实环境。

Most of the ongoing research efforts can be laid out nicely across three axes: the number of skills an agent can do, the body forms, or embodiments, it can control, and the realities it can master.

那我们一个维度一个维度来聊。

So, let's take it one axis at a time.

今年早些时候，我主导了Voyager项目，这是一个在技能数量上实现大规模扩展的智能体。

Earlier this year, I led the Voyager project, which is an agent that scales up massively on a number of skills.

要说能支持无限创意的游戏，没有比《我的世界》更合适的了。

There's no game better than Minecraft for the infinite creative things it supports.

接下来跟大家分享一个有趣的事实。

And here's a fun fact for all of you.

《我的世界》目前有1.4亿活跃玩家。

Minecraft has a 140,000,000 active players.

这个数字有多惊人？比英国的总人口还多出一倍以上。

And just to put that number in perspective, it's more than twice the population of UK.

《我的世界》之所以这么火爆，就是因为它的开放性。

Minecraft is so insanely popular because it's open ended.

游戏里没有固定剧情逼着你走，你想干嘛都可以。

It does not have a fixed storyline for you to follow, and you can do whatever your heart desires in the game.

我们把Voyager放进Minecraft后，发现它能连续玩好几个小时，完全不用人管。

And where we set Voyager free in Minecraft, we see that it's able to play the game for hours on end, without any human intervention.

它能探索地形、挖各种材料、打怪物、制作上百种物品，还能不断解锁新的技能树。

It can explore the terrains, mine all kinds of materials, fight monsters, craft hundreds of recipes, and unlock an ever expanding tree of skills.

这背后的秘诀是什么？

What's the magic?

核心理念就是把编程变成行动。

The core insight is coding as action.

首先，我们用一个由热心社区开发的Minecraft JavaScript API，把三维世界转成文字描述。

First, we convert a three d world into a textual representation using a Minecraft JavaScript API made by the enthusiastic community.

Voyager调用GPT-4来写JavaScript代码片段，这些代码在游戏里就能直接变成可执行的技能。

Voyager invokes GBD4 to write code snippets in JavaScript that become executable skills in a game.

不过，就像人类工程师一样，Voyager也会犯错。

Yet, just like human engineers, Voyager makes mistakes.

它并不总能一次就写出正确的程序，所以我们给它加了一个自我反思机制来不断改进。

It isn't always able to get a program correct on the first try, so we add a self reflection mechanism for it to improve.

自我反思的反馈来自三个地方：JavaScript代码执行的错误、代理自身的状态（比如生命值和饥饿度），以及世界状态（比如地形和附近的敌人）。

There are three sources of feedback for the self reflection: the JavaScript code execution error the agent state, like health and hunger and the world state, like terrains and enemies nearby.

Voyager采取行动，观察这个行动对世界和自身造成的影响，反思如何能做得更好，尝试新的行动计划，然后不断重复这个过程。

Voyager takes an action, observes the consequences of its action on the world and on itself, reflects on how it can possibly do better, try out some new action plans, and rinse and repeat.

一旦某个技能熟练度达到一定水平，Voyager就会把它保存到技能库中，作为长期记忆。

Once the scale becomes mature, Voyager saves it to a scale library as a persistent memory.

你可以把技能库看作是一个完全由语言模型生成的代码仓库。

You can think of the scale library as a code repository returned entirely by a language model.

就这样，Voyager在Minecraft中不断探索和尝试，逐步递归地提升自己的能力。

In this way, Voyager is able to bootstrap its own capabilities recursively, as it explores and experiments in Minecraft.

我们一起来看一个具体的例子。

Let's work through an example together.

Voyager感到饿了，急需尽快找到食物。

Voyager finds himself hungry and needs to get food as soon as possible.

它察觉到附近有四个实体：一只猫、一个村民、一头猪，还有一些小麦种子。

It senses four entities nearby, a cat, a villager, a pig, and some wheat seeds.

Voyager 开始内心独白。

Voyager starts an inner monologue.

我该杀了那只猫还是村民来填饱肚子？

Do I kill the cat or villager for food?

这主意太糟了。

Horrible idea.

那小麦种子呢？

How about the wheat seed?

我可以靠种子种个农场，但那得花很长时间。

I can grow a farm out of seeds, but that's going to take a long time.

不好意思啊，小猪，就选你了。

So sorry, piggy, you are the chosen one.

Voyager 在背包里发现了一块铁，于是它从记忆库中调出之前学过的技能，打造一把铁剑，并开始学习一个新的技能：猎猪。

Voyager finds a piece of iron in its inventory, so it recalls an old skill from the library to craft an iron sword, and starts to learn a new skill called hunt pig.

现在我们也知道了，不幸的是，Voyager并不是素食者。

Now we also know that unfortunately, Voyager isn't vegetarian.

但还有一个问题没解决。

One question still remains.

Voyager是怎么做到持续不断探索的呢？

How does Voyager keep exploring indefinitely?

我们只给了它一个高层次的指令。

We only give it a high level directive.

那就是尽可能收集更多独特的物品。

That is to obtain as many unique items as possible.

Voyager自己制定了一套学习计划，逐步挑战更难、更新颖的任务。

Voyager implements a curriculum to find progressively harder and more novel challenges to solve, all by itself.

把这些结合起来，Voyager不仅能掌握技能，还能在过程中发现新的能力。

And putting all these together, Voyager is able to not only master, but also discover new skills along the way.

这些全都不是我们提前写好的。

And we did not preprogram any of this.

这全是Voyager自己的主意。

It's all Voyager's idea.

这就是我们所说的一生学习——代理永远充满好奇，永远追求新的冒险。

And this is what we call lifelong learning, where an agent is forever curious and forever pursuing new adventures.

与AlphaGo相比，Voyager在能完成的任务数量上大幅扩展了，但它在Minecraft里依然只控制一个角色。

Compared to AlphaGo, Voyager scales up massively on a number of things it can do, but still controls only one body in Minecraft.

问题是，我们能不能设计出一种算法，适用于多种不同形态的机体？

The question is, can we have an algorithm that works across many different bodies?

这时候，Metamorph登场了。

Enters Metamorph.

这是我在斯坦福参与共同开发的一项计划。

It is an initiative I co developed at Stanford.

我们打造了一个基础模型，能够控制的不只是一个，而是成千上万种结构各异的机器人。

We created a foundation model that can control not just one, but thousands of robots, with very different arm and leg configurations.

Metamorph能够应对各种机器人身体截然不同的运动结构。

Metamorph is able to handle extremely varied kinematic characteristics from different robot bodies.

这就是我们如何打造Metamorph的思路。

This is the intuition on how we create a metamorph.

首先，我们设计了一套专门的词汇来描述身体部位，这样每个机器人都像用这套词汇写成的一句话。

First, we design a special vocabulary to describe the body parts, so that every robot body is basically a sentence written in the language of this vocabulary.

然后，我们直接用一个Transformer模型来处理，就像ChatGPT那样，只不过Metamorph输出的不是文字，而是运动控制指令。

Then, we just apply a transformer to it, much like ChatGPT, but instead of writing out text, Metamorph writes out motor controls.

我们证明了Metamorph能够控制成千上万的机器人完成上楼梯、穿越复杂地形和避障等任务。

We show that Metamorph is able to control thousands of robots to go upstairs, cross difficult terrains, and avoid obstacles.

展望未来，如果我们能大幅扩展这套机器人词汇，我设想Metamorph 2.0将能通用化到机械手、人形机器人、机器狗、无人机，甚至更多领域。

Extrapolating into the future, if we can greatly expand this robot vocabulary, I envision Metamorph two point zero, we'll be able to generalize to robot hands, humanoids, dogs, drones, and even beyond.

相比Voyager，Metamorph在多机器人协同控制上迈出了关键一步。

Compared to Voyager, Metamorph takes a big stride towards multi body control.

现在，我们再往前进一步，把学到的技能和身体形态迁移到不同的现实环境中——这就是NVIDIA的仿真平台Isaac Sim。

Now, let's take everything one level further, and transfer the skills and embodiments across realities, enters Isaac Sim, NVIDIA's simulation effort.

Isaac Sim最大的优势在于，它能把物理仿真速度提升到实时的1000倍。

The biggest strength of ISACSIM is to accelerate physics simulation to 1000x faster than real time.

这就像是《黑客帝国》里那个虚拟对练道场一样。

So, it's very much like the virtual sparring dojo in the movie Matrix.

更厉害的是，Isaac Sim 能自动生成无数种不同的世界，每个都独一无二。

What's more, Isaac Sim can procedurally generate worlds with infinite variations, so that no two look the same.

这里有个有意思的想法。

Here's an interesting idea.

如果一个智能体能掌握一万种模拟环境，那它很可能就能直接迁移到真实世界——毕竟真实世界不过是第一万零一个场景，你品，你细品。

If an agent is able to master 10,000 simulations, then it may very well just generalize to our real physical world, which is simply the ten thousand and first reality, and let that sink in.

我们最终会迎来一个能在所有三个维度上都通用的单一智能体，这就是基础智能体。

We will eventually get to the single agent that generalizes across all three axes, and that is the foundation agent.

我认为训练基础智能体的过程，会跟训练 ChatGPT 非常相似。

I believe training foundation agent will be very similar to chat GPT.

所有的语言任务，本质上都是输入文本、输出文本。

All language tasks can be expressed as text in and text out.

不管是写诗、英译西，还是写 Python 代码，本质上都是一回事。

Be it writing poetry, translating English to Spanish, or coding Python, it's all the same.

ChargeGPT 只是把这套方法在海量数据上大规模地放大了。

ChargeGPT simply scales this up massively across lots and lots of data.

原理其实是一样的。

It's the same principle.

基础智能体接收两种输入：一个身体指令和一个任务指令，然后输出对应的动作。

The foundation agent takes as input an embodiment prompt and a task prompt and output actions.

我们通过在无数种现实环境中大规模训练它来实现这一点。

We train it by simply scaling it up massively across lots and lots of realities.

我相信，未来所有会动的东西，最终都会变得自主。

I believe in a future where everything that moves will eventually be autonomous.

总有一天，我们会发现，无论是《瓦力》《星球大战》还是《头号玩家》里的那些AI角色，不管它们出现在现实还是虚拟世界里，都不过是同一个基础智能体的不同指令而已。

One day, we will realize that all the AI agents across WALL E, Star Wars, Ready Player One, no matter if they are in the physical or virtual spaces, will all just be different prompts to the same foundation agent.

而这，朋友们，将是我们追求人工智能道路上的下一个重大挑战。

And that, my friends, will be the next grand challenge in our quest for AI.

理查德·里维斯对那些鼓吹极端男性气质的网络红人并不买账。

Richard Reeves is unimpressed by online influencers who peddle ideas about hypermasculinity.

你刚才说的是男孩和男人。

You're talking about boys and men.

我们其实已经为他们做了不少事。

We're actually doing stuff for them.

你在播客里说得头头是道，可你的政策主张在哪？

You're good on podcasts, but where's your policy agenda?

我们确实已经为男孩和男人做了很多事。

We've actually done a bunch of stuff for boys and men.

不好意思。

Sorry.

那你做了些什么？

What have you done?

关于下一代男性气质的思考。

Ideas about the next era of manhood.

接下来敬请关注NPR的《TED电台》。

That's next time on the TED Radio Hour from NPR.

欢迎收听并订阅《TED电台》，在你常用的播客平台都能找到。

Listen and subscribe to the TED Radio Hour wherever you get your podcasts.