OpenAI：当 AI 智能体使用电脑的速度比你更快时

我们现在已经让电脑使用能力达到了比普通人类更快的水平，而且我觉得我们离超越人类的水平也已经非常接近了。

We've gotten computer use now to a place where it's faster than the average human, and I think we're getting really close where it is superhuman.

你觉得哪些类型的任务已经准备好交给智能体来处理，而哪些还像是那种，嗯，还需要检查和纠正的？

What sorts of tasks do you think are ready to hand over to the agent versus the ones like, okay, it still requires some checking, correcting?

自从Astra问世以来，使用电脑的智能体现在可以做到人类能做到的任何事情。

Computer-using agents now since Astra is now capable of doing anything that a person can do.

我们花了很多心思思考，如何用更快、更好的模型来创造新的消费者体验。

We think a lot about what new consumer experiences can we enable with a faster, better model.

而在企业方面，这也自然让安全问题变得更加引人关注。

And then on the enterprise side, that's kind of naturally why the safety becomes more interesting.

我们如何提供一种能力，让人们可以控制人类参与的程度，以及如何确保可审计性？

How do we give the ability to control the amount of human-in-the-loop there is, and then the auditability?

这和我们之前构建的任何东西都不一样，开发者需要做一些工作才能采用它。

It's unlike any other thing that we've built where the developer has to do some work to adopt it.

需要有人来构建一个连接器或MCP服务器。

Someone has to build a connector or MCP server.

计算机使用现在什么都能干。

Computer use works with everything.

现在我们已经有了能力强的模型。

Now we have capable models.

它们的运行成本已经低到可以大规模部署了。

They're getting cheap enough to run at scale.

公司应该考虑自动化哪些工作呢？

What should companies think of automating?

我觉得可以分成两大类。

I think of it as in two big categories.

第一类是——

The first category is-

AI智能体现在能越来越熟练地使用我们常用的软件，点击按钮、浏览网页、完成任务，而且不需要专门的API。

AI agents can increasingly use the same software we do, clicking buttons, navigating websites, and completing work without dedicated APIs.

一旦它们在这些方面比我们更快，会发生什么变化？

What changes when they become faster at it than we are?

a16z的希玛·安布尔和法布里齐奥·塞拉菲尼，与OpenAI的阿里·温斯坦和詹姆斯·孙一起，探讨了使用计算机的智能体取得的飞速进展，以及为什么计算机使用可能成为AI的通用界面。

a16z's Seema Amble and Fabrizio Serafini join OpenAI's Ari Weinstein and James Sun to explore the rapid progress in computer-using agents, and why computer use could become a universal interface for AI.

他们讨论了智能体如何在传统企业软件、财务、采购和销售等领域自动化任务，以及推动这一进展的技术进步，包括为什么模型的配套框架能使其速度几乎翻倍。

They discuss how agents could automate tasks across legacy enterprise software, finance, procurement, and sales, alongside the technical advances driving progress, including why the harness surrounding a model can nearly double its speed.

他们还探讨了企业如何控制和审计这些智能体。

They also examine how enterprises can control and audit agents.

接下来会发生什么？

And what comes next?

AI主动在后台工作，实时与我们协作，并最终以超越专业人类专家的速度操作软件。

AI that proactively works in the background, collaborates with us in real time, and eventually navigates software faster than expert humans.

欢迎收听a16z播客。

Welcome to the "a16z" podcast.

我是希玛·安布尔。

I'm Seema Amble.

和我一起的是我的同事法布里齐奥·塞拉菲尼。

I'm joined by my colleague, Fabrizio Serafini.

八月的时候，我们写过关于使用电脑的智能体，以及它们如何开始在公司内部做真正的工作。

In August, we wrote about computer-using agents and how they're starting to do the real work inside companies.

今天，和我一起的是来自OpenAI的阿里·温斯坦和詹姆斯·孙，他们将帮助我们深入了解使用电脑的智能体目前的情况，并分享一些对未来的看法。

Today, I'm joined by Ari Weinstein and James Sun from OpenAI, who are going to help unpack what is going on with computer-using agents from their perspective and share some thoughts on the future.

阿里、詹姆斯，谢谢你们来做客。

Ari, James, thanks for joining us.

当然。

Of course.

很高兴来这里。

Happy to be here.

是的，谢谢邀请我们。

Yeah, thanks for having us.

阿里，你先开始吧，跟我们说说在Astra中，是什么让使用电脑的智能体取得了进展，真正实现了哪些突破？

Ari, kick it off and tell us a little bit about what in Astra has made computer-using agents move forward and what's really unlocked.

Astra是一个很棒的模型。

So Astra's an amazing model.

它擅长的事情太多了。

It's good at so many things.

在Astra之前，我们的计算机使用模型就已经非常出色了。

Our computer-use models were already so good before Astra.

比如Soul本身在计算机使用上就表现得很棒。

Like Soul itself is amazing at computer use.

所以根据我的经验，Astra确实在一些边缘问题上帮了不少忙。

So Astra really helps with a few things around the edges in my sort of anecdotal experience.

其中一个方面是，它平均速度更快，因为它完成同一任务时使用的思考标记更少。

One is, it is a little bit faster on average because it uses fewer thinking tokens to do the same task.

它在某些创造性工作上表现更好。

It's better at certain kinds of creative work.

比如说，在我们的发布视频里有一个演示，你在和它对话，它在Microsoft Paint上画画。

Like, for example, we had this demo where in our launch video where you were talking to it, and it's drawing things on Microsoft Paint.

它在那类事情上表现好得多，我觉得这种能力可能也会迁移到其他类型的创造性任务上。

It's way better at that sort of thing, which I think probably transfers to other kinds of creative tasks also.

最后我还发现，它在一些细节上更可靠了。

And then the last thing I find is it's just more reliable around the edges.

我感觉我更加信任它了。

I feel like I trust it a little more.

就是那种，99%可靠和97%可靠之间，有一点难以察觉但非常重要的差别，而我觉得它在某种程度上确实更高了一点，这感觉非常实在。

It's sort of the kind of thing where there's a little difference that is hard to perceive but really important between something being 99% reliable and 97% reliable, and I feel like it's sort of just a little higher in a way that feels pretty concrete.

对，而且我想补充一点，Astra 也是我们最对齐的模型。

Yeah, and I think adding on to some of those things, Astra is also our most aligned model.

说到信任的话题——

And so talking about trust a little bit-

如果你看它的系统卡片，它在抵御提示注入方面要好得多。

... if you look at the system card for it, it is much better at being resistant to prompt injection.

它更不容易采取任何非预期的行动。

It's much better at not taking any kind of unintended actions.

所以想想它在企业工作中的应用，特别是对于处理敏感事务的公司，无论是采购、与供应商沟通，还是与客户交流，你会觉得用它更安心，这真的非常令人兴奋。

And so when you think about it for enterprise work, especially for companies using it for sensitive things, whether it's procurement or talking to vendors, or talking to customers, you're going to feel a lot safer using it, and that's really, really exciting.

再来说说创意方面的一些有趣的事，我记得我们内部让Astra和5.6对战过电子游戏——

And then so some fun things on the creative side, I think internally we had Astra play video games against-

……结果每次都是它赢。

... 5.6, and it wins every time.

所以网上有一些很酷的推文，说Astra能30个小时左右通关《辐射》之类的游戏。

And so there are some cool tweets that people have made where Astra was able to beat Fallout in 30 hours or something like that.

所以说它在玩电子游戏方面也真的很厉害。

And so it's also just really amazing at playing video games.

虽然我不太确定这能转化成什么实际用途，但确实很令人兴奋。

So I know that probably translates to something, but yeah, very exciting.

关于这一点，可能稍微展开一下，消费级和企业级的电脑代理用例，在所有的辅助架构上含义非常不同——

Maybe just to double-click on that point, consumer versus enterprise use cases of computer agents, very different implications around all the scaffolding around-

……

...

我指的是那些能使用电脑的代理。

I would say the computer-using agents.

你们觉得哪里更让人兴奋——

How do you guys think about where things are more exciting-

——如果要在消费者和企业之间选的话？

... between consumer and enterprise, if you were to pick?

我觉得OpenAI的目标就是打造对每个人都实用的产品。

I think our goal at OpenAI is really about building products that's useful for everyone.

所以其实挺有意思的，因为Ari在这里确实更偏向消费者这边，而我则更多花时间在企业那边。

And so actually, it's interesting because Ari is definitely more of a consumer guy here, and then I spend a little bit more time on the enterprise side.

我认为消费者领域总是我们尝试突破可能性的地方。

I think the consumer is always the place where we try to push the frontier on what's possible.

正如Ari提到的，Astra比之前的模型快得多。

With Astra, like Ari mentioned, it's much faster compared to the previous models.

所以我们思考很多，如何利用更快更好的模型来创造新的消费者体验。

And so we think a lot about what new consumer experiences can we enable with a faster, better model.

而在企业方面，这自然让安全性变得更有趣，因为我觉得在企业中，当我们与大型公司交流时，电脑使用的用例其实相当明显，对吧？

And then on the enterprise side, that's kind of naturally why the safety becomes more interesting because I think at the enterprise, whenever we talk to large companies, I think the use cases for computer use is actually fairly obvious, right?

在企业内部，有很多传统软件，根本没有MCP工具。

Inside enterprises, there's lots of legacy software that doesn't have any MCP tools.

这些软件已经用了15年、20年了。

They are software that's been used for 15, 20 years.

所以很多时候，普通人用起来其实挺费劲的。

And so a lot of times, it's actually kind of hard to use for a normal person.

很多界面还是上个十年的。

Lots of UI that is from the last decade.

连一些关键功能你都不知道怎么操作。

And so you don't even know how to operate some of the key things.

所以我们看到有些公司开发技能，教模型更好地使用这些软件。

And so we've seen companies build skills that kind of teach the model to be able to use them better.

所以我觉得，可以这么理解：企业这边，能力可能已经具备一段时间了，公司们还在努力赶上，学着怎么应对；消费端呢，我们在探索新的用例，那些可能现在软件还实现不了的事情。

And so I think maybe the way to think about it is enterprise is where the capabilities have been there maybe for a little bit longer, and the companies are catching up with trying to understand how to deal with it, and consumers are where we're trying to figure out the next types of use cases that's not maybe possible today with software.

也许你可以多分享一下。

Maybe you can share a little more.

是的，消费者用例让我觉得特别有意思，因为就像是，你怎么让你的智能体帮你处理日常生活中的那些事情——

Yeah, I mean, the consumer use cases are so fun for me because it's kind of like, how can you put your agent to work doing the things in your daily life-

那些特别烦人的事情，能帮你省下大量时间，让你有更多时间专注于你真正感兴趣的事情？

... that are so annoying and save you tons of time and just have more time to focus on the things that you're excited about?

我觉得看Dot特别有趣，每个Dot都有自己的电脑，你可以让它在云端帮你做事。

I think it's been really fun to see Dot, where each dot gets its own computer, and you can ask it to do things for you in the cloud.

它有一个完整的浏览器，可以帮你买东西，可以帮你预约理发，还有其他你想让它做的任何事情。

It gets a full web browser, it can buy things for you, it can book haircut appointments, and anything else you want it to do.

所以这真的很有趣，而且我觉得未来还会有更多有意思的东西。

So that's been really fun, and I think there's a lot more on the horizon that's going to be really fun.

对，那正是我想问的关于Dots的问题。

Yeah, that was going to be my question on Dots.

James，我觉得你可能会用Dot来浏览产品反馈。

I think, James, you probably use a Dot to walk through product feedback.

如果你能多分享一些关于它的内容，那就太好了。

It would be cool if you could share more about it.

或者说，我特别好奇，你们那边是怎么用 Dots 的，然后计算机使用在你们用 Dots 做的事情里起到什么作用？

Or I'm just so curious to know, on your end, guys, how are you using Dots, and how does computer use play into what you can do with them?

我觉得 Dots 很酷，原因有好几个。

I think Dots is really cool for a few reasons.

第一个原因是它真的很主动，很多时候我都特别惊讶，不管是个人生活还是工作中，Dots 会突然指出某个我完全忽略掉的事情。

One is really proactive, and so I've been very surprised a lot of times when, in both my personal life and work, when Dots will just flag something that I just completely slipped through the cracks.

就在前几天，我们内部的邮件系统里，每当收到外部邀请邮件时，它其实不会出现在我们的日历上。

Just the other day, so internally, on our email system, whenever we get an external email for an invite, it doesn't actually show up on our calendars.

所以结果就是我错过了一堆外部会议。

And so I end up missing a bunch of external meetings.

现在 Dots 会帮我抓住这些情况。

And so Dots now is catching these things for me.

它就会说：“哎，你收件箱里有个邀请。

It's like, "Oh, there's this invite that's in your inbox.

它没在你日历上。”

It's not on your calendar.

你要不要我在你的Gmail里接受这个邀请，这样它就能显示出来？"

Do you want me to accept it in your Gmail so it shows up?"

所以这真是救了我一命。

So that's been a lifesaver.

虽然是小事，但非常重要。

It's a small thing, but it matters a lot.

我还可以让它主动去查看那些不太具体的任务，我觉得它在这方面做得特别好。

But I also have it proactively looking at, I think it's really good at proactively looking for tasks where the shape of it is not super, let's say, concrete every single time.

举个例子，我也设置了用它处理Slack。我们内部用Slack，消息非常多，一直在刷屏。

So an example that I also have it set up to do is we use Slack internally, and there's a ton of messages that's going on at all times.

以前我用Codex里的自动化来处理Slack任务，每半小时或一小时去提取信息，但最后我经常忽略掉很多通知，因为有时候有用，有时候没用。

And I used to have this Slack job using automations in Codex that's every hour or every 30 minutes go and try to pull things, and I've just ended up tuning out a lot of the notifications because it's just sometimes useful, sometimes a Dot.

有了Dots，我可以设定一个高层次的目标。

With Dots, I can give kind of a high-level goal.

比如像这样："嘿，如果我的汇报链里有人说了什么，就提醒我。"

It's like, "Hey, if anybody in my reporting chain says anything, flag it to me."

如果有人在给出一些关键反馈，或者对某些产品有自己的判断，那就尽管去做吧。

If anybody who's giving some critical feedback, use your judgment about certain products, go for it.

然后因为它有云电脑，我有一个挺酷的工作流程：当有人报告问题时，通常会有个 Sentry 日志。

And then because it has a cloud computer, I've had this pretty cool workflow where when someone reports, they usually have a Sentry log.

我可以用云电脑来实际操作我的本地浏览器，让 Dot 同时控制你的本地电脑和云电脑，登录 Sentry，拉取日志，分析并给我反馈。

I could have it go use the cloud computer, actually drive my local browser while the Dot's going to control both your local computer as well as the cloud computer, log into Sentry, pull the logs, analyze it, give me feedback.

这样就省掉了整个流程：判断哪些反馈重要、查看内容、去 Sentry 下载日志、再丢给 Codex 征求意见，所有这些现在都可以自动完成了。

And so it's saving that entire loop of figuring out which feedback's important, looking at it, then going to Sentry, downloading their logs, feeding it into Codex, asking for thoughts, and all that can now just be automated.

所以，是的，Dots 非常适合这种主动且长期运行的任务，它们每次的形状都不完全相同。

And so yeah, Dots is great for at least these proactive and long-running tasks where they're not the same shape every single time.

我们之前聊过一些关于可靠性、速度、令牌使用和效率的话题。

We talked a little bit about reliability, speed, token usage, and efficiency.

你们在不断改进发布内容的时候，是怎么考虑平衡这些方面的？

How do you guys think about balancing all of these things as you keep making improvements on what you're releasing?

我觉得我们对 OpenAI 的电脑使用功能非常兴奋。

I think we're really excited about computer use at OpenAI.

我们在这方面投入了很多，而且我们已经达到了一个我认为我们拥有最快、最便宜、最可靠的计算机使用产品的程度。

We've invested in it a lot, and we've gotten to a place where I think we have the fastest, cheapest, and most reliable computer use product.

而且我觉得我们还有很大的提升空间。

And I think there's actually so much further we can take that.

我们花了很多时间研究速度。

We spent a lot of time looking at speed.

速度真的很有意思，因为一方面，计算机使用的强大之处在于它能在后台执行任务。

Speed is really interesting because on one hand, the power of computer use is it can do tasks in the background.

所以在某些情况下，它花多长时间并不重要，因为你可以去做其他事情，而它同时为你完成任务。

So in some cases, it doesn't matter how long it takes because you can be going and doing other things while it's accomplishing the task for you.

但有时候，你也想看着它，确保它做对了。

But also, sometimes you want to watch it to make sure it does it correctly.

你得等它，因为它需要先完成这个，之后才能进行其他操作。

You need to wait for it because it needs to complete this before something else can happen.

所以速度非常重要，因为你不想有那种感觉："哦，我自己做可能还更快，更容易。"

And so the speed is so important because you don't want to get this feeling like, "Oh, I could have just done that myself, and it would've been so much easier."

所以我们现在已经让计算机使用这个功能达到了比普通人操作电脑还快的水平，我觉得我们很快就能达到超人类的水准，也就是说，比专业的电脑使用者还要快，可能就像我们在座的某一位那样。

So we've gotten computer use now to a place where it's faster than the average human at using a computer, and I think we're getting really close to a place where it is superhuman, meaning it is faster than an expert computer user, like probably one of us at this table.

我觉得那真的会带来翻天覆地的变化。

And I think that's going to actually be a sea change when that happens.

我觉得这会改变我们所有人跟电脑互动的方式，因为突然间，你能手动做的任何事情，它都能做得更快。

I think it's going to change the way we all interact with computers, because all of a sudden, anything that you could do manually, it could do faster.

所以除非你出于某种原因真想自己做，我觉得人们可能有很多理由想这么做，但在默认情况下，你可能更想让代理来帮你做。

So unless you really want to do it yourself for some reason, which I think actually there may be many reasons people would want to do that, but in a default case, you probably just want the agent to do it for you.

我觉得这能让我们打造出各种很酷的用户体验。

And I think that's going to let us build all kinds of cool user experiences.

所以我们经常思考速度这个问题。

So speed is something we think about a lot.

然后显然，我们的策略是提供多种不同质量和价位的模型，所以针对不同的使用场景，你可能想用不同的模型。

And then obviously we have a strategy of offering multiple different models at multiple different qualities and price points, and so for different use cases, you may want to use a different model.

同时，我们那些更便宜的模型在电脑使用方面也在不断进步，所以我觉得随着时间的推移，人们会根据任务的不同，选择使用哪种模型，这种差异会越来越大。

At the same time, our less expensive models are getting much better at computer use over time, and so I think there will actually be more variation over time in which models people want to use for which tasks.

我觉得这就是阿里刚才说的意思。

I think that's what Ari was saying.

还有一个想法是，我们怎么在安全、可靠性、自主性和速度之间，给客户更多的控制权？

I think another thought is how do we give customers control over the safety, reliability, autonomy versus speed?

如果模型很慢，你还可以看着它干活，但如果它比你慢，你就没法一直坐在这儿盯它，能做的任务也有限。

If you have-- When the model's slow, you can kind of watch it do stuff, but then you're limited to the tasks that you can do because you can't just sit here watching the thing doing it.

实际上这样做效率不高，因为它比你慢。

It's actually less effective because it's slower than you.

但是一旦你没法再去观察和审查它做的每一件事了，我们怎么能让企业，特别是企业和商家，有能力控制有多少人工介入，以及审计的权限呢？

But once you can get to a point where you can no longer observe and audit everything it's doing, how do we give, especially enterprises and businesses, the ability to control the amount of human-in-the-loop there is, and then the auditability?

我觉得几个月前，这是Astra的一部分。

I think a few months ago, it was part of Astra a few months ago.

作为Astra的一部分，我们推出了一套企业级控制，让企业可以选择，第一，有多少人工介入？

As part of Astra, we kind of launched a set of enterprise controls that let businesses choose, one, how much human-in-the-loop is there?

他们想要代理自动批准所有事吗？

Do they want the agent to automatically approve everything?

我们有一个非常棒的系统叫Guardian，或者应用里的自动审批功能，它是一个非常智能的模型，会检查电脑使用代理的每一个操作，来判断是否安全，是否符合最初的意图。

We have this really awesome system called Guardian, or I think Auto Approve in the app, that is a really smart model looking at every single action that the computer use agent is taking to decide if it's safe, if it's aligned to the original intent.

消费者可能会说，'我就想让它做所有事情，不想被要求批准。'

A consumer might be like, "I just want it to do everything, and I don't want to be asked for approval."

而企业则可以选择哪些类型的操作实际执行。

Whereas an enterprise is able to kind of decide which type of actions actually go through.

所以我认为，随着我们的电脑使用代理做事越来越超人类，控制这部分就变得越来越重要。

And so I think the control part becomes more and more important as our computer use agents become also superhuman at doing things.

完全同意。

Totally.

我觉得当你们试图推广更多企业用例时，雇主们对效率感到兴奋，但同时也担心——

I think as you try to roll out more enterprise use cases, I think employers are excited about the efficiency, but they're also wary of-

……到底释放出了什么东西。

... what's sort of being unleashed.

再说到另一边，你们有没有看到应用软件公司那边的反应，就是，现在有电脑使用代理在访问我们遗留的工作流软件。

Maybe even on the other side, have you guys seen the response from application software around, okay, now there's computer-using agents accessing my legacy workflow software.

你们有没有考虑过这个问题？

And do you guys think at all about that?

你们是不是更关注消费者或企业用户实际使用的那款智能体？

Are you more focused on just the agent that's being used by the consumer enterprise user?

是的，我觉得这个问题在开放网络上其实变得特别有意思。

Yeah, I think this problem becomes really interesting on the open web, actually.

我觉得在公司批准并使用的软件内部，实际上更直接一些——

I think when it's within software that is approved and used by the company, it's actually more straightforward-

因为就像，当然啦，我们百分之百希望大家能在里面自动化工作，不用自己去做那些繁琐的事情。

... because it's like, oh yeah, 100% we want to have ... people automate and do work in here that they don't have to do tediously themselves.

但一旦牵扯到互联网，我觉得就有几类不同的人，对吧？

But when it comes to the internet, I think you have a few different camps of folks, right?

一类是，我们的商业模式并不支持智能体来使用我们的产品，对吧？

One is our business model is not in line with having agents use our product, right?

你想想广告商、发行商，他们的商业模式完全依赖广告。

You think about advertisers, publishers whose business model is entirely on advertisement.

如果智能体替他们使用网站，那广告收入从哪来呢？

If the agent's using the website for them, then where's the ad revenue coming from?

我觉得在大多数地方，智能体的广告展示其实并不算真正的收入。

And I think most places, a agent's ad impression does not actually count towards real revenue.

所以这是个很有意思的问题，我们一直在和合作伙伴团队一起探讨。

And so that's an interesting question that we're always working through and working with our partnerships team on.

还有另一个问题，就是对于那些确实想拥抱计算机使用智能体的公司，很多都在采用一些标准，让人类和智能体可以一起使用网站。

There's another question around, for the companies who do really want to embrace having computer-use agents using their sites, a lot of them are adopting standards that let humans or people and agents be able to use it together.

比如WebMCP这个标准，我们就非常看好。

So, like WebMCP is a standard that we're really excited about.

我们已经在应用浏览器里加了支持。

We added support for it in the app browser.

所以你可以想象这些场景，智能体提升了网站的使用体验，但并没有完全取代它。

And so you can imagine these scenarios where the agent is elevating the experience of using the site, but is not entirely replacing it.

所以我个人觉得，上网浏览依然很有趣，对吧？

So, I think browsing the internet is still very fun, in my opinion, right?

我还是很喜欢购物。

I still like doing shopping.

我还是喜欢看看各种酒店、航班和地点，所以你可以想象这样的场景：智能体把所有繁琐的工作都做了，但给你留出一个更轻松的体验，让你自己来浏览，然后你会说：“好的，现在我要完成交易，然后输入我的信用卡信息，做这些事。”

I still like looking at all the hotels and flights and places, and so you can imagine places where the agent does all the tedious work, but it carves out an easier experience where you're doing the browsing, being like, "Okay, now I want to finish the transaction, and then put in my credit card and do all this stuff."

然后再把它交回给智能体。

Then you can hand it back to the agent.

所以WebMCP是一种有趣的方式，你与网站的交互仍然是网站本身，而不是API，但人机可以一起使用。

And so WebMCP is an interesting way where your interface with the website is still the site itself versus an API, but then the person is using it together.

这些就是一些想法。

And so those are some thoughts.

换个话题，我想稍微深入探讨一下，完成一项工作的真正含义——

Shifting gears, I want to double-click a little bit on what it means to do the job or-

...就是如何把整项工作从头到尾做完。

... for get the end-to-end job done.

我们很想听听你们的看法，哪些方法比较有效，你们怎么界定合适的、窄范围的、可重复的任务——

We'd love to hear a little bit from you guys around what works well, how do you guys define the right narrow, repeatable task-

你觉得哪些任务可以交给智能体来做，哪些任务还是需要用户在一边检查和纠正呢？

... and what sorts of tasks do you think are ready to hand over to the agent versus the ones like, okay, it still requires some checking and correcting, on the user side?

总的来说，我发现现在的模型基本上什么都能在电脑上做了。

I mean, generally, I've found the model to be pretty capable to do anything that you can do on a computer now.

我经常逼自己做一个练习，就是每当我有一个任务，我第一反应就是，能不能直接让智能体去做？

And one exercise that I try to force myself to do is that whenever I have a task, the first thing I think about is just, can I just have the agent do it?

当然，有些情况下我会比较谨慎，比如如果我要给别人发消息，我希望那感觉像是从我这里发出的。

Now, there are obviously some sensitivities around if I'm sending a message to someone, I want to feel like it's coming from me.

但总的来说，我觉得自从Astra推出以来，能操作电脑的智能体已经能做人能做的任何事了。

But in general, I think the way I think about it is computer-using agents now since Astra, is now capable of doing anything that a person can do.

之前有人会担心，比如任务的具体性和要求方面存在一些模糊不清的地方。

And there are questions around, I think before of like, oh, there are some squishiness around specificity and requirements.

我发现，现在当你让模型做什么事，如果描述得不够清楚，它会主动向你提问，就像我们推出Astra时引入的灵活输入功能，对吧？

I've generally found that the model is getting pretty good at whenever you're asking it to do something and there is some vagueness, it will actually, with Astra, I think we launched this flexible input, right?

所以它通常会问你，“嘿，我需要一些细节。”

And so we'll actually ask you about, "Hey, I want some details."

如果你没有及时回答，它就会继续往下做，然后在需要的时候再获取上下文。

And if you don't answer it in time, it'll just keep going, and then pull the context as necessary.

我觉得有些方面，如果你有重复性的工作流程，最好能够定义需求，至少类似于写产品规格说明。

I think there are some things around, if you have repeatable workflows, it's nice to be able to define the requirements are at least similar to writing a product spec.

但总的来说，除非有特别敏感的事情需要你全程参与，我现在几乎什么都用电脑使用模型来做。

But in general, I generally feel like unless there is something incredibly sensitive that you want to be in the loop for, I just reach for the computer-use model for everything I do now.

我很好奇你是不是也这样。

I'm curious if it's the same for you.

是的，我确实是这样。

Yeah, it is.

我正在努力培养这种习惯，就是遇到事情先想到用它，

I'm trying to build that impulse to reach-

因为它太好用了，而且有时候很容易陷入

... for it because it's so good, and sometimes it's so easy to get locked into the-

老的做事方式里。

... old way of doing things.

我觉得计算机使用这项技术最酷的地方在于它的普适性。

I think the really cool thing about computer use as a technology is the universality of it.

它适用于所有应用程序。

It works with every application.

这和我们之前构建的任何东西都不一样，那些都需要开发者做额外工作来适配。

It's unlike any other thing that we've built where the developer has to do some work to adopt it.

必须有人去构建连接器或MCP服务器。

Someone has to build a connector or MCP server.

而计算机使用则适用于一切。

Computer use works with everything.

它的工作方式和人使用应用程序的方式一样，如果你还没怎么接触过这项技术，我觉得这一点值得好好理解。

And it works by using the application the same way a human uses it, which I think is something worth understanding if you haven't spent much time with this technology.

计算机使用就像人一样操作电脑。

Computer use uses computers the same way a human does.

它让智能体拥有和你一样的所有能力，我觉得这非常强大。

It makes the agent have all the capabilities you do, which I think is so powerful.

对，而且我想补充Ari说的，部分原因是我慢慢不再想，这是不是计算机使用任务，或者不是计算机任务。

Yeah, and I guess adding on to Ari saying, part of it is, I think over time I've stopped thinking about it as like, am I using them on a computer-use task or a non-computer task?

很多时候它们是关联的，对吧？

Frequently, they're connected, right?

我们对计算机使用的看法是，它不是你要拿起来然后说“我想用它做点什么”的东西。

The way we think about computer use is that computer use is not a thing you pick up and just say, "I want to do something with it."

它实际上是模型用来做任何事情的通用API。

It's really the universal kind of API that the model has to do whatever it takes.

所以如果你有一个雄心勃勃的任务，想交给代理去做，很可能任务越复杂，它在过程中就越有可能涉及到计算机使用。

And so if you have an ambitious task and you want the agents to do it, likely the more complex it becomes, the more likely it will probably touch computer use at some point in that journey.

所以也许唯一我不去使用计算机使用的情况，就是当任务太简单了，我更愿意自己点两下按钮就搞定。

And so maybe the only time that I don't reach for computer use for some cases is if the task is so simple that then it's easier for me to just go and click two buttons and do it.

但随着计算机使用变得更快，甚至更快，那问题就变成了，也许我不需要自己去做。

But as computer use gets faster, even faster, then the question becomes, maybe I don't need to do that.

我只需要语音驱动，它就会和我一起使用计算机。

I just drive voice, and it just uses the computer with me.

所以这些想法也挺有意思的。

So those are some interesting thoughts there, too.

你觉得你有没有什么要跟企业用户分享的——

Do you think there's anything you have to share with an enterprise user-

……来让他们觉得放心呢？

... to get them comfortable with that?

很明显，你们完全是建立在计算机使用上的，但是——

Obviously, you guys are totally built on computer use, but-

……

...

我觉得有时候企业用户确实会有些犹豫——

I think there's obviously hesitance sometimes on-

完全同意。

Totally

……企业用户会想：好吧，是，人类点击操作可能会更差，但总觉得这样有责任归属的问题——

... the enterprise user of like, okay, well, yes, the human would click through it and probably do it worse, but there is that feeling of there's accountability-

而且如果那个人受过训练，再加上那些典型的反驳理由。

... and if that person's trained, and all the classic kind of rebuttals.

我很好奇——

I'm curious-

你需要说服大家什么呢？

... what you have to convince people around.

当然。

For sure.

最近跟很多企业聊过，我觉得最关键的是，我们把它分成两层来看，对吧？

Lots of conversation with enterprises over the last little bit, and I think the main thing is we think about it as two layers, right?

第一层就是，我们怎么建立确定性的控制？

The first layer is, how do we create deterministic controls?

我觉得这确实能让企业感到安心，就是当他们能——即使我们的模型是有史以来最对齐的，而且在防范提示注入和意外情况方面，我认为它比大多数人都做得好，因为它的智能非常高。

I think that's really what gives enterprises a lot of feeling of safety, is when they can-- even though our model is the most aligned ever, and it's I think better than most people at protecting against prompt injections and unintended things because the intelligence is so smart.

但我觉得，当企业管理员能基本上确定某些任务或某些数据源太过敏感而不能使用时，那种安全感就会油然而生。

But I think there is just a feeling of safety when the enterprise admins can basically say for certain tasks or for certain data sources, it's just too sensitive to be used.

所以我们就看到了企业方面的需求——所以我们发布了两样东西，对吧？

And so we see kind of enterprise-- So what we launched was two things, right?

一个是能够为你的智能体创建一个确定性的沙箱环境。

One is the ability to be able to deterministically create a sandbox around your agent.

也就是说，可以设定它能用哪些应用，不能用哪些应用，能访问哪些网站，不能访问哪些网站。

So being able to say it can use certain apps, it can't use other apps, it can use certain websites and not certain other websites.

第二个就是人类参与监督的环节。

And then two is this kind of part of a human in the loop.

哪些类型的操作需要人工介入呢？

What type of actions does it require a human to be intervened?

比如说，任何下载和上传操作，都需要人来批准。

And so we might say, okay, for any time there's a download and upload, a person has to kind of approve it.

每次访问新网站，也需要人来批准。

Any time it goes to a new website, a person has to approve it.

当然，这些都是企业可以自行调节的设置。

Now, again, these are all dials that enterprise can kind of go through.

所以这其实是一方面，另一方面就是向企业管理员展示我们内部运行的所有评估，看看这些模型有多对齐，甚至和内部及第三方评估相比。

And so it's really like that's one, and then two is really just showing the enterprise admins just how all the evals that we've run internally around how aligned these models are, even compared to both internal and kind of third party evals.

我觉得我们还有一些基础工作，是为了帮助人们对这类产品建立信任。

I think there's also some baseline stuff that we do across the board to just try to help people build trust with these kinds of products.

我们知道这太新了，对人们来说既神奇又有时会觉得有点陌生。

We know it's so new, it's such a magical and sometimes alien experience for people.

所以我们在产品设计上下了很多功夫，比如你可以随时看到光标，如果你想看它执行任务，它会以有趣的方式动起来，这样看起来不会无聊。

And so, we put a lot of work into the design of the product, so you can always see the cursor if you want to watch it perform a task, and it sort of animates in a fun way, so it's not boring to watch.

然后，就像詹姆斯说的，这对企业来说非常可定制，但对普通用户来说，我们在使用新的应用或网站前会征求你的许可，这取决于你所处的模式。

And then, like James was saying, this is very customizable for enterprises, but also for a regular user, we ask you for permission before using a new application or website, depending on which sort of modes you're in.

我觉得这有助于人们不会觉得它会在电脑上乱来。

And, I think that helps people sort of not feel like it's going to go do some runaway stuff on their computer.

你确实需要这种建立信任的时刻，先弄清楚它的感觉以及你希望它为你做什么，然后随着时间推移，你会觉得更自在。

You kind of want to have this trust-building moment where you figure out what it feels like and what you want it to do for you, and then you sort of feel more comfortable over time.

Codex的光标真可爱。

The Codex cursor is so cute.

我超喜欢这个。

I love it.

我知道这是个很小的细节，但看着它操作电脑会让人很有信心，也让整个体验变得更舒服。

I know it's a small detail, but watching it use the computer gives you confidence, and it makes the experience nice.

所以我觉得在用户体验方面，这确实是个值得投入的点。

So I think there is a kind of user experience point that's definitely worth investing on.

我很好奇，从更宏观的角度看，我们现在已经拥有能力很强的模型了。

And I guess I'm curious, in a broader sense, okay, now we have capable models.

它们的运行成本也低到可以大规模使用了。

They're getting cheap enough to run at scale.

在你看来，企业应该考虑把哪些工作自动化呢？

In your view, what should kind of companies think of automating?

你觉得在企业内部，哪些用例最有前景？

Where do you see the most promising kind of use cases within enterprises?

有没有一些工作流程，比如和ERP相关的，或者关于人类和智能体如何协作完成任务的那些流程？

Are there some workflows, maybe something around the ERP, maybe something around kind of how, again, humans and agents can work together to get tasks done.

但如果让你选，你最希望看到你的模型在哪些地方发挥作用？

But if you had to pick where are you most excited to see your models perform in the world?

从我的角度看，我很想听听阿里的想法，我觉得可以分为两大类。

I guess from my perspective, I'd love to hear Ari's, I think of it as in two big categories.

第一类是自动化那些繁琐的工作流程。

The first category is automating tedious workflows.

这个比较无聊。

That's the more boring one.

然后我给你举一些无聊的例子，比如——无聊就是机会。

And then I'll give you some boring examples like- Boring is opportunity.

无聊就是机会。

Boring is opportunity.

对，没错。

Yeah, exactly.

比如企业资源计划系统就是一个很大的领域。

Like ERPs is a really big one.

所以你能想到的所有那些缩写，ERP、CRM——所有那些系统记录。

And so all the acronyms you can think of, ERPs, CRMs- All the systems of record.

每一个系统记录，对吧？

Every single system of record, right?

所以我们见过采购软件这类东西。

And so we've seen folks, procurement software.

你要去找供应商，每年都要订购一堆实验室设备。

You're going to vendors, and every year you're ordering a bunch of lab equipment.

而不是用那个有上千行的巨大表格，然后运营团队去处理它……

Instead of having this giant spreadsheet with 1,000 rows that then the ops team...

我们跟一家公司合作过，他们通常的做法是，团队自己填写一个巨大的表格。

So it's worked with one company, it's like normally what they have done is they have this giant spreadsheet that gets filled out by the teams themselves.

然后运营团队拿到这个巨大表格，录入内部的采购软件，最后生成一堆订单条目。

Then an ops team takes that giant spreadsheet, puts it into an internal procurement software, and then that gets finalized as a bunch of lines.

接着采购团队拿着那个软件的输出去联系供应商。

And then a procurement team then takes that giant software output and then goes out and reaches out to vendors.

电脑可以自动完成所有这一切，对吧？

Computers can automate all of that, right?

所以采购是我们看到的一个大应用场景。

And so procurement is a big one we see.

财务是另一个我们看到的非常有趣的领域。

Finance is another really interesting category we've seen.

公司内部有很多不同的账本，对吧？

And so, there's a lot of different ledgers within a company, right?

有费用账本，还有其他各种账本。

And so you have ledgers for expenses, ledgers for all these things.

所以到了月底，会有很多对账工作。

And so at the end of the month, there's a lot of reconciliation.

而且很多财务软件没有API，不能快速提取数据。

And a lot of these financial software don't have APIs to be able to quickly pull things.

所以财务团队如果能用电脑来查看所有这些账本，进行对账，找出任何异常，标记给相关人或进行调查，那会很好。

And so the finance team being able to have computer use, look through all these ledgers, reconcile, pull out anything that's interesting, to flag either to a person or then do some investigation.

这也是个很无聊但很实用的流程。

Also a very boring but useful workflow.

对于销售团队来说，很多销售团队会用好几种不同的软件来处理转录、获取数据之类的各种事情。

For their sales team, a lot of sales teams uses a lot of different software for transcripts, for getting data, all these kind of things.

所以让计算机自动去发掘潜在客户，然后把这些信息输入到CRM或其他外联软件里，等等等等。

And so having computer use go and do prospecting and then going and putting that into a CRM or some kind of outreach software and then all that.

所有系统记录都是那个无聊的部分。

So all the system records is the boring part.

还有市场调研也是我们经常见到的一个领域。

And then market research is another one that we see a lot.

所以呢，在空闲时间里，我一直在做一个羽毛球视频标注软件。

And so, in my free time, I've been building this video annotation software for badminton.

我经常打羽毛球。

I play a lot of badminton.

我喜欢录下自己打球的视频，然后让代理去分析这些视频。

And I like to film myself playing the game and then having the agent kind of go and look at stuff.

然后我就想，好吧，我该怎么让这个东西真正有用呢？

And I was like, okay, how do I make this thing really useful?

市面上已经有一些针对匹克球、壁球和网球的软件，所以我就让电脑自动登录这些软件，创建账号，试用一下，截图给我反馈，然后再试着把这些功能整合回我自己做的应用里。

There's some software out there for pickleball, for squash, for tennis, and so I just had computer use, go log in, create an account on these softwares, try it, take screenshots, give me feedback, and then try to build it back into the app.

所以说，这些都是可以交给电脑去做的繁琐工作。

And so these are all kind of tedious workflows that could be done.

然后我觉得，更有意思的创造性工作，是让更宏大的项目成为可能，比如我们刚才提到的，模型现在在视觉设计上已经非常出色。

And then I think some of the more interesting creative work of enabling more ambitious work is, we talked about the model now being really good with visual design.

所以，电路设计、建筑工作、硬件工程、游戏设计，这些也是我认为Astro能够大展身手的领域，我们也非常兴奋。

And so, circuit design, being able-- Architecture work, hardware engineering, game design, these are also areas that I think we're really excited about where Astro can take.

现在企业不太会考虑这些，因为他们都在关注那些比较枯燥的机会。

Less common for enterprises to think about those right now because they're thinking about the boring opportunities.

但我相信，随着大家越来越了解模型的能力，我们很快就会看到更多这类应用，不过——我很好奇你们有什么想法。

But I think as folks understand the capabilities of the model more, that's where, I think we'll kind of see more usage next, but- Curious for any thoughts.

抱歉。

Sorry.

对，我觉得在消费者这边也差不多，有很多事情就是自动化那些繁琐的活儿，或者就是……

Yeah, I think kind of similarly on the consumer side, there's a lot of things where it's sort of automating drudgery or just-

……做那些你不想做的事情。

... doing things you don't want to do.

比如帮你填车管所的表格。

It's filling out DMV forms for you.

帮你退订收件箱里一堆邮件。

It's helping you unsubscribe from a bunch of emails in your inbox.

然后我觉得还有一些事情更像是，我听到很多人分享说，他们做了那些本来没时间或者根本不会想到去做的事情。

And then I think there's also things that are more like, I'm hearing from a lot of people stories of people doing things that they just wouldn't have had time or wouldn't have thought to do otherwise.

比如，我自己就在维护关于财务和饮食计划的电子表格——

Like, I don't-- for me personally, I'm maintaining spreadsheets about my finance and spreadsheets about meal planning-

是的

Yep

……而且我有一个朋友是私人教练，他正在为他的客户定制一些东西。

... and I have a friend who's a personal trainer who's making customized things for his clients.

而且这些事情中有一些在自己的个人生活中做起来会非常费时，但有了这个就变得特别容易。

And some of these things would just be so time-prohibitive to do in your personal life, but this makes it so easy.

我用的是苹果的 Numbers，这算是个比较冷门的表格软件，但模型依然表现得非常出色。

I use Apple Numbers, which is like a obscure spreadsheet program, but the model still knocks it out of the park.

我好像从来没听人用过苹果的 Numbers。

I don't think I've ever heard someone use Apple Numbers.

它其实是个出乎意料好用的表格软件。

It's a surprisingly great spreadsheet software.

然后我觉得还有第三类东西，这也是我最兴奋的，就是刚刚开始并即将出现的，那就是用一种全新的方式来使用你的整个电脑或界面——

And then I think there's a third category of thing which I'm kind of the most excited about, which is just starting, and I think on the horizon, which is really just having a different way to use your entire computer or interface-

——与软件交互。

... with software.

我觉得你在我们的 Astra 广告中看到了很酷的例子，人们只是用声音和电脑对话。

I think you saw in our Astra ad really cool examples of people just talk to their computer with their voice.

这是一种非常自然的电脑使用方式，而且我觉得随着时间推移，我们会看到软件本身的形态因为这项技术而改变，我对此非常期待。

It's such a natural way to use your computer, and I think over time we're going to see the form factor of software itself change due to this technology, and I'm really excited about that.

说到这个，软件形态的变化，嗯，确实是个有意思的话题。

On that point, okay, form factor of software changing.

想象一下，如果你是企业用户，你习惯了用那种三十年前建的笨重的ERP系统。

Imagine if you're an enterprise user, you're used to using whatever clunky ERP from, it was built 30 years ago.

现在有一种可能，你放出一群智能体去操作那个软件，你根本不需要操心。

There's a world in which you now unleash a set of agents that just operate that software, and you don't really care.

另一种可能呢，就是有新的AI原生的玩家，他们用计算机操作智能体——

There's another world in which folks are a new AI-native player is using a computer-using agent-

对。

Yep

……所以我想知道，你觉得世界会更倾向于保留传统系统，然后在上面加一层智能体，还是说计算机操作智能体可以成为新软件玩家的核心部分？

... and I'm curious if, do you think the world is going to go more in the direction of keep the legacy folks and just have this layer on top, or do you think computer-using agents can absolutely be part of new software players?

这个问题我其实想了很多，所以很想听听你们的看法。

And this is something I think a lot about, so I'm curious to get your guys' thoughts.

这确实是个有趣的时刻，因为同时，创建新软件的成本和复杂度都大大降低了，所以你——

It is such an interesting moment where simultaneously the cost and complexity of creating new software is so much lower, so you-

可以想象，软件会发生巨大的变化。

... can imagine software changing so much.

但同时，世界上还存在着大量的传统软件。

But also there exists so much legacy software in the world-

而计算机使用（代理）非常适合与这些传统软件交互。

... that computer use is so well-suited to interact with.

所以我认为这两种情况都会持续很长时间。

So I expect both of those things to continue for a long time.

你看看今天正在运行中的软件，很多都是十年前、二十年前、三十年前甚至四十年前编写的，我不认为这些会一夜之间消失。

If you look at the software that's in production today, a lot of it was written 10, 20, 30, 40 years ago, and I don't think that that stuff's going to go away overnight.

我确实认为，从某种意义上说，计算机使用（代理）的存在可能会给企业带来更多压力，促使他们让软件与代理更好地配合。

I do think that in some way the existence of computer use may put more pressure on companies to make software work better with agents-

因为所有软件默认都会与代理协作，我认为WebMCP就是一个很好的例子。

... because all the software is going to work with agents by default, and I think WebMCP is a great example of this-

而且我觉得未来还会看到更多类似的东西。

... and I think we'll see more stuff like that.

所以是的，我觉得两者都会非常重要，但我确实认为它会随着时间而变化。

So yeah, I sort of expect both to be super important, but I do think it'll kind of shift over time.

你怎么看，詹姆斯？

What do you think, James?

是的，我同意你的看法。

Yeah, I agree with you.

我觉得这里面有很多手艺和工艺，你明白这一点，因为阿里和他的团队打造了一款精美的产品。

I think it's like there's a lot of arts and craft, and you understand this because Ari builds a beautiful product with his team.

那个光标设计完全是他做的。

The cursor was all him.

所以我认为，打造人们能够握在手中的伟大产品，确实需要很多艺术和工艺。

And so I think there's a lot of art and craft into building great products that people can hold in their hand.

而且我不知道，对我来说，如果互联网变成了一个无脸的东西，只是智能体之间互相交流，人们不再使用任何东西，我会非常难过。

And I don't know, for me, if the internet becomes just this faceless thing that is just agents talking to each other and people are not using anything anymore, I would be very sad.

而且我不认为会是那样，因为工艺和匠人软件确实让人用起来感觉很舒服，因为设计它的人真的、真的很理解客户想要拥有的体验。

And I don't think it will be like that because there is something about craftsmanship and artisan software that just feels nice to use because the person who designed it really, really understands the experience that the customer wants to have.

我老婆最近开始玩一个棋盘游戏。

So my wife recently got into playing this board game.

我忘了那游戏叫什么名字了。

I forgot the name of it.

就是那种五子连珠赢的游戏。

It's where you connect five pebbles to win.

好像是叫五子棋吧。

I think it's like Omok is the name of it.

反正她因为跟朋友在线玩，想练好一点，但老是被虐。

And so anyway, she's trying to get good at it because she's playing online with friends, and she's getting her ass kicked.

于是我说：“行，

And so I was like, "All right.

我给你做个战术训练器，练练不同的开局，让你变厉害。”

I'm going to build you a tactical trainer for going through different gambits so you can get better at it."

比如这就是我做的第一个版本。

That's, for example, something that I created a first version.