本集简介
双语字幕
仅展示文本字幕,不包含中文音频;想边听边看,请使用 Bayt 播客 App。
当你让眼睛执行的任务呈指数级增长时,就会开始出现严重的问题。
As you're exponentially doing more things with the eyes, you're going to start having really bad actions happen.
最近我们就看到一些案例,比如智能代理不小心发布了本不该公开的代码和密钥。
And we've seen some of that happen lately with agents accidentally publishing code and tokens that they weren't supposed to.
企业们已经明显意识到,这种风险正在成倍上升,而且他们根本没法阻止这种趋势的蔓延。
Like, definitely, enterprises are starting to realize that risk has grown exponentially and that they don't have any way to stop the adoption.
现在他们必须采取措施,降低这些代理行为出现非法或错误操作的可能性。
They just now have to do something to reduce the chance of these agent actions being illegitimate or incorrect.
我们当然可以参考大量历史上代理的行为数据,但如今的企业并不愿意把这类数据交给Anthropic或OpenAI,因为他们清楚,这些公司对数据极度渴求,肯定会拿去训练模型。
But we're allowed to look at a lot of historical data of how these agents have behaved, but enterprises today are not willing to have Anthropic or OpenAI give that historical data because they know these are very data hungry companies that will want to train on that data.
大家好,各位听众。
Hi, listeners.
欢迎回到《NoPriars》节目。
Welcome back to NoPriars.
今天我邀请到了Onyx Securities的联合创始人兼首席执行官马克西姆·巴·科根,这家公司位于以色列,由一群研究人员、数学家和工程师组成,专门开发用来监控AI代理的系统。
Today, I'm here with Maxim Bar Kogan, the co founder and CEO of Onyx Securities, an Israel based startup of researchers, mathematicians, engineers building agents to watch the AI agents.
我们会聊聊专用模型训练、理念体系、对齐研究,以及以色列在安全和人工智能领域的生态。
We talk about specialized model training, mythos, alignment research, and the Israeli ecosystem in security and now AI.
欢迎。
Welcome.
马克斯,非常感谢你参与这次对话。
Maxim, thanks so much for doing this.
谢谢。
Thank you.
很高兴能来。
Pleasure to be here.
现在大家对安全问题,以及人工智能对安全的影响,比几个月前关注得多多了。
Everyone is much more concerned about security and the impact of AI on security than they were certainly a few months ago.
两年前你刚创办公司时,大家普遍认为的风险就是聊天机器人版的数据防泄漏。
The consensus risk story two years ago when you started the company was basically like DLP for chatbots.
员工到底往ChatGPT里输入了什么?
Like, what are employees putting into ChatGPT?
现在的情况明显不是恐慌,但已经接近全市场范围的恐慌了。
Now we have clearly something that is not quite panic, but close to market wide panic.
你们刚开始的时候,是怎么决定押注智能体行动的?
How did you decide to bet on agent actions when you started?
我觉得对我们来说,关键转折点是 AutoGPT。
Look, I think for us, the pivotal point was AutoGPT.
AutoGPT 让大家的想象力,包括我们自己的,彻底放飞了,因为它
I think AutoGPT kind of let everyone's imagination, including ours, run wild because it was a
你能跟听众们再讲讲那是什么吗?
Can you remind listeners what that was?
当然可以。
Sure.
AutoGPT,虽然我不太清楚背后是谁开发的,但我绝对是它的铁粉。据我所知,他们是第一个打造出真正自主智能体的人,这个智能体基于大语言模型运行。
So AutoGPT, and I'm sorry if I don't know the guy behind it, but a huge fan, they created the first, as far as I know, the first really autonomous agent running on LLMs, right?
也就是说,这个智能体能让大语言模型不只是生成文字,而是决定该做什么,然后通过 API 接入工具去执行那些操作。
So, an agent that would let LLM not generate text, but decide what to do and then give that agent an API access to do that thing, a tool to do it.
然后我们就这么循环执行。
And then we'd do that in a loop.
理论上,这能让代理完成非常复杂的事情,任何人在电脑上能做的,它都能做。
So, it basically, in theory, could let agents do very complicated things, anything a person could do on a computer.
当然,当时实际效果并不好,太早了,模型还不够强,GPT-4也还没达到那个水平,但我觉得它让大家看到了一个可能的未来——如果模型足够强大,用同样的结构,我们就能拥有能力超强的代理替我们做事。
Now, granted it didn't work that well, it was too early, the models were not good enough, GPT-four was not good enough, but I think it did give everyone a glimpse into the future of what if the models were good enough, and then basically using that same structure, we could have very capable agents doing stuff for us.
我觉得,如今的Cloud Code在很多方面,跟当年的AutoGPT其实很像。
I think that was, in many ways, cloud code today is not dissimilar to AutoGPT back then.
他们可能又有点太早了,模型还没准备好,但方向是对的。
I think they were a bit early again, before the models were ready, but the concept was right.
让我印象最深的是,那时候我就已经看得两眼发直了。
And the thought that sticked with me was I was very eye pilled even back then.
我当时就在想,天啊,模型迟早会比我们聪明得多。
So, I was thinking, oh my god, models are gonna be way smarter than us.
等那一天真的来了,我们该怎么去监管那些比我们还聪明、能力还强的代理呢?
When that happens, how do we oversee these very smart agents that are smarter than us, they're very capable?
我们怎么才能放心让他们替我们做事呢?尤其是当他们开始管理一些至关重要的事务时。
How are going to feel easy about them doing stuff for us, especially when they start managing really important stuff.
有一天,他们可能在管理你的供水、供电,甚至整个电网。
One day they're managing your water supply and your electricity, your power grid.
对吧?
Right?
那你怎么去控制他们呢?
How do you control them?
那种想法一直萦绕在我心头,让我特别着迷。
And that was like the thing I was kind of obsessed about that thought.
不过我当时也太早了。
I was also too early.
那时候,企业根本还没在用任何智能代理。
So, I think at the time, enterprises were not using any agents.
市面上几乎看不到什么智能代理。
There were hardly any agents out there.
那时候我跟很多安全产品的买家聊,他们都说:兄弟,你太早了。
And talking with a lot of security buyers at the time, they were like, oh, dude, you're way too early.
这事儿根本不会发生。
Like, this is not something that's gonna happen.
我问过
I asked
你这个问题我也问过。
you the same question.
我说,你在没钱之前,真有人会做这事儿吗?
I said, is anyone going to do this before you run out of money?
我觉得我确实很可能在那之前就没钱了,因为你说得没错。
I think there was a good chance that I would've run out of money before because I think you were right.
当时确实有点运气成分,但后来市场真的出现了。
Like, I think there was an element of chance here, but then I think the market did happen.
突然间,出现了能处理长期任务的推理模型。
So, we had suddenly reasoning models that could do long horizon tasks.
我们推出了Cloud Code,它成了第一个被广泛使用的自主代理,之后又推出了Cowork和OpenCLO。
We had Cloud Code, which became the really first widely used autonomous agent, and then we had Cowork and OpenCLO.
现在我们开始看到,尽管当初人人都不敢做这种高度自主的代理,但大家都转而去搞那些功能受限的低代码平台,全靠连接器拼凑,结果这些平台能力太有限,根本没带来预期的效率提升。
And I think we're starting to see now that these types of agents that are very autonomous, even though everyone was afraid to build them, everyone started building these low code platforms that were much more limited, much more based on connectors, those platforms ended up being quite limited, so we didn't get the productivity gains from those limited platforms.
但当我们真正开始享受到这些完全放开的代理带来的惊人好处时——它们能干各种活,内置的限制少得多,连大型企业都决定要采用它们。
But when we started getting the crazy benefits from these very unleashed agents that could do everything, that had much less controls baked into them, and even very large enterprises decided they're gonna adopt it.
实际上,Anthropix的收入主要来自企业客户,他们花钱让Cloud Code来完成过去由开发者手动做的大量工作。
Know, Anthropix revenue is coming from enterprises that are paying for cloud code to do a lot of the work that developers used to do.
这大概就是我们当初的起步方式,幸运的是,高度自主的代理出现得恰到好处,没让我们等太久。
That was a bit of kind of how we started, and we definitely were in luck that very autonomous agents appeared before it was too late.
所以你能简单说说吗?我觉得在当前这个AI时代,要理解‘部署’到底意味着什么,以及能力上发生了哪些变化,几乎是不可能的,但又特别重要。
So can you describe a little bit just because it's, I think, both close to impossible and then very useful in this period of AI to think about what is deployment right now and then, you know, what's changing about capability?
Onyx产品现在到底能做什么?用一句话概括,再谈谈你对长期愿景的思考?
What's the one liner on what the Onyx product does today, and then, like, how you think about long term vision?
现在,Onyx主要就做两件事。
Today, like, Onyx is really does two things.
第一,我们训练模型并构建能够监督其他代理的系统。
Number one is we train models and build agents that can oversee other agents.
这样做的目的是确保我们能识别出,这些我们正在采用的AI所执行的大量操作都是合法的,因为这类操作的数量正在呈指数级增长。
And the goal of that is to say, okay, we need someone to be able to tell that all of these actions that are now happening by these AIs that we're adopting are legitimate because the number of these actions is growing exponentially.
过去我们认为有用的方法,比如人工介入,现在根本行不通了——因为操作量可能增加一百倍、一千倍,甚至上百万倍。
And so, things that we thought might be useful in the past, like a human in the loop, now that you're going to have 100x, 1000x, a million x of these actions, that's not going to work.
我们把这个能力产品化,推出了一个叫AI控制平面或安全AI控制平面的系统,帮助客户发现所有正在运行的AI和自主代理,把它们接入Onyx,实时监控这些AI的行为,避免随着AI任务量激增,出现严重的错误操作。
And then we take that capability and we basically productize it in a product that we call the AI control plane or the secure AI control plane, where we come to Interpret and say, hey, let's find all of your AIs and autonomous agents and hook them up to Onyx to this system where we can oversee what your AIs are doing, so that you don't run into the risk of, as you're exponentially doing more things with AIs, you're going to start having really bad actions happen.
最近已经发生过一些案例,比如代理擅自行动导致服务中断,或者不小心发布了不该公开的代码和密钥等等。
And we've seen some of that happen lately with downtimes that were caused by agents doing their own thing, agents accidentally publishing code and tokens that they weren't supposed to, and so on.
企业现在终于意识到,这种风险已经呈指数级上升,而且他们根本拦不住AI的普及。
So, definitely, enterprises are starting to realize that that risk has grown exponentially and that they don't have any way to stop the adoption.
所以,他们只能想办法降低这些代理行为出错或不合法的可能性。
So, they just now have to do something to reduce the chance of these agent actions being illegitimate or incorrect.
对。
Yeah.
我觉得基础模型团队之所以大力投入代码领域,是因为代码本身能力超强,理论上迟早能实现所有软件能做的事。
I think one of the core reasons, obviously, the foundation model labs are going after code is because it is very powerful in general and can do, you know, in theory, all things software can over time.
但反过来,也正是因为它能做所有软件能做的事。
The flip side of that is it can do all things software can.
对吧?
Right?
我自己的经历就挺典型的,之前对代理太宽松了,结果它直接把数据永久删了,还搞得我们返工。
And so joyously am already in the camp of a having been over permissive with my agents such that it deleted data permanently and caused rework.
所以我现在觉得,
So I'm like,
哦,这样啊。
oh, okay.
得给它配几个‘守护者’才行。
I think I need some guardian spirits around it.
结合你们现在的部署情况,还有跟大企业交流的经验,目前的落地状态怎么样?
Given your deployments today and talking to large enterprises, what is the state of deployment?
对吧?
Right?
你觉得在这些范围更明确的、类似工作室的平台里,和那些自由发挥的编码代理之间,各自的使用情况大概是怎样的?
Like, much do you see that's within these more scoped, like, studio like platforms versus, you know, free riding coding agents?
在大型企业和不同行业中,你实际看到的应用情况有多少?
How much are you actually seeing in large enterprises and in different sectors?
是的。
Yeah.
目前,在我们典型的大型企业里,我们会把情况分成三类。
So, I think right now, in our typical enterprise, we break it down to three categories.
一类是各种SaaS平台,通常属于低代码类型,人们用拖拽的方式搭建代理,但这些其实并不是真正自主的代理。
So, we break it down to various SaaS platforms that are typically more low code, where people build agents in this drag and drop way, and they're not really autonomous agents.
我觉得它们更接近自动化工具。
I would think of them more as the automations.
然后是第一方代理。
Then there are first party agents.
人们在自己的云环境中构建应用,可能是因为这是公司内部需要的工具,或者是一款打算推向客户的智能型产品。
People are building in their cloud, potentially because it's an application they want inside the company, or even a product they're planning to release to the customers that is agentic.
第三类则是高度自主的编程代理和助手。
And then the third category is very autonomous coding agents and assistants.
在这些类别中,目前平均来看,超过一半都是高度自主的编程代理和助手。
Of these categories, I would say roughly at this point, over 50% is the autonomous coding agents and assistants in the average enterprise.
大约45%是那些低代码自动化工具,剩下2%才是真正由企业自己开发的第一方代理——毕竟,打造高效的智能代理难度大得多,而直接采用现成的或用低代码方式搭建要容易得多。
Then probably 45% is those low code automations, and the last 2% are really the first party ones that they're building themselves, because obviously it's much harder to build effective agents, and it's much easier to adopt agents off the shelf or build them with low code.
这正是我们目前观察到的情况。
And that's what we're seeing.
我们还认为,自动化工具的增长速度是最快的。
We do think that the automations are also the fastest growing category.
过去,只有开发人员会看到云代码在客户中像野火一样蔓延,现在我们发现,云协作者的增长甚至更快。
So, it used to be that only developers would see cloud code growing like fire in our customer base, and now we're seeing a cloud co worker growing even faster.
让我们自己都没想到的是,越来越多的企业开始把OpenCloud当作正式认可的工具来使用,因为公司CEO非常积极推动AI落地。
We're starting to see, to our own surprise actually, people adopting OpenCloud as a legitimate sanction tool in the company because the CEO is very driven to adopt AI.
所以我觉得,如今自主化的Azure无疑是增长最快的领域,而且通常没有任何管控措施。
So, I think that today, autonomous Azure is by far the fastest growing category, today typically comes without any controls.
企业现在每年在安全上的投入已经高达一千亿美元了。
So enterprises already buy, let's say, 100,000,000,000 of security today.
他们在终端、网络、云和身份等多个层面都部署了各种防护措施。
They have lots of different protections at the endpoint and network and cloud and identity domains.
那么,这些现有的防护手段对保护智能代理有用吗?还是完全没用?
What's relevant here for securing agents, or is none of it?
你是怎么看待现有这些防护体系的?
Like, how do you think about the existing protection set?
安全领域一向存在多种工具重叠使用的情况。
Security is always a space where you have some overlap between different tooling.
在这里,我们还要考虑到‘防御负债’这个概念。
In this case, and you have the concept of defensive debt as well.
所以,最好在技术栈的各个层级都设置防护,才能真正解决问题。
So, you want to have defenses at different levels of your technology stack to solve the problem.
不过话说回来,我觉得在这个领域,很多企业其实很被动,举个例子,身份权限这块。
That said, I think in this space, a lot of enterprises are kind of helpless because I'll take an example, the identity approach.
过去,如果我们公司有个软件系统,我们最首要的控制手段就是限制它的权限范围。
Traditionally, if we have a software system that's running in our company, our first and most important control will be to limit what permission it has.
不管发生什么,哪怕系统出错或者被攻破,它也做不了超出原本授权范围的事。
And then no matter what, even if it goes wrong, even if it's compromised, it can't typically do stuff that it was originally allowed to do.
但现在的自主AI、助手、编程代理不一样,我们反而希望它们拥有我们的权限——因为我们要让云代码帮我们做事,然后去吃午饭,回来一看,事情已经办好了。
But with these autonomous AIs, with these assistants, with these coding agents, we kind of want them to have our permissions because we want to tell Cloud Code to do something or Cloud Code work to do something, and we want to then go have lunch, we want to come back and see that it's done.
而且我们给它们的任务五花八门,根本没法精准定义一套合适的权限组合。
And we want to give it so many diverse tasks as well that we kind of can't find the right set of permissions to do.
这样一来,我们原来的身份安全系统就基本派不上用场了。
So, suddenly our identity security software is not very useful.
再比如终端安全或者API安全,如果我们告诉云代码:‘重建一下数据库,先删掉再新建’,这确实很高效。
Then if you think about endpoint security or API security, if we tell our Cloud Code that we want to recreate a database and it should delete it and recreate it, that's great.
这能帮我们的运维和平台团队省下大量时间。
That's gonna save our DevOps team and our platform teams a lot of time.
这是云代码的一大优势。
It's a great benefit of Cloud Code.
但如果云代码在处理无关任务时,突然觉得最好把我们的数据库删了再重建呢?
But if Cloud Code is working on an unrelated task and suddenly thinks that maybe the right thing to do is to delete our database and recreate it.
也许我们并不希望发生这种情况。
Maybe we don't want that to happen.
可惜的是,我们的终端或API安全工具根本不知道云代码在想什么,为什么要做这些事,对吧?
And unfortunately, our endpoint providers or API security tools, they don't know what cloud was thinking, why is it doing what it's doing, right?
所以,很多现有的工具缺乏足够的上下文,无法理解这些高度灵活且不可预测的系统在做什么。
So, a lot of these existing tools, they don't have the context to understand what these very flexible, unpredictable systems are doing.
如果你不为这类系统专门设计一些控制机制,那就只能要么过度限制它们,让它们对企业几乎失去价值,要么就会错过它们可能造成的许多严重风险。
If you're not building some kind of controls that are built for these systems, then you're either gonna end up limiting them a lot, making them almost much less useful to the enterprise, or you're going to miss a lot of pretty dangerous things that they might be doing.
作为一个有多年安全经验的人,遇到这种问题,我第一个很传统的想法是:这听起来像是个需要策略引擎的代理问题。
As somebody who has worked in security for a long time, my first very traditional instinct on a problem like this is like, that sounds like a problem for a proxy with a policy engine.
我们制定一些规则,再让这些规则变得更智能。
We make some rules and make the rules smarter.
比如,为什么这行不通?还是说你们试过这个方法?
Like, why doesn't that work, or did you try it?
我觉得有几个原因,主要是代理的集成方式这块。
There are a few things that, I mean, proxy's integration method, I would say.
所以,确实有一些AI系统,如果通过代理集成是最简单的方式,那我们会考虑这么做。
So, there are some AI systems where, like, you would want to integrate with a proxy if that's the easiest way to do it.
但首先,有很多系统在技术上根本不可行,因为现在的AI大多运行在云端、别人的基础设施上,或者在你的端点上,这时候代理并不总是个可行的选项。
But number one, there's a lot of systems where that's just not viable technically, because AI today runs on the cloud, on someone else's infrastructure, on your endpoint, and just proxy is not always an option.
第二点就是这个问题,好的,没问题。
And the second thing is the question, okay, great.
你通过代理在转发,所以你能看到数据。
You're proxying, so you're seeing the data.
你能看到数据,但这并不是最难的部分。
You're seeing it, but that's not the hard problem.
最难的部分是搞清楚我现在该怎么做。
The hard problem is understanding what I should do now.
结果发现,在AI系统的情况下,这才是关键问题。
Turns out that in the case of AI systems, that is the hard question.
到底该用什么机制来评估这些行为,判断它们是否安全?
Like, what is the engine that needs to underwrite these different actions and say if they're okay or not?
因为我们得弄清楚另一个AI系统在想什么、打算做什么,然后形成自己的判断。
Because we need to be able to understand what another AI system is thinking, what is it planning to do, and then have our own opinion on that.
更别说我们面对的,可能是世界上最有智慧的模型了,它们做的事本来就是对的。
And consider we're trying to understand some of the smartest models in the world are doing the right thing.
那我们凭什么去评判它们?
So, who are we to do it?
我们又该怎么准确地做到这一点?
How are we going to do it correctly?
对吧?
Right?
所以,这其实是个非常棘手的技术难题。
And so, that turns out to be a really difficult technical question.
Onyx 的一部分解决方案是训练自己的模型。
Part of the solution for Onyx has been training its own models.
对此你怎么看?
Like, what can you say about that?
如果你现在试着去构建一个系统,用来监督和控制其他智能体的行为,
If you try today, let's say we're trying to build a solution to oversee and kind of control how other agents are operating.
很多听众第一反应可能是:那我直接让 Cloud Code 去做不就行了?
Maybe the first thing a lot of our listeners might think is say, well, I'll just ask Cloud Code to do it.
从某种角度说,这想法没错,因为 Cloud Code 确实很强大。
In a sense, they would be right because Cloud Code is great.
我们可以让它为每个智能体都生成一个自己的副本,持续监控这些智能体的一举一动。
And maybe we can ask it to spawn a version of itself for every agent that we have and kind of keep monitoring everything that agent is starting to do.
一旦发现异常,就立刻介入干预。
And if you think that there's a problem, intervene.
这种做法显然很天真,也存在不少致命缺陷,我们待会儿可以聊聊,但它确实也有可取之处。
So, that approach, obviously it's pretty naive, and there are some ways in which it totally fails, we could talk about, but it has some merit to it.
所以,让有能力的代理去监督其他代理的行为,这听起来很自然,就像我们让有能力的人去监督其他人一样。
So, it does seem intuitive that it's a good idea to have capable agents reviewing what other agents are doing, same as we have capable humans reviewing what other humans are doing.
但问题来了:从成本、延迟和可靠性这些角度,你怎么让这套系统真正跑起来?
But then the problems that you're gonna run into is how do I make this work from cost, latency, reliability perspective?
如果你每运行一个代理,就得配一个专门负责监控的安全代理,那你的监控开销可能比你用AI本身的费用还高,对吧?
Because if I need to run an agent for every agent you're running as your security vendor, you're gonna be paying for me more than you're paying for your AI, right?
这样一来,这个方案基本就不可行了。
So, it's pretty much a dealbreaker.
而且,速度也会慢得让人受不了,你根本没法接受那样的延迟。
And also, it's going be so slow, so you're not going be happy with whatever latency you're going to get.
所以真正的挑战是:你该怎么判断,在什么时候才需要让这些智能代理介入,去检查当前发生了什么?
And so, the challenge then becomes how do I know what are the times where I need to interject with these smart agents to look at what's happening?
这时候,你真正该做的,其实是训练一些非常聪明的模型——等等,我纠正一下,不是特别聪明的模型,而是只擅长做一件事的模型。
And that's when actually what you want to do is you want to train very smart models that are, actually let me correct with that, very not smart models, but models that are just good at one thing.
它们体积很小。
They're very small.
它们几乎只能做一件事:判断是否需要让更智能的代理来检查一下。
They almost can't do anything else other than be able to say, should I have a smarter agent look at this?
如果你能把这种直觉很好地嵌入到这些小型模型里,让它们既不会漏掉重要情况,也不会频繁调用其他代理,就能实现一个完美的平衡——效率高、需要时有智能代理监控、成本低、延迟也低。
And if you manage to bake in that intuition into those small models well, in the sense that they don't miss a lot of stuff and they don't call that other agent too much, then you can get to a really good balance of we're very performant, we have smart agents overseeing things when needed, our costs are low and our latency is low.
但这就成了新的挑战,因为随着前沿模型越来越聪明,问题也变得越来越复杂,你必须确保自己这边的模型依然小巧、高效,并能持续判断:现在就是该出手的时候。
And then that becomes the challenge because you need to make sure that as the frontier models get smarter and the hardnesses become more evolved, you need to be able to have models that are on your side that are small and effective and continuously being able to say, now is the time.
就是这个行动,我觉得该有人仔细看看了。
This is the action where I think someone should take a closer look.
这就是为什么Onyx专门训练这类模型,这也是我们在这一领域做的最困难的事之一。
And that's why Onyx trains models for this purpose, and it's you know, most of the hard things that we're doing in this space.
是的。
Yeah.
你和我都喜欢下快棋,我觉得Guardian系统有点像这么回事。
You and I actually both love to play blitz chess, and I look at Guardian as a system that's a little bit analogous.
说真的,我们俩在真正的对局里,都没指望能赢过马格努斯。
Like, it's not clear either of us is going to be competitive with Magnus in a real game.
但如果你玩得够多,用对了数据,关键是在时间压力下快速做出直觉判断,这其实已经变成另一种游戏了。
But if the if you play if you play enough times with the right data and all you have to do is make intuitive decisions under time pressure very, very quickly, it's actually a different game.
对吧?
Right?
你觉得这么说有道理吗?还是我有点牵强了?
Do you think that makes sense, or am I reaching here?
是的。
Yeah.
说实话,我之前没这么想过,但确实有很多相似之处。你看世界上顶尖的棋手,他们下的大多数棋步都是凭直觉。
Actually, I didn't think about it, but, yeah, there's a lot of analogies because I think if you look at top chess players in the world, most of the moves that they make are intuitive.
他们并不总是往前算很多步。
They don't calculate forward.
他们看了太多棋局,下过太多对局,早就对什么才是好棋有了感觉,所以即使不深算,也不觉得这步棋风险太大。
They've seen so much games and they've played so much games that they already have a good sense of what is their right move and that they're not taking too much risk here by taking this move without calculating.
而当你回看这些对局时,偶尔他们会突然停下来,花很长时间去仔细计算各种可能性,因为他们知道,这一步是整盘棋的关键。
And then if you look at those games, every once in a while, they do stop for suddenly a really long period of time to just calculate forward a lot of options because they know this is a critical move in the game.
这有风险,你得认真思考自己在做什么,然后做出正确的决定。
There's risk, You need to think through what you're doing and you need to decide correctly.
我觉得这非常相似。
I think that's very similar.
这不就是高效利用计算资源的方式吗?
It's the efficient way to run computation, right?
没必要在不该花心思的地方浪费太多精力,但在高风险的情况下,就得全力以赴地投入。
You don't want to spend too much intelligence where you don't have to, and you want to spend a lot of intelligence, overwhelmingly a lot, in situations where there's high risk.
你们团队现在主要在以色列。
You guys are a team mostly based in Israel today.
全世界都认可,以色列有一批顶尖的安全人才,他们从军方和进攻性安全领域出来,像你们这样的连续创业者也不少。
I think the world has accepted that there is a cohort of amazing Israeli security talent that comes out of the military and offensive security and then repeat entrepreneurs like you guys.
但我觉得Onyx的基因有点不一样。
I think the DNA at Onyx is a little bit different here.
你的联合创始人之前是做合成数据的,还在英伟达工作过。
Your cofounder, came out of building synthetic data and working at NVIDIA.
你们觉得Onyx特别擅长的是哪类人才?
What would you characterize the talent that Onyx is particularly good at?
现在以色列真的有人在训练前沿的AI模型吗?
And then, are people actually training interesting frontier models in Israel now?
以色列在AI这块起步可能稍晚一点,但发展得特别快。
So, of I think Israel started maybe a bit late in the game, but it's catching up quickly.
现在以色列已经有好几家顶尖公司,正在打造世界级的模型、一流的AI基础设施,甚至自研芯片。
So, I think there's now amazing companies in Israel building world models, building AI infrastructure that's top of its glass, building chips.
总的来说,以色列在AI领域正变得越来越强,我们很自豪能参与其中。
So, I think Israel, in general, is becoming very strong in AI, and we're proud to be a part of that movement.
你说得对,我们公司融合了网络安全和人工智能的双重基因,这正好反映了我和吉尔的背景。
And I think you're right, our company has a very mixed DNA between cyber and AI, which kind of reflects mine's and Gil's backgrounds.
我们公司大部分研究人员和工程师,都来自以色列情报部门的一个单位,那里专门研究数学、网络安全以及两者的交叉领域。
Most of the people in our company, most of our research engineering, come from a unit in the Israeli intelligence where we actually deal with math and cyber and the intersection thereof.
所以,我们吸引的人才类型,也自然体现了这种背景。
And so, I think it is also reflected in kind of the type of talent that we bring in.
我觉得这有几个重要原因。
I think it's important for a few reasons.
最重要的是,我们长远来看不只想做一家安全公司。
The first and foremost is that we want to be more than just a security company long term.
我们觉得要真正解决这个问题,必须有深厚的AI能力,但问题本身并不仅仅是网络安全。
We think that to solve this problem well, it's going to require deep AI expertise, but then the problem is not just cybersecurity.
真正的问题是:我们该如何长期掌控先进的AI?
The problem is how do we control advanced AI long term?
就算你抛开企业安全和现有各种控制机制的漏洞不谈,单从最根本的角度看,这个问题本身就非常重要。
And that problem, even if you just forget about enterprise security and the different gaps in various controls that they currently have, first principle is that problem just sounds very important to me.
所以我认为,这会是至关重要的。
So, I think it will be crucially important.
如果未来出现价值十万亿的AI公司,我们觉得应该由一家不直接提供AI的公司来监督并协助控制AI的行为。
If you have AI companies that are $10,000,000,000,000 companies, we think you want a company that is not the vendor of the AI itself to oversee and help you control what AI is doing.
我们认为,这正是一个价值超千亿美元的巨大机会,值得打造一家真正重要的公司。
And we think that's an opening that's a $100,000,000,000 plus opening for a really important company.
再想想要长期控制先进人工智能需要什么,我们现在还只是刚摸到边儿,长远来看,我们得更清楚地了解这些模型在运行时内部到底在想什么、发生了什么。
And then if you think about what it's gonna take to control advanced AI long term, then we're just scratching the surface because long term, we're gonna have to also understand much better what models are thinking, what's happening on the internals of these models as they're operating.
这也是我们研究重点所在。
That's also a lot of where our research is focused.
目前业内对这个问题分歧挺大的,尤其是那些思考机制可解释性、或者研究能否更好理解模型的人。
So, the industry is quite divided on this issue, I mean, amongst the people who think about whether or not mechanistic interpretability or research into better understanding models is possible?
这本身就是一个悬而未决的问题。
Like, that's a question.
所以你是相信这个方向的。
So, it's something you believe in.
我们觉得,这方面已经取得了不少实质性的进展。
We believe that there's been a lot of strong progress in that direction.
我们相信,搞懂模型内部的权重和激活状态,理解这些系统的数学结构,至少会是解决方案的一部分。
We believe that understanding the internal weights and activations, what is the internal structure, the mathematical structure of these systems, is gonna be at least part of the solution.
在很多方面,我们认为——当然,这可能只有到真正实现时才能确认——以我们现在的智能水平,要快速看透一个大型语言模型的内部结构,其实相当困难。
And in many ways, we think that, and this is maybe, we'll only know when we get there, but we think that for our level of intelligence, it's kind of difficult to understand very quickly what is the internal structure of a large language model.
模型内部结构到底是什么样的
What is the internal structure of way Our it's an
以人类的智能水平,或者你们模型的摄像头水平来说?
level, like, human intelligence or our level of your model's camera?
明白了。
Okay.
人类的智能水平。
Human intelligence.
哦,对。
Oh, yes.
是的。
Yeah.
我觉得确实如此。
I think like yeah, exactly.
我觉得作为人类,要理解权重和激活值的含义,可能仍然非常困难,包括机制可解释性这方面。
I think as humans, it might still be very difficult to understand what weights and activations mean, and maybe mechanistic interpretability.
听起来好像这太难了,甚至根本不可能做到。
It seems like, oh, maybe that's too hard or shouldn't be possible.
但随着我们开始拥有在某些重要方面比我们更聪明的模型,我们相信自己将能更有效地追踪机制性能力。
But as we're starting to have models that are much smarter than us, at least in some important ways, we think that we'll be able to start tracking mechanistic capability much more effectively.
而且我觉得,从长远来看,这将极大地帮助我们理解智能本身——不只是监督,而是真正弄清楚智能是什么、它是如何运作的。
And I think it's gonna be extremely rewarding, by the way, long term for understanding intelligence in general, like not just overseeing, but just understanding what intelligence is, how it works.
更聪明的模型和没那么聪明的模型之间有什么区别?
What's the difference between the smarter model and the less smart model?
我完全同意,理解、信任、保障和监管这些超级智能AI的机会非常巨大。
I completely agree that the opportunity to understand and trust and secure and govern these super intelligent AIs is a very large opportunity.
如果回过头来看今天,作为安全人员,我觉得我必须赋予你们所有权限,也得让这些公司拥有同样的理解和掌控力。
If we just scroll back today, the security person in me says, Well, then I have to give you all the permissions and understanding that I have to give these companies too.
你怎么能让客户、你正在合作的财富百强企业,或者那些技术原生公司——大家都非常重视自己的安全和业务——去信任你们呢?你们现在才不到一百人,对吧?
How do you get customers or the Fortune 100s you're working with already or tech natives, everybody cares about their own security and business, to trust you now as a You're like less than 100 people, right?
没错。
Right.
我觉得这本来是不可能发生的事。
I think it's one of those things that should not be possible.
理论上,财富百强的公司根本没有理由跟我们合作,我们到底是谁呢?
So, in theory, there's no reason why a Fortune 10 or 20 company would work with us because who are we?
我们才成立两年。
We're a two year old company.
就是几个懂数学和网络安全的人组成的团队。
We're like a few people you know, who've done math and cyber.
但我觉得,只有当他们的痛点足够强烈时,这种机会才会出现。
But I think it's an opening that only happens when their pain is very strong.
他们的痛苦太真实了,所以一看到我们这家刚从隐身状态出来的公司,就会想:天啊,这正是我每天都在面对的问题。
So, their pain is so strong that they're gonna say, oh my god, I just saw this company come out of stealth, but it's a problem that I have daily.
于是他们直接给我们打电话。
So, I'll give them a call.
然后突然就有大客户主动找上门——这无疑是创业者最想看到的局面。
And suddenly, get inbound from these large customers, which is of course the best thing you could hope for as an entrepreneur.
我认为这反映出他们意识到,这个领域里的很多初创公司还都很小、很新,但未来一定会冒出一家巨头,所以我们想尽早找到值得押注的那一个。
And I think it reflects, in my opinion, their understanding that a lot of the startups in this space are still small and new, but there's gonna be a huge company here, and we wanna find the right horse to bet on.
因此,我们会去仔细研究这些公司。
So, we're gonna take a look at these companies.
其次,如果我们什么都不做,用不了多久,这个问题就会直接拖垮我们的业务。
And number two, that if we don't do anything, then in a very short time, this will disable our business.
归根结底,搞安全的人干的其实是保住收入的活儿。
At the end of day, security people are in the business of revenue preservation.
他们清楚,这其实是在两种风险之间做选择。
They understand that this is between the two risks.
他们更愿意和一个有潜力但还处于早期的团队合作,而不是坐以待毙。
They want to partner with someone that's promising and early rather than not do anything.
除了代理在他们系统各处的行动之外,我认识的每一个首席安全官和工程负责人,都在为一件事揪心:这些编程工具让发现漏洞的成本急剧下降。
The other thing besides agent actions across their surface area that every CSO I know is freaking out about and every engineering leader is freaking out about is the I would just describe it as the plummeting cost of vulnerability finding with these coding tools.
没错。
Yes.
神话
Mythos.
这给那些遭受攻击的供应商带来了不少问题。
And that has caused a number of issues for vendors that are being compromised.
你觉得人们应该怎么应对这个问题?
How do you think people should react to this other issue?
我觉得所谓的‘神话’其实是这样的:十年前,要是有人跟我说自动化漏洞研究能实现,我会觉得这简直是天方夜谭,至少得等个二三十年。
I think mythos is really like, if you took me ten years ago, automated vulnerability research looked like a dream that would take twenty, fifty years to happen.
也许是因为我们在以色列情报机构做过大量这类工作,总爱夸自己干的活儿有多难。
Maybe it's because we were doing a lot of that in the Israeli intelligence, and we like to pat ourselves on the shoulder of how difficult the job is that we're doing.
但那时候真觉得遥不可及,结果转眼间就一下子全来了。
But it did look really far, and suddenly it's coming all at once.
所以我认为,首先,市场并没有过度反应。
And so, I think that, first of all, the market is not overreacting.
我觉得这标志着安全团队所面对的格局发生了巨大变化。
I think this is a huge change in what this means for security teams.
如果你今天是个务实的安全人员,你就明白必须迅速行动。
If you're a pragmatic security person today, you understand that you need to move very quickly.
你的策略可能是:先尽快做些临时修补,缓解眼前的威胁。
Your strategy might look something like, I need to do the fastest quick fixes I can to mitigate the immediate risk.
比如,针对已经发现的漏洞,赶紧打补丁或者部署缓解措施。
So, maybe I'll invest in whatever the vulnerabilities that have been found, let's try to mitigate for them, whether it is through patching or through mitigating controls.
但真正的解决方案——每个大型企业的安全负责人心里都清楚——是得把基础架构搭好,从根本上规避风险。
But then the real solution, and every security leader at large enterprise knows it, is that we need to have the foundational pieces in place to avoid those risks.
基础的第一步,是把身份认证牢牢控制住。
And the foundational pieces are we need to have identity locked down.
得有防火墙。
We need to have a firewall.
得有终端检测能力。
We need to have endpoint detection.
而对于企业里不同的资产类型、不同的系统层级,都需要对应的基础安全机制。
And for different asset classes in your enterprise, for different parts of your stack, there's a different foundational security mechanism that you need in place.
对于你现在面临的AI攻击面,或者公司内部的AI系统,同样需要一套基础的安全解决方案。
For the AI attack surface that you now have, or for the AIs in your company, you also need a foundational security solution.
我们在这个领域扮演的就是这样的角色。
That's kind of the role we play in that space.
所以在为神话级模型及其后续发展做准备时,你需要部署大量基础安全工具,来加固企业各个部分,而我们在AI领域正是做这一块的工作。
So, as part of your preparation for Mythos level models and beyond, you're going to need a lot of foundational security tools to fortify your different parts of the enterprise, and we're playing that part in the AI space.
你对蚂蚁和OpenAI在这一领域推出的Glasswing和Daybreak的分阶段 rollout 有什么看法?
Do you have a point of view on the phased rollout or controlled rollout with Glasswing and Daybreak from Ant and OpenAI in this area?
我没有特别强烈的意见,但我想,如果能确定短期内不会有人发布神话级模型,那倒是好事,这样我们就有足够时间去积累经验、制定应对方案、在业内推广,并确保不会出现航空公司停飞、电厂瘫痪这类灾难性后果。
I don't have a strong opinion, but I think it's on the one hand, if we knew that there's not going be anyone who's going to release a mythos level model soon, I think that would be great because it gives enough time for me to prepare, to build the know how, to build the playbooks, to share that around in the community, and to make sure that we're not starting to see airlines go down and power plants go down and really disastrous effects that could happen.
但问题是,如果有人提前搞出了神话级模型,回头一看,这就会是个巨大失误——因为我们本可以提前让企业有选择地快速行动,让更多公司接触到神话级技术。现在呢,因为中国已经出了神话级模型,其他人却毫无防备,完全束手无策。
The problem is that if anyone gets to a mythos level model earlier, then in retrospect, it would look like a huge mistake because we could have at least given companies the choice to start moving very quickly and give more companies access to mythos, now they're all vulnerable because there's a Chinese model that's mythos level, and there's nothing they can do about it.
所以,我希望我们能顺利实现渐进式的发布。
So, I think hopefully we manage to do the gradual rollout correctly.
我强烈建议扩大能接触到这项技术的公司范围,让获取门槛变得更低。
I would really encourage that we expand the amount of companies that get access to this and make it much easier for people to get.
我建议大家都做好心理准备,这些模型迟早会出现。
I would advise everyone to assume that these models are coming anyway.
眼下你能做的,就是投资于这些基础性防护措施,防止未来在系统中发现漏洞后引发连锁反应。
The only thing you can do right now is to invest in these foundational controls that will stop the downstream effects of these vulnerabilities that are going to be found in their systems.
在大型企业里,还有人坚持不碰这些技术吗?
Do you see in large enterprises any holdouts?
对吧?
Right?
其实我最近没怎么跟人聊过这个话题。
And I would say I actually haven't spent a bunch of time talking to people about this recently.
不过我记得一两年前,有些大公司还明确说,要全面禁用这些技术,等安全了再说。
But I remember a year and a half, two years ago, there were large companies that just said like, We're going to ban all of this stuff until it's safe.
是啊,现在几乎看不到这种态度了。
Yeah, I hardly see that anymore.
我觉得在金融行业,还是有一些公司对允许使用什么有比较明确的立场。
I think in the financial sector, there's some companies that are more opinionated on what they allow.
他们还是允许使用智能代理,但可能会更细致地限制,比如只允许使用这两款工具。
They still allow agents, but there may be like more granular as to like, maybe we're only gonna allow these two tools.
我个人觉得,未来能脱颖而出的公司,一定是那些愿意支持多种工具的公司,因为整个技术格局变得太快了。
I personally think that the companies that are gonna do well are the companies that are gonna allow a lot of different tools because the landscape is changing so quickly.
如果你一年前押注OpenAI,那绝对是世界上最稳妥的选择,可现在Anthropic的模型和工具反而更出色了,再过一年,说不定又冒出个新玩家,工具更厉害。
If you bet on OpenAI a year ago, that would have been the safest bet in the world, but suddenly Anthropic has much better models and better tools, and potentially a year from now, there's someone else with much better tools.
所以,这么做确实有代价,但我觉得大公司风险承受能力本来就该不一样。
So, I think there is a price to pay, but I think if you're a large company, your risk profile is and should be different.
初创公司当然希望智能代理包办一切,因为赢了就赚大了,输了也没什么可损失的。
When you're a startup, you want to have your agents do everything for you because you have everything to gain and you have nothing to lose.
但大公司就不一样了,比如摩根大通。
When you're a large, we're in JPMorgan.
你背负的风险太大了,完全可以慢一点,稳扎稳打地去利用AI带来的好处。
You have so much to lose, and you can maybe take a bit more time to gain what you can gain from AI.
顺便说一句,摩根大通其实是非常快就采纳新技术的。
And by the way, JPMorgan is the adopt a guy very quickly.
我认为,公司规模越大,在采用AI时越应该采取更细致的策略。
I think it is okay for companies to have a nuanced view the bigger they are on how they're adopting AI.
那你个人是怎么看待这个问题的?
How do you think about that question for yourself?
比如风险承受能力、推进速度,环境变化这么快,再加上你看到产品范围不断扩大,而这里的科研方向本身就已经相当庞大了。
Like, risk profile, pace, the environment's changing very quickly, and then, you know, you see a lot of problems growing the scope of the product, and the research thesis here is already quite large.
我们在AI安全领域算是运气不错,因为确实有不少供应商。
We are kind of in luck in the AI security space because, yes, there are lot of vendors.
新科技层出不穷,但到2026年AI运行的两大核心支柱,这几年其实没怎么变。
There's a lot of new technologies that are coming up, but the two core pillars of how 2026 AI works have not changed in the last few years.
我们依然主要依赖大语言模型,跟几年前相比并没有本质区别。
So, we're still using largely LLM foundation models that are not entirely dissimilar to how they were a few years back.
构建智能体的方式也基本没变,还是靠大模型来决定该调用哪些工具,并生成相应的指令。
And we're still building agents in pretty much the same way where we have an LLM decide what are the tool calls that we're going to make and generate those.
正因为如此,像我们这样的公司,现在就能基于这两个基础模块,快速拓展到各种应用场景,同时保持核心技术的简洁和专注。
And so, that does allow a company today like us to scale to a lot of different applications that are utilizing these two primitives while still keeping the core technology that we're developing fairly lean and focused.
当然,明天可能会出现全新的大语言模型范式,或者全新的通用人工智能范式,这就是为什么我们虽然对2027年AI的样貌有明确看法,但并不固执己见。
Now, of course, there's always a risk that tomorrow there's a completely new LLM paradigm that could happen, or a completely new AGI paradigm that could happen, And that's why we do try to We have strong opinions loosely held about what does AI look like in 2027.
我们对2026年可能有个比较清晰的判断,但对2027年,我们保持非常开放的态度,因为接下来两年,我们得先看看通用人工智能和超人工智能到底会变成什么样。
We maybe have a good picture for 2026, but for 2027, we're very open minded, and we think that's the right stance to be for the next two years until we see what does AGI, ASI look like.
你觉得你们正在解决的这些问题——比如对模型的信任和治理——实验室本身有可能做得了吗?还是说这本质上是个结构性问题?
Do you see the set of problems you're addressing, trust in the models, and governance of them, as something that the labs could ever do, or do you think it's a structural thing?
我这么问是因为,现在湾区的创业圈最常被问到的问题就是:假设模型能力持续提升,或者实验室本身变得更加雄心勃勃,那他们为什么不做同样的事呢?
I ask because the number one question amongst the startup ecosystem in the Bay Area today is, you know, if you assume capability improves or, you know, when the labs just gets hungrier from their already currently ambitious stance, why wouldn't they do this too?
所以,我也想听听你的看法。
And so, I ask you the same question.
今天,如果你是个普通用户,或者是个安全产品采购方,有些情况下你并不想完全信任那个卖给你产品的人。
Today, if you're a private person or if you're a security buyer, there are some places where you don't want to trust the same person that you're buying it from.
比如说,你买一辆车,你不会让卖车的人自己来认证这辆车是安全可靠的。
So, you know, maybe, you know, if you're buying a car, you're not going to have the same guy that you're buying it from certify that the car is good.
你会找第三方来做这个检查。
You're maybe going to have someone else do it.
如果你是安全团队,你也不会相信产品供应商说他们的产品不会破坏你的环境。
And if you're a security team, you're not going to trust the vendor of a product to tell you that this product is not going to mess your environment.
你会希望有个独立的第三方,他们的全部业务就是准确告诉你这东西是否可靠、是否正当,而且必须是对的。
You're going to want to have an independent party whose whole business depends on telling you that this thing is correct and being right, that this thing is legitimate and being right.
所以,这个领域里有一种生物信任的心理机制,对我们非常有利。
So, there's biorep psychology in this space that I think really goes in our favor.
然后,我觉得还有些核心问题,比如模型为什么会出错?
And then I think there's the core problems, like why are models even making mistakes?
智能代理为什么会出错?
Why are agents even making mistakes?
所以我大致可以把这些问题分成两类。
So, I would broadly categorize it into two things.
一类是这些模型的智能很不稳定,经常会犯一些很荒谬的错误。
One is, there's the jagged intelligence of these models, and there's sometimes very silly mistakes that they make.
我觉得这类问题最终会解决。
And I think that problem will go away.
我觉得未来模型会越来越聪明,犯的低级错误也会越来越少,而我们的职责不再是防范这些低级错误。
I think we're heading for much smarter models that make less silly mistakes, and our role is not going to be to prevent silly mistakes.
这些错误会由模型厂商自己解决,因为他们有很强的动力去做这件事。
That will be taken care of by the model vendors because they're very incentivized to do it.
我觉得另一个快速增多的问题是,模型并不是在犯低级错误,而是会从自己的角度出发,做出某种独立的、甚至可以说有点自我意识的判断,而这种判断未必和你的期望一致。
I think what is the other fast growing category of things that we're seeing malice do wrong is places where they're actually not making a thing that is like a silly mistake, but more, I would say, have an independent, you would even say, semi aware or semi conscious perspective on what should happen, and that perspective might not always align with your perspective.
我觉得这个问题是随着模型越来越聪明才逐渐显现的。
And I think that is a problem that we've seen grow hand in hand with models getting smarter.
也许越聪明,就越容易有独立的想法,越有自我意识,这本来就是自然的。
Maybe just the way it is that as you get smarter, you have more independent thoughts and you're more conscious.
而这个问题,即便是大厂商现在也很难解决。我们之所以能更清楚地识别和发现这类问题,关键在于我们被允许做的一些事情,他们是不能做的。
And I think that problem is actually seemingly very hard to tackle today, even for the large vendors, and one of the key things that are making it easier for us to understand and detect these things versus the other vendors is that we're allowed to do certain things that they're not.
比如,我们可以查看这些智能体过去的行为数据,但企业现在并不愿意让Anthropic或OpenAI拿到这些数据,因为他们知道这些公司数据胃口很大,肯定会拿去训练模型。
So, for example, we're allowed to look at a lot of historical data of how these agents have behaved, but enterprises today are not willing to have Anthropic or OpenAI give that historical data because they know these are very data hungry companies that will want to train on that data.
所以,相比厂商,我们能获得更多的上下文和更大的自由度,能更清楚地判断哪些行为偏离了常态,这一点对解决这个问题至关重要。
And so, I think there are some ways in which you are given more context and more latitude to know if something is happening that is wrong compared to the past, compared to how these agents typically behave and so on that the vendors don't have, and is really important in solving this problem.
最后我想说的是,你面对的不是一个供应商。
And the last thing I'll say is that you're not dealing with one vendor.
我们正走向一个由众多供应商共存的世界,原因有很多。
So, we're heading for a world where there's a multitude of different vendors for many reasons.
出于成本考虑,人们会使用开源模型,因为更便宜;同时也会选用在不同任务和成本结构上表现更优的模型。
You're going to have, for cost reasons, open source models that people are going to use because it's cheaper, and you're to have models that are better at different tasks and at different cost profiles.
因此,指望所有供应商都提供同等水平的安全保障是不现实的,尤其当你在快速采用新技术时,这些新供应商显然还没来得及完善这些机制。
And so, it is going to be unrealistic to expect all the vendors to provide the same level of security and to assume that as you're trying to adopt technology very quickly, especially coming from new vendors that obviously have not yet built out all of that.
所以,我认为这些因素使得大型实验室几乎不可能彻底解决这个问题。
So, I think these are the reasons why I think it would be very difficult for this problem to be just completely solved by the large labs.
顺便提一句,考虑到硅谷或非安全领域的人可能不了解,你们是在特拉维夫打造这个产品的,对吧?
Just to close and also thinking about what people in Silicon Valley or outside of security may not know, you're building this from Tel Aviv.
没错。
Right?
我认为全球最具对抗性思维的团队之一,就在以色列生态里,比如8200部队、Wiz、Armis、Island、NSO集团。
I think one of the deepest adversarial thinking benches in the world is the Israeli ecosystem, eight thousand two hundred, Wiz, Armis, Island, NSO Group.
对吧?
Right?
你觉得那些在安全领域之外、尤其是各大实验室里的研究人员、工程师和商业人士,对于安全和对齐方面需要做的事情,还缺少哪些认识?
What do you think that the researchers, engineers, business people in, the tech ecosystem outside of security and then in the labs in particular are missing about what needs to happen in security and alignment,
也就是
which
你在这里所讨论的内容。
is what you're talking about here.
在开发安全产品时,真正重要的是理解安全团队是如何运作的,我觉得以色列在这方面确实很有经验。
What is really important when you're building security products in general, and I think what people in Israel have really good know how in, is just understand how security teams work.
因为归根结底,无论你解决的是什么技术问题,你最终都是在为个人或组织打造工具。
Because at the end of the day, no matter what is the technical problem you're solving, you're building a tool for people, for an organization.
这些组织都有特定的架构。
That organization has a certain structure.
里面有不同的团队,也有特定的信息和责任流转流程。
There are certain teams, there are certain flow of responsibilities of information.
为这样的用户打造一款产品,不仅要解决技术问题,还要让他们真心喜欢,这真的很难。
And creating a product for this audience that doesn't just solve the technical problem, but they actually love is really hard.
你必须真正关心这些不同岗位的日常,还需要你的团队里有那些过去为他们做过产品的人,对他们了如指掌。
You need to really care about just the day to day of these different functions, and you need to have people in your ecosystems that have built products for them in the past that know them like they know their best friend.
比如,他们早上进办公室时做什么,喝咖啡时想什么,打开哪些系统,老板希望他们做什么,同事期待他们怎么配合,哪些事会让他们受表扬,哪些事会惹他们生气。
Like, they know what they do when they step into the office in the morning, they drink their coffee, what are the systems they're opening, what is their boss wanting from them, what are their colleagues wanting from them, what are they gonna get praised for, what are gonna get mad for.
然后,你要把这些细节完全融入到你的产品里。
Then you need to take that and make it as your product.
我觉得,以色列人之所以能做好这一点,是因为他们跟这些买家和终端用户接触太多了,这是他们现在特别擅长的地方。
And I think that's, I think today, one of the kind of really hard things that people in Israel learn to do because they've had so much contact with these buyers and end users.
我只想提醒大家,多去了解安全人员的日常,多一点好奇心。
And yeah, I would just encourage people to be much more curious about the day to day of security people.
这话听上去老生常谈,但这些人每天都在保护我们,阻止攻击者偷钱、窃取数据,维持着数字世界里的正常生活。
It's a cliche to say it, but these people are actually saving us daily from attackers stealing our money, taking our data, and they're kind of keeping a way of life as it is in this digital world.
所以,我真心希望全球的安全团队能得到更多理解和支持。
So, yeah, I think more love to security teams around the world.
展开剩余字幕(还有 25 条)
我想让你把这一点和你之前告诉我的另一件事对上,马克斯,你说过你是我在以色列见过的最信奉AGI的人。
I'm going to ask you to just square that with something else you've told me, Maxim, which is you're the most AGI pilled person I'm going to meet in Israel.
你的话里隐含着一种信念,那就是在未来几年里,我们仍然需要防御性的安全团队。
Embedded in what you said is a belief that we will continue to have defensive security teams for some number of years.
所以,你真的这么认为吗?
So, you do believe that?
我觉得安全团队也会变得越来越强大。
I do think that security teams are also going to become completely high powered.
但我相信,不久的将来,它们都会由AI代理来运作,就像知识型工作中的其他一切一样。
But I do think that they're going to be run by AI agents, like everything else in the knowledge workspace in the near future.
但我确实认为,保持务实很重要。
But I do think that it's important to be grounded.
今天,当我推销产品时,我的客户还是人类,只是附带一些代理工具。
And today, when I sell a product, I sell it to a human audience with a few agents.
顺便说一句,我们也在努力让我们的系统对代理来说更易用。
And by the way, we also invest in making our systems very convenient for agents to use.
重要的是,我要专注于为今天购买产品的用户打造极致的体验。
And it's important that I focus on delivering an amazing experience today for people who buy the product today.
当使用人群逐渐从人类转向代理时,我们就得不断优化,让代理能更顺畅地完成工作。
And as that audience becomes more agents than humans, it will be important for us to evolve and to make it work really well for agents doing the work.
所以,我认为核心原则其实没变。
So, I think the core principle is the same.
我们得时刻提醒自己:最终用户是谁?他们的体验如何?
We need to really be reminded of who is the end user, what is their experience.
对人类来说,就是别塞给他们太多无关的信息。
For human, it might be not overwhelming them with too much information that is irrelevant.
对代理来说,就是别在对话中浪费太多上下文令牌。
For an agent, it might be not wasting too many tokens in their context when we talk to them.
说到底,可能本质上是一回事。
Maybe it's the same thing, really.
因此,我们必须始终清楚谁在使用系统,以及怎样才能为他们提供最好的体验。
So, I think it's important that we always manage who's using the system and what will be the best experience for them.
太棒了。
Awesome.
非常感谢你做这次访谈,马克斯。
Thanks so much for doing this, Maxim.
谢谢你的认可。
Appreciate it.
非常感谢你,莎拉。
Thank you very much, Sarah.
在Twitter上关注我们:NoPriersPod。
Find us on Twitter NoPriersPod.
如果你想看到我们的脸,记得订阅我们的YouTube频道。
Subscribe to our YouTube channel if you want to see our faces.
在Apple Podcasts、Spotify,或者你常用的平台订阅本节目。
Follow the show on Apple Podcasts, Spotify, or wherever you listen.
这样你每周都能收到新一期内容。
That way, you get a new episode every week.
还可以在 nopriors.com 订阅邮件或查看每期的文字稿。
And sign up for emails or find transcripts for every episode at nopriors.com.
关于 Bayt 播客
Bayt 提供中文+原文双语音频和字幕,帮助你打破语言障碍,轻松听懂全球优质播客。