本集简介
双语字幕
仅展示文本字幕,不包含中文音频;想边听边看,请使用 Bayt 播客 App。
我真心相信,现在正是聚焦于想法和理念的时刻,思考什么才能真正保护并驾驭这项强大的技术。
Truly believe that right now is a moment to focus on ideas and ideas around what would work for really safeguarding and and god reeling this very powerful technology.
所以现在出现的两极分化,是因为人类倾向于与人结盟,而不是与理念结盟。
And so the polarization that's happening is because there is a propensity as humans to align with people rather than aligning with ideas.
我认为现在正是时候,真正开始独立于个人来看待这些理念。
And I think this is the moment in time to really start looking at the ideas independent of individuals.
而这正是我对整个行业以及参与这场讨论的人们的期望。
And and and that truly is my ask of the industry and the folks who are engaging in this discourse.
在本期节目中,我采访了Credo AI的首席执行官兼创始人纳维娜·辛格。
In this episode, I talked to Navina Singh, CEO and founder of Credo AI.
作为曾在微软、高通以及多个政府机构工作过的前工程师,纳维娜帮助我们剖析了在关于发展速度、治理、监管和创新存在如此多对立观点的当下,AI行业是如何应对安全问题的。
As a former engineer who's worked with Microsoft, Qualcomm and various government agencies, Navina helped us unpack how the AI industry is navigating safety at a moment when there are so many polarized views about pacing, governance, regulation, and innovation.
我今年早些时候在一个AI会议上认识了纳维娜,当时就决定一定要请她来上播客,因为她在AI安全和治理这个热门话题上有着非常冷静客观的见解。
I met Navina at an AI conference earlier this year and had to have her on the podcast because she has such a level headed perspective on this hot button issue of AI safety and governance.
她是以创始人的身份来谈这个问题的,过去六年来,她一直在围绕帮助全球一些最具标志性的组织安全、负责任地扩展AI来打造自己的事业。
She comes at this as a founder who's been building a business around helping some of the world's most iconic organizations scale AI safely and responsibly over the past six years.
如果你想了解世界上一些最重要的组织在实施人工智能时,真正发生了什么,你绝对不能错过这一集。
If you want to understand what's really happening on the ground as some of the world's most important organizations implement AI, you won't wanna miss this.
好的。
Alright.
那么,现在开始。
So here it is.
这是我们与Credo AI的Navina Singh的对话。
Our conversation with Navina Singh of Credo AI.
好的。
Alright.
欢迎收听《Deep View Conversations》播客。
Welcome to the Deep View Conversations podcast.
今天我有位特别的嘉宾,Navina Singh。
I have a special guest with me today, Navina Singh.
Navina,请向那些不熟悉Credo AI和你工作的人介绍一下你自己吧。
Navina, why don't you introduce yourself for those who aren't familiar, with Credo AI and with what you do?
跟大家聊聊你最近在忙些什么吧。
You know, tell them a little bit about, how you are working on, what you're working on these days.
杰森,非常感谢你邀请我。
Jason, thank you so much for having me.
我是纳维娜·辛格,Credo AI的创始人和首席执行官。
I'm Navina Singh, founder and CEO of Credo AI.
我们是AI治理平台的领先提供商,同时也提供程序化服务。
We are the leading provider of AI governance platform and also programmatic.
为那些正在构建、部署和使用大量人工智能的企业提供前沿的治理专业知识。
Provide forward deployed governance expertise to enterprises that are building and deploying and using a lot of artificial intelligence.
我六年前创办了这家公司。
I started the company six years ago.
实际上,我们开创了AI治理这个类别。
We actually created the AI governance category.
在那之前,我在微软、高通等一些最重要的公司做了近二十年的工程师,负责产品开发。
And prior to that, spent almost twenty years as an engineer building products at some of the most consequential companies like Microsoft, Qualcomm.
我一直也是开源的坚定支持者。
I've also been a pretty big advocate for open source.
我曾在Mozilla董事会任职,帮助推出了Mozilla AI。
I was on the board of Mozilla, helped launch Mozilla AI.
尤其是在这个AI时代,看到开源和开放权重的作用,我真的很兴奋。
And I'm just so excited to see, especially in this age of AI, the role of open source and open weights.
过去五年,Jason,我也有机会把技术和产品专长真正带入政策领域,尤其是当我们开始考虑全球AI标准、全球AI安全措施和监管的时候。
In the past five years, Jason also have had the opportunity to really bring in the technical expertise and product expertise to policy, especially as we start to think about global AI standards, as we start to think about global AI safeguards and regulation.
所以过去我有幸参与国家AI委员会,为上一届政府白宫提供建议,并且与我们的盟友密切合作,深入思考全球AI治理协调意味着什么,不仅对前沿实验室,也对那些构建和购买AI的企业。
So I've had the honor in the past to be part of the National Commission on AI and advising the White House in the previous administration and also working very closely with our allies on really thinking through what does global coordination on AI governance mean, not just for the frontier labs, but also for the enterprises that are building and buying AI.
对我来说,Credo AI自成立以来的个人使命一直聚焦于如何确保AI服务于人类。
And for me, I think the personal mission of Credo AI since our founding has been really focused on how do you ensure AI is in service of humanity.
我们通过产品、人才和专业知识,为这个生态系统带来监督和问责能力。
And we do that by really bringing in oversight and accountability capabilities through product, through people, and through expertise to this ecosystem.
非常好。
Very good.
你知道吗,Navina,今年夏天我们在AI活动上见过面。
You know, Navina, you and I had a chance to meet at the AI for event this summer.
我特别喜欢那次对话。
And I loved the conversation so much.
有意思的是,我们当时已经聊了一些关于治理、安全AI的话题,比如怎么让AI更可靠,怎么克服大语言模型固有的挑战,特别是对于大型企业和受监管行业来说,这些都是你们一直在做的事情。
You know, it's interesting because we were already talking a little bit about governance, about safe AI, how do you make AI reliable, how do you overcome some of the challenges that are inherent in LLMs if you are a large business, a regulated industry, all of those things, the stuff that you all have been working about.
当然,我们也聊了开放模型、开源生态系统这些内容。
Of course, we talked about open models, we talked about open source ecosystem, all of the things.
那次对话真的非常愉快。
And we had such a great conversation.
我当时就想,一定要请你来上播客。
Was like, okay, I'd love to have you on the on the podcast.
一切都非常棒。
And all of that was was fantastic.
然后就在上周,这个话题突然就火了,就是关于暂停或放缓AI发展的讨论。
And then this last week, you know, thing this whole topic kind of went through the roof, right, around the pause or slowing down AI.
现在这已经成了全国新闻,甚至是国际新闻了,对吧,大家都在问安全的人工智能应该是什么样的?
And now it's become national news, right, international news of, you know, what does safe AI look like?
甚至有人怀疑这是不是个骗局?
Is it a hoax even?
各种说法都有。
All of these things.
这周我写了一篇文章,说最大的挑战其实不是搞清楚谁对谁错,而是对话已经变得太两极化了,而且我们好像陷入了一种非此即彼的立场,要么选择人工智能安全,要么选择创新,两者不能兼得。
And I wrote a piece this week that one of the biggest challenges is not really figuring out who's right or wrong, but the fact that the dialogue has gotten so polarized and that we have taken this sort of position that you can either have AI safety or you can have innovation and that you can't have both.
我不得不这么想,这就是为什么我迫不及待想和你聊这个话题,因为在我的社论里我说那是个伪命题,问题的一部分就在于把它当成一个选择题。
I have to think, this is why I couldn't wait to talk with you about this because in my editorial I said that that's a false choice and part of the problem is thinking of that as a choice.
你怎么看这个问题?
Where does that land with you?
我觉得这可能是你经常要面对的事情之一,就是如何从思维模式的角度来思考这个问题。
I have to think that that's some of the thing that one of the things you confront pretty regularly is how to think about this just from as a mental model.
所以,Jason,过去四天里的这些讨论确实很有意思,但对我们来说并不新鲜。
So, Jason, it's really interesting to see the discourse in the past four days, but it's not new to us.
你知道吗,这是我们过去六年一直在经历和讨论的话题。
You know, this is the discourse we've been living and breathing for the past six years.
显然,不仅仅是过去四天,而是今年发生的加速发展,对行业来说是一个有趣的时刻。
Obviously, the acceleration that has happened, not just in the past four days, but this year's, is an interesting moment in time, for the industry.
所以我觉得你提到了几个我想回应的点,然后我们可以更深入地讨论你提到的那些权衡。
So I think you bring up a couple of points that I do want to address, and then we can dive deeper into the trade offs that you were talking about.
好的。
Okay.
我真的相信,现在是一个专注于想法的时刻,专注于那些真正能保护和管理这项强大技术的想法。
I truly believe that right now is a moment to focus on ideas and ideas around what would work for really safeguarding and and god reeling this very powerful technology.
所以现在出现的两极分化,是因为人类有一种倾向,就是更愿意跟人站队,而不是跟想法站队。
And so the polarization that's happening is because there is a propensity as humans to align with people rather than aligning with ideas.
我认为现在是时候真正开始独立于个人来看待这些想法了。
And I think this is the moment in time to really start looking at the ideas independent of individuals.
这也是我对行业和参与这场讨论的人们的真正请求。
And then that truly is my ask of the industry and the folks who are engaging in this discourse.
话虽这么说,我觉得现实情况还是有一些的。
Having said that, I think there are certain realities on the ground.
第一个现实情况是,除非你亲自开发过前沿模型,除非你亲手打造过这些高性能系统,否则旁观者很难真正了解这些先进系统的能力。
The first reality on the ground is, you know, unless you've built frontier models and unless you've built these highly capable systems, I think it's very difficult for a bystander to know the capabilities of these advanced systems.
所以现在也是一个该倾听前沿顶尖研究者们看法的时刻。
And so this is also a moment in time to listen to what the leading researchers are seeing at the frontier.
这就引出了我的第二点。
And I think that brings me to my second point.
过去四天里,大家关注的焦点主要是达里奥·阿莫代伊写的那篇关于前沿节奏的文章。
The moment in the past four days has been really focused on the essay that Dario Amadai has drafted around pacing the frontier.
很重要的一点是,要理解他特意选择用“节奏”而不是“放缓”这个词,背后是有用意的。
And it's really important to understand there's a very intentional choice behind the use of word pacing and not slowing down.
说到节奏,其实我觉得他这篇文章的标题要是改成“治理前沿”就更贴切了,因为整个讨论的核心就是这个。
So pacing really, you know, and as as I think about it, I I wish he had written the paper with the title governing the frontier because that's what the entire conversation is about.
之所以说现在是个关键时刻,是因为过去三四个月里发生了几件事,我觉得业界和那些不在AI圈子里的人也都明白:第一,前沿实验室里的AI已经变得非常强大,以至于它越来越能帮助研究人员去开发AI了。
And the reason this becomes a very pivotal moment is because there's a couple of things that have happened in the past three to four months, which I think the industry and the folks who are not in the AI community understand is, one, that we've reached a point where AI has become so capable in these frontier labs that it is increasingly helping researchers build AI.
而且,你知道,这基本上就是所谓的递归自我改进。
And, you know, that's basically called recursive self improvement.
而且它已经开始以非常实际的形式出现,以至于Anthropic目前大约25%的代码是由AI编写的。
And it is starting to appear in very practical forms, so much so that around 25% of the code at Anthropic is right now being written by AI.
所以我觉得当你开始思考这个动态时,当能力加速到这种程度,随之而来的问题是,我们是否应该把计算资源用于递归自我改进?
And so I think when you start to think about the dynamic, when the capability has accelerated to such a point, the question that begs is, should we be using compute resources for recursive self improvement?
或者,我们是否有机会将这些计算资源重新导向其他有用的功能性事物,因为AI已经非常非常强大了。
Or here, do we have an opportunity to redirect those compute resources to other useful functional things because AI is already very, very capable.
所以这是这篇文章背后一个关键方面。
So that's one key aspect of what's underlying this this essay.
这篇文章背后第二个关键方面是安全事件,这对很多人来说是个警钟,就是OpenAI的代理,不是单个代理,而是一群代理,基本上利用了Hugging Face。
The second key aspect of what's underlying this essay is the safety incident, which basically is an eye opener for many, is the incident where OpenAI agents, not single agent, but a swarm of agents, basically exploited Hugging Face.
而这件事之所以非常非常重要,不是因为最终发生的那个利用,而是那些利用的序列。
And the reason that is very, very monumental is because of not the exploit that ended up happening, but it was a sequencing of that exploits.
首先,这些代理没有被赋予互联网访问权限,而它们找到了获取权限的方法。
First, the agents weren't given Internet access, and they discovered a way to obtain it.
然后它们没有被赋予任何智能体之间的通信能力。
Then they weren't given any sort of inter agent communication.
你猜怎么着?
And guess what?
它们想出了如何利用内部包管理器来创建一个消息留言板。
They figured out how to use an internal package manager to create a message message board.
然后它们又想出了办法,共同制定通信策略来利用其他智能体。
And then they figured out a way to jointly figure out a communication strategy to exploit other agents.
它们最终能够入侵Hugging Face的凭据。
They were able to then hack into Hugging Face credentials.
它们能够利用其他未知的漏洞。
They were able to exploit other unknown vulnerabilities.
我觉得最吸引我注意的是,尤其是在Black Hat上有一个回顾,非常有意义,就是当这些智能体群体开始协调和通信时,它们总能找到达成目标的方法。
And I think what was really fascinating for me tracking, especially there was a retro done at Black Hat, which was very meaningful, is that as these swarms of agents started to coordinate and communicate, they always found a way to get to their objective.
它们最终不仅入侵了监控基础设施和研究基础设施,还入侵了评估基础设施。
They ended up not only hacking into the monitoring infrastructure, the research infrastructure, but also the evaluation infrastructure.
这之所以关键,是因为我们真的需要把能力和意图分开,这就涉及到治理问题了。
And the reason this becomes pivotal is we really need to separate the capability from intent, and this brings in governance into it.
没关系。
It's Okay.
现在这些系统展示出的方法,让它们能够像人类一样行动。
It's these systems now are demonstrating methodologies where they are able to literally like humans.
人类,我们拥有一种美妙而惊人的能力,那就是你和我可以交流。
Humans, we have this beautiful, amazing power that you and I can communicate.
我们能在想法上达成一致。
We can align on ideas.
我们能召集众人。
We can rally the troops.
我们能通过故事真正改变世界。
We can use our stories to really transform the world.
我觉得那个根本事件背后,是一个更大的故事。
And I think that fundamental incident is a bigger bigger story.
Hugging Face事件之所以重要,是因为它真正把AI控制从纯粹的理论对齐辩论变成了一个治理问题。
And the Hugging Face incident matters because it's really turning this AI control from just a theoretical alignment debate into truly a governance problem.
所以总结一下,我真的希望那篇文章能聚焦于前沿治理,因为这才是当下正在发生的事情。
So just to summarize, I think I I truly do wish the essay was focused on governing the frontier because that is what's happening in this moment.
而且,治理前沿和治理企业有不同的方式。
And then, again, there's different ways to govern the frontier and govern the enterprise.
我真的希望我们能达成共识,倾听彼此,找到真正利用这项技术为我们服务和带来好处的途径。
And and I I really hope that we can align on ideas, listen, and and figure out the pathways to really utilize this technology in our in our favor and our in our benefit.
谢谢。
Thank you.
你知道吗,这真是个很好的观点,语言在这些事情中的力量,尤其是在这样一个充满争议和两极分化的时刻,我们总是想方设法划定地盘,然后誓死捍卫那个立场。
You know, it's such a great point that the the power of language in these things, especially in such a charged kind of polarized moment, you know, we we find ways to carve out territory and then sort of defend that position to the death.
而在对话中,我们需要的是给每个人一个参与进来的机会。
And in dialogue, what we need is we need sort of an opening for everyone to find a way to participate.
每个人都有他们的想法,他们希望从中得到的东西。
Everybody has their ideas, the things that they want to see out of something.
在这种情况下,政府希望美国的人工智能生态系统继续发展,保持领先地位,处于前沿。
In this case, the government wants to see the AI ecosystem in The US to continue to advance and be at the cutting edge and be at frontier.
当他们听到像“放缓”或“减速”这样的词时,正如你所说,他们会认为这会阻碍他们保持美国的目标。
When they hear words like pacing or slowdown, to your point, I think they then interpret that as something that is going to inhibit their goals of keeping The U.
美国。
S.
生态系统处于非常强大的地位。
Ecosystem in a very strong place.
而且存在恐惧,这是一种合理的恐惧,如果你在某些情况下过早进行监管,支持这种做法的论据是,如果过早监管,可能会扼杀创新。
And there is fear and it is a reasonable fear that if you if you do regulate in some cases too early, the argument for for that is that if you regulate too early, like, you can stymie, you know, innovation.
但这并不是一个零和游戏。
But it isn't a zero sum game.
比如,也有一种观点认为,当治理和监管运作良好时,它会创造一些边界,使得快速前进成为可能,就像你有一辆直线加速赛车,周围有围栏一样。
Like, there is also a case to be made and sometimes that's not being put out there as clearly right now is that when governance and regulation works well, it creates sort of some a set of boundaries that make it possible to go really fast to just in the same way as if you had a, you know, drag racer, right, that had a fence around it.
它不用担心,因为它知道有围栏把它保持在安全的地方,所以它可以非常非常快地行驶。
Well, it doesn't have to worry because it knows that it sort of has this fence keeping it in a in a sort of a safe place so it can it can go really, really fast.
这是我们面临的最大挑战之一。
So that's one of our biggest challenges.
我想听听你的看法,过去一周里,随着各方立场逐渐固化,对话的质量有什么变化?比如,我们是应该放缓前沿技术的发展,还是加强监管,又或者认为不应该设置护栏,因为那样会让美国落后。
I guess I would love to hear from you how the quality of the dialogue has advanced in the past week as we've sort of seen some digging in of the positions, you know, of like, do we need to pace the frontier or regulate versus like no, we need to be careful not to put guardrails around this because then The U.
有人担心中国行动非常迅速,而且在开源方面采取了非常不同的立场。
S, the fear is that China is moving very quickly and has taken a very different position, especially around open source.
他们有一个生态系统,实验室之间有一些共享,他们进展非常快。
And they have an ecosystem where there is some sharing in between those labs, they are moving very rapidly.
所以从这个角度看,有人担心如果美国放慢脚步或者停下来整顿内部,就会在竞赛中被超越。
And so there is this fear from that standpoint that if The US slows down or stops to sort of get its house in order that it it will get lapped, you know, in the in the race as it were.
所以我很想听听你的看法,因为我知道你和参与这些对话的各方都有联系。
So I would love to hear if you are hearing because I know you're also quite connected with the parties who are involved, you know, in in these conversations.
你是否听到,关于如何在继续推进的同时设立安全保障的对话还在继续?
If you're hearing that there is still a conversation that's continuing about how do we set safeguards whilst continuing to advance.
我希望你能给我们一些希望,表明对话并没有完全停止。
I I'm hoping that you can give us some hope that the dialogue hasn't completely, you know, stopped.
当然。
Absolutely.
我觉得现在正是有很多健康对话的时候。
I think this is the moment that there's a lot of healthy conversation.
我想为观众们澄清一些事情,因为我觉得还是有很多误解。
I I wanna unpack something for the audience because I think there's a lot of, again, misinformation.
治理不等于监管。
Governance is not regulation.
首先,我想先把这两者区分开来。
First and foremost, I wanna start with with separating that out.
好的。
Okay.
当你思考信任的谱系是什么样的,在这个信任谱系上,有不同的方式可以通过验证这些系统是否正常工作、这些机构是否有效运作、以及这些机构背后的人是否负责来注入信任。
When you think about the spectrum of what does the spectrum of trust look like, On that spectrum of trust, there's different ways to sort of inject trust by verifying whether these systems are working, whether these institutions are working, whether the people behind these institutions are being accountable.
而这个谱系基本上包含了几种机制和几种工具。
And that spectrum basically includes a couple of mechanisms and couple of instruments.
比如说,信任光谱的一端是你自己的测试和声明。
For example, once one part of the spectrum is your own self testing and attestation.
作为产品公司,每个开发产品的组织都有责任进行自己的测试和声明,确保推向公众的产品确实符合公众的要求。
As a product company, every organization that is building products has the responsibility to do their own testing and attestation and making sure that the products that are being put out in the public actually meet the requirements of the public.
它们对公众是安全的。
They're safe for public.
它们遵守消费者保护法。
They are following consumer protection laws.
它们有相应的责任。
They have liability associated with it.
所以,无论你是哪家前沿实验室,你都不能把不安全的产品推向市场。
So whether whoever frontier lab you are, you have to make sure that you cannot be putting products out in the market that are not safe.
因为这是你在信任光谱上的第一个组成部分,是你对自我测试和声明的责任。
Because that's your first component of, on that spectrum of trust, your responsibility around self testing and attestation.
随着你继续推进,还有第二个组成部分。
As you progress, there's a second component.
这跟我们在金融服务行业看到的很相似,如果你愿意这么理解的话。
It's very similar to what we've seen in financial services, if you will.
第二道防线是进行独立的、但仍然是内部的测试,由一个完全独立的团队来做红队测试,检验你是否不仅满足了用例方面的要求,还建立了正确的证据方法。
There's the second line of defense where you actually do independent, still internal testing, where you have a completely separate group that is going to be doing red teaming, that is testing whether, you know, so you're meeting the requirements not only from the use case perspective, you've sort of created the right evidence methodology.
你把发现的问题都记录下来了。
You have documented the findings.
你在组织内部作为独立方,透明地、真正地去搞清楚问题。
You're transparently, like, really figuring out as an independent within the organization.
你是否在做该做的事情,是否有正确的测试流程?
Are you doing the things and are you having the right testing procedures?
所以在AI领域,现在有很多关注点放在红队、紫队等机制上。
And so within AI, there's a lot of focus right now on red teaming, purple teaming, etcetera, mechanisms.
再进一步,你就不必是公司内部的人了,但你能成为外部人员吗?
And then you progress a little bit further, which is now you don't have to be part of the company, but can you be an outsider?
这就是业界现在所谓的“嵌入式评估者”的概念。
And that's the notion of what's called being called to right now in the industry, the embedded evaluators.
你可以作为一个完全独立的测试机构,被邀请进入前沿实验室来实际测试这些系统。
That you can be a very separate testing organization and you're brought in into a frontier lab to actually test these systems.
但这里的区别在于,不仅仅是测试技术系统。
But the difference here is it's not just testing the technical systems.
还要对照组织所做的承诺进行测试。
It is also testing against the commitments that the organization has made.
如果作为一个前沿实验室,我承诺要发布系统卡,那么这个外部实体现在不仅要测试我的系统,还要检查我是否真的发布了那张系统卡。
If I, as a frontier lab, am gonna say that I'm gonna be reporting on system cards, this outside entity is now testing my systems, but also saying, because, did I publish that, you know, system card or not?
所以,我认为这是审计和测试的结合,一方面从组织层面看你承诺了什么,另一方面从技术角度看你的系统表现如何。
So it's a combination of, I would say, audit and testing, where you're looking organizationally what you've committed to, and then you're looking from the technical perspective, how are your systems performing?
这就是信任光谱中的第三个组成部分。
And so that's a third component of, you know, the spectrum of trust.
然后你可以进一步扩展,引入独立审计师来测试这些嵌入式评估者发布的方法论。
And then you can extend that a little bit more further where you bring in independent auditors to test the methodologies that are being published by these embedded evaluators.
这就是为什么整个审计生态系统会建立起来,就像金融服务领域那样。
And then that's why the entire audit ecosystem was created, like for financial services.
再把这个范围拉大一点,可能就会涉及到监管了。
And then you stretch that spectrum a little bit further, and then you might have regulations.
好消息是,Jason,我们已经有很多现成的法规仍然适用于AI。
The good news there, Jason, is we already have a lot of regulations that still apply to AI.
而且我觉得很多人忘了,比如说,你有没有歧视性的做法?
And and I think many folks forget that, you know, things like, do you have discriminatory practices?
你有没有遵守消费者保护法?
Do you have are you you know, consumer protection laws?
你有没有遵守这些法律?
Are you abiding by those?
这些法律是存在的,每个开发AI系统的公司都必须遵守现有的法律。
Those exist, and every company that is building AI systems has to adhere to those existing laws.
然后,是的,我们都需要考虑前沿监管,尤其是在国际协调方面,因为AI带来了新的风险和新兴威胁。
And then, yes, there is frontier regulation that we all need to think about, especially on international coordination because with AI comes new kinds of risks, emergent threats.
还会出现新的合规要求,这可能会导致新的监管形式。
There's new kinds of compliance requirements that show up, which might result into new form of regulation.
所以我之所以想跟你拆解这个信任光谱,杰森,从自我声明——也就是你们公司、前沿实验室的责任——一直到监管,是因为治理很多时候被误解为,哦,那就意味着你要走向监管了。
So the reason I wanted to unpack the spectrum of trust for you, Jason, which goes from self attestation, that is your company, your frontier labs responsibility, all the way to regulation is because governance many a times is like, oh, that means you're going towards regulation.
当你思考这个光谱时,杰森,真正有趣的是,前沿实验室——我真心相信,随着这些公司考虑上市,它们将要对公共机构负责。
And when you think about this spectrum, Jason, what becomes really interesting is frontier labs, I I I truly do believe that as these companies think about going public, they are gonna be held accountable to public institutions.
它们自己做测试和评估,并加强治理,这一点非常关键。
It is really critical that they do their own testing and evaluation and robustify their governance.
我会监督Anthropic和OpenAI在这方面的表现。
And I'm gonna be holding Anthropic and OpenAI accountable to that.
但同时,它们也在做这些事。
But at the same time, they're doing that.
所以这就是为什么我们需要其他机制,比如嵌入式评估者。
So that's why we need these other mechanisms of embedded evaluators.
我们需要一个非常活跃的独立验证组织。
We need a very vibrant independent verification organization.
我们可能需要新的法规,但现有的法规也仍然适用。
We need maybe new regulations, but there are also existing regulations.
所以,Jason,我希望这能让你对这个光谱有个概念。
So I hope, Jason, that gives you a sense of the spectrum.
它不是单一的东西。
It's not one thing.
这就是为什么,正如我在对话开始时提到的,现在想法很重要,因为我们必须处理测试和评估、证据验证,以及所有这些不同的机制,以便尽可能多地注入信任,尤其是在这些系统将要做出重大决策的地方。
And that's why, as I mentioned at the onset of this conversation, that right now ideas matters because we have to tackle testing and evaluation, verification of evidence against all these different mechanisms to inject as much trust as we can into these systems and and where these systems are actually gonna be making consequential decisions.
所以,我听到你说的一件事是,你知道,一开始,我们不一定需要新的法律或法规,对吧,来开始治理,走上正确治理这些事物的道路。
So one of the things I I heard you say is that, you know, to start, we don't necessarily need new laws or regulations, right, to start on the governance, on the path to governing these things well.
这样说对吗?
Is that fair to say?
我会说,现有的法律和法规已经适用了。
I would say that there are existing laws and regulations that already apply.
而且,是的,我们也应该考虑新的法规。
And, yes, we should be looking at new regulations as well.
但我认为,在某种程度上,在某些人中间,有一种……
But I think there's a somehow, there's a there's among certain folks there.
他们认为自己的人工智能产品目前不受某些法规的约束。
They think that their AI products are not right now governed by certain regulation.
我觉得这个区别很重要,需要指出来。
And I think that distinction is important to make.
好的。
Okay.
你对监管俘获这个想法怎么看?
What do you think about the this idea of regulatory capture?
这是过去一周经常被提到的一个话题。简单来说,监管俘获指的是像谷歌、OpenAI、Anthropic这样的大型实验室主动提出要帮助政府理解技术。
That's one of the things that's come up over this past week a lot, which is that and for the audience, regulatory capture is this concern that the the big labs, Google, OpenAI, Anthropic, they've offered to help government understand.
他们甚至提出了自己的框架。
They've even proposed frameworks.
其中最著名的就是谷歌DeepMind和戴密斯·哈萨比斯提出的那些想法和框架。
So the most famous of which was the one by Google DeepMind and Demis Hassabis offering some ideas, some frameworks.
而且你也看到OpenAI和Anthropic也提出了一些想法。
And you've seen it from you've seen some ideas offered by open AI and Thropic as well.
但说到监管俘获,大家担心的是这些公司会以某种方式影响监管的制定,从而保护自己的市场份额和地位,让竞争对手更难进入,包括一些开源竞争对手,他们可能没有那么多资源去雇佣很多人,去跟监管机构打交道,或者跟独立的观察员一起参与组织的工作。
But with regulatory capture, concern is that these companies are going to shape the form of regulation in a way that will protect their market share, will protect their position, and will make it harder for competitors, including some of the, dialogue has raised, including some open source competitors who may not have the resources to also have people on staff to work as many people, to work with regulators, to work alongside maybe independent observers being part of the organization.
所以很想听听你对监管俘获的看法,这应该引起多大程度的担忧,以及这在对话中占了多少分量。
And so would love to get your perspective on regulatory capture, how much of a concern it should be, and how much of that is coming up as part of the dialogue.
所以,这又是一个非常微妙的对话。
So, again, very nuanced conversation.
如果你看到我没有选边站,那是因为这确实是个很微妙的话题。
And if you see me not taking sides, it's because it is a nuanced conversation.
话虽如此,我对监管俘获确实很担忧,而且这是真实存在的问题。
Having said that, I worry about regulatory capture a lot, and it's a real thing.
好的。
Okay.
所以,如果权力集中在少数几个实验室手里,我确实会担心。
So I do worry if there is, you know, the power is consolidated in the hands of a few labs.
这确实是个值得担忧的问题。
That is a true worry.
这也是我大力支持开源的原因之一。
And I think this is one of the reasons I'm a big proponent of open source.
我真心觉得,美国和我们的盟友投资开源,是非常关键和基础的一步。
And I truly believe that there's, you know, as as United States and also our allies investing in open source is really critical fundamental step.
所以,监管俘获甚至能阻止年轻公司和初创企业在这个生态系统中崭露头角,让它们无法与那些最终占据生态大部分版图的实验室竞争。
So, capture can prevent even young companies, you know, and startups trying to show up in this ecosystem, prevent them to be competitive with some of the labs that then end up owning a big part of what the what that ecosystem looks like.
我想给听众的另一个观点是,当你在技术发展上遥遥领先时。
I think the flip side that, again, I might offer to the audience is when you are so far ahead in terms of technological development.
想象一下你在攀登珠穆朗玛峰,而且是第一个攀登的人。
So imagine you're climbing Mount Everest, and you're the first one to climb Everest.
现在你到达了顶峰。
Now you've reached the summit.
你在攀登途中看到了别人没看到的东西,因为他们还没登顶。
You've seen things on the journey to summit that others haven't because they still haven't climbed the summit.
所以你可以开始把信息传回给他们,给他们指明通往顶峰的道路。
So you can actually start to communicate information back to them, giving them pathways to how they can get to the summit.
但杰森,你得记住,人类真的是很有趣的生物,因为你不光要看他们的专业能力,还得看他们的动机。
But as you have to remember, Jason, that, again, humans are very interesting, I would say, creatures because you have to weigh not only their expertise, but also their incentives.
所以,如果我现在站在珠穆朗玛峰顶上,我发明了一种新鞋想多卖点,那我告诉那些正在爬山的人的信息就是:买这双鞋,因为只有这样才能登顶。
So if I'm now at the top of this Mount Everest and I have created a new kind of shoes that I want to sell more, now the information that I can relay to the people who are climbing the summit is buy these shoes because that's the only way you can reach the summit.
因为我已经登顶了,而且我是专家。
Because I have already reached it and because I'm the expert.
但我也有卖更多鞋的动机。
But again, I have the incentive to sell more shoes.
所以你可以想象,你需要在专业能力和动机一致性之间做个平衡。
And so as you can imagine, there is a little bit balancing that you have to do from expertise perspective, but also from the perspective of incentive alignment.
我觉得现在,摩擦就出在这里。
And I think right now, that's where the friction is happening.
我真心相信,那些AI实验室、前沿实验室的研究人员确实见过很多、造过很多。
I truly do believe that the AI labs and the frontier labs and the researchers there have seen things and have built things.
我们应该听这些专家的,因为他们已经登顶了。
We should listen to those experts because they've reached the summit.
同时,是的。
At the same time, yes.
他们都有动机吗?
Do they all have incentives?
我们都有动机。
We all have incentives.
每个人都是从不同的角度出发的。
Everyone is arriving at it from different places.
所以这确实是这个讨论中提到的对齐挑战和对齐问题。
So this truly is alignment challenge and alignment problem that is being discussed in this discourse.
监管捕获是真实存在的吗?
Is regulatory capture real thing?
因为如果权力集中在少数前沿实验室手中,最终会发生什么,还有另一种姿态,尤其是在我们服务的企业中。
Because if it's the power is consolidated in the hands of a few frontier labs, what ends up happening, There's this other posture that is happening, especially in the enterprises that we serve.
我们通常服务财富五百强企业,涵盖受监管和不受监管的行业,如医疗、保险、制药、消费品零售。
We generally serve Fortune five hundreds in regulated and unregulated sectors like healthcare, insurance, pharma, CPG retail.
所以现在所有企业都在尝试使用基础模型。
And so all the enterprises right now are trying to utilize foundation models.
他们都在尝试利用前沿能力。
They're trying to utilize frontier capabilities.
但同时,他们也在问自己一个问题:我应该租用智能,还是应该拥有自己的智能?
But at the same time, they are asking themselves this question, should I be renting intelligence or should I be owning my own intelligence?
因为我不想把自己的所有权力都交出去。
Because I don't wanna go give away all my power.
所以你可以想象,有很多激励在冲突,同时也有很多专业知识。
And I think so as you can imagine, there's a lot of incentives that are at clash along with a lot of great expertise.
所以在这个时刻,倾听变得非常关键。
So this is the moment in time that listening becomes really critical.
最后我想留给你三件事:是的,监管俘获是一个巨大的担忧。
And then so the three things I would, like, leave you with is, yes, regulatory capture is a massive concern.
我认为投资开源非常重要。
I think it's really important that we invest in open source.
但与此同时,前沿实验室拥有我们应当倾听的专业知识,至少应该关注一下,然后共同做出明智的决策,真正弄清楚如何在这项变革性技术上共同推进我们的企业、社区和国家的发展。
But at the same time, the frontier labs have the expertise that we should be listening to, and then at least paying attention to and then making informed decision coming together to really figure out how do we progress our enterprises, our communities, and our country forward collectively on on this transformational technology.
好的。
Alright.
我想稍微聊聊开源的话题,然后想问问关于Credo AI的事情,因为你们正在做的一些事情我觉得特别有意思,尤其是那些前沿部署工程师或顾问相关的。
I wanna talk a little bit about open source, and then I wanna ask a little bit about Credo AI because there's some stuff that you're doing specifically that I think is interesting, especially sort of forward deployed engineers or consultants.
这在你成为2026年热门词汇之一之前,你们就已经在做了一段时间了。
It's something you've been doing for a while before it became one of the hot terms of 2026.
不过我们先聊聊开源吧,因为你在这方面有些背景。
Let's talk about open source for a second, though, because you have some background there.
你是开源的倡导者。
You are an advocate for it.
我们目前看到的一个现象是,AI代理显然出现了爆炸式增长。
One of the things that we're seeing is that there's been obviously this explosion of AI agents.
大家也很担心token成本的问题。
There's been a lot of concern about token costs.
这推动了对本地部署AI、开源和端侧设备的兴趣。
That's driven some interest in on prem AI, in open source, on device.
与此同时,更多受监管的行业开始对使用AI感兴趣,而他们有着非常严格的数据主权和数据治理方面的顾虑。
At the same time, more regulated industries are getting interested in using AI, and they have some really strict data sovereignty and data governance concerns.
所以对他们来说,采用开源模型、进行定制化、在本地运行,比发送到云端更能让他们信任。
And so for them, taking an open source model, customizing it, running it locally, they have a lot more trust in that than they do sending it to the cloud.
而且这样还能省下不少钱。
And it also saves them a lot of money.
这很有价值。
It has a lot of value.
沿着同样的思路,你可以基于这些开源模型,打造自己的领域专用模型。
And then along those same lines, you can take it and make your own domain specific model based on some of those open source models.
我们已经在汤森路透、CrowdStrike和Salesforce这些公司身上看到了这样的做法。
And that's what we've seen from folks like Thomson Reuters, CrowdStrike, Salesforce.
他们都采用了这些大型模型,然后针对网络安全、商业及业务流程、法律等领域进行了定制化,我提到的这三家公司就是这样的例子。
They have all taken these large models, and then they have made them domain specific to cybersecurity, business and business processes, and to legal, all of those kinds of things in the three cases that I mentioned.
所以开源模型现在势头非常强劲。
So open source has an enormous amount of momentum behind it.
但同时也有人担心,很多开源模型还是中国的研究机构遥遥领先。
And yet there is also the concern that still a lot of it is the Chinese labs are very far ahead in open source.
他们正在迅速追赶美国模型,而且速度越来越快。
They are rapidly catching up to US models and faster and faster.
不过我们也看到开源模型得到了更广泛的支持。
But we are seeing some more broad based support for open source models.
比如来自NVIDIA和Mistral的支持。
So from NVIDIA, from Mistral.
很多人不知道,谷歌和OpenAI也有开源模型,而且已经在使用了。
A lot of people don't know that Google and OpenAI also have open source models that are out there and are being used.
那么你看到开源生态系统在哪些方面正在多元化发展,并且充满活力呢?
So what are you seeing in terms of the open source ecosystem diversifying and growing in some energy?
另外,在企业采用这些模型方面,你看到了什么?就像你刚才说的,他们决定要拥有自己的智能。
And then what are you seeing in terms of enterprises adopting them, doing just as you're saying, like deciding that they want to own their own intelligence?
杰森,这个问题问得真好。
What a great question, Jason.
我觉得确实,整个行业,尤其是我们合作的企业,正处在一个非常有趣的权衡点上。
And I think truly, the industry, especially the enterprises we work with, are at a very interesting point of trade offs.
他们正在努力弄清楚如何在能力、成本和掌控之间做取舍。
And they're trying to figure out how do they trade off between capability, cost, and control.
顺便说一句,这三样你不可能全都要。
And by the way, you can't pick all three.
你只能选两样。
You have to pick two.
所以有时候,就像说,我想尽可能快地推进。
So in some cases, it's like, I want to move as fast as possible.
我非常看重高性能系统,也非常在意成本,那我愿意放弃掌控。
I deeply care about highly capable systems, and I deeply care about cost, I'm willing to give up on control.
你就得做出某种选择。
You make a certain choice.
但在某些情况下,你会说,不行。
Whereas in some cases, you're like, nope.
我想要高度受控的系统,因为我身处受监管的行业,而且我在乎成本。
I want highly controlled systems because I'm in a regulated sector, and I care about cost.
至于能力,我不需要最新最强的Astra模型。
Maybe capability, I don't need the latest and greatest Astra model.
我用旧版的GPT就挺满意的。
I'm happy with the older versions of GPT.
所以,杰森,你可以想象,我们正处在这个权衡的关键节点,而且我们正亲眼看到许多客户在做出这些选择。
So as you can imagine, Jason, we are at this really pivotal point of trade offs, and we are seeing live in front of us many of our customers making these choices.
话虽如此,我觉得最吸引人的地方——这也是Credo AI的工作变得真正重要的地方——就是无论开源、开放权重还是专有的前沿模型,治理都必须能够应对所有这些选择。
Having said that, I think what I find fascinating, and this is where, you know, Credo AI's work becomes really consequential, is irrespective of open source, open weight, proprietary frontier models, governance has to be able to solve for all those choices.
对。
Okay.
因为归根结底,当你作为一家企业决定要依赖前沿模型,把它们引进来,然后部署到你在意的用例上时,你仍然必须能够拆解你整个人工智能物料清单,并评估风险。
Because at the end of the day, when you as an enterprise have decided that I'm going to lean in on the frontier models, I'm gonna bring those in, and then I'm going to deploy that for a use case I care about, you still have to be able to unpack your entire AI bill of materials and figure out the risk.
你还得在整个物料清单上搞清楚机会和合规问题。
You have to also figure out the opportunity and the compliance across that entire bill of materials.
同样,如果你选择用,比如说,Thinking Machines 的开源权重模型,你会觉得,好吧,我想拥有自己的智能。
And similarly, if you choose to use maybe, let's say, an open weight model from thinking machines and you're you're like, okay, I want I want to own my own intelligence.
但与此同时,这给你带来的好处是,你不仅能拥有自己的智能,而且你现在必须嵌入很多控制,这些控制本来可以从前沿实验室那里得到的。
But at the same time, what that affords you is the ability to sort of not only own your own intelligence, but you have to now embed a lot of control which you otherwise would have got from the frontier labs.
所以,Jason,我觉得我们又处在一个非常有趣的权衡地带。
So I think, again, Jason, we are in this very interesting land of trade off.
我现在更大的担忧,也是我们关注的重点,就像在产品开发中遇到的技术债或设计债一样。
My bigger concern right now and things that we are focused on is similar to in in product when you have, like, a tech debt or design debt.
现在发生的是治理债正在急剧增加。
What's happening right now is the governance debt is increasing quite drastically.
好的。
Okay.
治理债增加的原因是前沿能力在加速发展,而我们仍然处于那个拐点,即将看到变化速度呈超指数增长。
And the reason that governance debt is increasing is your frontier capabilities are accelerating, and we are still at that inflection point where we are gonna see the rate of change super exponential.
在治理方面的投入和治理能力的建设,其实并没有跟上前沿能力的发展速度。
The governance investment in governance and the investment in governance capabilities haven't really kept pace with the frontier capabilities.
结果就是,你看到这个差距越来越大,这就是治理债务。
And as a result of it, you're seeing this growing gap, this governance debt.
我觉得这就是我想要开始着手减少的债务,这也是Credo AI存在的原因。
And I think that's the debt that I want to start focusing on reducing, and that's why Credo AI exists.
大家好。
Hey, everybody.
感谢收听本期节目。
Thanks for listening to this episode.
简单说两句,然后我们继续回到对话。
Quick note, and then we'll get you back to conversation.
我们每周都很高兴能为大家带来这些内容。
We love bringing you this content every week.
我们一直在想办法,希望能给你带来最大的价值,帮你理解AI如何改变商业、改变世界。
We're always trying to figure out how we can deliver you the most value to help you understand how AI is transforming business and transforming the world.
如果你喜欢这个节目,有个简单的方法可以回馈一下,也能帮别人了解这个节目。
If you're enjoying the show, there's an easy way for you to give a little value back and help others learn about the show as well.
如果你在Apple Podcasts或Spotify上,给我们打个分,留个评论吧。
If you're on Apple Podcasts or Spotify, drop us a rating and leave us a review.
如果你在YouTube上,点个赞,订阅一下,或者留个评论。
If you're on YouTube, hit like, subscribe, or leave us a comment.
就这样。
That's it.
只需要一分钟,但帮助很大。
It only takes a minute, but it's a huge help.
所以先谢谢你的支持。
So thanks in advance for pitching in.
现在回到节目正题。
And now back to the show.
你最后提到Credo AI,我很高兴,因为接下来我想聊聊这个。
I'm glad you brought up Credo AI at the end because I'd like to talk about that next.
因为上个月我们聊的时候,你让我印象很深的一点是,你在这个领域已经深耕了一段时间。
Because when we talked, you know, last month, one of the things that I was really impressed with is that you've been working on this for a while.
你创立公司就是基于这个想法。
You founded the company on this idea.
所以我很想请你跟听众们详细聊聊这件事。
And so I'd love for you to unpack that with the audience.
你是什么时候开始的?
When did you start?
为什么?
Why?
以及,在帮助公司以清晰的方式掌控AI方面,你经历了怎样的过程,让他们在AI发展道路上能有明确的前进方向,尤其是在那些已经有严格法规和指导方针来管理数据的行业里,这一点尤为重要。
And what's been the journey to helping companies really get their arms around AI in some really clear ways so that they have sort of a very clear path forward in their journey in AI that's especially important in industries where there are very important regulations and guidelines already in place for the ways they manage data and that.
所以,你已经帮助这类公司好几年了。
So you have been helping, you know, these kinds of companies for several years now.
那就跟我们聊聊这段历程吧。
So talk a little bit about that journey.
杰森,这段经历真的很有意思。
Very interesting journey, Jason.
正如我在加入Credo AI之前提到的,我当时在微软开发一些非常令人兴奋的AI产品。
As I mentioned prior to Credo AI, I was at Microsoft building some really exciting AI products.
我觉得有趣的是,当你总是站在技术演进的最前沿时,你会开始看到未来的趋势。
And I think the interesting thing is when you're always at the front end of technological evolution, you start to see what's coming.
但即便如此,我也没有预料到它会来得这么快。
And even I didn't anticipate how fast it would come at us.
不过在2019年,我有一个想法:如果我现在看到的AI进步,哪怕只是线性增长,而不是指数增长,总有一天,AI开发者们将需要确保他们构建的是安全、合规、可信赖的AI系统。
But back in 2019, a thesis that I had was if the advancements that I can see right now in AI, even if they just linearly increase, not even exponentially increase, there is going to come a point where the AI builders will need to make sure that they are building safe, compliant, trustworthy AI systems.
与此同时,那些来自风险与合规领域的AI治理者,将能够接触到AI技术,从而真正开始成为开发者。
And at the same time, the AI governors who are coming from risk and compliance will get access to AI technology that they can actually start becoming builders.
当这种情况发生时,AI开发者和AI治理者之间的鸿沟将会消失。
And when that happens, the gap between AI builders and AI governors will collapse.
为了以一种真正有趣的方式构建安全可信的技术,我称之为AI前沿的“最后一公里”问题,也就是:我们能否真正将治理嵌入到AI开发的整个生命周期中?
And to make that really, you know, interesting way to build safe and trusted technology, I call it the last mile problem in the AI frontier, which was, can we actually embed governance throughout the AI development life cycle?
回到2019年,你知道,大家一听到治理这个词,就会翻白眼,因为会觉得,天哪,你会拖慢我的创新速度。
And back in 2019, you know, everyone, the minute you use the word governance, it's like rolling your eyes because it's like, oh my god, you're gonna slow my innovation down.
没错。
That's right.
这很有意思,我当时非常刻意地选择了AI治理这个词来定义我们正在创建的类别,尽管我知道这对我们的公关和品牌并不有利,因为在我们还没进门之前,你就已经认定治理会拖慢我的速度。
And it's very interesting, and I very methodically chose the word AI governance for the category we were creating even though I knew that the PR and branding would not work for us because the minute even before we've entered the room, you've made an assumption that governance is going to slow me down.
我觉得现在情况已经变了,治理成了AI增长的驱动力。
I think fast forward to now, governance is the growth driver for AI.
我们确实看到,我们服务的企业,无论是大量采购第三方基础模型,还是大量采购第三方应用,或者自己开发AI,那些一直与我们合作并在治理上投入的公司,他们确实看到了在利用AI获得投资回报、营收增长以及利润增长方面的有趣增长。
We actually are seeing that the enterprises we serve who are either buying a lot of third party foundation model pro you know, models, or they're buying a lot of third party applications, or they are building their own AI, the companies that are actually have been working with us and have invested in governance, they're seeing this interesting growth in actually leveraging AI to get the ROI, the top line growth, as well as bottom line.
他们能更快地开发产品。
They are able to develop products faster.
他们企业内部的生产力也在提升。
They're seeing increased productivity within their enterprises.
由于透明度,他们能更好地留住客户,同时也能更快地获取新客户。
They're able to, because of transparency, retain customers longer as well as acquire customers faster.
所以你看到的是,治理为那些押注于风险管理、合规和监管基础的组织解锁了非常有趣的机会,他们实际上能更快地在AI上获胜,无论是内部员工,还是客户和其他利益相关者。
So you're seeing this very interesting, I would say, opportunity that governance is unlocking for organizations who are betting on having that foundation of risk management compliance and oversight, that they're able to actually win faster with AI, both internally with their employees, but also with their customers and other stakeholders.
非常好。
Very good.
你觉得你们未来的道路是什么样的?
What do you feel like the path looks forward looks like, you know, going forward for you all?
你跟我提到过这个前置部署员工的想法。
One of the things you mentioned to me is that this idea of forward deployed, you know, employees.
提醒我一下你们组织里怎么称呼这个。
Remind me what you call it from your organization.
是的,当然。
Yeah, absolutely.
你知道,我真的想大力赞扬Palantir和Alex,我认为他们确实是创建所谓前置部署工程师的奠基者。
You know, I do wanna give huge credit to Palantir and Alex, and I think they really were foundational in creating what's called forward deployed engineers.
前置部署工程师背后的想法是,我能否把我的工程师嵌入到这些非常复杂的企业环境中,基本上了解足够多,以便我能做出更好的产品。
And the idea behind forward deployed engineers was, can I actually embed my engineers in these very complex enterprise environments and basically learn enough about them so that I can make better products?
Credo AI之所以成为第一家开发类似模式但专注于治理的公司,我们称之为“前沿部署治理”。
The reason Credo AI became the first ones to actually develop a similar model but for governance, we call it forward deployed governance.
我们的前沿部署治理分为两部分:前沿部署治理咨询和前沿部署治理工程。
So our forward deployed governance has two parts, forward deployed governance advisory and forward deployed governance engineering.
好的。
Okay.
这个模式借鉴了Palantir的理念,但前沿部署治理团队的重点是深入了解企业的AI成熟度和AI治理状况。
And it's built on a similar idea like Palantir, but the forward deployed governance team is really focused on really understanding the AI maturity and the AI governance posture of an enterprise.
所以我的前沿部署团队会进入企业,很快就能判断出他们处于哪个级别,是一级吗?
And so my forward deployed team will go into an enterprise and very quickly, we'll be able to figure out, are they at level one?
这意味着他们连政策都没有。
That means they they don't have policies.
他们甚至没有相关人员。
They don't even have people.
他们甚至不知道如何进行治理。
They don't even know how to do governance.
展开剩余字幕(还有 143 条)
一直到第五级以上,他们有成千上万个智能体在自主运行,需要监督和治理才能完全自主。
All the way to, are they at level five plus where they have thousands and 10 thousands of agents running autonomously, and they need oversight and governance to be completely fully autonomous.
所以从一级到五级,前线部署治理团队的重点是,第一,了解你处于哪个级别。
So going from level one all the way to level five, the forward deployed governance team is really focused on understanding one, where you're on that level.
然后第二,帮助你通过人员、流程或我们的产品,真正把治理监督带到你的组织里。
And then secondly, help you either with people, process, or with our product to really bring that governance oversight to your organization.
杰森,你可以想象,这是我深思熟虑的事情——随着AI能力的进步,实际上是人类的工作方式和流程跟不上。
And as you can imagine, Jason, you know, this is something I deeply think about is as AI capabilities are advancing, it is actually the human, our ways of working and our processes, that's not keeping up.
所以我们合作的很多组织都在努力弄清楚,怎么让我的员工更懂AI?
So many of the organizations that we work with are really trying to figure out how do I make my workforce more literate?
怎么确保每个员工的token使用量真的能说明我的员工在积极拥抱AI?
How do I make sure that the token per worker is actually an indication of, you know, the my my employees are actually being AI forward?
怎么确保我有合适的防护措施?
How do I make sure that I have the right guardrails in place?
因为对于上市公司来说,你可以想象,如果AI搞砸了,把产品推向市场,做出影响医疗、金融服务、就业决策的后果,那风险太大了。
Because for public companies, as you can imagine, there is a lot at stake if you get AI wrong and you're putting products out there that are making consequential healthcare, financial services, employment decisions.
而且,回到刚才说的,有很多法律法规你必须遵守。
And again, going back to there are lot of laws that you have to adhere to.
所以我们的前沿部署治理团队,重点就是帮你用人工智能转型和增长业务,从治理作为增长驱动力开始。
So our forward deployed governance team is really focused on helping you sort of transform and grow your business with artificial intelligence by starting with governance as a growth driver.
非常好。
Very good.
你们有多少这样的团队在外面部署?
How many do you of those do you have that are deployed out there?
我们是个小团队,大概90人。
We are a small team of about 90.
可以说,超过12%的团队成员是负责前沿部署治理的。
More than, I would say, 12% of my team is forward deployed governance.
所以,再说一遍,我们是一家A轮创业公司。
So, again, we are a series a startup.
就靠这90人的强大团队,我们在改变世界。
And with a mighty team of 90 people, we are changing the world.
一次一位前沿部署治理专家。
One forward deployed governance expert at a time.
很好。
Very good.
好的。
Alright.
那关于对话的现状呢?
How about in terms of the the state of the dialogue?
你知道,你参与了很多这类事情。
You know, you you are involved in many of these things.
我记得,你经常去华盛顿。
As I remember, you know, you you take trips to Washington.
因为你那家小而强大的系列A初创公司已经和许多机构、各种规模的组织合作,所以你参与了很多对话。
You you are part of a lot of the conversations because of the role your small but mighty, you know, series a startup has already doing with many of these with many of these agencies, organizations of all sizes.
你觉得现在的前进道路是什么样的?
What do you feel like the path forward looks like, you know, right now?
有没有理由感到担忧?
Is there reason to be concerned?
有没有理由感到希望?
Is there reason to be hopeful?
你觉得现在的对话进展到什么程度了?
Where do you feel like the state of, you know, this conversation is?
因为很明显,这已经成了全国乃至国际新闻,而且短期内不太可能消失。
Because obviously, it has now become this sort of national, international news, and it is something that is unlikely to be going away anytime soon.
你觉得最终会走向哪里?会是什么样子?
Where do you feel like it's it's gonna net out and what does it look like?
前进的道路是什么?
What's the path forward?
所以,Jason,你提了一个非常好的问题。
So, you know, it's Jason, you bring up a really good point.
我觉得自从Credo AI成立以来,我们在生态系统中扮演了一个非常有趣的角色,因为我的观点是,治理要跟上AI能力的发展,顺便说一句,这并非因为治理投入不足,我们稍后可以聊聊这个。
I think since the founding of Credo AI, we've played a very interesting role in the ecosystem because the thesis that I had was for governance to keep pace with, you know, AI capabilities, which by the way, it's not because of the lack of investment in governance, and we can talk about that in just a bit.
我觉得我们的角色不仅仅是打造优秀的产品,还要在这个生态系统中起到协调者的作用。
I think our role was really not just around building the great products, but it was also being sort of orchestrator in this ecosystem.
所以正如你所说,我们在企业领域确实有很好的进展。
So to your point, we have obviously great inroads into enterprises.
我们与政策制定者、监管机构和标准制定机构有很多合作,从NIST一直到ISO。
We work a lot with policymakers and regulators and standard setting bodies all the way from NIST, although, you know, to ISO.
我们还与前沿实验室合作,并与他们保持持续的沟通。
We also work with frontier labs and our constant conversation with them.
所以,我们作为协调者的角色,实际上是要把治理标准融入到能够推动企业转型的产品中。
So our role as an orchestrator is really to enable how do we bring governance standards into products that can scale enterprise transformation within these enterprises.
我之所以从这一点开始,是因为这就是我们未来的方向。
And the reason I started with that is where we are headed.
而你问的是一个创始人,创始人总是处于既乐观又警惕的状态,因为这就是创始人的角色。
And and you're asking a founder who always operates in a constant state of optimism and paranoia because that's the role of a founder.
当然。
Of course.
当然。
Of course.
对我来说,是的,我确实有一种担忧,就是这些AI能力发展得太快了。
For me, yes, there is a paranoia around, wow, these AI capabilities are advancing so fast.
作为一名工程师,我亲眼看到,当完全自主的智能体每天可能完成我工作中很大一部分时,而且我们在Credo AI也在关注每个员工配备的智能体数量,你能感受到那种变化的速度。
As an engineer, I am seeing firsthand that when you have fully autonomous agents doing maybe, you know, a huge percentage of my work on a daily basis, and and, you know, we are looking at agents per worker within Credo AI as well, you're seeing that pace of change.
它正在达到指数级的增长点。
It is getting to that exponential point.
所以我觉得这种担忧其实源于,天哪,这是不是治理债务?
So I think the the paranoia really stems from, oh my god, is that governance debt?
我们能不能弥补它?
Are we gonna be able to close it?
我现在不知道,如果没有正确的投资,没有我在信任光谱上提到的各个方面的正确关注,我们怎么能做到。
And I don't know right now how we can without the right investment, without the right focus across the spectrum of trust that I mentioned.
但作为创始人,我也乐观地思考,这对我们来说是一个多么好的机会,第一,通过正确的投资,我们可以构建更好的治理产品。
But then, as a founder, I also think about optimistically, what a great opportunity for us to, one, actually, with the right investment, build better governance products.
这也是我每天专注的事情,不只是让Credo AI成功,我觉得整个治理、保障、安全生态系统都会一起崛起。
And and I think this is my day in, day out focus, not only Credo AI winning, but I think this is the moment the entire governance assurance security ecosystem is all going to rise.
这对我们来说是个绝佳的机会,不仅是为了繁荣的未来,还能让每个人在经济上受益。
And what a great opportunity for us from, you know, not just like a prosperous future, but economically benefiting everybody.
我觉得这波浪潮会让所有船只一起升高。
I think the the the this wave is going to make all the boats rise together.
第二个乐观的点是,我们在国际层面看到了大家团结起来,讨论如何合作制定国际标准,来衡量这些前沿实验室,以及当它们在企业中部署时,特别是在用例层面,应该是什么样子。
The second optimism point is the rallying we are seeing at the international level on having this discourse as to how do we collaborate on international standards for what good looks like for measuring these frontier labs, but also when they deploy in enterprises, especially at the use case level, what does that look like?
第三个让我非常兴奋的乐观点是,我觉得这是研究界、AI研究社区、像我们这样的AI专家,以及政策制定者和监管者之间的一个很好的汇聚。
And then the third point of optimism that I get very excited about is I think this is a really nice coming together, even the different perspective of the research, AI research community, AI experts like us, and the policymakers and regulators.
我觉得我们越是能倾听、交流想法、达成共识,就越能更好地为这项基础技术设置护栏和治理,这项技术会一直存在,而且不会很快减速。
And I think the more we can listen and exchange ideas and align on ideas, the better it is going to be for us to guardrail and govern this foundational technology that is here to stay and not going to slow down anytime.
我觉得这不仅是对我们国家,也是对全人类来说,一个非常重要、不仅具有竞争力而且有益的努力。
And I think it's gonna be a very important, not only competitive, but beneficial, endeavor for for our country and for humanity.
你知道吗,我们上次见面时,你记得你当时把这件事放在了一个更大的视角里。
You know, when we met last time, remember you putting it in perspective.
你上次提到了一些数字,让人对治理的挑战有了更直观的认识。
You had some numbers that put in perspective the challenge of governance.
你能提醒我一下那些数字吗?就是那些显示资源挑战的,你之前说的那种艰难处境。
Can you remind me what they were there's some resource challenges that show, you know, the uphill battle that you were talking about before.
哦,对。
Oh, yes.
从哪儿说起呢?
Where to start?
我们还有多少时间,Jason?
How many more hours do we have, Jason?
但我觉得具体来说,有三点我想指出来。第一,看看投入前沿AI能力和AI应用层的资金量。
But I think just like concretely, I think there's three things that I would love to point out too is one, if you look at the amount of investment that's going into frontier AI capabilities as well as the AI application layer.
最近Gartner发布了一项很有意思的研究,说我们今年在那些前沿AI能力和应用上大概要花两万亿美元。
There was a very interesting research which was put out recently by Gartner that we are spending about $2,000,000,000,000 just this year on on that frontier AI capabilities and and application.
相比之下,投入到治理、安全和可观测性上的总额大约只有五亿美元。
And by comparison, the amount that is going collectively into governance, security, and observability is around, you know, $500,000,000.
所以这个比例大概是五千比一,这样你就能理解我们在能力上投入了多少,而在治理上又投入了多少。
So that's like a 5,000 to one ratio just to give you things in context of how much we are investing in capability versus how little we are investing in governance.
顺便说一下,在创业生态圈里,风投资金的情况,我还没做过具体分析,但我确实想做。
And and that, by the way, in the in the startup ecosystem, the VC dollars, I have not done that analysis, but I do want to.
那也是一塌糊涂。
That is also abysmal.
这是第一点。
So that's one thing.
第二点其实更让人大开眼界,而且还在变化中,就是在企业内部,我经常跟上市公司董事会交流。
The second thing which actually is even more eye opening and it is shifting is within an enterprise, when you look at I speak to a lot of public company boards.
大多数上市公司董事会,由于历史原因,花了一些时间,但现在他们都有网络安全方面的董事了。
Most of the public company boards, given historical context, took a little bit time, but now they have cybersecurity, you know, board members.
他们对安全非常重视。
They have a lot of focus on security.
他们也有独立审计,但他们还没有真正解决如何设立一个AI治理的专门角色,而这个问题现在至关重要。
They have independent audit, but they haven't really grappled with how do I create AI governance focus role, which is paramount right now.
所以现在,因为AI影响到业务的方方面面,很多公司都成立了专门的委员会。
So there are now, right now, committees of individuals involved because AI is impacting all parts of your business.
这是很好的第一步,但我认为这需要转化为企业在AI应用和AI治理上的实际投入比例。
Great first step, but I think that needs to translate into how much is an enterprise spending on AI adoption versus AI governance.
你知道,我们可以分享一些数据,但我觉得这个比例依然非常低,不到1%。
And, you know, we can share some numbers, but I think it's, again, very abysmal, less than 1%.
我认为这种情况正在改变,这对我们的业务来说是好事,我们也看到了变化,但速度需要更快。
And I think that is changing, which is great for our business, and we are seeing that, but it needs to happen faster.
最后我想说的第三点是关于AI素养的。
And then the third thing that I'll leave you with is just, I think, in terms of AI literacy.
我觉得你连怎么用都不懂,就更谈不上治理了。
I don't think you can govern something that you don't know how to even use.
所以我们看到真正原生AI和前瞻性AI组织的数量,
So we are actually seeing the number of AI native and AI forward organizations.
我觉得还不到1%。
They are like less than, I would say, 1%.
上市公司以及大型企业需要在AI素养方面投入更多,这样员工才能受益于这项技术,同时也能以安全、合规、可信的方式使用它。
And the public companies, as well as large enterprises need to invest more in AI literacy so that your employees can benefit from this technology, but also do so in a safe, compliant, trust trusted manner.
非常好。
Very good.
好的。
Alright.
Navina,这些数据确实让人警醒,但同时也充满希望,就像你提到的,戴上企业家的帽子,看到了其中的机遇。
Navina, very sobering stuff, but also, hopeful as you, touched on your your put your entrepreneur hat on and, saw the opportunity there.
我还想向观众澄清一点,就是在这种情况下,治理是一个总括性的概念,它也包括网络安全、隐私等等这些方面。
I one thing I wanna make sure to to clarify as well for the audience is, you know, governance is an umbrella in this case that is including cybersecurity, privacy, all of those things as well.
为了帮助大家理解,我想说明一下,治理并不仅仅等同于监管。
And just to help frame it so that we can help people so that the frame isn't just governance equals regulation.
在这种情况下,治理意味着安全、保障、隐私、数据主权,以及所有这类事情。
In this case, governance means safety, security, privacy, data sovereignty, all of these kinds of things as well.
完全同意,Jason。
Absolutely, Jason.
我知道,你知道的,我是个吃货,所以我就用食物来打个比方吧。
I know, you know, again, I'm a big foodie, so I'll leave you with the food analogy.
治理和监督其实就是你整个组织、AI部署、用例以及运行时的问责机制。
I you know, governance and oversight is really around accountable mechanisms across your organization, across your AI deployments, your use cases, and across runtime.
它就像这三层蛋糕一样层层叠加,全面审视你在数据主权、隐私保护、安全态势、可靠性、稳健性、内部测试和监控方面所做的决策和选择。
And the way that it layers in across these three layers of cake is it holistically looks at what are what are the decisions and choices you're making in your data sovereignty, in your privacy posture, in your security posture, in your reliability, robustness, in your internal testing, in your monitoring.
它收集所有这些信息,然后纵观组织、用例和运行时这三层蛋糕,帮你决定作为企业应该做什么、不应该做什么。
And it gathers all that information and then looks across this three layer cake of organization, use case level, and runtime to make a sort of like a a decision on what you should as a business be doing versus not doing.
而真正重要的是,我们经常强调的一点是,治理就是大脑。
And what becomes really important is something we share a lot is governance is the brain.
它决定做什么以及为什么做。
It's deciding the what and the why.
你应该构建什么,为什么要构建,以及如何为你的客户和不同的利益相关者部署它。
What you should be building, why you should be building it, how should you be deploying it for your customers, for your different stakeholders.
而安全、可观测性、隐私这些,就是肌肉。
Whereas the security, the observability, privacy, these are the muscles.
它们实际上是在运行时发现问题,然后把信号反馈给大脑,说别再这样做了,或者我允许这样做,这些是你必须做出的改变。
They are actually at runtime figuring out what things are going wrong and feeding that signal back into the brain to say, let's not do it again, or I'm allowing it, and these are the changes you have to make.
所以治理确实是一种整体姿态,它从技术层面反映了企业所承诺的内容。
So governance really is that holistic posture, which informs very technically what you as a business have committed to.
而且你有没有证据和验证来支持你的系统和代理在运行时和用例层面真正履行了这些承诺?
And do you have the evidence and verification supporting that your systems, your agents are actually holding, those commitments responsible at runtime at use case level?
非常好。
Very good.
好的。
Alright.
我要问你两个每次节目结尾我都会问所有嘉宾的问题。
I'm gonna ask you the same two questions I always ask at the end of every episode of all of our guests.
第一个问题是,你现在正在使用的AI工具是什么?
The first one is what is the AI tool that you are using right now?
你显然接触了很多创始人和公司,见识很广。
You obviously have access to a lot of founders and companies and you see a lot.
那你现在正在用的、希望更多人知道的工具是什么?
But what is the tool that you are using right now that you wish more people knew about?
这个问题有点难,因为你知道,我是Claude的忠实用户,也是重度用户。
That is a tough one because, you know, I am a very avid and then a super user of Claude.
从我们的编程工作到制作设计模板和视频,都用它。
So everything from our coding endeavors to building, you know, design templates and videos.
我觉得Claude是我每天都会用的首选工具。
I would say that Claude is my go to day in, day out.
另外,我也对GPT的能力印象深刻,尤其是Astra模型。
I am also actually very impressed with, you know, GPT capabilities, especially Astra models.
所以目前这两个是我的首选。
So those are my two go to right now.
除了这些,我没用过其他工具,因为它们确实能满足需求。
I have not used any other tools beyond that because they actually meet the purposes.
如果它们不满足,我实际上会用它们来构建代理。
If they don't, I actually end up building agents using that.
非常好。
Very good.
那在AI时代,有一个承诺是,AI能自动化掉很多琐碎的工作,很多我们不想做的事情。
How about, you know, in the age of AI, one of the promises was like, oh, AI is going to be able to automate away a lot of the the grunt work, a lot of the things that we don't wanna do.
所以想法是,你可能会有更多时间出去走走,但眼下这种情况似乎不太可能。
And so the idea is maybe you would sort of touch grass a little bit more, but the likelihood of that seems pretty low at the moment.
我认识的越来越多用AI的人,现在反而工作得越来越多。
The more people I know that are doing more and more with AI tend to be working more and more at the moment.
所以,我喜欢问领导者一个问题,因为现在领导者都在讨论如何优化时间以获得最大杠杆。
And so, one of the things that I love to ask leaders are, because leaders now are talking about how do I optimize my time for maximum leverage?
我通常怎么利用这些工具的机会来优先处理最重要的机会?
How I usually, how do I use my opportunities with these tools to prioritize my highest opportunities?
你对领导者有什么建议?
What's your recommendation for leaders?
对于在AI时代试图最大化时间杠杆的领导者,你最好的建议是什么?
Your best tip for leaders who are trying to maximize, you know, get maximum leverage for their time in the age of AI?
好问题,Jason。
Great question, Jason.
你知道,我经常跟我的团队说,要专注于三件事,因为那是我们现在的超能力。
You know, I always tell my team is focus on three things because that's our superpowers right now.
第一,在某个领域深耕,成为专家。
One is develop deep expertise in one domain.
因为归根结底,真正重要的不是我们能不能跟AI比产出,也不应该去比。
Because at the end of the day, what's gonna become important is these we cannot compete and we should not be trying to compete with AI in terms of creating outputs.
但那些产出有效吗?
But are those outputs valid?
准确吗?
Are they accurate?
这真的需要靠我们自己的专业判断。
Will really require our own expertise.
所以把大语言模型当成一个裁判,让它来评判其他大语言模型的产出。
So think about LLM as a judge, where LLM is judging the outputs of LLMs.
我真心相信,如果我们在某些领域深耕专业能力,我们就能更好地与AI合作。
I truly believe that we if we develop our expertise in certain areas, we can actually work better with AI.
而且人类和AI的结合会强大得多。
And the human AI combination is gonna be so much greater.
所以我总是告诉我的领导们,专注于在某些领域建立深厚而广泛的专业知识。
So I always tell my leaders, focus on building expertise deep and wide in certain areas.
第二件事,就是在品味上大力投资。
The second thing, big investment in taste.
因为我真的相信,创造产出很容易,但创造美观、有用的产品,靠的是品味。
Because I truly do believe creating outputs is so easy, but creating beautiful products, useful products, it's a matter of taste.
而且我觉得人类天生就有这种能力,怎么说呢,就是热爱美的能力。
And I think humans have this fundamental, I would say, ability to have that, you know, to, you know, to to love beauty.
所以我认为现在正是我们应该更注重品味的时候,这样才能做出正确的产出。
So I think this is the moment in time that we should actually be leaning into our tastes a little bit more so that we can build the right outputs.
然后我要求我的领导们花大量时间做的第三件事,就是判断力。
And then the third thing that I ask my leaders who spend a lot of time on is judgment.
因为归根结底,是的,你知道,我举个例子,我创建了一个智能体,它基本上是在模拟我的董事会,在我参加董事会会议之前,它会帮我准备好所有可能被问到的问题,关于我的公司,来自董事会的各种情况。
Because at the end of the day, yes, you know, I I, for example, have created an agent which literally is sort of replicating my board and is, before I go into a board meeting, setting me up with all the different scenarios that are gonna be posed to me about my company from the board.
然后我发现,在那一刻,判断力非常重要,这样才能以真正提升我作为CEO和创始人运作方式的方式与AI协作。
And then what I have found is in that moment, the judgment matters so much to be able to collaborate with AI in a way that is going to actually amplify the way I operate as a CEO and founder.
所以我认为,投资于专业知识、品味和判断力,是我们人类和AI一起获胜的方式,而不是只靠其中一方。
So I think investing in expertise, taste, and judgment is the way that we win human and AI together, not one or the other.
Navina Singh,非常感谢你来到这里。
Navina Singh, thank you so much for being here.
在你非常忙碌的时候,能有这样一场精彩的对话,而且我们正处于行业中的一个非常关键的时刻。
What a great conversation during a very sort of busy time for you, and what a, you know, a very powerful moment, you know, that we're in in the industry.
感谢你所做的一切工作,也感谢这次对话。
Thanks for all the work that you're doing, and and thanks for conversation.
Jason,非常感谢你邀请我。
Thank you so much for having me, Jason.
关于 Bayt 播客
Bayt 提供中文+原文双语音频和字幕,帮助你打破语言障碍,轻松听懂全球优质播客。