为什么现实世界的AI比基准测试更难 | EliseAI的Mario Martone

我没想到会这样，但即使是从弦理论转过来，我也得说我们现在研究的问题的复杂度可能更高了——

I did not expect that, but even after coming from string theory, I would say the complexity of the problem we're working on is probably higher now-

哇

Wow

……比以前更高。

... than it was before.

一个90%时间都能正常工作的AI系统，在论文里可能看起来很厉害，但当你要处理数百万条关于人们住房和医疗的对话时，剩下的那10%就变成了一个巨大的问题。

An AI system that works 90% of the time might look impressive in a research paper, but when you're handling millions of conversations about people's housing and healthcare, that remaining 10% becomes an enormous problem.

在本期节目中，EliseAI的应用研究主管Mario Martone加入MTS，解释将AI从实验室带入关键行业需要做些什么。

In this episode, EliseAI Head of Applied Research, Mario Martone, joins MTS to explain what it takes to move AI from the lab into essential industries.

Mario在理论物理学领域深耕了15年，研究过黑洞和弦理论这类问题。

Mario spent 15 years in theoretical physics, studying problems like black holes and string theory.

现在，他把同样的研究思维应用到生产环境的AI上，他的团队需要决定什么时候用大语言模型，什么时候传统机器学习其实更好，以及如何评估在数百万次真实交互中运行的系统。

Now he applies that same research mindset to production AI, where his team has to decide when to use an LLM, when traditional machine learning is actually better, and how to evaluate systems operating across millions of real-world interactions.

他们深入探讨了EliseAI的语音技术栈、微调开源模型的工作，以及公司如何自动化工作流程，让一个人从找公寓到看房，最后搬进去的整个过程都变得顺畅。

They get into EliseAI's voice stack, its work fine-tuning open-source models, and how the company has automated workflows that can take someone from looking for an apartment to touring it and eventually moving in.

对马里奥来说，智力上的挑战并没有变得更容易。

And for Mario, the intellectual challenge hasn't gotten easier.

他认为，他的团队在生产型人工智能中处理的一些问题，可能比他在理论物理学中遇到的问题还要复杂。

He argues that some of the problems his team is tackling in production AI may be even more complex than the ones he encountered in theoretical physics.

大家好，欢迎回到MTS。

Hello everyone, and welcome back to MTS.

今天，我们请到了博士博士。

Today, we are joined today by Doctor Doctor.

我们有两位博士。

We have two PhDs.

两位博士。

Two PhDs.

所以，再次介绍，双博士，马里奥·马托内。

So again, a double doctor, Mario Martone.

他是EliseAI的应用研究主管，这家公司为住房和医疗保健等关键行业构建对话式人工智能，而且他有理论物理学的背景，是一位物理学家。

He is the Head of Applied Research at EliseAI, a company building conversational AI for essential industries like housing and healthcare, and he has a background in theoretical physics, a physicist.

他现在带领一个团队，把最新的机器学习成果应用到现实问题中。

He now leads a team applying the latest machine learning advancements to real-world problems.

马里奥，欢迎来到MTS。

So Mario, welcome to MTS.

谢谢。

Thank you.

太好了。

That's great.

对，欢迎来到节目。

Yeah, welcome to the show.

我确实想了解一下你的背景，因为你最初是教授和理论物理学家，现在你实际上把这些知识都应用到了人工智能领域。

I did want to ask a little bit about your background because you started off being a professor and theoretical physicist, and now you're actually applying all of that learning into applied AI.

我刚才在直播前我们聊了几句，说到为什么把这些技能应用到像住房和医疗这样重要的行业如此关键，这些可以说是最重要的行业了。

And I was just sharing, we were having a little conversation right before we went live about why it's so important to use these skills and apply them, especially to essential industries like housing and healthcare, probably arguably some of the most essential industries.

所以我想先了解一下你，为什么会有这样的转变，为什么选择EliseAI，你从中带走了什么？

And so I wanted to just start off learning a little bit about you, why that changed, why Elise AI, and what did you take with you?

嗯，我觉得你说得对，这确实是个很大的差距。

Well, I think I agree with you that it's a big gap.

准确地说，我花了15年时间研究宇宙、黑洞和弦理论。

So just to be precise, I mostly worked for 15 years on trying to understand the universe, black holes, string theory.

太酷了。

So cool.

这些和我在EliseAI每天做的事情完全不同。

Things that are very different from what I do daily on EliseAI.

但我认为有一件事其实非常重要。

But I think there is something that actually is quite important.

我们正处在AI飞速发展的时代，未来会怎样真的很难说。

We are living in this moment where AI is moving so fast, and it's really unclear what the future will hold.

所以有很多悬而未决的问题。

And so there are extremely open questions.

虽然研究弦理论时事情进展很慢，比现在慢得多，但这里就像砰砰砰，每天都在变化。

And so the similarity that I can see, while obviously, working on string theory, things take forever, is much slower than the pace, and here it's like boom, boom, boom, every day there's a change.

但问题是，那些能够真正深入思考问题、非常有创造力、不断提出新想法的人，会有巨大的优势，因为AI每天都在变化。

But there is a question of the people who are able to really think deeply about the problems, can be very creative, can constantly come up with new ideas, have a huge edge, because AI is a new field every day.

所以，我觉得正是这一点让我走到了今天，也让我能够产生很大的影响。

So that's the thing that really, I think, brought me here and allowed me to really have a big impact.

那你给我讲讲这个领域的情况吧。

And tell me a little bit about the space.

有一件很重要的事，我可能有点偏见，因为我们上周刚在纽约证券交易所请了Mina Song来聊，我们一直在讨论关键行业，以及为什么把这种智能真正应用到这些行业里非常重要。

One of the big things, I'm a little bit biased because we just had Mina Song on last week at the New York Stock Exchange, we were talking all about essential industries and why it's so important to actually use this intelligence and get it diffused inside these industries.

仅仅在实验室里拥有更强的模型，并不一定意味着这种智能的影响能真正扩散到现实世界，扩散到实际的行业中去。

Just having stronger models on their own inside labs doesn't necessarily mean that the impacts of that intelligence actually gets diffused into the real world, into actual industries.

事实上，对我来说，我本来有一个选择，因为我当时对进入工业界有点兴趣，比如我可以尝试加入某个前沿实验室。

And in fact, for me, one alternative I could have done, because I was kind of interested in going to industry, is to, for example, try to join one of these frontier lab.

但在前沿实验室做的研究，虽然和理论物理明显不同，但相似度要高得多。

But the research that you do at frontier lab, it's kind of obviously different from theoretical physics, but it's much more alike.

你是在解决那些大问题。

You're working on these big problems.

但我真正想要的是在现实世界中产生影响。

But one thing I really wanted is to have an impact in the real world.

我觉得Elise在这方面是一家非常独特的公司，因为虽然这些大语言模型的能力在不断增强，但我们每天都在面对真正的挑战，那就是这些模型做出的决策会带来什么样的实际后果。

And I think Elise is such a unique company in that respect because you can have these LLMs that are getting growing capabilities, but we are facing every day with the real challenges of what it means for have consequential decisions.

所以实际上，Elise的模型为我们的产品所做的决策，不仅仅是屏幕上随机出现的内容，而是会对人们的生活产生实实在在的影响。

So actually, the decisions that the LLMs make for Elise, for our products, they're not just some random things that appear on the screen, but drive impacts on people's lives.

这种影响可能是非常积极的，也可能是非常消极的。

And that could be very positive or very negative.

这正是让我感到兴奋的地方，也是我觉得Elise是一个如此独特的工作环境的原因。

So that's the thing that really excites me, and I think makes Elise kind of such a unique space to be in.

完全同意。

Completely.

我也很兴奋。

It excites me, too.

你的角色有什么独特之处？你在Elise AI做应用研究。

What's unique about your role, so you do applied research at Elise AI.

我觉得现在Elise AI的技术栈，包括预测续约风险，也包括管理语音延迟。

I think right now, Elise AI's tech stack, it includes predicting renewal risks, and it also includes managing voice latency as well.

所以，因为你处在应用研究负责人这个层面，我想知道你能不能多讲讲这个技术栈，以及这些工程问题具体难在哪里。

And so because you're at this layer where you're the head of the applied research angle, I wonder if you could just tell me a little bit more about this tech stack and what specifically is really hard about these engineering problems.

有很多事情都非常难。

There is many things are very hard.

我想说你刚才提到的那些就是最好的例子。

I would say that you touched upon some of the kind of the best examples.

我们在语音方面起步很早。

We were very early on, on voice.

我们已经做了三年了。

We've been in it for three years.

这也是我喜欢Mina和Tony以及他们愿景的原因之一。

And that's one of the things I love about Mina and Tony and their vision.

我们确实是在时机还太早的时候就开始做了。

We definitely started when it was way too early.

所以我们是在技术还不成熟的时候，就努力想让它运转起来。

So we were trying to make it work when the technology wasn't there.

但语音处理真的非常难。

But voice is extremely difficult.

你要在短短一秒半的时间里，完成几十甚至上百个操作。

You're talking about doing tens, if not hundreds of operations in a span of a second and a half.

而且你还提到了另一个点。

And you're also touching upon another thing.

我们确实需要非常精准地理解人们的行为。

We really need to make sure that we understand ... people's behavior in a very precise way.

这也是Elise独特的地方，你提到预测续约风险，我们结合了前沿AI，也就是基于大语言模型的那一块。

So that's another thing that is unique about Elise is that when you touched upon predicting renewals, we have a mixture of cutting-edge AI in the more LLMs frontier base.

但我们也大量使用传统机器学习，因为我们有海量数据。

But we also use a lot of traditional machine learning, because we have a ton of data.

在这种情况下，大语言模型并不是最佳解决方案。

And so for that, LLMs are not really the answer.

所以我们有——

And so we have-

……真的是一整套工具，我们正在全力使用它们。

... really a full spectrum of tools that we're using, and we're using on full force.

比如说，对于续订预测，我们使用的是我们内部构建的非常复杂的深度神经网络，但这和LLM无关。

So for example, the predictions of renewals, we're using really elaborate deep neural network that we built in-house, but it has nothing to do with LLMs.

所以对我来说，作为应用研究负责人，这是一个相当有利的位置，因为我不必只盯着一个方向。

So for me, as a head of applied research, it's kind of a privileged position because I don't have to be tunnel vision on one thing.

我真的可以看到整个人工智能领域的进步。

I can really see the advancement of the entire field of AI across the board.

等等，跟我多讲讲你看到的这些进步，尤其是因为我觉得人们试图把LLM应用到所有这些问题上，但并不是所有问题都是完全概率性的。

Wait, tell me a little bit more about this advancement you're seeing, especially because I think people try to throw LLMs at all these problems, and not all of these problems are fully probabilistic.

你需要一些确定性的角度。

You need some deterministic angles.

你需要混合所有这些不同的方法。

You need a mix of all these different things.

所以你就能从宏观层面俯瞰整个AI研究领域的发展方向，然后还能实际应用到公司里。

And so you get to have this bird's eye view of just what's happening at a macro level, where the entire field of AI and research is going, and then you get to apply it actually at the company.

那你再多讲讲这个吧。

So tell me a little bit more about that.

而且不仅如此，其中一个重要的事情就是做决策。

And then, not only that, but one of the major things is to make decisions.

比如说，我们遇到一个问题。

Or like, okay, we have a problem.

这是个问题。

This is a problem.

比如预测一个想续租的人续租的概率。

Renewing, producing a probability for a person that wants to renew their lease.

我们要不要用LLM？

Should we throw in an LLM?

我们要不要用深度神经网络？

Should we throw a deep neural network?

我们要不要用非常简单的机器学习模型？

Should we throw very simple machine learning models?

我觉得这就是我和其他人早期必须做出的那种决定。

And I think that's the kind of decisions that me and other people have to make early on.

所以在这里，就像很多最近加入AI领域的人一样，他们只有一个解决方案，就是LLM。

And so here, it's like many people who have joined AI very recently, they just have one solution for everything, LLMs.

我理解这种感觉，因为这些LLM能生成很好的答案，看起来非常聪明。

Which I understand the feeling because these LLMs can generate great answers, and it seems they're incredibly smart.

但当你深入表面之下，你会发现有些东西其实并不好。

But then when you dig a little bit deeper in the surface, you see that some of the crap is not really good.

而且有些问题，比如你提到的，确定性不存在。

And there are certain problems, for example, having, as you mentioned, determinism is not there.

对于相同的输入，LLM无法保持相同的行为。

Trying to have the same behavior given the input, LLMs fail.

概率也是LLM很难处理的事情。

And probabilities is also something that the LLMs have a really hard time to elaborate.

所以我需要看一个问题，试着理解两件事。

So I need to look at a problem, try to understand two things.

第一，什么是最有影响力的？

One, what's the most impactful?

第二，因为我们也必须快速交付，对吧，什么解决方案能让我们尽可能接近最优结果，但又是最快的——

And then two, because we also have to ship at a very fast pace, okay, what's the solutions that can really get us almost as close as possible to the optimal outcome but in the fastest-

——最快的方式？

... way possible?

然后我们就得在那上面加倍投入。

And then we have to double down on that.

你能给我一个例子，说明你在团队里实际用的这个方法吗？

Can you give me an example about this actual method that you're using within your team?

这样你就能看出传统产品在构建这些东西时可能哪里出了问题，尤其是在风险很高的情况下。

So you're able to identify what is wrong with the way that maybe traditional products are trying to build these things, especially when there's high risk here.

你不能搞错。

You can't get it wrong.

就算准确率是99%，那也远远不够。

Even if it's 99% accurate, that's not good enough.

必须达到99.999%，他们还开玩笑说，每多加一个9，就像多爬一座山。

It has to be 99.999, and they joke that every 9 that you add is another hill to climb.

挺有意思的，我还在看一些学术论文，那里就能看出真正的差别。

It's kind of funny because I still read some papers from academia, and there you see really the difference.

他们谈可靠性的时候，会说，如果大语言模型能到90%就不错了。

When they're talking about reliability, they talk, okay, if the LLM can take it up to 90%.

然后我们说，90%？

And then we say, "90%?

那意味着每年1600万通电话里，会有大量的失误。

That means for like 16 million phone calls per year, it means an enormous amount of failures.

这完全无法接受。

That is totally unbearable."

所以你真的能看到，那些从理论上思考这个问题的人，和那些真正为这些错误买单的人——也就是我们——之间的差别。

So you can really see the difference between people who are thinking about this problem theoretically and people who actually pay the consequences of those mistakes, us.

我们每天都会收到大量的投诉，而且这些错误对人们的生活影响也很大。

We get inundated with complaints and also the impact on people's lives.

所以我们做的事情，我觉得非常特别，至少对我来说，是真正让我投入、让我坚持下去的动力，那就是组建一个多元化的专业团队。

And so what we did, which I think is quite exceptional, and to me at least, is what really engages me and really keeps me going, is to build a mixture of expertise.

在Elise，我们有顶尖的学术人才，包括我自己，还有顶尖大学的教授，以及顶尖的开发者。

So at Elise, we have top academic talent, including myself, professors of top universities, as well as top developers.

所以这个团队能全方位地看问题，从编程到更学术的问题都能深入理解。

So you have a team that has all eyes, and so they can understand deeply from coding to more academic questions.

我们尝试用所有这些工具来构建评估体系，来理解——评估体系其实就是回答你问题的关键。

And we try to use all these tools to build evals, to understand-- evals is kind of the answer to your questions.

我们每天都有数百万次的对话。

We have these millions of conversations every day.

我们要做决策。

We make decisions.

我们能不能大规模地理解这些决策的影响呢？

Can we understand, at scale, the impact of those decisions?

一旦你开始这样做，并且系统性地去做，结果就会反过来告诉你答案。

And then once you start doing that and you're doing it systematically, the results speak back to you.

所以你真的能看到其中的差别。

So you really see that difference.

如果你只是小规模试试，比如五六个对话，看起来都挺好的。

If you just try it at small scale, five or six conversations, all look quite good.

但一旦规模上去了，到了百万级别，你就会发现不同的方法之间差异非常大。

But at scale, when you get to the millions, then you see that different methods diverge significantly.

不，这真的很有意思。

No, it's pretty interesting.

这几乎就像你把科学方法用在了产品设计上，以及你实际构建产品的方式上。

It's almost like you're applying the scientific method to product design and how you actually build the product.

我完全同意你的看法。

100% agree with you.

而且我坚信，科学方法是人类思想史上最伟大的想法之一。

And I'm a strong believer that the scientific method is one of the best ideas that the human concept ever generated.

我还想跟你聊聊Elise的AI语音模型，因为你特别擅长处理语音延迟的问题。

I want to talk to you a little bit, too, about Elise's AI voice model as well, because you specifically are able to manage voice latency.

我知道你们在语音和通信方面的工作，甚至比一些前沿实验室开始得还早。

I know you guys were working on voice and communication way before even some of the frontier labs were working on that.

但其中一个主要问题就是延迟，因为这不仅是软件问题，还涉及硬件。

But one of the big issues with it is this latency issue, because it's not just a software problem, it's also hardware.

你们要考虑的因素很多，尤其是你们——我记不清具体数字了，但你们的互动客户数量是数亿级别的。

You have all these considerations, especially you guys-- I don't remember the exact number off the top of my head, but in terms of the customers that you have who are interacting, it's hundreds of millions.

那是多少来着——

What was the-

我们大约有五百万——

We have about five million-

五百万

Five million

……每月电话量。

... phone calls a month.

每月五百万通电话。

Five million phone calls a month.

所以每年大约六千万，这个数字相当高了。

So about 60 million a year, which is pretty high.

没错。

Exactly.

对于这些通话，你不能让语音与消费者互动时出现那种奇怪的延迟。

And for those, you can't have this weird latency between how voice is actually interacting with these consumers.

那你是怎么看待这个问题的？

And so how do you think about this problem?

我很想听听你的想法，还有你团队的想法。

I'd love to get into your head, into your team.

那么在Elise AI，你们是怎么处理这个问题的？

So how do you think about this problem at Elise AI?

你们是怎么拆解它的？

How do you break it down?

你们是怎么决定先解决哪个问题的？

How do you figure out what's the first thing you guys should work on to tackle it?

所以，还是那句话，不幸的是，这有两个参数。

So again, unfortunately, this is- So there are two parameters here.

一个是，完全了解当前的技术栈。

There is one, knowing exactly the current stack.

就是说，你完全清楚当前的技术栈是什么。

Like you know exactly what the current stack.

这个技术栈大部分是我自己搭的，所以我有非常独特的视角。

I built it mostly largely myself, so I have a really unique vision.

还有一个很大的优势，我得说，我们完全是靠自己搭建的。

And another huge advantage to that, I have to say, we built it entirely on our own.

所以我们对它的每一个部分都了如指掌。

So we have full understanding of every single part of it.

所以你有完整的技术栈，你知道瓶颈在哪里。

So you have the full stack, like you understand where the bottlenecks are today.

哪些步骤耗时最长，对吧？

What are the steps that takes the longest, right?

然后，你是对外视角，接触很多不同的供应商、不同的变化，比如微调开源模型，或者更新的语音转文字、文字转语音模型。

And then, you are the outward-facing perspective, where you have a lot of different providers, different changes, like fine-tuning open sources or newer speech-to-text or text-to-speech models.

所以你有一系列可能的解决方案，然后试着把它们和当前的技术栈匹配起来。

So you have a list of possible solutions, and you try to match them with the current stack.

基本上，思路就是，如果我们当前技术栈里确实有很糟糕的部分，而又有新的解决方案可以应用，那就全力推进。

And basically, the idea is that, okay, if we obviously have something that is really bad in our current stack, and there is a new solution that you can apply, then you go in full force.

否则，就要不断理解和排序，根据市场上已有的解决方案，判断我们技术栈的哪部分可以改进。

Otherwise, you're trying to constantly understanding and prioritizing, okay, given the solutions that the market has built, this piece of our stack can be improved.

我还想说，我们为了加快这个过程，还做了一件事，就是建立了一套实验机制。

And I'm going to say that another thing that we did to speed up that process is that we built a way to experiment.

所以我们有一个非常紧密的反馈循环，每当有新东西出现。

So we have a very closed loop where something new happens.

有人在Slack上给我发消息说：“嘿，我在X上看到这个帖子。”我不一定非得相信它。

Somebody on Slack messages me and says, "Dude, I saw this post on X." I don't really have to believe it.

我甚至都不看那些基准测试。

I don't even look at the benchmark.

我们做的事情就是，好，把它搭起来。

The thing we do is like, okay, let's set it up.

在内部跑个实验。

Let's run an experiment internally.

这样我们就能看到每个模型在我们自己内部基准上的表现，而那些外部的是不能信的。

So we can actually see a performance of every model on our own internal benchmarks versus the other ones that you cannot trust.

等等，这太酷了。

Wait, that's so cool.

所以它从Slack进来，然后自动——这是全自动的吗，还是你手动介入，然后——

So it comes in on Slack, and then it automatically-- Is this all automated, or do you come in and then-

我们确实有人工参与，但我们正在努力把它完全自动化。

We do have some human in the loop, but we are working on automating it actually fully.

哇，这太酷了。

Wow, that's so cool.

那接下来，跟我聊聊你的研究背景吧。

And then, so tell me a little bit about this research background that you have.

你是伦敦的教授，对吧？

You're a professor in London, right?

在顶尖大学之一。

In one of the top universities.

那跟我说说，你从整个框架中学到了什么，特别是对理论研究，尤其是理论物理研究的方法。

And so tell me a little bit what you took from that entire framework for how you do research, especially theoretical physics research.

然后这些方法是怎么实际应用的，尤其是和你的团队一起？

And then how does it actually apply, especially with your team?

你们内部是怎么实际使用这些相同的方法的？

How do you actually use those same methods internally?

所以再说一遍，有两件事。第一，显然，拥有一定的数学技术实力和理解数学是有好处的，这确实有帮助。

So again, the two things is, one, obviously, there is a gain of having some sort of technical strength in mathematics, and just understanding math, and it helps.

还有编程，这些东西实际上很有用。

And programming, those things are actually useful.

但你也知道，提出一个新想法是什么感觉。

But also, you understand what it's like to come up with a new idea.

理论物理在推理和思考方面处于人类的前沿。

Theoretical physics is at the cutting edge of humanity in terms of reasoning and thinking.

所以，提出新想法并不容易。

So coming up with new ideas, it's not easy.

你怎么才能获得真正改变一切的洞察力呢？

How do you get an insight that really changes things?

如果容易的话，每个人都能做到，所以这需要一种过程。

If it was easy, everybody would have it, and so there is a kind of a process.

然后这样做非常有用，我们再把这种方法带回来，对吧？

And then this is very useful to do it, and then we bring it back, right?

比如，我们就把这种方法用在了微调上。

So, for example, we brought this to fine-tune.

过去大约八个月里，我们大力投入的一件事就是尝试开源模型。

One of the things that we invested on heavily in the last eight months or so is to try open-source models.

所以当时的情况是，好，现在我们有机会真正自己动手做微调了。

And so the thing was, okay, now we have the possibility that we try really fine-tuning on our end.

那我们该怎么做呢？

So how do we do it?

在哪里做？

Where should we do it?

有哪些新东西是我们真正该尝试的？

What are the new things that we should really try?

而且因为我们有独立产生想法的背景，你也知道你已经解决了世界上最难的问题之一，所以你不会害怕自己动手去解决。

And because we have the background of being able to generate ideas independently, and you know that you've tackled one of the hardest questions that are out there, you're not scared of tackling it yourself.

于是我们组建了一个研究团队，开始全力攻克这个问题，而且我们真的在训练自己的模型上取得了惊人的成果——

And so then we set a group of researchers, and we started heavily tackling that, and we have been able to actually get incredible results with training our own models-

好的

Okay

……并且把它投入到了生产中。

... and delivering it to production.

这真是太令人兴奋了。

That's very exciting.

好的，感谢你的分享。

Well, thank you for sharing that.

我很好奇，你前几天在LinkedIn上发的另一篇帖子，讲的是那些研究背景很出色但对现实世界影响甚微的情况。

I'm curious as well, you posted on LinkedIn this other post the other day, and it was talking about brilliant research backgrounds with minimal real-world impact.

这也正是我们刚才聊到的。

And this is also what we were talking about right before.

那就是我。

It was me.

直到三年前，我就是那样的人，是的。

It was me like up until three years ago, yeah.

没错。

Exactly.

所以我觉得这真的很棒，尤其是你提到的，我现在能感受到你的热情，你对于把所有这些严谨的科学方法应用到像住房和医疗这样重要且实际的行业中去，感到非常兴奋。

And so I think that's really cool, especially as you mentioned, and I feel your energy right now, just getting so excited about applying all this scientific rigor, but to real-world and essential industries like housing and healthcare.

那你们Elise AI团队现在面临的最难的问题是什么？你们正在攻克哪些最棘手的问题？

And so what are some of the hardest problems you say your team at Elise AI right now, what are some of the hardest problems you guys are working on?

我会说有两个。

So I would name two.

一个是，比如，大语言模型发展得很快，对吧？

So one is, for example, LLMs are moving fast, right?

所以现在一个很合理的问题是，六个月后，哪些现在解决不了的问题，用大语言模型就能解决了？

So now one of the very legitimate question is, six months from now, what problems will be solvable with LLMs that are not solvable now?

然后再推到一年后，以此类推。

And then push it to a year from now and so forth.

所以我们要不断优先考虑，哪些事情可以用大语言模型来解决，哪些可以自动化。

So constantly prioritizing what are the things that we could solve with LLMs and we could automate.

再说一次，我们说的不是数学题。

And again, we're not talking about math problem.

我们说的是现实世界中的实际应用。

We're talking about real-world iterations.

所以你得确保，哦，这里有个真正棘手的问题，对数百万套公寓来说是个巨大的痛点，而现在我们真的能着手解决了。

So you have to be sure that, oh, here there is something that is really hard, that is a real huge frictions for millions of apartments, and now we can actually iterate.

这是我们一直在花时间处理的事情之一，而且这是高风险的决策，因为行动得早、行动得快，能让我们在六个月后处于更有利的位置。

So this is one of the things that we're constantly spending time on, and it's a high-stake decisions because moving early and moving fast allows us to be in a much better position six months from now.

然后另一件我觉得特别难的事情是，Elise 发展得很快——

And then the other thing that I think is extremely difficult is that, again, Elise is growing fast-

但我们公司还只有几百人，而我们运营的规模是每天自动化处理一百万通电话，还有几十万封邮件和短信。

... but we're still 600, a few hundred people in the company, and we operate at a scale with we automate millions, 1 million phone calls a day, hundreds of thousands of email and SMSs.

所以我们真的建立起了完全自动化的闭环，完全自动化。

So we really have developed loops which are fully automated, completely automated.

这当然很棒，很了不起，很辉煌。

And now this is wonderful, it's great, it's glorious.

但另一方面，你真的需要建立一些评估机制，这样你才能知道。

But on the other hand, you need to really build some evaluations so that you know.

我们有大量的对话在没有人——

We have tons of conversations that go on without our-

哇

Wow

...输入。

... inputs.

这些影响，再说一次，涉及住房和医疗。

And these impact, again, these are housing and healthcare.

这不是随便什么小事。

These are not some random thing.

这些真的会影响人们的生活。

These actually impact people's life.

如果我们犯错，就会带来重大后果。

There's consequential changes that happen if we make a mistake.

所以建立这些评估机制，让我们能用很少的人，比如十个人，每天就能了解对话进行得怎么样。

And so building those evaluations that we are able to really gather with few people, with 10 people, they're able to get an understanding every day how the conversations are going.

这真的是一项非常艰巨的任务。

That's a really, really hard task.

哦。

Oh.

说实话，这个我们基本上已经攻克了。

Which we have largely cracked, to be honest.

给我举几个具体的例子吧。

Give me some specific examples there.

那有没有什么例子，是你把整个流程都完全自动化的？

So what's an example of something that you fully automated this entire loop from?

哦，基本上，整个……语音通话这块都已经自动化了。

Oh, basically, the entirety of- ... voice calls is out of it.

比如说，我们自动化的两个主要流程，一个是租赁，就是那些对公寓感兴趣的人。

For example, the two main loops that we've automated, one is leasing people that are interested in apartments.

他们基本上可以从浏览网站、决定对公寓感兴趣开始，一直到搬进去，整个过程都能完成。

They basically can do the entirety of starting from looking at the website and deciding that they are interested in apartments to moving in.

这真的可以做到完全自动化。

It could be kind of literally fully automated.

比如说，你可以预约看房，然后AI会给你发一个密码，你到房子那儿，用密码开门进去。

For example, you can book a tour, then the AI sends you the code, you can go to the house, then you open the house with the code.

你直接跟AI互动，AI会给你介绍房子。

You interact with the AI, the AI is explaining you the house.

所以整个过程都是你自己完成的——

So, you're doing the whole thing-

哇

Wow

对吧？

... on your own, right?

另一件事是住户打电话来报修。

The other thing is for resident, calling in to have any issues.

这个环节我们还没能完全自动化，因为要完全自动化的话，需要机器人来配合。

There, there's still a gap that we've not been able to automate because we need robotics to fully automate that.

因为有些维修工作需要实际的人工操作，所以大语言模型在这方面确实做不了。

Because some of the fixes require actual human labor, and so LLMs can't really do that.

不过我们已经把很大一部分自动化了，因为现在你可以提交一个工单，说“好的，请修一下这个东西。”

But we have automated a large chunk of that because you can now submit a work order telling, "Okay, please fix this thing."

AI能够判断这个问题是否需要立即处理。

The AI is capable to understand whether or not it requires a fix immediately.

然后它会升级处理，找到维修商，打电话联系相关人员。

Then escalates it, finds a vendor, calls up a person.

最后一步就是真人，实际的人，上门来帮你修好。

And then the last piece is a person, actual physical person, coming into your house and fixing it.

也许过一两年我们就能把整个过程都自动化了。

Maybe we'll get to automating the whole thing in a year or two.

哇，这挺有意思的。

Wow, that's pretty interesting.

那我想了解一下你们接下来的计划。

So I want to know a little bit about what's next for you guys.

我和Mina聊过一个大话题，就是关于垂直扩展和水平扩展。

One of this big conversation I was having with Mina is around this vertical versus horizontal scaling.

我觉得Elise AI的独特之处在于纵向深入整个技术栈，尤其是在住房领域，因为你们就是从那里起步的，你们把每一个问题都彻底解决了，除了——

I think what's unique about Elise AI is vertically down the stack, especially when it came to housing, because that's where you guys started, you fully went through every single problem, especially except-

没错

Exactly

……

...

我猜是机器人真正走进你家门。

I guess the robotics physically going into your home.

我们会做到的。

We'll get there.

但你们会做到的。

But you'll get there.

而现在你们在更横向地应用它，所以现在也在医疗保健领域深入发展。

And now you're applying it more horizontally, so now also going really deep down in healthcare.

下一步是什么？

What's next?

你有没有考虑过进入另一个行业，还是继续在这个垂直领域深耕？

Are you thinking of another industry or going deeper on this vertical angle?

目前我们手头的事情已经很多了。

So right now we have our hands full.

我现在在做医疗健康。

I'm in healthcare.

这确实很不容易。

It's a lot.

医疗健康和住房——

Healthcare and housing-

这两个领域

These two in here

——加起来占了美国GDP的40%。

... is like 40% of the GDP in the US.

所以我们还有不少发展空间。

So we have room to grow a little bit.

而且，我们确实发现，打造有意义的AI最好的方式，是因为Elise有一个独特之处，就是我们拥有这些非常了不起的东西。

And especially, we do find that our best way to build AI that matters is because one of the things that's unique about Elise is that we have these incredible things.

我们有像我这样的理论物理学家，花了15年时间仰望星空，思考黑洞。

We have theoretical physicists like myself who have spent 15 years looking at the stars and thinking about black holes.

再加上他们积累的，再加上非常深入的领域理解和商业知识，对吧？

And combined with, they have acquired, combined with very deep domain understanding and business knowledge, right?

所以我们确实拥有这种独特的技能组合。

So we really do have this unique mixture of skills.

所以我们想继续沿着这个方向前进。

And so we want to continuously move that way.

所以我们希望根据客户的意见，来决定下一个大赌注是什么。

So we want to be driven by what is next big bet based on what our clients say.

比如说，我们现在面临的一个很重的问题，就是真正降低经济适用房的运营成本。

So one of the things, for example, that we are now facing pretty heavily is to really cut down the cost of running affordable housing.

这看起来是个巨大的问题，因为我们发现，经济适用房非常复杂，但又极其有影响力，而且对最弱势、最没有资源的那部分人群来说，尤其重要。

This seems like a massive problem because we are finding that affordable housing is a hugely complex, yet extremely impactful and relevant for especially the component of the population that has the least means, the ones that is most vulnerable.

所以我们真的很想让他们更能负担得起。

And so we really want to be able to make them more affordable.

而且这非常复杂。

And it's extremely complicated.

基本上完全没有自动化。

It's basically not automated at all.

目前这个行业里的技术深度几乎为零。

The amount of tech depth that is currently in the industry is nearly non-existent.

所以我们正在非常积极地尝试——而且这是一个巨大的部分——

And so we are really aggressively trying-- and it's a massive component-

……整个房地产行业的一部分。

... of the entirety of real estate.

所以如果我们能大幅降低经济适用房的成本，那将产生巨大的影响，会立刻惠及消费者，让他们的生活好得多。

So that would be massively impactful if we could bring down the cost of affordable housing significantly, so that will impact consumers immediately and make their lives much better.

我知道，这挺有意思的。

I know, that's pretty interesting.

一些最难的问题最能吸引顶尖人才。

Some of the hardest problems attract the top talent.

在结束之前，我还想问你最后一个问题。

And one last thing I wanted to ask you before we go.

我知道你的团队正在招人——

I know your team is hiring-

是的

Yes

——我想更清楚地了解你们在找什么样的人。

... and I wanted to just better understand what you're looking for.

你之前提到，你的团队里有来自不同背景、拥有不同技能的人。

You said earlier that you have a mixture of people with many different backgrounds, who have a mixture of skills.

你来自学术界。

You come from the academia world.

我只是想更深入地了解一下——

I just want to better understand-

是的

Yes

……在团队发展方面，你们在找什么样的人？

... what are you looking for when it comes to how you grow your team?

我们之所以主要在旧金山和纽约招人，尤其是在技术方面，是因为我们想要最优秀的人才。

So, the reasons why we are mostly hiring, especially on the technical side in San Francisco and New York, is because we want the best.

我们的标准相当高。

We're pretty high standards.

我们主要看重两点：我们想要最优秀的工程师。

And the two main things, we want best engineers.

有大量的工程工作需要完成，而我们在这方面人手严重不足。

There's a ton of engineering that needs to be done, and we are so short on that.

特别是在旧金山，我们的第二个中心发展得非常快，但我们还需要更多人。

And in particular, in San Francisco, our second hub is growing fantastically well, but we need more.

我们就是发展得不够快。

We just can't grow hard enough.

其次，我们特别看重研究人员，尤其是我个人。

And then secondly, we love, in particular me, obviously, researchers.

这类人通常有很强的研究背景，但他们内心真的渴望把自己的想法和深度思考转化为实际的影响力。

And the background there is people who have a very strong research background, but they really have this fire in them to wanting to use their ideas, their depth of thought in actual impact.

所以这种人才其实很难找，就像独角兽一样稀有。

So these are kind of very difficult, kind of unicorns to find.

他们既要有出色的学术背景，又要真正热衷于把研究成果应用到现实世界中。

People who have exceptional academic background, but yet they are really interested in driving that into real world.

也就是说，研究人员要明确专注于改善人们的生活，让生活变得更好。

And so researchers with a clear focus on impact people life and make it better.

这听起来真让人兴奋。

That's exciting.

好的，非常感谢你今天来参加我们的节目，Mario。

All right, well, thank you so much for joining us today, Mario.

谢谢。

Thank you.

这真是一次非常精彩的对话。

This is such a fascinating conversation.

我觉得特别有意思的是，这也是我和Elise的第二次对话了，能听到你的观点，还有Mina的观点。

I think what's really interesting getting to hear also, this is my second conversation right now with Elise, and getting to hear from your perspective, also Mina's perspective.

你们有着不同的背景和方法，你从理论物理的背景出发，把深厚的学术功底应用到我们现在在AI领域面临的一些最棘手的问题上。

But just the different background and approach that you have coming from that theoretical physics background, and then applying rich academia to some of the toughest problems we have now in AI.

我想对大家说，这完全出乎我的意料。

And I want to say to the people out there, I did not expect that.

但即使是从弦理论转过来，我觉得我们现在处理的问题的复杂性可能比以前更高——

But even after coming from string theory, I would say the complexity of the problem we're working on is probably higher now-

哇

Wow

……从某种意义上说，比以前更高。

... in a way, than it was before.

即使是从弦理论转过来的？

Even coming from string theory?

我是说——

I mean-

确实挺难的。

It's pretty hard.

这可是独家消息。

You hear it here first.

马里奥，非常感谢你今天来参加我们的节目——

Well, Mario, thank you so much for joining us today-

谢谢

Thank you

……在MTS上。

... on MTS.

感谢收听本期a16z播客。

Thanks for listening to this episode of the a16z podcast.

如果你喜欢这期节目，记得点赞、评论、订阅，给我们评分或留言，并分享给你的朋友和家人。

If you like this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family.

更多节目，请前往YouTube、Apple Podcasts和Spotify观看。

For more episodes, go to YouTube, Apple Podcasts, and Spotify.

在X上关注我们，账号是@a16z，也可以订阅我们的Substack，地址是a16z.substack.com。

Follow us on X, @a16z, and subscribe to our Substack at a16z.substack.com.

再次感谢收听，我们下期节目再见。

Thanks again for listening, and I'll see you in the next episode.

本信息仅供教育用途，不构成任何买入、持有或卖出投资或金融产品的建议。

This information is for educational purposes only and is not a recommendation to buy, hold, or sell any investment or financial product.

本播客由第三方制作，可能包含付费推广广告、其他公司提及以及与a16z无关的个人。

This podcast has been produced by a third party and may include paid promotional advertisements, other company references, and individuals unaffiliated with a16z.

此类广告、公司和个人均未获得AH Capital Management LLC、a16z或其任何关联公司的认可。

Such advertisements, companies, and individuals are not endorsed by AH Capital Management LLC, a16z, or any of its affiliates.

信息来源于发布之日被认为可靠的来源，但a16z不保证其准确性。

Information is from sources deemed reliable on the date of publication, but a16z does not guarantee its accuracy.