为什么多邻国成功了（与首席执行官路易斯·冯·安对话）

哦，是的。

Oh, yeah.

我们于2012年上线。

We launched in 2012.

但上线时我们没有赚到任何钱。

But when we launched, we made no money.

我们没有广告，用户也无法向我们付费。

We didn't have ads and you couldn't pay us.

我的意思是，我们没赚到钱，因为我们根本没有办法盈利。

I mean, we made no money because there's just no way for us to make money.

我们就是没把付费功能放上去。

We we just didn't put it there.

这种情况一直持续到2017年左右。

And that was true up until about 2017.

我们在2016年开始考虑盈利模式。

We started thinking about monetizing in 2016.

事情是这样的，我们刚刚从当时称为谷歌资本、现在叫资本G的机构获得了融资。

What happened was we had just raised funding from, at the time it was called Google Capital but it's now called Capital G.

我们刚从他们那里融了资，我不记得具体估值了，大概是十亿美元左右，但完全没有收入。

We had just raised funding from them and I don't remember the valuation, it's about a half $1,000,000,000 valuation, but with no revenue whatsoever.

来自那家机构的合伙人莱拉·斯塔迪飞到了我们在匹兹堡的办公室。

And the partner from there, Layla Sturdy, flew to Pittsburgh where we were.

她让我们喝醉了，然后说：你们必须从明天开始赚钱。

And, you know, she got us drunk, she said, you gotta start making money, like, tomorrow.

当然，作为创始人，你很清楚一个秘密：比赚很多钱更好的，就是一分钱都不赚。

And, of course, as a founder, you clearly knew the secret that the only thing better than making a lot of money is making no money.

中间的一切都不好。

Everything in between is not good.

是的。

Yeah.

没错。

Exactly.

不。

No.

那就是秘密。

That is the secret.

于是第二天我去办公室，这挺有意思的，因为在头几年里，我们根本没有赚钱，甚至根本没打算赚钱。

And and so went to the office the next day, and it was pretty interesting because for the first several years, we made no money, and we weren't even really trying to make money.

我们招聘的每个人都非常有使命感。

And everybody that we were hiring was very mission driven.

我们招聘了所有这些热爱免费教育理念的人。

We we were hiring all these people that were that loved the idea of giving free education.

所以当我来到办公室，告诉所有人：嘿。

So when I came to the office and I told everybody, hey.

我们必须想办法赚钱。

We have to figure out how to make money.

人们都很震惊，他们说：但为什么？

People were pretty shocked, and they were like, but why?

但我会说，总得支付你们的薪水吧。

But I'd say, well, kinda have to pay your salaries.

他们说，你这整个期间一直在付我们薪水。

And they said, you've been paying our salaries this whole time.

是的。

Yeah.

对。

Right.

到目前为止都没问题。

Hasn't been a problem so far.

是的。

Yeah.

你为什么不能继续做你一直在做的事呢？

Why can't you just continue doing whatever the hell it is you've been doing?

我不得不解释，你不能永远靠融资维持下去。

And, know, I had to explain you can't raise funding forever.

在这里，我们要举办一个研讨会。

Here, we're gonna do a seminar.

我们会进行一场关于晚期资本主义的Duolingo环节，你知道的，比如

We'll do a Duolingo session on late stage capitalism and, you know, like

你知道，这太疯狂了。

You know, it was crazy.

花了大约六个月的时间，才说服公司员工赚钱其实并不是邪恶的。

It took about six months to convince the employees of the company that it was in fact not evil to make money.

最终，人们被说服了。

Eventually, people got convinced.

我认为人们被说服的主要方式是，每个人都明白，如果我们赚钱了，就能为使命投入更多资金。

And I think the main way in which people got convinced is I think everybody understood that if we made money, we could invest more in the mission.

我认为每个人都觉得，哦，好吧。

And I think that everybody was like, oh, okay.

好吧。

Okay.

所以你的意思是，我们仍然可以把大多数人投入到真正打造更好的产品上，而不是专注于赚钱？

So you mean we can still put most people to work on actually making a better product as opposed to making money?

我说，当然可以。

And I said, sure.

当然可以。

Sure.

今天，我们依然如此，这就是我们的目标。

That's what we're gonna Today's day, it remains true.

实际上，公司里专注于赚钱的员工比例，相比其他人来说是很小的。

Mean, the fraction of people that are working on making money at the company is relatively small compared to the rest.

说服人们接受这一点花了大约六个月的时间。

It took about six months to convince people that we should do that.

另一个复杂的问题是，当时我们还不确定如何在不违背我们希望大多数人仍能免费学习这一原则的前提下赚钱。

The other thing that was complex is at the time we weren't sure how to make money in a way that respected the fact that we wanted to still to be the case that most people could learn for free.

那时候，Spotify正在成长，等等，但免费增值模式是否有效还不太明确。

You know, at the time Spotify was kind of growing, etcetera, but it was not super clear that this freemium model worked.

这种免费增值模式是这样的：你可以免费使用，但可能需要看一些广告；如果你不想看广告，就需要付费。

This freemium model, which is like, you can use it for free, but you may have to see some ads, and if you don't want to see ads, you have to pay.

当时我们并不确定这种模式能否行得通。

It wasn't clear that we could make that work.

结果证明，这种模式是可以行得通的。

It turns out you can make it work.

这太棒了。

It's great.

它运作得非常好。

It worked really well.

这种模式的成功对我们来说非常有利，因为它完全契合我们的使命。

And the fact that that worked was really good for us because it's very mission aligned.

基本上，你可以完全免费地学习。

Basically, you can learn entirely for free.

你可以从不向我们付费的情况下学习，但如果你想关闭广告，就可以选择付费。

You can learn without ever having to pay us, but if you wanna turn off the ads, can pay us.

这非常符合我们的使命。

That's very mission aligned for us.

但我们很幸运，这一点成功了。

But we're fortunate that that worked.

当时真的不太清楚能否做到。

It really was not clear back then.

这太有趣了。

It is so funny.

现在这简直是一个基本的假设，但在2017年、2018年，YouTube上你根本没法付费。

That's like just such a basic assumption now, but even 2017, 2018, YouTube, you couldn't pay them.

你只能看广告。

You could only watch ads.

Netflix上，你只能付费。

Netflix, you could only pay them.

你不能看广告。

You couldn't watch ads.

实际上，大卫，我认为谷歌贡献者这个功能是存在的。

Actually, David, I think Google contributor did exist.

天啊。

God.

我都不知道那是什么。

I don't even know what that was.

它是YouTube Premium的前身，可以在谷歌的所有平台上使用，你只需支付一笔费用，这笔费用相当于谷歌从你观看的所有广告中获得的收入。

It was the predecessor to YouTube Premium, and it worked across all their properties where you could basically pay the amount that Google would make on you from all the ads that you would see

跨所有平台？等等。

across Wait.

把所有AdWords广告都去掉？

All the strip out AdWords?

这不可能。

There's no way.

我知道。

I know

确实有。

it did.

我想除了AdWords之外，其他都包括了。

I think it might have been except for AdWords.

我不确定。

I don't know.

我应该把这件事的细节弄清楚。

I should get my facts straight on this.

我曾经是Google Contributor的付费用户。

I was a paying Google contributor customer.

我其实没什么可说的，但名字叫'Contributor'就已经说明了很多。

I had no I mean, says a lot that it was called contributor.

这真有趣。

That's funny.

无论如何，对我们来说，当时并不清楚这能否成功，但它确实成功了。

Well, in any case, for us, it was not super clear that this was going to work, but it worked.

这有战略意义吗？

Was it strategic at all?

有没有网络效应的因素？

Was there a network effect component?

比如，不给增长设置任何障碍是不是更好，因为用户越多会带来越多用户之类的？

Like, is it good to not put any inhibitors on your growth rate because more users begets more users or something like that?

我坚信，我们能够实现如此快速增长的原因在于我们没有过早盈利。

I am a huge believer that the reason that we have been able to grow so much is because we didn't monetize early on.

但我不能建议每家公司都这么做。

Now I cannot recommend that to every single company.

我的意思是，对某些公司来说，你可能确实应该尽早盈利。

I mean, for some companies you probably should monetize early on.

但在我们的情况下，发生的是这样一件事。

But in our particular case, what happened was this.

我们上线了。

We launched.

因为我们没有赚钱，所以也无法在营销上花钱。

Because we weren't making any money, we also couldn't spend money on marketing.

我们没法做效果营销，因为效果营销就是你花30美元给谷歌，他们就给你带来一个用户。

Like we couldn't do performance marketing because performance marketing is this thing where you, like, you can pay Google $30 and then they send you a user.

但对我来说，花30美元去获取一个用户却无法从他身上赚到任何钱，这简直太蠢了。

But to me, that seemed really dumb to spend $30 to get a user when we were not gonna make any money off of that user.

你希望从那个用户身上赚到超过30美元。

You wanna be making more than $30 from that user.

是的。

Yes.

或者不管是多少。

Or whatever it is.

你总得赚得更多才行。

You wanna just do more than that.

所以我们没法在营销上花钱。

And so we couldn't spend money on marketing.

我们没有赚到任何钱。

We weren't making any money.

我们有一群工程师，那我们该做什么呢？

And we had a bunch of engineers, so what were we gonna work on?

我们唯一决定要做的就是改进教学和提升用户留存。

And the only thing we decided to work on was teaching better and increasing user retention.

就这么简单。

That's it.

因为这是我们唯一能做的。

Because that's the only things we could work on.

所以我们从2012年到2017年基本上都在优化用户的留存率。

So we spent from 2012 to 2017 basically optimizing retention rate for users.

我们做了大量的优化。

And we optimized it a lot.

比如，我们优化了第一天留存率，也就是用户首次使用后，有多少比例的人会在第二天或次日回来。

I mean, for example, we went day one retention, so that's the retention of people when they show up, what fraction of them come back the second day or the next day.

我们刚推出时，这个数字是13%。

When we first launched that number was 13%.

也就是说，你在用户注册当天就失去了87%的用户？

As in you lose 87% of your users the day that they

嗯，他们可能之后还会回来。

Well, they may come back another day.

有87%的人恰好没有在第二天回来。

87% happen to not come back the day right after.

他们可能之后会回来，但这就是我们观察到的情况。

It could be that they come back later, but that's what we saw.

在这段时间里，我们不知道做了多少次A/B测试，但到2017年，这个数字已经达到了大约50%。

We ran who knows how many AB tests over this period of time, but by 2017 that number was, I don't know, probably about 50%.

我可能记不太清具体数字了，但我们确实大幅提升了所有留存指标，因为这是我们唯一能专注的事情。

I may be misremembering exactly the numbers, but we were able to drive all of our retention numbers by a lot because that's the only thing we could work on.

如果我们没这么做，很可能就会花大量时间优化广告投放流程，以及各种看似能赚钱却未必能真正推动增长的奇怪事情。

Had we not done that, we probably would have spent all our time optimizing our ad pipeline and optimizing all kinds of weird stuff that ends up making you money, but I don't think helps you grow that much.

我认为，事后看来，这是一个非常明智的决定。

And I think we were in retrospect, this was a very smart move.

当时，我们并没有意识到这有多明智。

At the time, I don't think we were thinking how smart this was.

我们只是顺其自然地做了。

We just kinda did it.

它为你创造了一个持久的资产。

It gave you a durable asset.

我的意思是，你现在拥有一个能够最大限度提升用户留存率的产品，这意味着你所建立的用户基础以及最终赚钱的机制要稳固得多。

I mean, the the fact that you now have a product that that ensures that you have fully maximized the potential for a user to stick around, it means the user base you've built and the machine by which you eventually do make money is far more durable.

没错。

That's right.

我也相信这一点。

And I believe that.

当然，现在我是带着 hindsight 才相信这一切的。

Now, of course, I believe all of this in retrospect.

那时候，这些事情我们根本没想那么多，就是觉得，我不知道。

At the time, none of this you know, we were just like, I don't know.

做个大家喜欢的东西。

Make a thing that people like.

那也是工作。

That was also work.

你当时是以五十亿美元的估值在融资。

You were raising money at a half $1,000,000,000 valuation.

那时候，钱在某种意义上是免费的。

This was when money was free in some sense.

就是觉得，来，拿更多钱吧。

It's just like, here, have more money.

但它确实奏效了。

But it it worked.

好的。

Okay.

所以你开玩笑说你根本没有做任何营销。

So you joke that you weren't doing any marketing.

但从某种意义上说，这确实是事实。

And then in some sense, that's true.

但从某种意义上说，又不是这样。

In some sense, it's not.

你们有那只绿色的猫头鹰。

You have the green owl.

你们一直以来都有一个出色的品牌。

You have had a great brand forever.

卡内基梅隆大学的两位博士，怎么会想到设计出一只可爱的绿色猫头鹰？我可不觉得卡内基梅隆是世界上顶尖的设计与营销学府。

How did a couple PhDs from Carnegie Mellon, of all places, come up with a cuddly Green Owl and build a I don't think of CMU as wonderful as an institution as it is as, like, the top design and marketing place in the world.

他们恰好有一个非常出色的设计项目。

They happen to have a really good design program.

但你看，这一切是随着时间慢慢演变的。

But look, this evolved over time.

我们当时只知道一件事。

We did know one thing.

我们确实想要一个吉祥物。

We did want a mascot.

早期我们就想，看啊，一个吉祥物 somehow 会对我们有帮助。

Early on, we were like, look, a mascot, somehow this will help us.

我们并不确切知道会怎么帮到我们，但我们确实想要一个吉祥物。

We didn't know exactly how, but we we did want a mascot.

最终我们选了猫头鹰，因为在西方文化中，至少它象征着智慧。

Eventually, chose an owl because of in Western cultures, at least, a symbol of knowledge.

它最后变成绿色是因为我的联合创始人讨厌绿色，我觉得如果它是绿色的会很有趣。

It ended up being green because my cofounder hates the color green, I thought it would be funny if it was green.

所以这就是它为什么是绿色的原因。

So that's why it was green.

现在你的整个办公室都出现在镜头后面了。

And now your whole office looked behind you.

是的

Yeah.

猫头鹰的第一个版本并不太可爱，或者说，老实讲，长得不太好。

First version of the owl was not very cuddly or I mean, it was kinda not great looking, honestly.

随着时间推移，它不断改进。

Over time, it evolved.

我们聘请了一些非常出色的插画师。

We hired some amazing illustrators.

我们聘请了这位至今仍在我们公司工作的格雷格·哈特曼，他现在是我们艺术总监。

We hired this this guy who still worked for us, Greg Hartman, who's, you know, our head of art now.

他逐渐创作出一只极其优美的猫头鹰形象，不断进化。

He just made an amazingly beautiful owl over time that that evolved.

所以，这是一方面。

And so that's one thing.

我们只是打造了一个看起来很棒的优质素材。

We just made a really good asset that looks good.

另一件发生的事是，它开始展现出个性，但这个个性是偶然形成的。

And then the other thing that happened is it started getting a personality, but the personality was a little bit accidental.

它获得个性的第一个方式是，最初这只猫头鹰根本不在应用里。

The first way in which it gets to the personalities, the owl was actually not in the app at all at first.

人们看到它的唯一地方是应用图标，那是一个非常近的猫头鹰脸部特写。

The only place where people saw it was the app icon, which is kind of a very close-up version of the owl's face.

此外，当你收到通知时，图标会显示出来，看起来就像是猫头鹰发来的。

And then also, when you get a notification, it has the app icon so it looks like it comes from the owl.

所以，猫头鹰个性的最初来源，其实就是我们在通知里写的内容。

So, the first place where the personality for the owl came up was basically what we said in the notifications.

我们开始在通知里写一些奇怪的话，我想人们正是因为这些内容，才逐渐赋予了它一些个性。

And we started saying some weird stuff in the notifications, and I think people started giving it a little bit of personality because of that.

然后，通知又发生了另一件事。

Then, another thing happened with the notifications.

在某个时候，我们决定：不能再这么频繁地发通知了，所以如果用户连续五天不活跃，我们就停止发送通知。

At some point we decided, look, we can't be super spammy, so we're going to stop sending notifications after five days of inactivity.

所以如果你五天不使用这个应用，我们就停止推送。

So if if you don't use the app for five days, we'll stop.

然后我们想到，既然要停止发送通知，

Then it occurred to us, well, we're gonna stop sending a notification.

不如干脆告诉你一声。

We may as well tell you.

对。

Right.

还能因此赢得好感。

Get the credit for it.

是的。

Yeah.

这样人们就知道我们是好人了。

So people know we're good guys.

这就是我们的想法。

This is what we thought.

所以最后一次通知，第五天我们写的，只是说：这些通知似乎不起作用了。

So that last notification, the fifth day we wrote it, it just said, these notifications don't seem to be working.

我们暂时会停止发送这些通知。

We're gonna stop sending them for now.

但正因为如此，互联网上开始出现各种梗，比如猫头鹰如何痴迷于让你回来，如果你不回来它就会杀了你——但你知道，那是社区的创作。

Because of that though, it started giving rise to all kinds of memes on the Internet about, like, how the owl was obsessed with getting you to come back and he would kill you if you didn't come back and but that, you know, that was the community.

我们并没有那样做。

We didn't do that.

之后，我们意识到可以利用这些梗。

After that, we realized that we that we could push on those memes.

于是我们开始拿这些梗自嘲。

And so we started making fun of ourselves with those memes.

这效果非常好。

And that has worked really well.

在某个时候，我们想到了找一套猫头鹰服装，让人可以穿上的那种。

And at some point, we got the idea of getting an owl suit, a suit that somebody could wear.

起初，我们认为这在招聘活动中有用，比如去大学招人时，猫头鹰服装会出现。

At first, we thought it was good it was gonna be good for recruiting events, like going to try to hire people at universities and the owl suit would show up.

结果发现，让一个人穿上猫头鹰服装在TikTok视频里做些奇怪的事情，能吸引很多人观看。

Turned out you could start putting somebody in an owl suit to do weird stuff in TikTok videos, and that gets a lot of people to watch it.

但这一点是偶然发现的。

And so but that was a little bit accidentally discovered.

事实上，第一个提出这个想法的是一个非常年轻的女性，扎里亚。当她提出时，我觉得这主意太蠢了。

And in fact, the first person who started doing this was a very young woman, Zaria, who when she proposed it, I thought this is such a dumb idea.

我显然完全错了。

I was obviously completely wrong.

结果证明，这为我们带来了全球范围内的广泛关注。

It turned out this has given us a lot of prominence in in the world.

你们有没有尝试量化过？我的意思是，这现在已经是Duolingo商业策略的核心支柱了。

Have you ever tried to quantify I mean, this is like a core pillar of Duolingo's business strategy at this point.

你们有没有尝试量化过，这些非付费媒体带来的商业影响？

Have you ever tried to quantify the business impact of having all this earned media?

是的

Yeah.

我的意思是，我们确实统计过。

I mean, we have.

我们相信，大约15%的用户来自这个渠道。

Our belief is that about 15% of our users come from that.

在任何一天，大约15%的新用户都来自于

On any given day, roughly 15% of our new users come from

那只猫头鹰，姑且这么叫吧。

The owl, for the lack of a better word.

某种猫头鹰相关的东西。

Some sort of owl owl thing.

大多数用户是通过朋友推荐来的，但大约15%来自这些奇怪的内容。

The majority of them come from people telling their friends about it, but about 15% come from this weird stuff.

为了印证你之前提到的其他数据，如果真的是15%，而你有1亿用户，那么1500万用户的价值是多少？

And just to multiply some other numbers together that you've talked about, if it really is 15% and you have a 100,000,000 users, is what's the value of 15,000,000 users?

如果每个用户的付费获取成本真的是30美元的话，那就是总成本。

Well, if it's actually at at 30 a pop would be the paid cost to acquire them.

那是多少？

That's what?

4.5亿美元。

$450,000,000

但你只算了活跃用户。

Well, you're only taking the active ones.

在谷歌上获取一个用户，成本就是30美元，就这么简单。

$30 on Google to get a user, period.

一个可能不会长期留存的用户。

A user that may not stick around.

所以实际成本比这还要高。

So so it's more than that.

这可能是我——我不知道确切数字，但肯定是数亿美元的收入。

This is probably been I, know, I don't know the exact number, but it's hundreds of millions of dollars of of earnings.

他是历史上最有价值的AI。

He's the most valuable Al in history.

我不知道这是否属实，但它确实为

I don't know if that's true, but it's been really good for

我们带来了巨大收益。

us.

这太有趣了。

It's fascinating.

好了，各位听众。

Alright, listeners.

现在是一个绝佳的机会，来感谢我们非常兴奋的新赞助商——Sierra。

Now is a great time to thank a new friend of the show that we are very excited about, Sierra.

是的。

Yes.

我们非常高兴能与布雷特、克莱以及他们那里的整个团队合作。

We are thrilled to be working with Brett, Clay, and the entire team over there.

那我们为什么对Sierra感到兴奋呢？

So why are we excited about Sierra?

嗯，多年来我们在制作《Acquired》的过程中学到的一点是，一家优秀公司的标志往往是其客户体验。

Well, one of the things that we've learned from making Acquired over the years is that a great company is often defined by its customer experience.

是的。

Yep.

但做到优秀很难。

But being great is hard.

与客户沟通成本很高。

Talking to customers is expensive.

虽然网站和应用程序很不错，但它们也相对缓慢和笨拙，而且你的客户还得花时间去学习它们。

And while websites and apps are great, they're also kinda slow and clunky, and your customers have to learn them.

是它们在学习你，而不是你在学习它们。

They don't learn you.

Sierra改变了这一切。

Sierra changes all that.

他们构建了面向客户的AI代理，能够完成各种惊人的任务，比如找到理想的房子、挑选电视节目、办理抵押贷款、配送沙发、退鞋、为患者进行医疗认证、订购信用卡、阻止订阅者取消服务，等等。

They build customer facing AI agents that can do an insane range of things, like finding the perfect home, or picking TV shows, or originating mortgages, shipping a sofa, returning shoes, authenticating patients for health care, ordering credit cards, saving subscribers from canceling, and on and on.

在成立仅两年的时间里，他们已成为领先的对话平台，吸引了ADT、Clear、Minted、Ramp、Redfin、Rocket Mortgage、Safelight、SiriusXM和Wayfair等数百家杰出公司，这些公司都信赖Sierra来提升客户体验。

In just two years since founding, they've become the leading conversational platform with hundreds of incredible companies like ADT, Clear, Minted, Ramp, Redfin, Rocket Mortgage, Safelight, SiriusXM, and Wayfair, all trusting Sierra for their customer experiences.

Sierra的设计足以满足财富500强企业的需求，包括医疗保健和金融服务等高度监管的行业，但它同样非常适合任何企业，包括你的公司。

Sierra was built to be powerful enough for Fortune 500 companies, including heavily regulated industries like health care and financial services, but it really works great for any business including yours.

使用Sierra，你只需构建一次AI代理，几周内即可在电话、聊天、短信、WhatsApp、电子邮件等多个渠道部署，并支持30多种语言。

With Sierra, you can build your AI agent once and deploy it everywhere within weeks on the phone, in chat, SMS, WhatsApp, email, all in over 30 languages.

你甚至可以将其发布到ChatGPT上。

You can even publish it to ChatGPT.

凭借他们独特且高度对齐成果的定价模式，你只需为Sierra带来的实际价值付费——包括提升客户满意度和问题解决率、降低成本以及增加收入。

And with their unique and insanely aligned outcomes based pricing model, you only pay for the value that Sierra delivers, increased customer satisfaction and resolution rates, lower costs, and higher revenue.

Sierra让全球优秀的企业能够每天每时每刻都展现出最佳状态。

Sierra enables the great companies of the world to show up at their best consistently every minute of every day.

事实上，我们对Sierra的评价极高，以至于大卫和我都投资了这家公司。

And in fact, we think so highly of Sierra that David and I even invested in the company.

要了解如何利用AI打造更出色、更人性化的客户体验，请访问 sierra.ai/acquired，并告知他们是本和大卫推荐的。

To find out how you can build better, more human customer experiences with AI, visit sierra.ai/acquired, and tell them that Ben and David sent you.

好的。

Okay.

我还会不断回到一些类似启发式的问题上。

I'm gonna keep coming back to my, like, what's a heuristic type questions.

一家初创公司来找你，说他们打算采用类似的策略。

A startup comes to you with the idea that they're going to run a similar strategy.

对于这种情况，什么时候能成功、什么时候不能成功，你有什么建议吗？

Any advice for when it could work or not?

还是说这纯粹是可遇不可求的幸运，我完全不知道它什么时候能奏效？

Or is this pure lightning in a bottle, I have no idea when it will work?

有些东西我觉得是可以复用的。

Some stuff I think you can reuse.

我们的策略有几个核心支柱。

There's a few pillars in our strategy.

我们还没谈到的一件事，我认为这占据了公司价值的近一半，那就是我们对A/B测试的执着。

One thing that we haven't talked about that I think accounts for probably half the value of the company is our obsession with AB testing.

我们通过不断的A/B测试走到今天，我估计每年会进行大约2000次各种各样的A/B测试——这个数字是我随便说的。

We have AB tested our way to where we are now and we have probably run, I'm gonna make this up, probably about 2,000 AB tests per year of all kinds of things.

因此，我们整个公司的架构都是围绕着A/B测试建立的。

And so we had just built the company around our entire product organization.

这不仅仅是产品经理、工程师和设计师的事。

That's not just product managers, engineers, designers.

整个系统的设计目标就是尽可能快地上线A/B测试。

The whole system is built around shipping AB tests as fast as possible.

这对我们的效果非常好，我认为你们在任何地方都应该这么做。

That has worked really well for us, and I would say you should do that everywhere.

如果你做的是消费者产品，那当然应该做A/B测试；但如果你做的是B2B SaaS产品，那就该用别的方法。

If you have a consumer product, I mean, you shouldn't be AB testing stuff if you have like a B2B SaaS thing, do something different.

但对于用户量大的消费者产品，你几乎可以通过A/B测试提升任何你能衡量的指标。

But for a consumer product that has a lot of users, you can really AB test your way to improve almost any metric you can.

你应该这么做。

That you should do.

我想给你的另一个建议是，持续聘用优秀的人才。

The other thing that I would give advice to is hiring really good people that you should continue doing.

关于免费增值模式这种策略，即暂时不盈利，我来告诉你为什么这可能不再有效。

In terms of the specific strategy of having a freemium thing that you don't monetize for a while, I'll tell you one of the reasons why that may not work anymore.

我们自己也遇到了这种情况，而且我们看到这种现象无处不在。

We have found ourselves, but we have seen it all over the place.

我们很幸运，2012年就推出了这款应用。

We were fortunate that we put out an app in 2012.

当时应用市场还没有定型。

The app market was not yet set in stone.

如今市场已经基本定型了。

It's pretty close to set in stone these days.

这并不是100%确定，但如果你看看今年和去年全球前50名应用的榜单，会发现它们非常相似。

It's not a 100%, but if you look at what are the top 50 apps globally for the year this year versus last year and you compare, they're pretty similar.

可能有一两个变了，但总体上还是很相似。

And maybe one or two changed, but they're pretty similar.

和前一年类似，也和更早一年类似。

And it's similar to the year before and it's similar to the year before.

到目前为止，进入这个市场已经非常困难了，我认为我们在2012年采用的策略今天不会再奏效，因为应用市场已经基本定型了。

It's just at this point, it's pretty hard to enter and I don't think that the strategy that we employed in 2012 would work today because the app market is pretty set in stone.

我的意思是，事实就是如此。

I mean, it's just just kinda is.

是的。

Yep.

那关于通过巧妙的创意获得数亿美元的免费口碑传播呢？

What about the strategy of something clever earning you hundreds of millions of dollars of sort of free earned media?

如果你能做到，那当然很棒。

It's great if you can do it.

当然，Duolingo的许多员工都展现了极高的创意，但我觉得我们在几个方面运气不错。

There's of course a lot of cleverness in this that many employees at Duolingo did, but I think we were lucky in a few ways.

我认为我们作为一家教育公司，获得了做一些其他公司可能无法做的事情的许可，因为很难说语言学习对你有害。

I think the fact that we're an education company gave us license to do some stuff that many other companies may not have been able to do because it's kind of hard to say that language learning is bad for you.

这意味着我们可以在抖音上做一些大胆的内容。

So that means that we can do risque stuff on TikTok.

但如果是一家试图做金融科技之类业务的公司，我认为人们更容易指责他们做坏事。

But if a company that is trying to do like, I don't know, like some sort of fintech company or something, I think I think it's much easier to accuse them of stuff.

我认为这对我们有帮助，大家都会说：我们是好人。

And I think that helped with us, just come on everybody, we're the good guys.

我认为我们能做很多大胆的事情，是因为很难对语言学习说三道四。

I think we were able to get away with a lot of stuff because it's hard to say bad things about language learning.

没错。

Right.

如果我跟你争论，说你的利率是合理的，或者说是掠夺性的利率，

If I'm coming at you with, you could argue it's a fair interest rate or you could argue it's predatory interest rate.

如果我有一个吉祥物拿着刀坐在那里说：多用我们的应用，那肯定要出问题。

If I have a mascot that's sitting there with a knife saying, use our app more, it's it's not gonna go well.

没错。

Exactly.

我认为我们从本质上是在做教育这一事实中获益良多，因此人们能够接受这些内容。

I think I think we benefited a lot from the fact that ultimately, we're doing it for education, so people are okay to to take some of this.

对。

Right.

太有趣了。

Fascinating.

顺便说一下，当我们和其他公司交流时，每个公司都想和我们聊聊。

By the way, you know, another place, when we talk to other companies, every company wants to talk to us.

并不是因为他们能从我们这里学到什么，而是因为他们觉得，‘你们这群人真特别’。

Not because they learn from us, but because they're like, oh, you guys.

是的。

Yeah.

你们还挺有趣的。

You guys are kind of fun.

这很好。

This is good.

而且没人把我们视为竞争对手。

And nobody thinks of us as competitive.

这对我们来说真是太好了。

Well, that's great for us.

因为每次我们跟任何随机公司的大人物聊天时，我女儿都很喜欢你们。

And it's because almost every time we talk to whoever, fancy person at random companies, my daughter loves you.

你知道，我觉得我们从中受益良多。

You know, I think we benefited a lot from that.

对。

Right.

这太疯狂了。

Which is crazy.

像其他市值一百亿美元的公司，有哪家能这么说吗？

Like, many other $10,000,000,000 companies could that be said for?

像你是一家价值一百亿美元的公司，几乎可以想见，理所当然会有一些人把你视为死敌。

Like, if you're a $10,000,000,000 company, almost you'd think, like, de facto, you're gonna have people who are your mortal enemies.

再说，现在每个人都觉得是绿色的。

Again, because everybody's like, ah, green now.

是的。

Yeah.

对。

Yeah.

这真是一个关于竞争对手的有趣观点。

So it's an interesting comment on competitors.

到目前为止，你大概很少会召开会议，专门讨论那些老牌语言学习公司，以及如何从他们那里抢夺用户吧？

At this point, you probably don't have a whole lot of meetings about legacy language learning companies and saying, how can we acquire users versus them?

你觉得谁是你们争夺用户的主要竞争对手？

Who do you think about as competitors for your users?

我们的看法是，我们竞争的是用户的时间。

I mean, the way we see it is we compete for time.

所以我们的竞争对手是Instagram、TikTok等。

And so our competitors are Instagram, TikTok, etcetera.

这就是我们争夺用户时间的对象。

That's that's who we compete for time.

顺便说一下，我们正在输。

And by the way, we're losing.

我的意思是，我们应用的使用时长远少于TikTok或Instagram等平台。

I mean, we're we have much less time spent on our app than TikTok has or or Instagram, etcetera.

但当我们观察用户并发现他们停止使用Duolingo时，我们会问他们：你为什么停止使用？

But when we look at our users and we see that our users maybe stop using Duolingo or something, we ask them, you know, why'd you stop?

最常见的回答是：是的，你知道吗？

The most common answer is like, yeah, you know what?

我花在随便哪个社交媒体网站上的时间更多了。

I'm spending more time on pick your random social media site.

我花在Instagram上的时间更多了。

I'm spending more time on Instagram.

就这样了。

And that's it.

竞争对手就是任何占据他们时间的东西。

The competitors is whatever takes their time.

好的。

Alright.

不过，你提到了一个我们在这期播客里只聊了一分钟的话题，但每个我接触过的与Duolingo有过互动的人——包括前员工和你参加过的其他播客——都一直在谈实验、A/B测试和数据驱动。

Well, you planted a seed of something that we talked about for, like, one minute in this podcast, but every person I talked to who's ever interacted with Duolingo, former employees and other podcasts you've been on, is all about experimentation, AB tests, being data driven.

关于你们推送通知背后的AI以及这些系统如何运作，已经有很多文章深入探讨了，想了解的人可以去读那些优秀的博客。

There has been lots of inks spilled on this for anyone who wants to read excellent blog posts about the AI behind your push notifications, and how all those systems work.

我有个宏观层面的问题想问你。

One high level question for you.

作为领导者，你是如何在以数据和实验为导向的文化与直觉导向之间取得平衡的？

How do you as a leader balance a cultural tenet being data and experimentation driven versus instinct driven?

我们实际上很特别，两者兼有。

We're actually weirdly both.

我来告诉你这个想法的来源。

I'll tell you where this comes from.

我们很早就意识到，纯粹的实验可能会把你引向非常糟糕的方向。

We realized early on that pure experimentation can lead you to really bad places.

如果你完全不加思考，只是盲目地优化某个指标的话。

If you kind of just don't think about it at all, you just you're just gonna optimize some metric.

这可能会让你陷入非常糟糕的境地。

That can lead you to be really bad places.

我听到的关于这一点的最佳故事之一，我们在多邻国经常提起，是关于我们从Groupon的创始人那里学到的，而他恰好来自匹兹堡。

And one of the best stories that I heard about that, and we repeat this a lot at Duolingo, is something that we learned from the founder of Groupon who happens to be from Pittsburgh.

他住在匹兹堡。

Mean, located in Pittsburgh.

是的，没错。

Happens Yes.

来自

Be from

我们是 Descript 的重度用户。

We're huge Descript users.

是的。

Yeah.

对。

Yeah.

他是个很棒的人。

He's a great guy.

他给我们讲了这个故事，我知道，现在这个故事可能已经有点变了，因为我现在记得的，可能是杜乐ingo内部流传的版本，而不是他当初原原本本讲的内容。

He told us this story, you know, I'm gonna tell it, at this point, it probably has changed a little bit because I, you know, I probably remember now what is in the lore of Duolingo as opposed to exactly what story he told us.

但他跟我们讲了一个关于 Groupon 每天发送邮件数量的故事。

But, you know, he told us a story about emails that Groupon sent per day.

所以你知道，Groupon 的模式是，一开始你有一个优惠，他们的规定是每天只能发一封邮件，包含一个优惠。

So, you know, Groupon basically, you've got a deal and the idea is at the beginning, you could only they could only send one email per day with one deal.

他对此要求很严格。

He was strict on it.

就是说，每天只发一封邮件，因为发超过一封就是垃圾邮件。

It's like, only send one email per day because more than one email is spammy.

后来，某个有想法的产品经理出现，说：如果我们发两封呢？

At some point, some enterprising product manager shows up and says, what about if we send two?

安德鲁·梅森说：绝对不行。

Andrew Mason says, no way.

就是不行。

Like, no.

我们只发一封。

We're only gonna send one.

然后这位产品经理提出了一个非常出色的反对理由。

And then this product manager gave this amazing argument against it.

他说，我们并不是想同时推出两封邮件。

He said, we're not trying to launch two.

我们只是想了解一下。

We just wanna know.

你反对知识吗？

Are you against knowledge?

当然，没有人反对知识。

And, of course, nobody's against knowledge.

所以，你知道的，他说用这个来平衡。

So so, you know, he so he said using this as even.

好吧。

Fine.

于是他们测试了两次。

And so they tested two.

猜猜当他们测试两次时发生了什么？

Guess what happened when they tested two?

两次比一次好得多。

Two is way better than one.

于是他们开始发送两次。

So, they started sending two.

然后呢，既然两个更好，那就测试三个吧。

And then like, well, if two is better, test three.

结果发现三个比两个好，四个比三个好，以此类推。

Turns out three is better than two, and four is better than three, etcetera.

他们通过A/B测试一路试到了某个数字，可能是七、八之类的，直到这个频道无法使用，因为用户把它标记为垃圾信息，然后就彻底垮了。

And they AB tested their way to like, I don't know what number, seven, eight or something, until the channel stopped working Because people marked it as spam and then it it died.

这个频道倒闭了。

The channel died.

但这就是A/B测试的问题所在。

But so this is the thing about AB testing.

你必须在A/B测试之上，真正遵循常识。

You have to you have to really do the common sense thing above the AB test.

所以我们在多邻国的做法是，采用一个两步流程。

And so what we do at Duolingo is we have we have this two step process.

我的意思是，第一步是在你进行A/B测试之前，顺便说一句，这时候我们是反对盲目依赖知识的。

I mean, the first one is before you even AB test something, and by the way, here we are against knowledge.

有些事情我们就是不想知道。

There are certain things we just don't want to know.

因为这样做太诱人了。

And it's because it's too tempting to do it.

所以第一步是在进行A/B测试之前，我们有一个叫做产品评审的过程，我想在很多其他公司也会有类似的做法。

So the first step is before we do an IV test, we have this process called product review, which I, you know, I assume happens in many other companies.

但在这里，每一个进入应用的改动都要经过产品评审。

But in this case, every single change that goes to the app gets product review.

产品评审就是公司里的一群领导者组成的团队，包括我，大概有五个人。

And product review is basically a set of leaders in the company that includes me, there's about five people.

我们会审查每一个改动，并且必须达成一致才能上线。

And we look at every change, and we have to agree for it to go out.

我们会阻止很多改动上线，以防止出现这种情况。

We stop a lot of things before going out to prevent this type of stuff.

这里，要依靠直觉和基本常识。

Here, use intuition and just common sense.

我们说：不行。

We're like, no.

没人想收到三万两千条通知。

Nobody wants to receive 32,000 notifications.

这太蠢了。

That's just dumb.

即使A/B测试这么建议。

Even if the AB test says so.

我认为这对我们帮助很大。

I think that has helped us quite a bit.

你有没有过过去刻意回避某些信息的例子？比如，你决定不想知道的某件事？

Do you have an example of a way in which you've been anti knowledge in the past, Something you decided you didn't want to know?

哦，是的。

Oh, yeah.

我的意思是，最近就有人想了解，如果用户打开应用时弹出一个占据整个屏幕的广告，会发生什么。

I mean, pretty recently, somebody wanted to know what happens if you put an ad that takes over the whole screen when somebody opens the app.

比如有个产品经理想知道这个。

Like a product manager wanted to know that.

我们当时说，是的，但我们不会这么做。

And we were like, yep, but we don't.

尤其是在变现方面，这非常诱人，因为变现时你就会做实验。

Particularly with monetization, it's so tempting because what happens with monetization is you run the experiment.

然后实验结果会说，如果你推出这个实验，每年能赚5000万美元。

And then it says it says something like, if you launch this experiment, it will make you $50,000,000 a year.

你一看，心想，这不错。

And you look at it and you're like, that's nice.

然后财务团队发现这件事发生了。

And then the finance team figures out that this happened.

比如财务团队的人看了这个实验，突然CFO就出现在我的会议室里，说：喂，每年5000万美元啊。

Like, somebody from the finance team looks at this experiment and suddenly the CFO is in my conference room or something and he's like, yo, $50,000,000 a year.

你必须上线这个。

You gotta launch that.

我们尽量不去知道这些货币化实验的答案。

We try not to know the answer to many of these monetization experiments.

当你戴上CEO或产品负责人的帽子时，这些事情就显而易见了。

When you wear your CEO or your your head of product hat, these things are sort of obvious.

而其背后是一种隐含的假设，即尊重用户、让用户对产品产生良好感受，会带来未来的潜在收益。

And and what's underneath it is this implicit assumption that there is a future dollar value associated with treating the user with respect and having a feel good association with your product.

本质上，如果我们这么做，所有潜在的未来收入都会受到威胁。

It's basically like all of the potential future dollars we could earn are jeopardized by doing this.

如果你做了这件事，就是在提前实现收入。

It's we're pulling forward revenue if you do something

很难量化。

very hard to quantify.

但这是很难量化的。

But it's very hard to quantify.

这涉及复杂的预期价值计算。

Complicated expected value calculation on it.

是的

Yeah.

你有没有尝试过量化那种品牌价值或未来价值呢？

Have you ever tried to quantify that brand value or future value to to

我们没有。

We haven't.

没有。

No.

我们没有。

We haven't.

我们只是有一群人——我说的是五个左右，但实际上是大约五位领导者，他们对这类事情有很好的直觉，比如‘不行，这太过分了’或者‘这不好’。

What we just have is a group of I'm saying five, but it's on the order of five leaders that have a very good gut feeling about, no, this is this is too much or no, this is not good.

我觉得他们的直觉相当不错，我想这么说。

And I think their gut feeling is quite good, I would say.

我们有时会遇到这个问题，你知道，我们会做一些实验，必须做出权衡。

We run into this problem sometimes, you know, we do run an experiment where we have to make a trade off.

比如，我们正在进行某个实验，因为我们知道，如果这么做，我们会失去一万用户，但能赚更多钱之类的。

Like, for example, we're in some experiment because we know if we do this, we lose 10,000 users but make this much more money or whatever.

我们从来不知道该如何做这些权衡。

We've never known how to make these these trade offs.

所以通常情况下，我们会优先考虑用户：如果某项改动会减少我们的活跃用户数量，除非它能让我们赚到五亿美元，否则我们根本不会上线。

So generally, what we do is we go for users, as in if something decreases the number of active users that we have, unless it's gonna make us $500,000,000, we're really not gonna launch it.

这很有趣。

It's interesting.

我在笑呢。

I'm laughing.

你知道，本常开玩笑说要用这句话来对付我。

You know, Ben jokes about using that line on me.

这适用于任何企业。

This applies to any business.

比如对我们来说，如果你告诉本和我：好吧，目标函数是最大化Acquired的价值，我们十分钟内就能列出一大堆我们清楚知道该怎么做、而且能赚到巨额利润的事情。

Like, for us, if you told Ben and me, okay, go objective function, maximize the value of Acquired, we could come up in ten minutes with a whole list of things that we know exactly what we would do, and it would make an obscene amount of money.

欢迎收看本期《Acquired》每日节目。

Welcome to this daily episode of Acquired.

是的。

Yes.

这在接下来的几个月内会有效，但不会持续几年。

It would, but for the next few months and not for the next few years.

我甚至不确定这是否是实现价值最大化的最佳做法，但我们绝不会这么做，因为这会摧毁我们的初心。

I'm not even sure that that like, that might be the value maximizing thing to do, but we would never do it because it would just kill our hearts.

你懂的？

You know?

比如

Like

对。

Yeah.

这一点也同样重要。

That that also matters a lot.

是的。

Yeah.

不过这挺有趣的。

It's funny though.

你完全可以慢慢煮青蛙，而多邻国的产品经理们已经非常擅长慢慢煮我的内心。

Your heart you can boil a frog quite well, and product managers at Duolingo have gotten really good at boiling my heart slowly.

五年前我觉得无法接受的事情，现在却觉得好多了，因为它们总是一点点突破界限，循序渐进地让你妥协。

Things that I think I was not okay with five years ago, I think I'm a lot more okay with because they just kind of push the envelope every slippery slopes, they they get you.

但你最后会不会觉得，好吧。

But do you end up feeling like, okay.

你知道吗？

You know what?

他们说得有道理。

They had a point.

像，我不再

Like, I I no longer

是的。

Yeah.

有时候我确实对某些事情判断错了。

Sometimes I've definitely been wrong about stuff.

我的意思是，这肯定是真的。

I mean, that's that's for sure.

但我仍然认为，从长远来看，做对用户最有利的事才是真正更好的选择。

But I still claim in the long term, it really is better to do what's best for the user.

我不是说用户要求的东西，因为用户总会要求一些奇怪的东西。

And I don't mean what the user's asking for, because the users ask for weird shit.

我是说你认为对用户真正有利的东西。

I mean what you think is better for the user.

我认为，从长远来看，我很有把握这确实是价值最大化的做法。

I think I think that in the long term, I'm pretty sure that's that's value maximizing.

我觉得这特别适用于创始人主导的公司，也就是我们会做我内心直觉认为对用户最好的事。

I think this is a pretty specific thing to founder led companies, which is we will do what I personally feel in my gut is best for the users.

就这样，每次都稳稳拿下。

And like that, just carrying the day every time.

在专业的管理环境中，这种情况不会发生。

That does not happen in a professional management setting.

也许不会。

It may not.

让我感到高兴的是，五年前，这完全是我的直觉。

What I'm happy about is that I would say five years ago, it was solely my gut.

但我觉得现在有一群领导者——我们的产品负责人吉姆、首席设计官西米——他们的直觉已经和我的几乎完全一致了。

And I think at this point there is a group of leaders, Jim, our head of product, Simi, chief design officer, that whose gut somehow is like identical to mine at this point.

可能只有1%的差异，但我觉得如果我被公交车撞了，这些人的直觉也相当靠谱。

Maybe 1% different, but I feel pretty good about the fact that if I were hit by a bus, these people's guts is pretty good.

所以我们之前聊过一点，关于你有一个漏斗底部的把关机制来控制发布内容。

So we've talked a little bit about this idea that you have a bottom of funnel governor on what goes out.

我们也提到过，漏斗中段有一个A/B测试。

We've talked about that there's a mid funnel thing that's the AB test.

我想你过去曾经说过

And I think you've said in the past

最新的数字是五五开。

The latest number is fifty fifty.

五五开。

Fifty fifty.

好的。

Okay.

每年都很接近五五开，也就是说，真的就是五五开。

It is yearly close to fifty fifty, as in, like, it really is fifty fifty.

不是48.52美元。

It's not $48.52.

我解释不了。

I can't explain.

就是那种情况，我只想说：真的吗？

It's one of those where I'm just like, really?

50.50美元？

$50.50?

拜托，各位。

Come on, people.

这必须有些不同。

It has to be something different.

但现在它非常接近五五开。

But now it's pretty close to fifty fifty.

这就引出了这个问题。

So that gets to this question.

为什么随着时间推移，你们的直觉对哪些实验会成功变得越来越准？

Why is it that over time, your collective gut has gotten better about what experiments will succeed.

它变得更好的方式是我们获得了实验的结果。

The way it has gotten better is because we get the results of the experiments.

我们的直觉本质上是一个学习机器。

Our gut is kind of a learning machine.

我们正在学习。

We're learning.

我们觉得，就是这类东西。

We're like, yeah, that type of stuff.

到目前为止，我们已经有多年的经验。

At this point, we have years of experience.

一直从事产品评审的那些人，现在已经做了好多年了。

The people that have been doing product review have been doing it for years now.

我们积累了多年的经验，知道这类改动会产生这类效果。

We have years of experience knowing this type of change does that type of thing.

我们在这方面相当擅长。

And we're pretty good at it.

我们不是百分之百准确，但在这方面相当擅长。

We're not a 100%, but we're pretty good at it.

你不仅知道事情会朝哪个方向发展，还能判断哪些改动影响更大。

Not only do you know which direction things are gonna go, but you also know which ones have more impact.

举个例子，我们知道课程结束后的页面——也就是完成一节课后看到的页面——对用户第二天是否回来至关重要。

So, example, we know that the session end screen, so that is the screen that you get after doing a lesson, matters a lot in terms of getting you to come back the next day.

但我们知道，像课程中间的页面这样的其他页面，对用户第二天是否回来几乎没有任何影响。

But we know that other screens, like a screen in the middle of a lesson, almost matters not at all in terms of getting you to come back the next day.

所以我们逐渐学会了哪些页面和应用的哪些部分对特定指标更重要。

So we've just kind of learned what what screens matter and what parts of the app matter for certain metrics.

我们对这一点已经有了非常准确的直觉。

We just have a really good gut on that.

对于大型实验和小型实验，你们的处理方式是否不同？毕竟在小型实验中，你们的直觉可能缺乏足够的训练数据？

Do you have to treat big experiments differently than little experiments where your gut may not have as much training data in it?

是的。

Yeah.

有些实验比其他实验复杂得多。

And there's some experiments that are a lot more complicated than others.

有些事情需要我们花很长时间才能完成。

Some stuff takes us a long time.

首先，我们现在还有非常先进的数据科学方法，可以告诉我们某些实验应该运行多长时间。

Well, first of all, now we also have pretty sophisticated data science that tells us for how long to run certain experiments.

我们对这个特定实验应该运行很长时间也有很强的直觉。

We also have a pretty good gut feeling about this particular experiment should run for a pretty long time.

顺便说一句，我们运行的大多数实验都是持续两周。

Most experiments we run, by the way, run them for two weeks.

两周内我们就能获得足够的数据来判断方向。

We have enough data in two weeks to know where to go.

但有些实验我们会运行大约六个月，因为我们知道，一开始你会看到非常大的差异，但后来人们会适应这个东西，差异就会变小。

But there are some experiments that we run for about six months because we know that you're gonna see a very big difference at the beginning than later because people get used to the thing.

你们公司的核心能力中包含非常重的统计学成分。

There's a very heavy statistics component to the core competency of your company.

这个统计学能力是从哪里来的？

Where did that come from?

我的意思是，最初的许多员工基本上都是卡内基梅隆大学的计算机专业人才。

I mean, the first many employees were basically computer nurse from Carnegie Mellon.

我想这就是来源。

I think that's where that came from.

我们入职时就理解了统计显著性这类概念。

We understood things like statistical significance coming in.

是的。

Yeah.

有道理。

Makes sense.

这和你的众包研究有关联吗？

Does it feel related to your crowdsourcing research?

还是说你请来了一位关键人才，然后让他成为公司的核心支柱？

Or does it feel like you you brought someone on who was a super key hire and then, you know, made that a a pillar of the company?

我不确定这是否直接源于我的众包研究，但确实是公司早期的几位员工。

I don't know if it's exactly my crowdsourcing research, but it's generally the first few people that worked at the company.

我们对统计学有很好的掌握。

We had a pretty good grasp of statistics.

但我可以告诉你的是，这很有趣。

What I will tell you though is that it's funny.

这对大多数事情都是如此，我认为这并不是多邻国独有的。

This is true for most of the things, and I don't think this is unique to Duolingo.

对于大多数事情，我们当时对统计学有相当好的理解。

For most things, you know, we had a pretty good grasp of statistics.

这很好。

That's good.

但我们现在做分析的复杂程度，和当初相比，简直不可思议。

But the level of sophistication that we do analysis today compared to that is insane.

我的意思是，我们现在有拥有统计学博士学位的人，他们真正懂行，而不是像我们当初那样，只是自称对统计学有‘相当好的理解’。

I mean, we now have people with PhDs in statistics who actually, honest to God, know what they're doing as opposed to we had quote unquote a pretty good grasp of statistics.

早期我们摸索如何教学时，我们大致有个教学的想法。

Early on when we were figuring out how to teach things, you know, we kind of had an idea of how to teach things.

如今，我们已经聘请了数十名拥有第二语言习得博士学位的人，他们在学习成果等方面的专业水平，和我们当初的做法相比，简直是天壤之别。

By now, you know, we've hired dozens of people with PhDs in second language acquisition and the level of sophistication that they have in terms of learning outcomes and stuff like that, compared to what we were doing at the beginning is night and day.