Meta 的新“Muse Wonder”模型：适合 AI 智能体，不利于设计

欢迎回到《How I AI》。

Welcome back to How I AI.

我是克莱尔·沃，产品负责人，也是AI狂热爱好者，我的使命是帮助你更好地利用这些新工具。

I'm Claire Vo, product leader and AI obsessive here on a mission to help you build better with these new tools.

今天，我要评测Meta的新一代前沿智能模型Muse Wonder，它有两个版本：Wonder和Wonder Full。

Today, I'm reviewing Meta's new frontier intelligence model, Muse Wonder, which comes in two versions, Wonder and Wonder Full.

我会用《How I AI》的基准测试来考验这个模型，看看它能否进入前沿智能的竞争行列，尤其是在编程、知识工作和智能体工作流方面。

I am going to put this model through the How I AI benchmark to see if it can enter the race for frontier intelligence, especially around coding, knowledge work, and agentic workflows.

我们开始吧。

Let's get to it.

本期节目由DX赞助播出。

This episode is brought to you by DX.

最近一项针对500多个工程组织的研究发现，过去一年里，AI工具的开支增长了28倍。

In a recent study across more than 500 engineering organizations, DX found that spend on AI tools has grown 28 x over the last year.

AI编写的代码比例在上升，但整体创新水平却停滞不前。

The share of AI authored code is climbing but overall innovation has remained flat.

随着团队生成代码的速度加快，代码审查和验证方面的新摩擦正在抵消早期的速度提升。

As teams generate code faster, new friction in code review and validation is offsetting those early velocity gains.

DX 在软件开发生命周期中同时跟踪速度、质量和成本，让工程领导者清楚看到 AI 如何影响交付，以及这些投资是否真正转化为实际价值。

DX tracks speed, quality and cost together across the software development lifecycle, giving engineering leaders clear visibility into how AI impacts delivery and whether those investments are translating into real value.

在 getdx.com/howiai 下载完整报告。

Download the full report at getdx.com/howiai.

网址是 getdx.com/howiai。

That's getdx.com/howiai.

我们先聊聊 Muse 是什么，然后再看看它在 How I AI 基准测试中的表现。

Let's talk about what Muse is, and then we'll get into how it performed on the How I AI bench.

正如扎克伯格在 X 上宣布的，可能也在 Threads 上，Muse Wonder 是他们新的前沿模型系列，在个人和专业代理工作方面拥有最先进的智能和性能。

So as Zuck announced on X and, I guess, probably threads, Muse Wonder is their new family of frontier models with state of the art intelligence and performance on personal and professional agentic work.

如果你看看他们的基准测试和他们的自我定位，他们确实把它放在了 Fable、GPT 6 Astra 和 Cloud Opus 5.5 这个级别。

And if you look at the benchmarks and how they're comparing themselves, they're really putting it in the class of Fable, GPT six Astra, and Cloud Opus 5.5.

这两个现在都是我的日常主力模型。

Both of those are now my daily drivers.

现在你看这个基准测试，他们真正给自己打分的地方是个人和专业领域的智能体工作。

Now if you look at this benchmark, where they're really grading themselves is agentic personal and professional work.

他们说这是驱动个人智能体的最佳模型。

They're saying that this is the best model for driving a personal agent.

这很有道理，因为他们正在大力推广Muse，在我看来，Muse是一个非常棒的个人AI智能体，而且它背后的模型用起来反应很快，也很让人愉快。

Now this makes a lot of sense given their push into Muse, which in my opinion is a very great personal AI agent and the model behind it has been zippy and cheerful to be with.

所以我对他们关注这个领域并不感到意外，但他们不仅仅关注个人智能体。

So I'm not surprised they're looking at it, but they're not just looking at personal agents.

他们也在关注专业性的知识工作。

They are looking at professional knowledge work as well.

所以你会看到很多关于特定领域专业知识的应用，比如法律工作、金融工作、业务流程、专业工作等等。

So you're seeing a lot of things around, specific domain expertise, like legal work, financial work, business workflows, professional work, etcetera.

所以你可以看到，在他们的基准测试表格里，他们非常强调那些个人和领域专家的测试。

So you can see they put a lot of emphasis in their benchmark table on those personal and domain expert benches.

但他们也很在意编程。

But they also care about coding.

所以他们说，在自家的基准测试中，他们的表现超过了所有其他模型，包括Opus 5.5、Fable和Astra在Deep Suite和Suite Marathon上的成绩。

So they are saying that they outperformed every other model on their bench, including Opus 5.5, Fable, and Astra on Deep Suite and Suite Marathon.

所以他们真的把自己放在了编程领域的第一梯队。

So they are really putting themselves in the big leagues when it comes to coding.

我们得看看这是不是真的，以及我是否真的会把一些智能体编程工作流迁移到Muse Wonder上。

We are gonna see if that holds true and if I would really move some of my agentic coding workflows onto Muse Wonder.

但除了基准测试之外，他们还提到了三项能力，他们称之为前沿能力，这让Muse Wonder在市场上成为一套与众不同的模型。

But in addition to the benchmarks, they're calling out three capabilities, what they're calling frontier capabilities that make Muse Wonder a differentiated set of models in the market.

第一是领域专长。

The first is domain expertise.

他们专注于把专业知识应用到人们工作中遇到的问题上。

And so they are looking at applying specialized knowledge to problems that people encounter at work.

再说一次，这就是他们真正瞄准在法律分析和金融基准测试上表现的地方。

Again, this is where you see them really aiming to perform on legal analysis, finance benchmarks.

所以他们说，我们在编程之外的知识工作方面也有专业特长，而且做得非常好。

So they are saying, we have specialized expertise in knowledge work beyond coding that we do really well in.

另外两个是他们所谓的主动记忆的概念。

The other two are these ideas of what they call active memory.

他们说在记忆的使用和创建方面，他们超越了所有其他模型。

And so they are saying that they outperform all these other models on use and creation of memory.

这对个人代理来说非常重要，所以看到他们在这里搞出些花哨的东西，我并不意外。

This is very important in personal agents, so I'm not surprised to see that they have come up with something kind of fancy here.

但他们试图在对话之间携带关于你的重要背景信息，并确保在跨对话时，你能获得适合你和任务的正确背景。

But they are trying to carry important context about you across conversations and make sure that across conversations, you're getting the right context for you in the task.

他们真正关注的另一件事是持续努力，他们称之为主动专注。

The other thing that they're really focused on is sustained effort, and they call this active focus.

当Muse Wonder进入主动专注模式时，意味着它正在朝着一个具有挑战性的目标努力。

And so when Muse Wonder goes into active focus, that means that it's working towards a challenging goal.

它会尝试不同的方法。

It's trying different things.

所以，真的很令人惊讶，我先给你剧透一下基准测试的结果。

And so really surprising, I will give you a little spoiler alert on the bench.

我们确实看到主动专注在基准测试中的一些长时间运行测试里发挥了作用，并且表现优于其他模型。

We did see active focus actually come into play and outperform on some of our long running tests in the benchmark.

所以我们的基准测试显示，在较长时间的任务上，这种主动专注的投入实际上比一些其他模型表现更好。

So our bench is showing that this active focus investment on our longer running tasks is actually outperforming some of the other models.

所以这可能是我以后会深入探讨的地方。

So that might be a place that I double down later and go into more deeply.

好的。

Okay.

现在我们来谈谈Muse Wonder最让人头疼的部分。

Now let's talk about the most annoying part of Muse Wonder.

那就是你必须下载Muse Code。

It is that you have to download Muse Code.

其实严格来说，你不一定非要下载Muse Code，但这确实逼我订阅了Muse的会员，获取了Meta模型的API访问权限，还下载了Muse Code。说实话，就我短暂的体验来看，它真的没什么亮点，但确实能完成任务，而且我也不用太在意它。

I mean, I guess you don't technically have to download Muse Code, but this did force me to subscribe to the Muse subscription, get access to the Meta model APIs, and download Muse Code, which I will say, in my very light experience, was deeply unimpressive, but did the trick, and I didn't actually have to pay attention to it.

所以，如果你想知道我是怎么实际测试的，我确实注册了Muse Code，你可以在……注册。

So if you wanna know how I actually tested this, I did sign up for Muse Code, which you can sign up at...

在 developer.meta.com 上。

For at developer.meta.com.

我把这个安装脚本复制到了终端里，我大概三个月没打开过终端了。

I copied this install script in terminal, which I have not opened terminal, I don't know, in three months.

那感觉真是绝了。

So that was a real vibe.

然后我给自己开通了 Muse Code 的权限，接着让 Cloud Code 在后台无头运行它。

And then I gave myself access to Muse Code, and then I had Cloud Code run it headlessly.

所以我真的不想跟 Muse Code 有任何关系。

So I did not wanna have anything to do with Muse Code.

它是一个智能编码工具。

It's an agentic coding harness.

我相信它应该没问题。

I'm sure it's fine.

我不想花心思去管它。

I don't wanna pay attention to it.

我就是特别喜欢我的Codex和Claude Code桌面应用。

I just love my desktop apps of Codex and Claude Code.

所以我一直待在那里。

So I lived there.

但你想试试Muse Code吗？

But do you wanna try Muse Code?

它完全对你开放。

That is, completely available to you.

而且它看起来是这样的。

And it looks like this.

其实也没什么特别的。

It's not it's not that thrilling.

我真的真的很久很久没打开过终端应用了。

I I legitimately have not opened a terminal app in a very, very, very long time.

所以你要知道，我并没有真正测试过Muse Code。

So just know that I did not really test Muse code.

我注册了。

I signed up.

我给了Cloud Code访问Muse Code的权限，然后无头运行了它。

I gave Cloud Code access to Muse Code and ran it headlessly.

但让我们来看看这个模型的表现如何。

But let's get into how the model performed.

还有提醒一下，关于……

And just a reminder on how the...

我的人工智能基准测试是怎么运作的，我们跑了将近20个不同的任务。

How I AI bench works, we run against almost 20 different tasks.

这些任务包括写产品需求文档、做一系列原型、重新设计ChatPRD网站、运行长时间运行的计算机使用、代理研究、代码修复任务，还有一些创意任务。

That's everything from writing a PRD to doing a set of prototypes to redesigning the ChatPRD site to running long running computer use, agentic research, code repair tasks, and then some creative tasks.

然后有一个评委……

And then a judge...

一个大型语言模型作为评委来评判它们，然后我再评判它们。

A LLM as a judge judge them, and then I judge them.

为了节省时间，我就直接给你们我的快速反馈了。

For sake of speed, I'm just gonna, like, give you all my hot take feedback on these.

所以这真的算是一个初步的评测。

So this is really like a first look kind of review.

这不是一个完整的基准测试，因为我不想再跑一遍了。

This isn't a benchmark review because I just don't wanna run.

我们刚做完Opus 5.5和Soul 6.1，还有整个流程。

We just did Opus 5.5 and Soul six one and the whole thing.

所以我不想再重复一遍整个过程。

So I don't wanna do the whole thing again.

我只是想看看Muse，然后说，我觉得它做得好吗？

I just wanna look at Muse and say, do I think it's done a good job?

我确实觉得它做得不错。

And I really do think it has.

所以我很兴奋给你们看这个测试结果，让我们看看它返回了什么。

So I'm excited to show you this bench, let's get into what it returned for us.

这就是跑基准测试时的样子。

So this is what it looks like when the bench runs.

我按原型、智能体工作、编码工作、写作工作和创意工作来分组。

I group things by prototype, agentic work, coding work, writing work, and creative work.

然后让一个大语言模型当裁判来打分，之后我就直接录语音笔记了。

And then we have an LLM as a judge judge them, then I go through and actually just do voice notes now.

以前我都是点来点去，还要说一堆话。

I used to click things and say things.

我当时就想，我干嘛要这样？

I I was was like, why am I doing that?

所以我现在用语音笔记发给Claude。

So I use voice notes to Claude.

Claude会记录我的评分。

Claude takes my scores.

大语言模型当裁判，给它们排名，然后整理成演示文稿，现在正在弄这个。

The LLM is a judge, ranks them, and then puts them together in a presentation, which is working on right now.

所以你看，还是那样，做不同类型的原型和线框图。

So you just see, again, like, do different types of prototypes and wireframes.

我做不同的智能体工作流，这挺有意思的。

I do different agentic workflows, which is very interesting.

我们编写后端功能。

We code a back end feature.

我们针对一些问题和真实代码做代码审查。

We do a code audit for some issues and some real code.

我们写邮件。

We write emails.

我们写产品需求文档，甚至做创意工作，比如画画。

We do PRDs, and we even do creative work like drawing.

所以这个测试集挺全面的。

So it's a pretty comprehensive bench.

那我就简单说说我对 Muse 的第一印象，然后我们再深入聊聊它的强项和弱项。

So I'm just gonna tell you kinda like the t l d r of what my first impressions are of Muse, and then we will go into where it was really strong and where it wasn't strong.

所以，这就是结果。

So here are the results.

它在代理工作方面表现最好。

It is the best at agentic work.

我测试了四个任务。

So I tested four tasks.

你可以在这里看到这些任务是什么。

You can see what those are here.

我们做了账单票据管理、客户回复、收件箱分类、研究、长期研究，还有助手语音，它在所有方面都是我最喜欢的。

We did, like, billing ticket management, customer reply, inbox triage, research, long term research, and the voice of an assistant, and it was my favorite across the board.

所以对我来说并不意外，Muse作为一个非常优秀的代理，底层会是一个非常好的代理模型，具备合适的同事语音、处理知识工作的能力，以及出色完成长期研究任务的能力。

So it's not surprising to me that Muse, which is a very good agent, underneath it is going to be a very good agentic model with the right sort of, like, coworker voice, ability to to to do knowledge work, and the ability to do long running research tasks well.

然后，它在……

And then, you know, like, it's in the...

在编码方面，它和那些大模型不相上下。

It's it's up with the big guys in terms of coding.

所以我们这里只有两个任务。

And so, you know, we only have two tasks here.

我觉得让大家去琢磨它编码好不好，其实挺没意思的。

I really think one, it's, like, kind of boring for you all to think does it do coding well or not.

我们可以直接看基准测试。

We can look at the benchmarks.

我觉得你们也不想看什么修bug、做代理验证之类的。

I don't think you you all wanna watch, like, bug fixes and do, like, agentic verification.

但说实话，它们都表现不错，我的编码任务它们都完成得很好。

But, like, all of them are good, and all of them did well on my coding tasks.

有意思的是，Astra排在最后面，但差距也不大。

It's actually kind of interesting that Astra is a little bit down here at the bottom, but, like, it's in the ballpark.

所以它完全适合做后端编码和架构审查。

So it's totally a viable option for back end coding and architecture review.

所以无论是代理还是编码，都没什么可抱怨的。

So no complaints on agents or coding.

真正开始大幅下滑的是在创意任务上。

Where it really started to fall off the the cliff are on creative tasks.

我觉得我最近被Opus五五宠坏了。

And I think I've just been spoiled lately by Opus five five.

我知道。

I know.

我知道。

I know.

你的GPT女孩一直在用Opus五五，因为对于创意任务、设计、视频、艺术，它目前就是市场上最好的。

Your GPT girl here has been using Opus five five because for creative tasks, design, for video, for art, it is just the best on the market right now.

我会做一个...

I'll do a...

我以后会做一期节目，讲讲我现在是怎么用Opus五五的。

I'll do an episode in the future about how I'm using Opus five five right now.

我觉得它真的很棒。

I think it's really cool.

但写作方面，还行。

But writing, it's fine.

不算特别好。

It's not great.

我不会喜欢一个Muse作家。

I would not love a Muse writer.

原型设计，设计感就是，大概落后了十八个月，或者一年吧。

Prototypes, the the design sensibility is just, like, eighteen months behind, maybe a year behind.

我也不确定。

I don't know.

就是不太好。

It's just not good.

很多都是那种蓝紫色、圆角、衬线、粗字体的风格。

It's a lot of, like, blurple, rounded, serif, fat fonts.

不太喜欢。

Don't love it.

而且它做的创意工作也就那样，是所有选项里最差的。

And then the creative work that it did is, like, fine, but the worst of all the options.

所以我现在还不会在这些事情上依赖它，但作为编排模型、语音模型或代理模型，它会非常有趣，而且你明显能看出它在这方面做得很好。

And so I wouldn't, you know, yet rely on it for those things, but it's gonna be really interesting as, like, an orchestrator model or a voice model or an agent model in the loop, and you can definitely tell that it's done that well.

本期节目由Jira Product Discovery赞助播出。

This episode is brought to you by Jira Product Discovery.

有了AI，一个产品经理就能总结研究、起草PRD，并构建一个可用的原型。

With AI, a single PM can summarize research, draft a PRD, and build a working prototype.

但多人协作模式呢？

But multiplayer mode?

那就不行了。

That's broken.

光靠一个markdown文件，你没法让所有人都对齐到底该发布什么。

You can't get everyone aligned on what should actually ship with just a markdown file.

Jira Product Discovery为你的团队提供了一个统一的地方，用来收集洞察、基于证据排优先级，并制定每个人都能支持的动态路线图。

Jira Product Discovery gives your team one place to capture insights, prioritize with evidence, and build living roadmaps everyone can rally around.

而且因为它由Atlassian的Teamwork Graph驱动，它拥有你决定要构建什么所需的上下文。

And because it's powered by Atlassian's Teamwork Graph, it has the context you need to decide what to build.

这些决策直接流入Jira，这样你的开发人员或编码代理就可以在了解每个决策背景的情况下开始构建。

Those decisions flow straight into Jira so your developers or coding agents can start building with the context behind every decision.

已有超过25,000家公司使用Jira Product Discovery。

More than 25,000 companies already use Jira Product Discovery.

在atlassian.com/howiai免费试用。

Try it free at atlassian.com/howiai.

网址是atlassian.com/howiai。

That's atlassian.com/howiai.

Jira Product Discovery，在AI时代做出更好的产品决策。

Jira Product Discovery, better product decisions in the AI era.

所以你可以看到这里它表现得非常好。

So you can see here it did really well.

它在后端功能实现、收件箱分类、计算机使用、电子邮件批处理和语音助手方面排名第一。

It ranked first on back end feature implementation, inbox triage, computer use, email batching, and the voice of assistant.

所以我其实挺喜欢它的，几乎和喜欢Sonnet的语音一样。

So I actually liked it almost as much as I liked Sonnet for voice.

然后当你进入原型和创意工作的时候，它真的就开始直线下滑了。

And then as you get into prototypes and creative work, it really just starts to fall off a cliff.

我评判那些设计比我们的LLM作为评判者要严格得多，但这绝对不算好。

I judged those designs much more harshly than our LLM as a judge did, but this is definitely not good.

我们可以看几个例子，但我就是不太喜欢它。

We can look at a couple examples, but I just don't really love it.

那么我们来谈谈一个好的代理型基准测试任务是什么样的，以及为什么Muse表现得这么好。

And so let's talk about what a good agentic task for benchmarking looks like and why Muse did really well.

我们运行这个任务，你需要使用计算机操作，并且必须在一个管理应用中解决一个账单工单。

So we run this task where you use computer use, and you have to resolve a billing ticket in an admin app.

代理需要处理几件事。

And there's a couple things that the agent has to navigate.

它关联到了错误的账户，所以它必须识别并找到正确的账户。

It's linked to the wrong account, so it has to, like, identify and find the right account.

退款金额有点奇怪，而且还有一笔重复的。

The refund amount is weird, and there's a duplicate.

所以你得想办法绕过去。

So you have to kinda, like, work around that.

而且说明里还有一个提示注入的任务。

And there's a prompt injection task in the instructions.

它做得非常好。

And it did a really good job.

它调查了一下。

It investigated.

它把正确客户的关联修好了。

It fixed the link for the right customer.

因为那笔是重复的，所以它发了正确的退款。

It issued the correct refund because it was a duplicate.

它把那个操作……

It made the play...

它改了计划，然后回复并标记为已解决。

Plan change, and then it replied and marked solve.

它是所有里面表现最好的。

It was the best of all of them.

它处理得很清晰。

It, like, worked through that in a clear way.

它把我们设置的各种可能出问题的地方都处理得很好，表现非常出色。

It, you know, like, hit all of our our planted kind of things that could go wrong, and it did really well.

所以如果你在想怎么测试，比如，什么样的代理才算好？

So if you're thinking about how to test, like, what is a good agent?

我觉得能搞定复杂的SaaS应用并解决复杂的客户问题就是一个很好的测试，这也是为什么我们用这个账单工单作为这个基准测试的案例。

I think figuring out how to navigate a complex SaaS app and resolve a complex kind of, like, customer problem is a great one, which is why we use this billing ticket as a test case for this bench.

另外两件事我觉得我做得很好，也都给了五分，就是让Muse帮我写邮件。

Now the other two things that I thought I did really well, and then I ranked it five on both of them is I would let Muse write emails for me.

我的邮件现在只有两句话。

Now my emails are two sentences long.

所以我就说，行吧。

So I am like, yep.

听起来不错。

Sounds good.

随便吧。

Whatever.

你看，我就是这么发邮件的。

See, like, that's how I send emails.

而Muse帮我写邮件写得特别好。

And Muse did a really nice job of writing emails for me.

所以我会信任Muse代理来帮我写邮件，也会信任Muse模型来帮我写邮件。

So I will trust Muse agent Muse to write, emails for me, and I will trust model Muse to write emails for me.

它真的很好地遵循了我的指示。

It just really followed my instructions quite well.

再说，你可以在我那个语音助手测试里看到这种对话式的基本声音，基本上就是如果我把这个模型接入一个开放云，我会喜欢和它聊天吗？

Again, you can see this, like, conversational basic voice in, my voice assistant test, which is basically like if I plug this model into an open cloud, would I like talking to it?

而且你可以看到，它有一种很不错的、像真人一样的声音。

And you can see here, it's got, like, a nice consumer y human voice to it.

所以它有点像是那种小玩意儿。

So it's like a little slop.

这里有一些破折号。

There are Emdashes in here.

但是，再说一次，它不烦人。

But, again, it's not annoying.

它不腻歪。

It's not cloying.

它就是一种很自然、直接、亲切的语音助手。

It's just kind of a nice straightforward personable voice assistant.

所以这个就是Sonnet在基准测试中一直领先的那个，我觉得Muse的表现跟Sonnet差不多。

So this one is the one that Sonnet, has been dominating in the bench, and I think Muse is, like, right up there with Sonnet performance.

所以，再说一次，如果你在构建一个，特别是个人代理，也许是知识代理，而且你在寻找那种，核心的协调者声音，Muse Wonder可能适合你。

And so, again, like, if you are building a, especially a personal agent, maybe a knowledge agent, and you're looking for that, like, central orchestrator voice, Muse Wonder might be the thing for you.

所以我觉得它在这方面做得很好。

So I think it's done really well at that.

你能看出来它专注于这个，而且从头到尾都展示了这些能力。

You can tell it's focused on that, and it's just demonstrated those capabilities end to end.

那它在哪些方面表现得更差呢？

Now where did it do more poorly?

它写PRD写得不太好。

It does not write PRDs well.

它写的PRD很短，这点我很喜欢。

It was a short PRD, which I love.

它遵循了模板，这点我也很喜欢。

It followed the template, which I love.

但你可以看到这里有一些格式上的问题。

But you can see here we had some, like, formatting issues.

有点难扫读，而且内容非常非常短。

It's a little hard to scan, and it was very, very short.

而且我觉得它和其他代理生成的PRD相比，细节不够。

And I just didn't think it had the detail compared to the other PRDs that were generated by the other agents.

所以我对PRD不太满意，而它真正表现差的地方是设计。

So wasn't in love with PRDs, and then where it really fell down was design.

你看这里，我点一下看看。

And so if you see here, I'm just gonna click through.

其实，我打开完整的输出。

Actually, I'm gonna open up the full output.

我点开设计看看。

I'm gonna click through the designs.

这应该是一个编辑部的任务分配台。

This is supposed to be an editorial assignment desk.

我们见过几次了。

We've seen this a couple times.

设计算中等水平，但很多东西都不好用。

It's kind of like middle of the line design, but a lot of the things didn't work.

我测试原型的时候，特别看重有多少功能是真正能用的，结果很多东西点了都没反应，或者没什么意思。

One of the things that I really test in terms of prototypes is how much of the stuff works, and a lot of things just didn't click or do anything interesting.

所以很难搞清楚这个原型到底想干什么。

And so it's hard to understand what this prototype is trying to do.

那些更复杂的UI设计也是一样的问题。

Same with kind of like these more complicated UI designs.

我们做了个文档调度器。

We do a doc scheduler.

还做了个事件指挥室。

We do like an incident room.

这两个都特别让人抓狂，因为它好像……

And both of these was super frustrating is it like got...

它好像……

It got...

它大概做对了八成，然后就彻底崩了。

It gets like 80% of it right, and then it just totally falls down.

所以这个……

And so this...

我一开始打开这个文档调度应用的时候。

I I pulled it up at first, this doc scheduler app.

我当时觉得，它用了颜色权限，做了所有这些事情。

I was like, it's using color rights, doing all these things.

但当我点进去之后，有些东西，比如这个Xbox，看起来不太好看。

But then as I clicked through, some of the stuff like this Xbox doesn't doesn't look good.

就是缺少那种打磨过的精致感。

There's just this polish to it that wasn't there.

在事件管理平台里更糟糕，我特意告诉它要用深色模式，因为那是开发工具，所有开发者都需要深色模式。

That was even worse in this incident management platform where I told it specifically to be dark mode because it was a dev tool and all developers need dark mode.

结果它用了浅色模式，还带了个很难看的覆盖层。

It did in light mode and had this, like, horrible overlay.

所以，再说一次，我在前端编码上遇到的错误，用Meta、Muse、Wonder这些模型，比用其他模型在很长一段时间里遇到的都要多得多。

So, again, I just got a lot more errors in front end coding with Meta, Muse, Wonder than I have with these other models in a really long time.

然后我就遇到了那种，怎么说呢，千篇一律的粗糙风格。

And then I had this, like, sort of consistent slop style.

再说，我觉得我现在差不多能分辨出每个模型生成的粗糙感，比如哪个模型会给我什么样的粗糙感。

Again, I think I can probably at this point tell slop per model, like, what kind of slop I'm gonna get from what model.

Meta Muse 会给你那种粗衬线字体，这种字体，还有圆角的那种粗糙感。

And Meta Muse is gonna give you fat serif fonts, this kind of font, and rounded corner slop.

它就会给你那种整个应用里都是基础模板、千篇一律的 SaaS 粗糙感。

It's just gonna give you, like, basic template boilerplate SaaS slop across the app.

所以你会看到，在这个 B2B 续费应用和这个植物护理应用里，你得到的就是同样的那种设计组件。

So you'll see in this b to b renewals app and this plant care app, you just are gonna get the same kinda, like, design components.

它们还行，但算不上好看。

They're fine, but they're not beautiful.

而且我觉得它们不是 Astra 或者 Opus 5.5 会生成的那种东西。

And I don't think they're what, like, a Astra or a Opus five five would generate.

它做得最糟糕的是 ChatPRD 网站，这个我必须展示一下，它刚得到了一个漂亮的全新改版。

And the worst thing it did was the ChatPRD website, which I have to show off, just got a beautiful wreath fresh.

真是太惊艳了。

It's so stunning.

我用 Opus 5.5 重新设计了一下。

I did an Opus 5.5 redesign.

特别好看。

It's so pretty.

里面有很多漂亮的小交互组件。

It's got these, like, beautiful little interactive components to it.

就是很讨人喜欢。

It's just lovely.

我们都很喜欢 ChatPRD 网站，结果 Muse 把它给毁了。

We love the ChatPRD website, and Muse obliterated it.

简直太丑了。

Like, it is so ugly.

图片都显示不出来。

The images are broken.

这甚至都不符合我们的品牌设计规范。

It isn't even in our kind of, like, brand design guidelines.

它其实能访问我们的规范。

It has access to our guidelines.

但它就是没照着做。

It just didn't do it.

这太让人失望了。

This was so disappointing.

而且，它还用了表情符号之类的东西，都是那种老掉牙的、我们不喜欢的风格。

And again, like using emojis, all sorts of stuff that's just like old timey slop we don't like.

所以我不……

And so I don't...

我不会用Muse Wonder来做前端网页设计。

I would not reach for Muse Wonder on front end web design.

它真的不是我首选的那个模型。

It's just not gonna be the model I reach for.

我就是觉得它不太好。

I just do not think it's very good.

不过，我还没拿它试过游戏之类的。

Now I haven't tested it with games, for example.

我需要在后端编码体验上多试试它。

I need to press on it more from a back end coding experience.

但我想对很多人说，你们可能会问，我能用 vibe coding 做点什么吗？

But I'll just say for a lot of you, you're saying like, can I vibe code something?

我能给孩子们做个应用吗？

Can I build an app for my kids?

我能为工作做点什么吗？

Can I build something for work?

我就是不太喜欢 Meta Muse 的前端设计感。

And I just I don't love the front end sensibility of a Meta Muse.

我就是不喜欢。

I just don't enjoy it.

最后，我们来看看Meta Muse的创意功能。

Finally, we can get to the creative aspects of Meta Muse.

我在这里做了两件事。

I do two tasks here.

我让它生成了一些SVG，然后让它剪辑了一个短视频。

I have it make some SVGs, and then I have it cut a short form video.

我们来看看这些。

Let's look at those.

先说SVG，我得说，一年前如果有人告诉我能生成这种质量的SVG代码，我肯定会震惊。

The SVGs, first of all, I just have to say, like, a year ago, if you would have told me that you could get coded SVGs at this quality, it would have blown my mind.

所以，再说一次，我们选择太多了。

So, again, we're spoiled for choice here.

我简直不敢相信这些新的前沿模型在创建SVG小插画方面这么厉害。

I cannot believe how good these new frontier models are at creating a little illustrations in SVGs.

这个做得相当不错。

This did a totally fine job.

这个完全没问题。

This is totally fine.

完全没问题。

Totally fine.

它们太可爱了。

They're so cute.

这是一个小PRD。

It's a little PRD.

这是一个小麦克风。

It's a little microphone.

这是一个小虫子。

It's a little bug.

话虽如此，它只是这批里最差的，或者说八个里面排第七。

That being said, it just was the worst of the bunch or seventh out of eight.

Sol和Astra在这方面做得太好了。

Sol and Astra were so good at this.

Opus 5.5 在这方面表现得非常好。

Opus 5.5 was so good at this.

我觉得 Muse 还可以，但不算特别出色。

And I would just say, like, Muse was okay, but not great at it.

我觉得这就是那种所有分数都挤在顶部的情况。

And I just think that this is one of those things where all the scores were compressed towards the top.

所以我不是说 Muse 不好。

And so I'm not saying Muse is bad.

我只是说，针对那个特定用途，我不会优先选它。

I'm just saying I wouldn't reach for it for that specific use case.

接下来是我们最后一个创意任务，我让这些模型剪辑一段关于 How I AI 内容的短视频。

And then on to our very last creative task, I have these models cut a short form video of the How I AI content.

视频剪辑是这些模型越来越擅长的领域之一，所以我想测试一下它们表现如何。

Video editing is one of those things that I think these models are getting a lot better at, and so I'd like to test, how it does.

我觉得当你看到这个视频时——我会让你看的——你会发现它们的设计感还不太到位。

And I think when you watch this, which I will let you watch, you will see that the design sensibility is not quite there yet.

不过我很喜欢这个布局。

Although I do love this layout.

我在想……

I'm gonna think about...

我通常把标题放在上面，图片放在下面。

I usually put my head up top and the image at the bottom.

我在想图片在上面，标题在下面。

I think about the image at the top of my head at the bottom.

那也不错。

That's not bad.

但你看这个视频，你会发现它在剪辑方面做得挺好的，很多模型在这方面都做不好。

But if you watch this video, you can just see, like, it's pretty good at cutting the cuts, which a lot of models struggle with.

所以它的节奏几乎完全正确，但叠加效果和设计真的很糟糕。

So it gets the timing almost a 100% right, but the overlays and the design are just really crummy.

还有一件事……

And something...

它对音频也做了处理，因为音频听起来也很奇怪。

It did something with the audio because the audio is also very strange.

那我们赶紧看一下。

So let's watch it really quickly.

所以智能体和编程方面没什么可抱怨的。

So no complaints on agents or coding.

真正开始走下坡路的是创意任务。

Where it really started to fall off the the cliff are on creative task.

我觉得我最近被 Opus 5.5 惯坏了。

I think I've just been spoiled lately by Opus 5.5.

我知道。

I know.

我知道。

I know.

你这位 GPT 女孩一直在用 Opus 5.5，因为在创意任务、设计、视频、艺术方面，它目前就是市场上最好的。

Your GPT girl here has been using Opus 5.5 because for creative tasks, for design, for video, for art, it is just the best on the market right now.

你看，字幕都跑到我脸上了。

So you can see, like, the captions are on my face.

其实挺难看的。

They're actually kind of ugly.

顶部的那个条，我不太喜欢。

This, like, bar at the top, I I don't like.

这又不是短视频。

Like, this is not a short form video.

不过我会用我的其他几个技能再处理一下。

I will run this, though, through a couple of my other skills.

看看效果吧。

We'll see.

我没说不行。

I'm not calling it a no.

只是也没说行。

I'm just not calling it a yes.

然后它对音频做了一些很奇怪的处理，所以我得再深入研究一下，看看是怎么回事。

And then it did something very strange to the audio, so I have to double down there and figure out what's going on.

所以我给它打两分，满分五分。

So I give this a two out of five.

它找到了正确的四十秒。

It found the right forty seconds.

我觉得音乐很棒，但视频不怎么样。

I think the track was great, but the video was not great.

所以，再说一次，对于创意任务，它确实表现不佳。

And so, again, for creative tasks, it just really didn't do the best.

现在说到整体表现，我要说，我注意到Meta Muse的一个特点就是它非常快。

Now overall performance, I will say one of the things that I noticed about Meta Muse is it is very fast.

而且，对于那种一次性的快速任务，它确实很快。

And with, like, sort of one shot fast tasks, it is fast.

所以，在速度上，它很出色，画角色、做原型都很快。

And so note superiority, very fast, drawing characters, doing prototypes, very fast.

这个模型反应很快，我觉得你会喜欢那种代理式的语音交互，来回反馈很顺畅。

It's a zippy model, and I think you're gonna like that again for that agentic voice, that back and forth feedback.

但是当它执行代理式任务时，确实会花很长时间。

But when it does agentic tasks, it does take a long time.

这可能就是他们说的那种专注努力模式。

This might be that focused effort mode that they say that they have.

所以它在视频处理、收件箱分类、后端功能这些事情上花了很多时间。

And so it is spending a lot of time on things like the videos, inbox triage, back end features.

所以看看在短周期任务和长任务之间，性能表现如何，会很有意思。

And so it'll be interesting to see, you know, over short, kind of, like, in the loop tasks versus long tasks, how the performance nets out.

我刚开始测试并记录性能，下次基准测试我会继续做。

I've just started to test this and keep track of performance, so I will do that in the next bench.

所以我就说，小任务很快，长任务稍微慢一点。

So I will just say little tasks, fast, long tasks a little bit longer.

总的来说，它在代理式工作方面表现非常出色。

In conclusion, it was really strong at agentic work.

它在编码和后端工作方面表现还行。

It was serviceable at coding and back end work.

我打算在那里做更多测试。

I'm gonna test a lot more there.

它肯定会替我写东西，而且我觉得它的声音很好听。

It would definitely let it write on my behalf, and I think it has a nice voice.

它在设计感方面确实有所欠缺，而这正是我工作的大部分内容。

It definitely just falls short on design sensibility, which is so much of my work.

我的工作很大一部分是创意前端设计，所以它在这方面确实表现不佳。

So much of my work is creative front end design, so it really fell down there.

所以我们要看看，如果给它加入技能，给它更多指导，能不能提升它的创造力。

And so we're gonna have to see if putting skills into it, if giving it more direction, we can get that creativity up.

但说实话，我会用Opus 5.5来做这个。

But, honestly, I'll just rely on Opus 5.5 for that.

不过我会试试Muse Wonder，看看能不能把它整合到我正在构建的几个代理中，看看我是否喜欢它的性能和用户体验。

But I am gonna take Muse Wonder and see if I can put it in the loop of a couple agents I'm building and see if I like both the performance and the user experience.