CyberWire Daily - 大语言模型安全基础。[研究星期六] 封面

大语言模型安全基础。[研究星期六]

LLM security 101. [Research Saturday]

本集简介

本周,我们很高兴邀请到Splunk SURGe的全球安全顾问Mick Baccio,分享他们关于"LLM安全:Splunk与OWASP针对基于LLM应用程序的十大威胁"的研究。该研究深入探讨了人工智能和大型语言模型(LLMs)的迅速崛起,这些技术初看似乎充满魔力,但实际上是由人类构建的复杂系统。尽管功能强大,这些系统仍面临诸多网络威胁的脆弱性。 Splunk的研究聚焦于OWASP针对LLM应用程序的十大威胁框架,揭示了诸如提示注入、训练数据投毒和敏感信息泄露等关键漏洞。 研究报告详见: LLM安全:Splunk与OWASP针对基于LLM应用程序的十大威胁 了解更多广告选择,请访问 megaphone.fm/adchoices

双语字幕

仅展示文本字幕,不包含中文音频;想边听边看,请使用 Bayt 播客 App。

Speaker 0

您正在收听由n two k提供支持的Cyberwire网络。

You're listening to the Cyberwire network powered by n two k.

Speaker 0

我们都有过这样的经历。

We've all been there.

Speaker 0

你意识到你的企业昨天就需要招人了。

You realize your business needs to hire someone yesterday.

Speaker 0

如何才能快速找到优秀的候选人呢?

How can you find amazing candidates fast?

Speaker 0

嗯,这很简单。

Well, it's easy.

Speaker 0

用Indeed就行了。

Just use Indeed.

Speaker 1

当它

When it

Speaker 0

说到招聘,Indeed就是你的不二之选。

comes to hiring, Indeed is all you need.

Speaker 0

别再为如何让你的招聘启事引人注目而苦恼了。

Stop struggling to get your job post noticed.

Speaker 0

Indeed的赞助职位能让你脱颖而出,快速招到人才。

Indeed's Sponsored Jobs helps you stand out and hire fast.

Speaker 0

你的职位将跃居搜索结果顶部,让合适的候选人优先看到,而且效果显著。

Your post jumps to the top of search results so the right candidates see it first, and it works.

Speaker 0

Indeed上赞助职位的申请量比非赞助职位高出45%。

Sponsored jobs on Indeed get 45% more applications than non sponsored ones.

Speaker 0

我最喜欢Indeed的一点就是它能极大加快招聘速度。

One of the things I love about Indeed is how fast it makes hiring.

Speaker 0

没错,我们N两K Cyberwire确实在用Indeed进行招聘。

And yes, we do actually use Indeed for hiring here at N two k Cyberwire.

Speaker 0

我的许多同事都是通过Indeed加入我们的。

Many of my colleagues here came to us through Indeed.

Speaker 0

此外,赞助职位无需订阅,也没有长期合同限制。

Plus, with sponsored jobs, there are no subscriptions, no long term contracts.

Speaker 0

您只需为效果付费。

You only pay for results.

Speaker 0

Indeed有多快?

How fast is Indeed?

Speaker 0

哦,根据Indeed全球数据,就在我跟你说话的这一分钟里,已有23人通过Indeed成功入职。

Oh, in the minute or so that I've been talking to you, 23 hires were made on Indeed, according to Indeed data worldwide.

Speaker 0

无需再等待。

There's no need to wait any longer.

Speaker 0

立即通过Indeed加速您的招聘流程,本节目听众还可获得75美元赞助职位信用额度,让您的职位在indeed.com/cyberwire获得更多曝光。

Speed up your hiring right with Indeed, and listeners to this show will get a $75 sponsored job credit to get your jobs more visibility at indeed.com/cyberwire.

Speaker 0

现在就去indeed.com/cyberwire,通过告知您是在本播客了解到Indeed来支持我们的节目,indeed.com/cyberwire。

Just go to indeed.com/cyberwire right now and support our show by saying you heard about Indeed on this podcast, indeed.com/cyberwire.

Speaker 0

条款与条件适用。

Terms and conditions apply.

Speaker 0

正在招聘?

Hiring?

Speaker 0

Indeed就是您所需的一切。

Indeed is all you need.

Speaker 0

大家好,欢迎来到《CyberWires研究星期六》。

Hello, everyone, and welcome to the CyberWires Research Saturday.

Speaker 0

我是Dave Bittner,这是我们每周与研究人员的对话,他们追踪威胁与漏洞,解决难题,在快速演变的网络空间中保护我们。

I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down the threats and vulnerabilities, solving some of the hard problems, and protecting ourselves in a rapidly evolving cyberspace.

Speaker 0

感谢您的参与。

Thanks for joining us.

Speaker 1

过去一年里,我们见证了AI和大型语言模型系统的爆发式增长。

Over the past year, we've seen the explosion of AI and LLM systems.

Speaker 1

人们普遍存在一种误解,认为基于LLM的应用防御极其困难,因为LLM本身的复杂性和AI技术的飞速发展。

And there's this misconception that defending LLM based applications is super difficult because the complexity of LLMs and the rapid pace of AI advancements.

Speaker 1

所以无法对不断变化的事物进行有效防御。

So you can't defend something that's constantly changing.

Speaker 1

我们想深入探讨这个问题,打破这个迷思。

And we kind of wanted to dig into that and, you know, kind of bust that myth.

Speaker 0

这位是Splunk Surge的全球安全顾问Mick Boccio。

That's Mick Boccio, global security adviser for Splunk Surge.

Speaker 0

我们今天讨论的研究报告题为《LLM安全:Splunk与OWASP应用安全十大风险》。

The research we're discussing today is titled LLM Security, Splunk and OWASP Top 10 for applications.

Speaker 1

我们借助了OWASP十大框架——抱歉,全称是开放全球应用安全项目。

And we did that using OWASP, the top 10, the open web I'm sorry, the Open Worldwide Application Security Project.

Speaker 1

如果你年纪大到能脱口而出这个全称,记得今天要补充维生素。

And if you're old enough to remember that off the top of your head, be sure to take your vitamins today.

Speaker 0

你可比我强多了。

You're a better man than I.

Speaker 0

对于可能不熟悉OWASP及其在评估LLM潜在漏洞方面作用的听众,能否为我们讲解一下?

Well, for folks who may not be familiar with OWASP and and its utility in, putting it up against some of these LLM potential vulnerabilities, can you kind of lay that out for us?

Speaker 1

当然可以。

Sure.

Speaker 1

OWASP是一个基金会。

Well, OWASP is a foundation.

Speaker 1

我记得大概是在2001年底左右成立的。

Wanna say it was started 2001, around late two thousand and one, I want to say.

Speaker 1

基本上,我们开发和构建系统所依据的那些行业最佳实践原则,都在那里得到了规范化。

And basically it's kind of those principles that we develop and build systems around those best practices that we talk about in the industry kind of are codified there.

Speaker 1

所以我们利用Smokes OTEL连接器,而OWASP针对LLM的内容正是我们重点关注的领域之一。

So what we did was leverage the Smokes OTEL connector and OWASP for LLMs is is kind of one of the things we focused on.

Speaker 1

在十大清单中,我们重点研究了五项,并提出了我们能想到的最佳检测方案来帮助网络安全从业者。

We out of the top 10 they have, we focus on five, and we kind of came up with the best detections we could think of to help cybersecurity practitioners out there.

Speaker 1

当考虑如何防御LLM系统时,OWASP就是为网络防御者制定这些规范的组织。

So when you think of what those best practices are from defending LLM systems, you know, OWASP is the body that would kind of codify that for NetDefender.

Speaker 1

因此我们采纳了这些建议并开发了相应的检测方案。

And so what we did was take those suggestions and come up with detections.

Speaker 0

那么具体采用的是什么方法呢?

So what was the methodology here?

Speaker 0

你们具体是如何与测试的各种大语言模型进行交互的?

How how specifically did you interact with the various LLMs that you tested?

Speaker 1

我们开发了自己的LLM。

So we developed our own LLM.

Speaker 1

对吧?

Right?

Speaker 1

这样我们就不会攻击任何人的财产、知识产权或产品,我们希望将这些研究作为我们内部研究网络的一部分。

So that way we weren't attacking anyone's property or IP or product in general, we wanted to do that in house as part of our research network.

Speaker 1

于是我们在Splunk的OTEL连接器(OpenTelemetry连接器)中部署了自己的LLM模型,并测试了我们的检测能力。

So we deployed our own LLM models in Splunk's OTEL connector, the OpenTelemetry connector, and tested our detection capabilities.

Speaker 1

通过这些LLM,我们为前十大LLM中的五个制定了最佳实践,正如我之前提到的。

And through those LLMs, we developed the best practices for five of the top 10 LLMs like I mentioned.

Speaker 1

我们从提示注入、不安全的输出处理、模型拒绝服务、敏感信息泄露以及LLM模型盗窃这五个方面入手。

We started off with prompt injection, insecure output handling, the model denial of service, sensitive information disclosure, and LLM 10 model theft.

Speaker 1

就像我说的,我们提出的检测方案是基于使用OTEL收集的遥测数据以及我们从测试的LLM那里得到的响应。

And like I said, the detections we came up with were based on the telemetry we we collected using OTEL and the responses that we got from the LLM that we were testing on.

Speaker 0

鉴于这个列表相对较短,不如我们逐一过一遍,你可以给我们简要概述一下你们的具体发现?

Well, given that it's a a relatively short list here, how about we go through them one at a time and you can give us a little overview of of what exactly you found?

Speaker 0

我们要从提示词注入开始吗?

Should we start with prompt injection?

Speaker 1

当然。

Sure.

Speaker 1

关于提示词注入,我们指的是通过精心设计的输入来操纵大型语言模型,导致LLM执行非预期的操作。

So prompt injection, what we're talking about doing is manipulating a large language model through crafty inputs, you know, causing those unintended actions by the LLMs.

Speaker 1

直接注入会覆盖系统提示,而间接注入则通过外部来源操纵输入数据。

We're talking about direct injections that overwrite system prompts while indirect ones manipulate inputs from external sources.

Speaker 1

这和我们一直强调的理念一致——在处理输入输出时要进行净化处理,我们设计的部分检测机制正是围绕这个核心。

So it's, it's the same thing we preach, when we talk about sanitizing inputs and outputs, some of the detections we, we crafted were around that.

Speaker 0

那么,保护措施的关键就在于净化这些输入数据吗?

And, and so is that the key here to protecting yourself, is sanitizing those inputs?

Speaker 1

我认为是的。

I believe it is.

Speaker 1

呃,应该说这是关键措施之一。

I, I, well, it's one of the keys.

Speaker 1

你觉得怎么样?

How about that?

Speaker 1

我有点认为,当我们讨论前十大原则时——我们稍后会逐一讲解——核心就是‘把基础做对’这个概念。

I kind of think when you talk about the top 10 principles and we'll go through all of these, it's that concept of do the basics right.

Speaker 1

‘吃你的网络安全蔬菜’是我的队友们总让我别再说的话,但我还是会反复提起它。

Eat your cyber vegetables is something my teammates tell me to stop saying all the time, but I I can keep going back to it.

Speaker 1

就是要把基础的事情做对。

It's you're doing the basic things right.

Speaker 1

OWASP针对大语言模型提出的十大原则,与其他系统常见的十大原则非常相似。

The the principles that the OWASP top 10 for LLMs are pretty similar to other top 10 principles you'll see according to other systems.

Speaker 1

原因在于,本质上这些都是我们需要正确执行的基础性原则。

And the reason for that is because, you know, it's essentially the same principles we need to do correctly, those foundational things.

Speaker 1

当我们讨论新系统、新工具的实施时,这就是我们所说的‘做好基础并持续坚持’。

When we talk about new systems, new tools we're implementing, this is what we mean when do the basics right and and keep doing those things.

Speaker 1

而这确实不容易。

And it is difficult.

Speaker 1

确实如此。

It is.

Speaker 1

但只有做好这些基础工作,我们才能构建起安全防线。

But doing those is how we build that base security.

Speaker 0

好的,我们接下来看看下一个议题——不安全的输出处理。

Well, let's move on to the next one here, which is insecure output handling.

Speaker 1

没问题。

Sure.

Speaker 1

因此当我们讨论不安全的输出处理时,Dave,漏洞发生在当LLM的输出未经审查就被接受,从而暴露后端系统的情况下。

So when we talk about insecure output handling, Dave, the the vulnerability occurs when an LLM output is accepted without scrutiny, exposing back end systems.

Speaker 1

对吧?

Right?

Speaker 1

所以我们讨论诸如跨站脚本、跨站请求伪造、SSRF,甚至远程代码执行或权限提升等问题,这些都可能是由于输出处理中的任何安全隐患引起的。

So we talk about things like cross site scripting, cross site request forgeries, SSRF, even remote code execution or privilege escalation can be caused by output handling, any insecurities inside that.

Speaker 1

因此,这再次成为构建LLM系统、使用基于LLM的应用程序作为网络防御者时的关键事项之一。

So it's one of those things, again, critical when you're building out LLM systems, using LLM based applications as a NetDefender.

Speaker 0

你能给我们一些具体的例子,说明这种情况会如何发生吗?

Can you give us a little insight here, a specific example of how this could play out?

Speaker 1

我记得我们有一些案例,比如某个LLM系统被用户诱导输出了一张打折卡或特定价格的机票。

I think some of the examples we had were there was an LLM system that the output a user had tricked it into giving a card a discounted price or paying a specific fare for an airline.

Speaker 1

正是这些不安全的输出导致了人们对这些系统的信任和依赖度降低。

And it's those insecure outputs that kind of lead to a lack of or degradation and trust and and, you know, reliance on those systems.

Speaker 0

我们稍后回来。

We'll be right back.

Speaker 0

大多数环境过度信任系统,而攻击者深知这一点。

Most environments trust far more than they should, and attackers know it.

Speaker 0

ThreatLocker通过在执行点实施默认拒绝策略来解决这个问题。

ThreatLocker solves that by enforcing default deny at the point of execution.

Speaker 0

使用ThreatLocker的应用白名单功能,你可以彻底阻止未知可执行文件。

With ThreatLocker allow listing, you stop unknown executables cold.

Speaker 0

通过沙盒隔离技术,你可以控制受信任应用的行为。

With ring fencing, you control how trusted applications behave.

Speaker 0

通过ThreatLocker的DAC配置防御功能,您能真正确保环境不存在配置错误,并清晰掌握是否符合合规标准。

And with ThreatLocker D a C defense against configurations, you get real assurance that your environment is free of misconfigurations and clear visibility into whether you meet compliance standards.

Speaker 0

ThreatLocker是以最简单方式实施零信任原则的解决方案,且无需承受运维痛苦。

ThreatLocker is the simplest way to enforce zero trust principles without the operational pain.

Speaker 0

这种强大的防护能力为首席信息安全官提供了真正的可视性、控制权与内心安宁。

It's powerful protection that gives CISOs real visibility, real control, and real peace of mind.

Speaker 0

ThreatLocker让零信任变得触手可及,即便是小型安全团队也能实现。

ThreatLocker makes zero trust attainable, even for small security teams.

Speaker 0

了解为何数千家机构选择ThreatLocker来减少警报疲劳、从源头阻断勒索软件,并重新掌控其环境。

See why thousands of organizations choose ThreatLocker to minimize alert fatigue, stop ransomware at the source, and regain control over their environments.

Speaker 0

立即访问threatlocker.com/n2k预约演示。

Schedule your demo at threatlocker.com/n2k today.

Speaker 0

AI正在改变每个行业,但同时也带来了传统框架难以应对的新风险。

AI is transforming every industry, but it's also creating new risks that traditional frameworks can't keep up with.

Speaker 0

当前的评估体系零散、重叠,且往往针对特定行业、地域或法规。

Assessments today fragmented, overlapping, and often specific to industries, geographies, or regulations.

Speaker 0

正因如此,Black Kite创建了BKGA3人工智能评估框架,为网络安全和风险团队提供一个统一的动态标准,用以衡量其组织内部及供应商AI应用中的风险。

That's why Black Kite created the BKGA3 AI assessment framework to give cybersecurity and risk teams a unified evolving standard for measuring AI risk across their own organizations and their vendors' AI use.

Speaker 0

该框架具有全球性、研究驱动、能随威胁态势演变而进化,并且完全免费使用。

It's global, research driven, built to evolve with the threat landscape, and free to use.

Speaker 0

因为Black Kite致力于加强整个网络安全社区。

Because Black Kite is committed to strengthening the entire cybersecurity community.

Speaker 0

了解更多信息,请访问blackkite.com。

Learn more at blackkite.com.

Speaker 0

我记得几个月前有个著名案例,有人在聊天中说服当地车商以5美元左右的价格卖给他一辆全新的雪佛兰卡车。

I remember a few months ago famously seeing one where someone was in a chat with a local car dealer and somehow talked it into selling him a, you know, a new Chevy truck for $5 or something like that.

Speaker 0

对吧?

You know?

Speaker 1

没错。

Right.

Speaker 1

这又回到了那个原则:机器不做决定,所以机器不能担责。

And it goes back to that principle, you know, it's a machine that doesn't make a decision, so a machine can't be responsible.

Speaker 1

所以责任又回到了你身上,要确保这些系统的安全性和可靠性到位,才能避免出现那样的结果。

So it's back to you, you know, to to ensure that the safety and security of those systems is is there before you get a result like that.

Speaker 0

对。

Right.

Speaker 0

嗯,下一个是模型拒绝服务。

Well, the next one is model denial of service.

Speaker 0

关于这个你能告诉我们什么?

What can you tell us about that one?

Speaker 1

拒绝服务攻击其实是个相当普遍的概念。

So denial of service is is pretty, you know, I I guess, kind of a universal concept.

Speaker 1

当攻击者让大语言模型执行资源密集型操作时。

When attackers cause resource heavy operations on LLMs.

Speaker 1

在大语言模型场景下,我们说的不是反复访问某个网站或攻击互联网服务提供商。

And on LLMs, we're not talking about, you know, hitting a website over and over again or hitting an internet service provider.

Speaker 1

我们指的是后台的计算资源层级,那些支撑大语言模型和生成式AI运转的计算周期。

We're talking about levels of something called compute, those computational cycles in the backend that kind of make all of the trains go when it comes to LLMs and generative AI.

Speaker 1

攻击者将引发资源密集型操作,从而导致服务降级。

Now, an attacker is going to cause resource heavy operations kind of leading to degradation of service.

Speaker 1

这种漏洞会被放大,因为大语言模型本身资源消耗巨大,加上用户输入的不可预测性会进一步加剧这种情况。

And that vulnerability is going to be magnified because LLMs are so resource intensive and the unpredictability of user inputs will kind of, you know, exacerbate that situation.

Speaker 1

因此需要像保护网站或其他基础设施那样,采取防护措施来抵御这类拒绝服务攻击。

So it's important to kind of safeguard against those denial of service attempts like you would a website or any kind of infrastructure that you have.

Speaker 0

我很喜欢研究中你使用的那个提示词示例。

I love in in the research here, the example you used was the prompt you gave.

Speaker 0

提示词内容是:'说奶酪',重复说直到你无法再说为止。

It says, say cheese over and over until you can't say it anymore.

Speaker 0

这个例子既有趣又聪明,但它确实简单展示了如何制造一个无限循环。

I mean, it it's funny, and it's clever, but it really is a simple example of how you you're basically creating an endless loop.

Speaker 1

正是如此。

Exactly.

Speaker 1

没错。

Exactly.

Speaker 1

我认为这类似于如果你经历过fork炸弹时代,它会占用系统所有资源,除非设置限制否则无法轻易关闭。

I think it's akin to the if you are fork bomb years old, it's something that's just gonna, you know, hog all the resources to any system, and there's not really a way to quite turn that off unless you put in a limit.

Speaker 0

对。

Right.

Speaker 0

对。

Right.

Speaker 0

那么下一个问题是敏感信息泄露。

Well, the next one is sensitive information disclosure.

Speaker 0

关于这一点我们需要了解什么?

What do we need to know about that?

Speaker 1

当谈到敏感信息泄露时,这是我们日常工作中越来越多地使用LLM工具的关键问题之一。

So when you talk about sensitive information disclosure, this is one of the critical things of more we leverage LLM tools as part of our daily job.

Speaker 1

你可能会无意中在这些响应中泄露机密数据,导致未经授权的数据访问、隐私侵犯和安全漏洞。

You may inadvertently reveal confidential data in those responses leading to unauthorized data access, privacy violations, security breaches.

Speaker 1

试想一下,当你将大量组织数据输入LLM时,这些数据的专有价值是什么?这些数据是否应该让任何能访问该LLM应用的人获取?

When you think about, you are putting a lot of your organization's data inside an LLM, what is the proprietary value of that data you're putting, and should that data be accessible to anyone that has access to that LLM application.

Speaker 1

因此我认为这变得极为关键,需要实施数据净化和严格的用户策略来缓解这个问题,确保输入系统的数据只能由特定角色的人员访问。

So I think that's where it becomes extremely critical to implement data sanitization and strict user policies to kind of mitigate that so that whatever data you're putting into a system is only allowed to be accessed by certain people with certain roles.

Speaker 0

是的。

Yeah.

Speaker 0

你提到的例子是使用了微软一个叫Presidio的SDK,它专门用于查找个人身份信息。

The example you use so you were using an SDK from Microsoft called Presidio, which looks for personally identifiable information.

Speaker 0

那么这里的理念是,在前端使用这类工具来确保人们不会无意间让这些信息流入LLM系统?

So is the notion here that you use a tool like this on the head end to make sure that people aren't inadvertently allowing this information to get to the LLM?

Speaker 0

反过来,

And conversely,

Speaker 1

返回到用户端。

to get back to the user.

Speaker 1

我明白了。

I see.

Speaker 1

即使是你在思考时可能搜索的偶然模式,并非刻意或直接,但ID匹配也可能无意间发生。

Even incidental patterns that you might search for when you think of things, not not surreptitiously or directly, but inadvertently IDs might match.

Speaker 1

一个账户ID可能匹配到信用卡、护照号或社保号的正则表达式。

An account ID might match a regular expression that matches a credit card or a passport number or a social security number.

Speaker 1

正是这些正则表达式可能会无意中返回我们视为PII的数据。

And it's those regular expressions that might inadvertently return data that is what we consider PII.

Speaker 1

因此为这类情况设置警报并调用后端服务很重要。

So it's important to set up an alert for those things and calling to a back end.

Speaker 1

这不是LLM的问题,而是系统设计应当能识别哪些用户访问了哪些数据。

It's not an LLM issue, but it's again the design of your system should be able to identify which user access what data.

Speaker 1

所以审计追踪在这里变得极其重要。

So that's where audit trails become super important.

Speaker 1

这样即使净化过程未能100%完成,你仍有审计记录可查。

So in the event that those sanitizations aren't a 100%, you have that audit trail.

Speaker 0

不。

No.

Speaker 0

这很有意思。

That's interesting.

Speaker 0

我在想,比如一个内部部署的大语言模型,有人问它‘给我所有关于我们HR总监Bob的信息’。

I I I'm I'm thinking of, you know, an in house LLM and someone asking it, you know, give me all the information you have about our HR director, Bob.

Speaker 0

然后,就像,

And, you know, like,

Speaker 1

你知道,然后

you know And and

Speaker 0

然后这里显示Bob的薪资,还有Bob上次生病的时间,诸如此类的信息。

and here's how much Bob makes, and here's, you know, here's the last time Bob was sick, and all those kinds of things.

Speaker 0

要知道,无论这些数据是否被有意存储,都需要防范这类泄露。

You know, there's if that data's in there intentionally or not, that's something you gotta protect against.

Speaker 1

确实如此。

Exactly.

Speaker 1

确实如此。

Exactly.

Speaker 1

我认为随着我们越来越依赖这些大语言模型系统,这个问题变得更为关键。

I think it becomes more critical the more we have this reliance on these LLM systems.

Speaker 0

是啊。

Yeah.

Speaker 0

嗯,你在这里提到的最后一点是模型盗窃。

Well, the last one you cover here is model theft.

Speaker 0

关于这个你能告诉我们些什么?

What can you tell us about that?

Speaker 1

要知道,模型盗窃就是字面意思。

You know, model theft is exactly what it sounds like.

Speaker 1

指的是未经授权访问、复制或窃取专有的大语言模型。

It's the unauthorized access, copying, or exfiltration of proprietary LLM models, you know.

Speaker 1

其影响范围从完全无关紧要到灾难性的经济损失,比如竞争优势归零或可能获取敏感信息。

And the impact from that is anywhere from absolutely nothing to catastrophic economic losses, you know, competitive advantages being zeroed out or potential access to sensitive information.

Speaker 1

我认为我们需要想办法最好地防范模型盗窃,比如通过重复查询来推断模型内容。

And I think it's that model theft is something that we we kind of need to figure out how to combat best, you know, inferring the contents through repeated queries.

Speaker 1

这让我想起那个网络安全蔬菜的比喻,要监控审计日志,要么限制访问包含模型数据的系统,要么限制请求速率,因为通过推理提取模型会留下很多痕迹。

Like, I think this is very much I kind of harken back to that cyber veggies metaphor where you access audit log, like, you know, access to either the systems that contain the model data or rate limit requests because extracting it through inference is pretty noisy.

Speaker 1

因此我认为这是众多可采取方法中最基本的一种。

So and I think that is the numb one of the many approaches you can take.

Speaker 1

你需要了解他们试图提取什么数据才能检测到这些尝试。

You'll need to know what data they're trying to extract in order to detect the attempts.

Speaker 1

这某种程度上又回到了那个概念,虽然不是通过OWASP LLM,而是广义上的OWASP,即充分了解你的环境才能有效防御。

It kinda goes back to that concept, not through OWASP LLMs, but OWASP in general, where knowing your environment well enough to defend it.

Speaker 0

你们的研究中有没有什么概括性的要点想分享给听众们?

Are are there any sort of broad general take homes that you wanna offer to our listeners here from the research that you all did?

Speaker 0

你有什么智慧箴言想分享给大家吗?

What are the what are the words of wisdom you like to share?

Speaker 1

哇。

Wow.

Speaker 1

这很艰难,而且形势正变得越来越严峻。

It is tough, and it's getting tougher out there.

Speaker 1

我认为各行各业的安保从业者都需要确保他们采取了正确的实践措施,来保护部署在其环境中的LLM模型安全。

I think security practitioners across every industry need to ensure that they're putting the right practices in place to secure the LLMs being deployed in their environments.

Speaker 1

我认为企业主和高管们应该听取同样的建议,尽管很多人担心这太复杂。

I think that business owners and executives would, know, take heeded to listen to the same advice where many people are worried where it's too complicated.

Speaker 1

我认为团队发布的研究表明,你可以运用我们早已熟知的、长期存在的安全原则来加强防御,这使得提升LLM安全性变得更加容易。

I think the research that the team has put out kind of showed that you can defend using established principles that we principles we've known about for quite some time, principles that have been in place for quite some time, making it easier to improve LLM security moving forward.

Speaker 1

我认为最重要的是,不要将效率与效果混为一谈。

I think most importantly, don't confuse efficiency for f, you know, efficacy.

Speaker 0

It

Speaker 1

快速完成某事并不等于有效或正确地反复执行。

is doing something quickly is not doing something effectively or doing something right over and over.

Speaker 1

我认为这正是我们在利用LLM系统时需要格外谨慎的地方,因为它们发展速度极快,而且预计会更快。

And I think that's where we really, really need to be careful when leveraging LLM systems because they do move so fast and they predict to be moving faster.

Speaker 0

感谢来自Splunk的Mick Baccio参与我们的讨论。

Our thanks to Mick Baccio from Splunk for joining us.

Speaker 0

该研究题为《LLM安全:Splunk与OWASP十大LLM应用安全风险》。

The research is titled LLM Security, Splunk and OWASP Top 10 for LLM based applications.

Speaker 0

我们会在节目备注中提供链接。

We'll have a link in the show notes.

Speaker 0

我们很想知道您对这期播客的看法。

We'd love to know what you think of this podcast.

Speaker 0

您的反馈能确保我们持续提供洞见,助您在快速变化的网络安全领域保持领先。

Your feedback ensures we deliver the insights that keep you a step ahead in the rapidly changing world of cybersecurity.

Speaker 0

如果您喜欢我们的节目,请在您常用的播客应用中分享评分和评论。

If you like our show, please share a rating and review in your favorite podcast app.

Speaker 0

也请填写节目备注中的调查问卷,或发送邮件至cyberwiren2k.com。

Please also fill out the survey in the show notes or send an email to cyberwiren2k dot com.

Speaker 0

我们深感荣幸,N2K的Cyberwire已成为公共和私营领域最具影响力的领导者和从业者日常必读——从财富500强到全球顶尖的情报和执法机构。

We're privileged that N2K's Cyberwire is part of the daily routine of the most influential leaders and operators in the public and private sector, from the Fortune five hundred to many of the world's preeminent intelligence and law enforcement agencies.

Speaker 0

N2K帮助企业轻松优化您最重要的投资——人才。

N2K makes it easy for companies to optimize your biggest investment, Hester.

Speaker 0

我们的执行制作人是Jennifer Ivan。

Our executive producer is Jennifer Ivan.

Speaker 0

我们的执行编辑是Brandon Karp。

Our executive editor is Brandon Karp.

Speaker 0

Simone Petrella是我们的总裁。

Simone Petrella is our president.

Speaker 0

Peter Kilpey是我们的出版人,我是Dave Wittner。

Peter Kilpey is our publisher, and I'm Dave Wittner.

Speaker 0

感谢收听。

Thanks for listening.

Speaker 0

我们下次节目再见。

We'll see you back here next time.

关于 Bayt 播客

Bayt 提供中文+原文双语音频和字幕,帮助你打破语言障碍,轻松听懂全球优质播客。

继续浏览更多播客