E063 - AI or Not - Chad Ratashak and Pamela Isom

Welcome to "AI or Not," the podcast where we explore the intersection of digital transformation and real-world wisdom, hosted by the accomplished Pamela Isom. With over 25 years of experience guiding leaders in corporate, public, and private sectors, Pamela, the CEO and Founder of IsAdvice & Consulting LLC, is a veteran in successfully navigating the complex realms of artificial intelligence, innovation, cyber issues, governance, data management, and ethical decision-making.

AI doesn’t need permission to enter your organization anymore because it arrives bundled inside the software you already pay for. That is why “we don’t use AI” can be the riskiest assumption of all. We sit down with Chad Ratashak, an AI risk leader and the Midwest Frontier AI Consulting enterprise leader, to talk about what actually changes when generative AI becomes a background feature in email, search, documents, and video calls.

We dig into the hard problem most people hand-wave away: hallucinations in large language models (LLMs) like ChatGPT. Chad explains why the models can produce convincing falsehoods, why that risk is not “solved,” and what the legal world learned from the Mata v Avianca case where fake citations became a real courtroom issue. From there, we get practical about verification habits, risk appetite, and the time cost of validating AI output so you know when AI is worth using and when it is pure friction.

Then we move into AI transcription and meeting notes: the difference between built-in Zoom or Teams transcription and third-party bots that “join” as attendees, how default settings can leak sensitive pre-meeting banter, and why AI transcripts can omit key objections or misattribute who said what. We also talk shadow AI, why training beats forced AI mandates, and how agentic AI and coding assistants can create outsized damage unless you design for failure with backups, least privilege, and resilience testing.

We close with the darker side: deepfake voice and live video filters enabling cryptocurrency scams, fake family emergencies, and evolving ransomware tactics. If you want a clear-eyed, actionable framework for AI governance and cybersecurity risk, press play, share this with a colleague, and leave a review with the one AI risk you want us to tackle next. 

Transcript

[00:00] Pamela Isom: This podcast is for informational purposes only.

[00:26] Personal views and opinions expressed by our podcast guests are their own and not legal advice.

[00:34] Neither health, tax nor professional nor official statements by their organizations.

[00:42] Guest views may not be those of the host.

[00:50] Hello and welcome to AI or not the podcast where business leaders from around the globe share wisdom and insights that are needed right now now to address issues and guide success in your artificial intelligence and that digital transformation journey.

[01:06] I am Pamela Isom and I am your podcast host.

[01:10] And so look, we have one of those special guests with us today. You'll see his name is Chad Ratashak.

[01:17] He is a Midwest Frontier AI Consulting enterprise leader.

[01:24] He is an AI risk leader.

[01:27] He is a leader of ransomware cryptocurrency. So Chad,

[01:33] welcome to AI Or Not.

[01:35] Chad Ratashak: Thanks Pamela for having me and give you some context on sort of how I got to where I landed.

[01:42] So I'm originally from Iowa.

[01:44] Like you, I lived in the D.C. area for a while and then got out of the high cost of living area.

[01:51] And so I started out my career worked for a couple NGOs working on refugee issues and countering recruitment by extremist organizations. I studied Arabic and worked on that and then I moved into defense contracting, worked for Leidos.

[02:09] Folks who were in defense contracting will know that name. Everybody else has no idea what they do except at the airport. I guess you'll see their their stuff scanning you.

[02:18] But I didn't have anything to do with that part of the business.

[02:21] I was an intelligence analys working on open source intelligence using Arabic. Actually worked on an early project detecting bot activity on social media using machine learning all the way back in late 2017.

[02:36] And then I moved over from defense side over to the financial sector, moved over to Wells Fargo, originally in the D.C. area and then Covid hit transferred to their Des Moines office.

[02:48] Same job on their financial crimes intelligence team working on complex financial crimes,

[02:55] things like ransomware, cryptocurrency related cybercrimes,

[03:00] things of that nature. And then of course as interest in generative AI like ChatGPT started to grow, I worked on that as part of my portfolio of work. Things like deepfake voices and biometrics as well as large language models LLMs.

[03:16] That's the general technology that ChatGPT is a part of now as Midwest Frontier AI Consulting I serve businesses around the country, but focusing specifically on the Midwest region,

[03:29] law firms and small medium businesses to help them understand generative AI risks and decide like kind of like the name of your podcast to choose whether or not to use AI So I don't push it, but I teach people how to use it and the risks and benefits and help them navigate the cost benefit analysis and whether it's appropriate for them or not.

[03:49] Pamela Isom: Well, so you gave me some background and yourself, which is quite interesting. The open source intelligence using Arabic, never thought of it.

[04:00] And then complex financial crime. So all I can say is I'm glad we're good friends.

[04:09] I want to know more about your entrepreneurial journey and your future plans. What can you tell me about that? Like how did you end up in where you are now?

[04:19] Chad Ratashak: Yeah, that's a great question.

[04:20] So one of the areas where I saw problems with large language models is the hallucination problem, which is simply that they'll make up very fluent, convincing, but false statements.

[04:34] Now, you might hear people saying that this problem is solved or that it's getting close to being solved, but actually the research from the AI labs themselves,

[04:43] like OpenAI, which makes ChatGPT,

[04:46] like Anthropic, which makes Claude,

[04:48] is that it's not solved and it's probably going to be a while before it's solved, if it's solved at all.

[04:54] It might be inherent to the nature of large language models that it's not solvable. And so that's concerning. Right. How do you use a tool that sometimes makes stuff up that's not true but is very convincing?

[05:05] And so thinking through that problem,

[05:07] one of the areas where this first came up in 2023 was in a case called Motta vs. Avianca, where an attorney cited cases that did not exist in a Southern District of New York case.

[05:20] And then the opposing counsel in that case said, we can't find these cases, they don't seem to exist. And they told the judge, and the judge went and looked and said, I can't confirm these cases exist either.

[05:31] And they went to the attorney and said, hey, can you produce these cases? Because we can't find any evidence that these cases exist. Well, the attorney then went and asked ChatGPT for the text of those cases and guess what?

[05:45] It dutifully bat out text of cases that don't exist, because that's part of the hallucination problem. AI is very good at continuing to just keep going along with fictional scenarios.

[05:57] Sometimes it'll push back. And AI is getting better.

[06:00] The newer AI models are getting better at pushing back against you if the question has a false premise, like, wait, you're asking me for the text of this question,

[06:07] this case, but the case doesn't exist, so I can't give it to you, that problem still remains, but it was worse in 2023 than it is now. So I don't want to say it's gone away.

[06:16] It is still a problem. But back in 2023, it was even worse. So anyway, this attorney gets this fake text to this case, submits it,

[06:24] and now it's not just that they cited a fake case, but they've given text that they're purporting to be the text of the fake case. And it's just nonsense Spat out by ChatGPT, but it looks good.

[06:37] So as you're evaluating AI tools, a lot of times people will say, oh, well, the AI is getting better. No,

[06:43] my framing is the AI is getting more capable.

[06:46] The reason I say more capable, not better, is because if it's better, better implies good, and getting better means problems are going away. But if the AI is getting better at tricking you into believing something is true when it's not, that's actually a bad thing.

[07:01] So more capable tells you, hey, the risks are still there. Maybe they're getting fewer and further between, but the risks that do get through are more likely to trick you.

[07:10] So now we got to rethink how do we use these tools? And I don't want to say with any of that. We don't use AI.

[07:16] I use AI. Pamela, I know you use AI, but we've got to be discerning about the risks and realistic and not just kind of hand wave. Oh, it's getting better.

[07:25] They'll probably fix all the problems by next year. We don't need to worry about it.

[07:29] No,

[07:31] that's not my message. So that's why I started with attorneys and actually education for attorneys, because it remains a problem in the legal profession as well as small businesses who are trying to juggle, wear a lot of hats and do everything themselves.

[07:46] They kind of have the same problem set.

[07:49] Yeah. And you and I both know that.

[07:51] Pamela Isom: Right, Exactly. Like we need the help sometimes. But the thing that I weigh out is,

[07:57] first of all,

[07:58] what is the situation that I'm going to resort to one of the tools for one of the agents for?

[08:06] And second of all,

[08:07] what's the appetite? What can I tolerate from a business perspective? Can I tolerate a little bit of variance here or there?

[08:18] And also the other thing I weigh in is the time that it's going to take to validate,

[08:26] because you have to go back and validate. So the question becomes, okay, do I want to use it for,

[08:32] like, I don't like using it for summarizing notes from meetings because it leaves out too much. Like, stuff that I think is important to me, it leaves out. And you're just like, by the time you go back and rectify and validate, you might as well have just done it yourself.

[08:45] Chad Ratashak: Exactly.

[08:46] Pamela Isom: A lot of people do. So. So that's where I think what you're saying is those are things that we have to weigh in.

[08:52] Because in that particular example that you gave, and it's a good business model. I appreciate your business model, by the way, in the example that you just shared.

[09:01] It's something that you would think that we would normally do. If I had a paralegal working for me and I assigned them to go and do this research, I'm still going to check on the data that they come back with.

[09:15] I may not check it line by line by line,

[09:18] but I'm going to check. If I'm a legal person and I have a paralegal working for me, I am going to check. That's just part of my due diligence.

[09:26] So why would we give it up?

[09:27] Because we rely on some tool.

[09:30] Chad Ratashak: Exactly.

[09:31] And the thing with the MATA case is if you replace ChatGPT with some guy named Bob, then you say, hey, the judge says Bob's lying, Opposing counsel says Bob's lying.

[09:42] What's your move?

[09:44] Oh, I'm gonna go ask Bob. Bob says he's not lying. And Bob gave me this case that looks totally legit. Now all of a sudden you're like, why are you still trusting Bob?

[09:53] Go ask literally anyone else.

[09:55] Pamela Isom: But.

[09:56] Chad Ratashak: But for some reason,

[09:57] that logic didn't transfer to ChatGPT. So when we treat these AI tools as if they're infallible.

[10:03] So that's my little rhetorical trick. If you just replace it with some guy named Bob and you wouldn't apply the same approach to validating its output. Don't ask the same AI tool to validate its own output when somebody else who knows what they're talking about has told you this seems wrong.

[10:21] Go check something else.

[10:23] Literally anything else. Although I will warn a second AI tool might actually go along with the first AI's falsehood. So that's a whole other category of risk that I also warn about.

[10:34] So you gotta watch out for that, too.

[10:37] Pamela Isom: Those are good points. Like I said, I appreciate that business model you have going on. We need it.

[10:41] So that tells me about your journey now and your. Your trajectory as well.

[10:47] So my next question.

[10:48] You have expressed appreciation and concerns about AI transcription in video conferences and other pathways.

[10:59] So what's going on? Tell me more about what's happened and what do you like, what you dislike and where's that risk that we need to mitigate?

[11:06] Chad Ratashak: Yeah, that's a great question. So there's kind of two. I'm going to put them in two buckets. There's your first party transcription. I'm being very general here, but I think it's helpful.

[11:16] The ones that are built into the tools themselves, you got Zoom or Teams or Google Meet that they might offer AI transcription that's built into the tool. The host of the program typically is the one writing the transcription.

[11:30] And then you've got your third party ones where the AI will do join the meeting, something like Otter or something like that. And it might appear as an actual attendee as if it were a person.

[11:42] And so the main concerns with the first person tools are that the defaults are often overly permissive, especially if you're in a sensitive field. I know that a lot of your listeners and a lot of folks we'd be talking to might be in federal contracting,

[11:59] so you might have higher information security standards. I talked to attorneys, they obviously have confidentiality concerns and things like that.

[12:07] And so I don't push a particular line and say you can't use these tools or you have to get rid of them or you can only use these settings. I lay out kind of the menu of options and explain them and the cost and benefit.

[12:21] And I do think there are different trade offs for different people have different risk profiles. Right. So there are different trade offs. But there are certainly people who tell me no way, like now that you've explained it to me, we don't want to use it.

[12:35] And there are other people who say, okay, like we need to turn off like these eight settings. We want it to only email to myself. I don't want it to send an email blast to every attendee of the meeting.

[12:47] And you might not even realize that that might be a default setting for certain Zoom meetings where you think maybe you're, you're giving a meeting and you have the pre banter like, like we did before, before the podcast.

[12:59] Right. You don't want that to go out in a transcript that gets emailed to every attendee of a meeting or something like that. But that can happen and you don't realize that.

[13:08] And that stuff could be more confidential, more private or just per, just personal about your life that you don't want some random person knowing and having in an email that's transcribed somewhere.

[13:20] And so those things you know are important. You also mentioned like the details that it chooses to focus on maybe it missed an important thing from the meeting, but it added recorded details about stuff that's really not that important about the meeting.

[13:35] However, if you're one person and you are very busy, it can be helpful to go back and look at the transcripts of a meeting that you just didn't have time to take the notes yourself and so forth.

[13:47] And so there can be use for it. I'm definitely not saying don't use it. I think also though, we need to as a whole society,

[13:57] like this isn't one person, but just everybody needs to talk about this more about what our norms are around,

[14:02] you know, of disclosure and stuff. Like we talked about it, you know, before the meeting. So I, you know,

[14:08] and I always talk to my clients about how I, for my clients, because of my position on it, I always tell them that by default I will not use it in meetings with them to kind of like live out my standard.

[14:20] But I don't say, therefore you can't use it with me. If you would like me to turn it on and have the notes, I'll do that. But by default when I start a meeting,

[14:28] I'll have it off as just a way of signaling like that's my position as least permission until least permission to the AI tool until I've talked to the client about it.

[14:42] But other times with these third party tools, sometimes people join your meeting and the person doesn't actually attend it because they can't make it for some reason, but they'll send their third party AI transcription, as you know, in their stead and you're like,

[14:57] what is this? Why is this person's AI transcription tool? Well, attending my meeting and sometimes that's just annoying to you, but other times that can be a real security risk because some of these third party transcription tools have had serious information leaks where I don't recall now off the top of my head which tool it was,

[15:15] but one of them had leaks where there was information about meetings where people from Disney and government officials and stuff. Not the same meeting, but different meetings where there were transcripts leaked because it wasn't properly secured.

[15:30] So there's information security concerns about these third party ones. And so if you're not thinking as an organization, like what is my policy towards this transcription? I think almost no organizations have, especially small organizations, I want to be clear.

[15:45] Obviously there's like very large organizations that have legal teams that have already gone line by line through all this kind of stuff. But you know, your small organization, you're hopping on all kinds of zoom calls with all kinds of people, and everybody's just using AI transcription by default.

[16:01] Sometimes multiple different transcribers on the same beating.

[16:04] And the sort of information security hygiene around that is just very lax. And so I think everybody needs to be thinking more seriously about that.

[16:14] Pamela Isom: I think it's the small and the large that need to pay attention to this, by the way,

[16:18] and revisit your transcription policies, because we probably didn't have them, other than meetings should not be recorded unless you have permission. Right. And now it's totally different.

[16:31] And so I would think that that's applicable to all businesses and legal teams and business units should have a transcription policy to address these types of emerging practices.

[16:47] Chad Ratashak: Yeah. And I think when it comes to larger organizations, too, a lot of times they'll have, you know, like governance committees or some kind of group that will make official decision where they have to have a quorum and they have to have minutes.

[17:02] And I would say for those types of situations, especially that or a board meeting, for example,

[17:09] that the AI transcription of that meeting is not going to be suitable for replacing minutes. That's my opinion. I'm not like, again, I'm not an attorney. I provide AI education for attorneys.

[17:20] I'm not an attorney. But just my opinion from the technology standpoint is that those aren't going to be adequate for meeting minutes to replace them. And the reason why I say that is for a couple different reasons.

[17:32] One,

[17:33] sometimes they just omit things that people said that were important.

[17:37] So if somebody's making, let's say, an objection, hey, I don't think we should do this. And that's going to come up later.

[17:43] Well, why did. Was there no discussion about whether or not we should make this decision if that's going to be a point of dispute on a meeting that is supposed to have minutes?

[17:52] Five people on the committee voted for, one against. This was their objection. And the objection isn't marked because it's not actually proper meeting minutes. It's just whatever the AI decided to transcribe.

[18:03] That's a problem.

[18:04] Another problem is that the AI will sometimes misattribute who said what. So it'll say Pamela said such and such to Chad, but it's actually the other way around. Chad said that to Pamela.

[18:14] Well, now, if I made the bad decision,

[18:17] but the transcript attributed it to you,

[18:20] you're on the hook for my bad call. And then if it is a bad call and somebody's getting in trouble for it, I can walk away and throw you under the bus.

[18:27] For it. That's not right.

[18:29] But that could happen. And I would not be surprised if we see situations in corporate governance where that happens in the near future,

[18:36] because one person made the. Tried to. Tried to railroad a decision and say, hey, this is what we're going to do,

[18:43] and that's it. But then the person that decision is attributed to in the AI transcription of the meeting is not the correct person who actually started on the call.

[18:55] That's a problem.

[18:56] And then there are subtler problems. Somebody makes an offhand joke or a sarcastic comment, and everybody on the call understands that to be sarcasm, but the AI doesn't pick up on it.

[19:06] The AIs are getting better at those subtle cues, but they're not perfect.

[19:11] So if somebody's just like, oh, yeah, we'll definitely do such and such, and then everyone laughs and they move on, but the AI records it as if it was said, seriously,

[19:21] that's a problem, too.

[19:23] So all of those things. Have you considered any of that?

[19:27] So if you're using AI transcripts in lieu of official meeting minutes,

[19:33] I would think through that, again,

[19:35] I agree with you.

[19:36] Pamela Isom: And that's. Those are interesting points.

[19:39] Here's something I want to talk about. So let's talk about companies that endorse the use of AI and those that don't.

[19:48] So we were talking earlier about how they'll endorse the use of AI,

[19:54] but they actually will say don't use AI,

[19:57] and they won't endorse the usage of AI,

[20:00] yet we're finding that they're using it anyway. So I want to use that example first before we get into those that actually endorse it and the things that they should be mindful of.

[20:11] Chad Ratashak: Yeah, absolutely.

[20:12] Pamela Isom: Yeah. Tell me more about that.

[20:14] Chad Ratashak: Yeah, so we, we talked about how, you know, and that kind of goes to the heart of what your podcast is named, like AI or not. Right.

[20:23] A lot of organizations have.

[20:25] Basically every organization is faced with this decision,

[20:28] do we use AI or not?

[20:31] And my message to any of your listeners is that the default decision if you do nothing is not we don't use AI. The default decision is we use AI without any policy or training.

[20:44] That's your default.

[20:46] You probably think it's defaulting to we don't use AI,

[20:50] but then you're being naive. And the reason is that AI, generative AI is being used. And I'm not using some. Funny. Technically, this is AI. Tricky definition.

[21:00] I'm talking about what you think of as AI. You're just not considering all the ways that AI has been added to existing software.

[21:09] So do you use Google Search? The AI overview can hallucinate.

[21:13] Do you use Microsoft Office? Almost certainly.

[21:17] Then the Microsoft copilot can introduce hallucinations. And even if it doesn't hallucinate outright falsehoods, it can just skew and kind of change your intent and your meaning of what you meant to say in written documents.

[21:31] Even if it doesn't outright change to falsehoods, it could still not be quite right what you meant to say. It can soften things you meant to say that were stronger or make things more intense than you intended to say that you meant to be more hedged or just omit things change,

[21:50] quotes,

[21:51] all that kind of stuff. Right? So that exists. We already talked about Zoom, right? Are you attending video calls? Almost certainly. As a small business owner, you're attending all kinds of video conferencing.

[22:02] So you're probably exposed to all that AI transcription,

[22:05] even if you're not using it, probably the other people on the other side of that are using it. So you're exposed to that AI transcription risk.

[22:13] You know, then you've got Adobe,

[22:15] you've got Salesforce. There's just all these different run of the mill,

[22:20] bread and butter software packages that have added generative AI to their offerings.

[22:26] They're already in the door.

[22:28] They came in through existing relationships, existing software,

[22:32] and they have the same risks as any generative AI, like hallucination risks. If they have AI agents, then certain, you know, destructive deletion operations or information leaks that AI agents can make, things like that.

[22:49] And so you need to think about what actually do you use what software? Instead of saying we use AI or we don't use AI, what software do we use? And does that software have generative AI features?

[23:04] And if we haven't consciously blocked those, disabled them, turned them off, told our employees they can't use them, gone through training on why they can't use them and why they're risky, then you don't actually not allow AI,

[23:16] you just kind of ignore it, which is not a good policy.

[23:21] It's actually not a policy at all.

[23:23] Pamela Isom: And you said it's like allowed with no policy.

[23:25] Okay, so basically what you are recommending or what you are experiencing is that it makes me think of shadow AI. And I was involved in a conversation and done some work and stuff with clients.

[23:39] And one of the discussions I had was, what are the contributors to shadow AI?

[23:45] And it could be that they don't feel like that the company endorses the use of AI, yet they need some information and they need it quickly.

[23:54] I'll just run this quick little scenario by the AI tool or I'll just use my personal device and get the information and transfer it in or something like that. But in any case, it is attributing to shadow AI where the information is still getting leaked and the tools are still getting utilized.

[24:19] There just aren't policies to support the guardrails and there aren't guidelines for you. So I'm in favor of doing what we can to support the use of it with education and training so that one understands how inferences from different data streams can be aggregated to cause a certain impact that we may want or that we may not want.

[24:49] Right. And so if you help people, so this goes to the workforce. And I think I heard this in what you were saying, but you can correct me here, but I think I heard of what you were saying is there's the people side of it and there is the workforce and the respect of the people that will encourage them to abide by the policies and even help shape the policies.

[25:12] If you support them and allow the use and if you are going to withhold the use in certain areas, explain why.

[25:23] But maybe complement that with where it can be used. I've said a lot, right. But that's kind of what I was reading and processing as you were talking.

[25:32] Chad Ratashak: Yeah. And training I think is so key. And that's why I focus on that in my business model is training the workforce. Because you can get a lot more out of $20 a seat for a trained workforce who understands what AI hallucinations are and knows what AI is useful for and what it's not useful for and what the risks are and how to avoid them.

[25:57] Then you can by spending thousands of dollars per employee per month with these crazy token maxing competitions. But they don't have any idea what they're doing. And somebody ends up deleting an entire in production software tool.

[26:14] Those are real destructive actions that actually happen.

[26:17] And if, if something like that happens to your company,

[26:21] it's not just a question of oh well, it was slightly negative roi. It might be existential for your company. Like there were. There was a recent example of a business that provided payment rails for car rental companies and they accidentally destroyed their entire software through AI coding agent that had too much agency and pushed the changes to production and deleted the entire production software on a weekend.

[26:50] So they couldn't. They. All their customers were without their records. All the car rental companies that use their service.

[26:58] I don't know what's going to happen to that company. But I have to think there's some company customers that switched or were looking to find another source after another payment alternative after that, and that could be the end.

[27:10] I'm not trying to make a prediction about that specific company, but just a mistake like that could be existential for a business. Right? So we're not just talking about, oh, it's 5% ROI on your AI investment or some little spreadsheet math type thing where it's just a minor tick up or down for the quarter.

[27:29] We're talking about things that could be devastating for your company's reputation and possibly destroy your business completely because you just ruin your reputation with your entire customer base if you, if you destroy an entire software product or something like that.

[27:48] Okay, now we have an AI use mandate. You're judged based on how much you use it. You have to use it and go,

[27:55] that's not the right approach. It's not going to be positive. It's not going to work. Well,

[28:00] on the other hand, like you can get a lot of value out of even cheap tools if you have people who understand and know how to do that trade off like you talked about, Pamela, like,

[28:10] do it the old fashioned way or do it the AI way, depending on if it makes sense.

[28:16] Fundamentally, you want your employees to make that cost benefit calculation because otherwise the other, the simpler, less. I talked about destructive AI coding agents. Sometimes that's not your failure mode.

[28:26] Sometimes your failure mode is everybody is spamming each other with worthless emails that they didn't write or read.

[28:32] Right. Which is still a failure mode. Right. Well, why didn't our workforce get more productive when we turned on all these AI licenses?

[28:39] Because all they did was send 20 times as many emails. And then they summarize those emails with AI and then they wrote responses to those emails with AI and nobody actually knew what was going on because nobody's actually reading email anymore.

[28:51] We're just spamming each other. Like,

[28:53] was that our goal? Did we pay all this money just to have people spam each other?

[28:58] We have spam filters for external spam, but then we just boosted our internal spam.

[29:03] That wasn't our goal. So if you do an AI transformation,

[29:07] what are you actually trying to accomplish? You need to teach your workforce what they can actually do with AI and not send emails. Don't send,

[29:14] write, write and summarize emails is what the AI will tell you it can do. That is a low hanging fruit and it's not very useful and it's not useful at scale.

[29:23] When Your entire workforce inside your organization is trying to do it.

[29:26] Pamela Isom: I agree. And I know that agentic AI, generative AI, agentic AI, AI, period. It, the, you know, everyone talks about how it increases speed is going to increase the,

[29:38] and expand your analytical reach and improve productivity to some extent, unless you're spending. And this is counterbalanced by the fact that you have to go, go along and double check everything.

[29:50] Right. So that's the balance that we have to weigh.

[29:53] And the risk tolerance, as we talked about,

[29:56] we have to think about how it accelerates the ways that information is combined.

[30:05] It's the information that gets combined that can lead to greater risks, greater successes, but also heightened risks. And you, I think that's one of the things that is coming through as I'm listening to you talk as well,

[30:21] because of the concern of the fact that it can combine information, which is going to require heightened governance to operate at that same pace. Right. Understanding where that information is going,

[30:37] who is the author of that information?

[30:40] Is it? And even thinking about inventory. So we used to talk about, and this is a question for you, but when I think about inventory and traditional inventory and how we always had to track our assets,

[30:54] how does that play when it comes to AI now? Right. So now,

[31:00] now we've got.

[31:01] It's not, it's laptops, it's servers, it's SaaS, your SaaS applications, it's your database in the past,

[31:07] but now you've got agents in the midst and some of these things, a code copilots that create code,

[31:15] all that. How do we deal with that?

[31:18] Chad Ratashak: Those are great questions. One thing that I start with is with companies where we're helping them use coding agents. And there are some companies where, where that makes sense, where it's worth the risk.

[31:32] It is risky though. I always start with explaining,

[31:37] and this is one of my explanations, is that it will delete something. It's not supposed to start from that assumption.

[31:45] That's my risk training.

[31:47] Okay, so then why would I even teach you to use this? If it's going to do that?

[31:51] You need to build the access that you give it with the assumption that at some point,

[31:59] not in a hundred years,

[32:01] not in 10 years, sometime in the next six months, assume that it's going to delete something it wasn't supposed to.

[32:07] Is the way that you gave it access to files and the way you're backing up files in places that the AI does not have access to,

[32:14] robust to the fact that it will delete something in the area where it has access to and it's little play area.

[32:22] If you grant it full disk access to your computer, guess what? You might as well consider that whole thing fried.

[32:28] And in some ways, this is bringing over some lessons from my ransomware, dealing with ransomware and other complex cyber crimes. The idea that you need to have multiple copies in multiple places,

[32:39] at least one of them offline.

[32:41] In this case, you want multiple copies of files and at least one of them outside of the agent's control.

[32:49] And so if you do, if you operate under that assumption and say it will delete your files, it will not. It might or it could, and I don't hedge, it's going to delete your files.

[33:02] It is going to happen.

[33:03] Treat it like it has already happened and then instead of having that oh, no moment, say we're okay because it's backed up.

[33:12] If you say that from the beginning, it is deleted, treated it as if it's accidentally deleted everything in this folder and it's going to, and I repeat that over and over, it's going to delete something.

[33:23] It is going to, it's going to happen.

[33:25] Don't treat it as a maybe or an if. If it doesn't, that's a happy coincidence or a happy accident.

[33:31] But it's not what you bank on. You build it with the assumption it's going to delete something, then you're going to be okay.

[33:39] Because the problem is, and this is an area I've talked, I've talked a lot when I give like keynote speeches and stuff, I'll talk about how I don't really care about what the error rate is as much as the damage that the AI can do when it causes an error.

[33:54] And so people will focus on, oh well, the AI hallucination rate is going down on these different benchmarks. It was, you know, 92 point whatever, now it's 97 point whatever.

[34:05] If it's going to be destructive to your business,

[34:08] if it's going to be existentially destructive or cost you tens of thousands of dollars,

[34:13] just treat it as 100%. Because the thing is,

[34:17] if it's 97% accurate and you're going to have an AI agent running thousands of times,

[34:24] you might as well treat it as a certainty. It doesn't really matter if it's 92 or 97 or 99,

[34:29] if it's going to be running over and over and over and over and over in this map. Many, many actions in parallel across multiple sessions. It only takes one of them to go awry.

[34:41] Now, I know I'm speaking kind of broadly and hand waving. Some of the numbers.

[34:46] Don't get too in the weeds about specific numbers. My point is that AI agents scale up their actions so fast and so large that we should just treat it as a certainty that they will fail and not operationalize.

[35:01] Like,

[35:02] 97 sounds so much better than 92. We shouldn't care about that in terms of how we design our defenses. Just treat it as a certainty that it's going to fail.

[35:12] And then are you okay when it fails?

[35:16] Pamela Isom: Which means that gets to our resiliency planning and our. I like to do adversarial. Right, adversarial testing. And so this gets into adversarial testing, resiliency planning,

[35:28] business continuity plans, and just more realistic planning for the integration of the agents. That's what you were saying, right? To plan it from the onset.

[35:39] Chad Ratashak: Yes.

[35:40] Pamela Isom: And you're testing for resilience.

[35:43] Chad Ratashak: Yeah. And actually in some ways,

[35:46] I'll add a caveat at the end, but in some ways, AI agents can themselves be helpful in brainstorming how to tabletop some of those scenarios.

[35:55] But it's an important prompting technique to make it so that you're trying to tear yourself apart. Because the AI is always trying to tell you what you want to hear.

[36:03] So if it thinks you're like, tell me how great my plan is, it'll say, wow, that is the best resiliency plan you thought of everything. There are no flaws. You are a genius.

[36:15] You're good to go.

[36:17] And.

[36:18] And then you'll be in trouble. But if you say,

[36:21] look,

[36:22] I'm really mad at my. It could be a plan you put together, but you say, look, I'm mad at my vendor because they missed some obvious things. Tell me what's wrong with this resiliency plan that you put together.

[36:32] But that's what you tell the AI. And then the AI is like, you're right, they missed this. Look at these five things they should definitely have thought of that they didn't put in there.

[36:40] And it'll tell you a completely different story. We're not used to computers working like that. Right.

[36:46] A computer that wants to tell you what you want to hear is not how we're used to normal. A database, an Excel spreadsheet doesn't change.

[36:54] You know what a sum function does based on whether you would like not to be in the red that that quarter or not. Right. But ChatGPT or Claude or Gemini will tell you, oh, no, you're doing great.

[37:05] You have a great business plan. You're going to be super profitable next quarter. Or, nope, that's a horrible plan. You should not invest in that, depending on how you frame the exact same facts.

[37:16] Pamela Isom: That is so important.

[37:18] I love that example.

[37:20] I love that example because that's how you get it. To tell you, I want to say the truth, but that's not true.

[37:26] Chad Ratashak: Right? Yeah. It's still. It might be overly harsh against your plan. Right. So that's a good point. It's not necessarily true, but it's adversarial. So, yeah.

[37:36] Pamela Isom: Yeah,

[37:37] this has been really helpful.

[37:39] Yeah. So I am excited that we had the opportunity to talk today. There's one area that I don't know if you want to go into or not, but you have ransomware and crypto.

[37:53] Your ransomware and cryptocurrency leader,

[37:56] is there anything. You talked on the ransomware a little bit. Is there anything else that you can share with the listeners that are contemplating AI or not? And just this whole subject matter,

[38:09] knowing that it's entangled in everything that we do.

[38:13] What is that connection or what can you share with us from a ransomware and cryptocurrency perspective?

[38:19] Chad Ratashak: Yeah, that's a great question. Well, one of the things that I worked on was the Department of Homeland Security had this program called the Public Private Analytic Exchange Program. And so I worked on the Ransomware Attacks on Critical infrastructure program in 2022, and I was the private sector leader for that.

[38:40] And then I participated in the second phase of the cryptocurrency group that was another group that also was in 2022. And they asked me to join them in 2023 for their second phase.

[38:53] And we talked about different AI enabled crime or, sorry, cryptocurrency enabled crimes.

[38:59] A lot of the crimes that are enabled by cryptocurrency over overseas romance scams and things of that nature are becoming more sophisticated because of AI. So where a scammer would have these long,

[39:13] long running romance scams, they're typically called pig butchering. That's a term that comes from Chinese. That's basically you're being fattened up over the long term. You can think of it as a long con.

[39:25] So it might take months before they say anything about money. And it just seems like they're your buddy that's chatting with you on a chat, you know, something like WhatsApp or texting you,

[39:34] or chatting online, sometimes on an online game or something, and then they try to talk to you about money later. It could be.

[39:42] Pamela Isom: We call that grooming, but go ahead.

[39:45] Chad Ratashak: Exactly. And so the grooming phase could be very long before they actually bring money up at all. And typically the payment will be in cryptocurrency of some kind.

[39:55] They might use crypto ATMs, they might use different investment platforms. They might use stablecoins. But the point is,

[40:02] there would be hesitancy. Oh, well. Will you do a video call with me? Well, I can't because I'm.

[40:08] Maybe they're pretending to be a US Service member overseas who's deployed. Well, I'm not allowed to because I'm deployed and I can't do video calls from where I'm at, or I'm on an oil rig out in the North Sea and I have bad Internet, so I can't do video calls.

[40:23] I can only do voice calls or whatever the story would. That's becoming less common now. They have live video deepfake filters where anyone can look like anyone live on a streaming call.

[40:35] How do you know? I've been Chad the whole time. Right. I mean, it's scary, but that's the world we're living in, right? So those scams are being enabled by AI.

[40:44] Already, organized crime groups,

[40:46] primarily in Southeast Asia, are carrying out these crimes for billions of dollars,

[40:52] affecting US Citizens and people around the world.

[40:55] And they're just getting more sophisticated with using AI,

[40:58] whether that's voice or video deepfakes.

[41:01] Sometimes they'll use family emergency or fake kidnapping. So they'll voice deep fake a family member and say, oh, help, I'm being kidnapped. Don't let them hurt me. And use that to pressure people to give money, typically parents or grandparents.

[41:16] Obviously very disturbing and unsettling for the people who have those calls, Even if you know rationally, like you're hearing me say this right now, if you get that call and you think it's your daughter or your son and they're being harmed,

[41:28] and 30 seconds later you call them, they're like, I don't know what you're talking about. I'm fine. You still have your heart racing in that extreme, intense stress because for 30 seconds you thought something horrible could be happening to my kid.

[41:40] You know,

[41:40] even when you knew in the back of your mind the whole time, it's probably that thing. It's probably not real. It's probably fine. You know, it's deeply disturbing to people that it happens to, even if they never send money, even if they never are deceived by it.

[41:52] So this is a really serious type of problem. And ransomware, unfortunately, is because of the nature of computer programming, becoming easier for people who don't have any sophisticated programming skills.

[42:05] The dark side of that is the bad guys who want to steal money but don't know how to code now can get into the game of writing ransomware as well.

[42:14] That risk has been a little bit overblown. It hasn't been as bad, I think, as it was originally anticipated. There's also some security work being done with the Mythos model that Claude released,

[42:27] where they haven't released it publicly, but they've done it kind of in the background, and they've done a lot of bug hunting in the private sector before releasing it publicly to sort of lock things down before they make the model available.

[42:39] So hopefully we get to sort of a balance where the defenders have the advantage.

[42:44] But it does seem like right now AI has given some advantage to the attacker, so hopefully that shifts.

[42:50] Pamela Isom: Okay,

[42:51] well, I guess what we can do to protect ourselves is kind of pay attention to some of the things that you said here, things that we may think are not relevant at all.

[43:03] I mean, you talked a lot about leaking information and somehow information getting spread abroad that we didn't intend to. And we want to be mindful of the recipients of the information,

[43:15] so helping us to be more intentional about how we disseminate information and aware of where that information is going, which is a challenge in itself nowadays.

[43:27] So I usually will close with asking the listener to share words of wisdom or call to action or a combination thereof. It could be a myth that you've heard that you want to go ahead and set straight.

[43:42] But as a governance leader,

[43:45] as definitely a cybersecurity and AI risk leader and entrepreneur,

[43:51] what are your words of wisdoms for us or a call to action?

[43:57] Chad Ratashak: Well, I would say that every organization really needs to think about how they're using AI, not if they're using AI, because you probably are using AI. Right. Even if it's as simple as using it in Microsoft Office or Google Search or something like that, or zoom calls like we're having right now to find that.

[44:16] And actually, so just this week, I finished refining a. And so this, I guess, is a good time to mention it.

[44:24] Working on a little questionnaire that helps businesses look at their unexpected AI risks that I call the Generative AI Reality Check. So it's just a short little quiz that kind of goes over some of the stuff that we talked about.

[44:38] So that's at Midwest Frontier AI Assessment,

[44:42] and people can take that little quiz and kind of get some feedback and some suggested reading for more information on my website that'll direct them to different blog posts that cover some of those risks.

[44:55] And stuff. And then if they're interested in more, they can book a follow up call and see if they want to do governance consults and stuff. But if not, they can just have that list and kind of some of the risks and kind of a reading list of things that would help them understand those risks of those software tools that they are already using.

[45:14] Pamela Isom: Well, I really want to thank you for being here. I appreciate the work that we have coming up together where we are looking and working with organizations to help them through this AI adoption journey and really doing some work with those that are looking to do business with government.

[45:34] So that's exciting because that is definitely a a target, right? It's a vulnerable area, but also an exciting area for citizens across the nation. Right. So we want to do what we do and we want to do it well.

[45:47] So we're going to share some of our lessons learned, some of our best practices and get some insights out there for potential government contractors and for those that are subcontracting to government leaders.

[45:59] So that's an exciting thing. And so I'm looking forward to working with you more there.

[46:03] And so thank you so much for being here and I appreciate you taking just time out of your busy schedule to chat with me and I'm sure the listeners will enjoy what you have shared with us today.

[46:16] Chad Ratashak: Thanks so much for having me.

[46:17] Pamela Isom: Thank you.