AI or Not
Welcome to "AI or Not," the podcast where digital transformation meets real-world wisdom, hosted by Pamela Isom. With over 25 years of guiding the top echelons of corporate, public and private sectors through the ever-evolving digital landscape, Pamela, CEO and Founder of IsAdvice & Consulting LLC, is your expert navigator in the exploration of artificial intelligence, innovation, cyber, data, and ethical decision-making. This show demystifies the complexities of AI, digital disruption, and emerging technologies, focusing on their impact on business strategies, governance, product innovations, and societal well-being. Whether you're a professional seeking to leverage AI for sustainable growth, a leader aiming to navigate the digital terrain ethically, or an innovator looking to make a meaningful impact, "AI or Not" offers a unique blend of insights, experiences, and discussions that illuminate the path forward in the digital age. Join us as we delve into the world where technology meets humanity, with Pamela Isom leading the conversation.
AI or Not
E018 – AI or Not – Renee Wynn and Pamela Isom
Welcome to "AI or Not," the podcast where we explore the intersection of digital transformation and real-world wisdom, hosted by the accomplished Pamela Isom. With over 25 years of experience guiding leaders in corporate, public, and private sectors, Pamela, the CEO and Founder of IsAdvice & Consulting LLC, is a veteran in successfully navigating the complex realms of artificial intelligence, innovation, cyber issues, governance, data management, and ethical decision-making.
Prepare to gain insights from Renee Wynn, a remarkable leader with a wealth of experience as a former NASA CIO and EPA leader. She joins us to share her unique journey from mission delivery to IT innovation within some of the most prominent government agencies. Discover how IT underpins NASA's global operations and pivotal cybersecurity collaborations with international partners. Renee's transition to the EPA reveals the thrilling complexities and challenges of IT leadership in government, shedding light on how innovation and collaboration drive mission success.
Explore the ethical and practical hurdles we face today with the democratization of AI tools. The rapid emergence of these technologies demands careful alignment with organizational missions and ethics, highlighting the importance of transparency and governance. Renee and I dive into real-world issues, like biases in facial recognition, underscoring the necessity for AI systems to resonate with a company's cultural values. Through rigorous testing and ethics boards, we aim to empower consumers and leaders to harness AI's potential, ensuring benefits while minimizing risks.
The final leg of our conversation turns to the transformative power of AI in healthcare. Generative AI is proving invaluable in remote medicine, providing diagnostic support to rural doctors and drawing parallels to space technology developed for astronaut health monitoring. We discuss the critical management of technical debt and the role of governance in steering digital transformation. As we conclude, we encourage listeners to embrace technological change, remain human-focused, and prioritize our planet even as we explore the boundless possibilities of space and AI.
This podcast is for informational purposes only. Personal views and opinions expressed by our podcast guests are their own and not legal advice, neither health tax, nor professional nor official statements by their organizations. Guest views may not be those of the host host.
Pamela Isom:Hello and welcome to AI or Not, the podcast where business leaders from around the globe share wisdom and insights that are needed now to address issues and guide success in your artificial intelligence and digital transformation journey. I am Pamela Isom and I'm your podcast host, and we have a special guest with us today Renee Wynn. Renee is an independent board director. She's a former NASA CIO. She also was at the EPA. We met during her tenure, during your tenure at NASA, and I'm so happy that we have maintained our relationship. I really am. You're a strong leader. You're a good role model. I appreciate you. You're a digital transformation leader, and I know you because of your excellence in cybersecurity. So there's more that I could say, but I'll let you tell your story. So, Renee, welcome to AI or Not.
Renee Wynn:Pamela, it's great to see you and it's great to be here with you today and thanks so much for asking and I'm so glad we have been able to stay in touch through all of these years and for the listeners to know you and I actually never worked together, we just ran in the same circles. I think a hidden secret in the IT world in the United States government is a lot of folks are very well connected with each other across all the agencies, even if you've never worked with them. And I love that about being part of the IT community in the US government because you were always a phone call away from help and you, pamela, put on the map working at Department of Energy, sort of an AI organization, which was wonderful that DOE did that and put you at the helm to really capitalize on your amazing career. And what a great way for you to end your career in the federal government as you move to this next phase. So thank you very much.
Renee Wynn:So for the listeners, a little bit about me. I always like to tell people what's the job of the NASA CIO, and it's quite simply this you deliver IT services across the globe and off the globe. The CIO's organization does provide some services to International Space Station, and the other part is is I led the effort to overhaul NASA's cybersecurity posture across the globe and off the globe, and NASA still operates in Russia today, which is always a bit of a surprise to everybody, but there was a few months ago both a launch and a return of an American astronaut on the Soyuz capsule out of Russia. So in many ways, I think space is a great human connector and if you're always talking, then you have a chance to resolve conflict, which is my great hope with the political and geopolitical situations that are going on around the globe today, the ones that are obviously in the news every day. But there's some that are going on that don't make the news every day, that are just as hard on the people there. So that's what I did at NASA, and I also have people forget and I actually forget too, because it was a while ago and that was I had about 25 years of my career, was at the Environmental Protection Agency and I came from mission.
Renee Wynn:And a warning to all your listeners if you complain a little too much about your IT services, you could end up in an IT organization. So, having offered a few observations. That is when I made the switch from mission delivery, program delivery at EPA to being the deputy CIO at EPA Because, like, okay, well, she certainly seems to understand it and maybe she'll bring her mission heart to delivering those services. And what's really funny is that one.
Renee Wynn:I learned how hard it was because so complicated and it's so complicated and it's largely driven by people, as you know. But the other part about moving and shifting there is the pace of the work and, boy, once I got comfortable with the pace, I loved the chaos and the pace of operations. It wears you out, as you know, but it can be really fun because you can make a big difference in mission and delivery and you get to solve some problems, sometimes at 2 and 4 am, and that's okay because you and your people realize that you're critical to mission delivery and it's always good to be reminded of that. So that's a little bit about me and how I came into the IT world and a warning to folks if they complain too much, they may end up with a job something enough, do something about it.
Pamela Isom:So that's that's yeah, and so and it's also funny, I find it I find it a little comical that you said the the operations, that you love the pace and chaos of operations. Okay, so I liked it earlier in my career, I loved it and that's what I did. Right, that's starting out in my career. And then I moved into the innovation realm and so I don't know if I really liked the operations, but I know that I always felt this sense of accomplishment because you're carrying the brunt of the organization. It's the operations crew that's keeping the organization running. So I appreciate the value of being in operations, but I don't know if I always like the pace and the chaos.
Renee Wynn:But you switching into innovation from operations. You have this great understanding and compassion and empathy for operations and how, frankly, to deliver innovation in a way where folks are receptive and it actually fits in the organization. I've had a number of conversations not everybody on the innovation side that was, oh well, just go do this and you think You're like what Exactly? You know you have a network that's across the globe and a hiccup could be very problematic, so that's why you have development environments to go play and production and environments are far more useful, but you got to figure some things out in these complex environments. So, yay, I know that you took a common sense approach to innovation and I think it's. The reason you did is just having that experience in the chaos of operation. So that's great.
Pamela Isom:Well, thank you, and that's probably good for people that are listening.
Pamela Isom:So, as you're looking at, when we're trying to figure out, like, how are we going about dealing with things that are going on today and trying to get back to some of the fundamentals, or even those that are just getting going, we have to remember that if you're going to be a strong innovator, you may want to look at transitioning to different roles within the department so that you get some experience right, and vice versa. So, when they switch roles, that's really helpful. I think you're right about that, though I believe that that's probably how and why I have the temperament that I had as an innovator. I believe that's a good assertion. But I want to thank you for your work and I want to thank you for the things that you have done. I sincerely, more than you realize, appreciate it. So I'd like to take time now and talk about governance, ai governance particularly. I'm curious as to your perspectives on, you know, are there things that we could be doing to make our lives and business leaders' lives easier, like, are there things?
Renee Wynn:we should be thinking about. I think that's absolutely right. So AI governance first. When you say the word governance, most people their eyes just glaze right over. Well, I'm going to use a NASA example and then I'm going to get into the AI governance because I'd like to set the stage of why governance matters.
Renee Wynn:So an element of governance at NASA was the Agency Program Management Council, apmc Make sure I got the letters right there. And James Webb Space Telescope came through the APMC and every line of business or line of mission within NASA, including mission support, sat at the table so I'm speaking for myself sat at the table to review that program, to ensure that program would be successful. And in many ways you have to also say that success is the reputation. When you're paid by taxpayer money, that reputational stakes and that success stakes, I think, go higher. And, nasa being rather iconic agency, people are always looking at what they do and mistakes are important to make and you have to create an environment that allows for mistakes. But you also have to learn from those mistakes and integrate those into what you do so that on the day of lunch we're about as mistake-free as you can get.
Pamela Isom:Yeah, right, like in your case, lunch means lunch.
Renee Wynn:Yeah, it's like, yeah, we're leaving the atmosphere and let's go, yeah, and so I bring that to light, because people have a tendency to say governance is bureaucracy. Yeah, and there are cases and in fact a lot of the cases where governance has become too much bureaucracy. And so it's important to assess on a regular basis your governance processes and can you move them with the right friction, with the right friction? Friction is good. You want to bring the right friction to the process so you don't go too soon or you don't go too late and you want to get the right voices at the table.
Renee Wynn:So, with AI governance, part of that process is to do the due diligence on AI. How are you going to use it? You need to have a purpose. It's not just a cool, shiny thing because AI could wreck your business. It's touching your data. It's bringing data potentially from outside your organization and bringing that into your organization. So there's a big trust piece with AI and can you believe what it's producing? At the other end? And I always like to say well, let's go to the basics first with AI, and what processes automated processes already do you have that you could bring AI to? So you already know the process. You know I was thinking workflow charts. You know, back there, when you had the little drawing tools to do that which I loved, and then- I remember those Were yours, the green ones, yep.
Renee Wynn:Yep, you know, make the boxes in that Well trained me to think that way.
Renee Wynn:And then I loved when the big Apple Macs came out and you could draw them instead of with the tools, instead of having to write them and then erase and then write and erase. Then take a look at what you've got and say, well, how can AI help us with our already digitized processes? It's a great place to start and what you can do through governance processes you ask across your enterprise, and so for me, I always think of the NASA enterprise. So, in science, what have you got already processed and how can this AI help you Mission support, hr, it, my organization procurement and then bring forward those that you serve to be part of the due diligence, the governance process, and then begin deploying it where you are best able. And there's a whole piece to infrastructure with AI, and so that's where you're going to turn for most CIOs within their own organization and make sure that your organization is ready to start on a few AI projects as you figure out best how to capture AI for productivity and continue to keep your organization thriving.
Pamela Isom:Mm-hmm, I agree with that. I have this excitement, but I have this concern that People and business business more so than individual consumers. I have two concerns. One I feel like consumers are forced to speak up. Speak out because things are happening, and forced upon us that if we don't, we're just going to get run over, and what I'm getting to is AI tools and privacy and protection of our data. I'm getting to all of that because of the way that these tools are being rolled out.
Pamela Isom:So we got tools that are number one. There's just like this influx of tools. You don't know which one to choose from. You're like, okay, I want to use AI. So this is.
Pamela Isom:This forces you to almost go back and think what is your, what is your mission intent, because there are so many tools. So that's number one is I'm concerned as a consumer that I'm not sure I'm being considered, but my other concern along those lines is the tools aren't really tested. So not only is there an influx of tools I don't think that they're I'm not making a blanket statement that they're not being tested but I feel like they're leaning on us to do the testing and quality assurance without informing us that, hey, we are releasing a version that is a test version only. So please don't try to make business decisions with this tool because it isn't there yet. We don't know that until we start seeing hallucinations and wild outcomes and then we get told.
Pamela Isom:So I have those two concerns and I want consumers to be empowered. So my take on it is back to two things again. One, the ethics board, so the governance boards. Going back to the discussion around ethics, are we too far removed? I mean we should be out there with the consumers. We should be out there with those stakeholders in the organization, understanding what they're going through, seeing, feeling, knowing what they're going through so that they are empowered and so that the leaders are proactive. I don't know if that's going on today. I want to push that more.
Renee Wynn:I think that's right and part of that governance process, as you pointed out, is this ethics and business values versus business value, and I'll call it mission value. Both you and I are government. We called it mission, business value and I'll call it mission value. Both you and I are government, we called it mission. So let's separate the mission purpose and begin with the ethics and the business values.
Renee Wynn:One again part of the governance process, and this is the due diligence part of it. So you've got to have your ethics folks and your cybersecurity, supply chain risk management folks, your supply chain, you know, maybe other supply chain expertise besides cybersecurity sit at the table as well as, I think, some other capabilities at the table to really evaluate what were the ethics, what are the ethics of our organization in bringing this in? United States government? The privacy piece is defined for us by law, which is great. It's defined for many businesses. You have the healthcare professions because they've got the HIPAA privacy laws, so you can go to, as part of the ethics, the laws that govern how you need to operate. So any publicly traded company, they have a list of compliance laws that they have to do. Well, those ought to get compared, you know, be part of this ethics review, business values review, that you do so once you've identified a tool, an AI tool that you think works with your ethics, the matching of it.
Renee Wynn:Where was it trained? Right, right An example the CIOs were all together and we got a briefing on the ethics of AI, and so this would have been eight, nine years ago, and the woman who spoke did her research and got her PhD out of Harvard and looked at the results of the ethics associated with facial recognition. So a complex thing to do, but it's only one aspect of what AI can do to you. And you and I, pam, as females most of the time would not have been recognized by these early facial recognition models as humans, because the models had been trained on the internet and most of the images out there that were training them and this is before Taylor Swift's big explosion and Beyonce's big you know they are GDP. Those two create blips, positive blips, in gross domestic production in some countries when they go visit. So when I say they are a phenomenon, they are a phenomenon.
Pamela Isom:This is pre-them.
Renee Wynn:So most of the images were male and predominantly white male. So when those models started to get applied to female faces different skin tones and colors it could not recognize it as human. So can you imagine if you hadn't done the research right and brought that in to rely on it or even test it? You would have been, I hope, curious about your outcomes and then go back and figure it out. So, in this day and age, do your research first so that when you start in on your due diligence on a particular AI provider I'm not going to mention any names that'll narrow down what you're going to have to focus on. So that's the ethics side, and then the business values.
Renee Wynn:How do you speak in your organization?
Renee Wynn:Right, and you want your tool, frankly, to mirror, or begin to mirror, the language of and the method by which you communicate in your organization, because people are going to start to think of this thing as a sentient being and therefore you've got to raise and align the LLM that you're using on your AI to match your business values, so it begins to speak the way you expect it to speak, because you and I know when somebody is in a influential position in an organization and they're in a cranky mood.
Renee Wynn:Most of the people around them feel that cranky mood Right? So your AI is in a cranky mood that day or always displays abrupt language, right? That's going to start to permeate and affect the culture of your organization. So these are hard issues that we're bringing up, but they are so key to the output and productivity that you can gain. And again, this goes back to governance and frankly, I put it free, it's governance, but it's also having the right team really doing the research on the model, the large language model or the product provider that you want to bring in to your organization. And these aren't easy, but they are fun, right? I mean, isn't doing hard fun?
Pamela Isom:hard fun For some people like me or used to be, I don't know, I need things easy nowadays. But yeah, no, no, I agree with you. I like what you said and I think it is governance and it is looking at that whole supply chain. So governance is going to see to it that we're looking across the supply chain, that we're evaluating what our supply chain is for the AI models also, the integrity of our supply chain. It seems like that there's an opportunity for this conversation really is turning into a governance conversation. It seems like there's an opportunity to strengthen the governance playbook and that we're starting to highlight some of those things that we should consider deeper today than one might be used to in traditional digital transformation projects.
Pamela Isom:Now, when you're saying what you're saying, I thought about democratizing AI. I know that it's important that in order for I said earlier that we want the ethics teams and the governance teams to understand what's happening in the environment. So, in order to do so maybe not necessarily the board of directors, but maybe someone's got to get out there and be with the ones on the ground right, someone's got to understand and make sure that all the communications is like it should be and that we're consistent in our communications. Then you also mentioned a common vernacular, so that business values are truly understood. So democratizing AI, then, is another area that I think we need to pay attention to, because we want the AI tools to be accessible, and we want them to be accessible to all, not just a group of people that's sitting over here, and so that's a tough issue to address, but that's going to help build that culture that we're talking about. So, when you mentioned culture earlier and working that culture and dealing with the culture dynamics, I think that one thing we have to think about is what does democratizing AI really mean, and what does it mean to make it accessible to all, because it's not good enough to just say, ok, ok, the tool is available. That's not good enough, right? So we got to make sure that is accessible to all. That is understandable. Maybe they look at what are the proper tools? How do we assess the tools? How do we go about guiding people on how to use the tools? So that's something that I think effective governance needs to start to pay attention to and also look at the different generations. I'm not sure we're looking at the generation.
Pamela Isom:So I had a situation a couple of weeks ago and it's a client and they wanted to use the tool that they're using and they said AI keeps popping up. And I was like how do you know that it's AI? Well, it just keeps popping up on my phone. And I was like, okay, she's like can you help me with this? Because I don't have time to deal with it right now, because I'm just trying to send a message to somebody, and clearly it was someone that wasn't familiar with how to use some of the other capabilities, but she knew it was an AI chatbot that's what it was, no doubt. And so she was like I don't want to use it. I don't want to use it. I think there needs to be more attention paid to all generations. What do you think about that?
Renee Wynn:Well, I think that's absolutely right and I'm going to sort of distill this down with any tool and, in particular, ai, so let's talk organizationally first and then let's talk personally. You have to do the training and having delivered that training and I'm confident, pamela, you've delivered that training People are like I don't want to take the training, just let me dive in. And for those folks that took the time to do the training and we would provide entry level maybe we'll call it the college level, the 101, up to the graduate degrees of training for various types of tool, and we would do it in person and you can do the on-demand videos. Training is so key with AI To your point. One is so people recognize if you've allowed this to be in your everyday usage, well, let's say, in the email system, so that people and what's going to happen out of it, and so this is where training becomes really critical and that way people know how best to use it, they understand that they are training it and you can coordinate off in your organization. So you're just training the model that you're using in your organization, not the global one. We'll call it that. So that's one side of the is education and training. Well, it's much easier to speak about and attempt to deliver in an organization than it is in our personal lives. We know across the globe there is a difference of access to the internet, and that difference in the internet access is a difference of understanding what's going on with AI. I'm just going to stay focused on that.
Renee Wynn:And so if you come into contact with AI for the first time and you've not been able to do some reading on it or watch videos on it or anything you know on the personal side, it could be frightening for some and it'll be energizing for others. And out in our regular everyday lives, this is harder. This is harder to get the training and if you rely on the news, you're going to see that it's only you know, depending on your news source. There's the one that's writing fear, fear, fear. And then there's always a set of people that are in the media that are writing do the research, but implore them to spread the word and share with others to say please do your research or you be an advocate for that.
Renee Wynn:You know, offer educational tips and stuff like that, because I don't know how we're going to reach every person and let's just make it in the United States, and I'll just narrow it down to the state of Virginia, I'm not going to be able to do it and that spectrum of understanding of AI is in the state of Virginia, just like it is in every state, and so there is this responsibility to get people to pay attention and get educated, and that's part of the democratizing the AI is understanding what it is, what it can do for you and what the dangers are associated with it. So and it's hard, this, I mean this is easy for you and I to talk to, because many times with IT, the methods for doing something people process and tools yeah, okay, that's it. But getting to the people, getting to an education level, getting that people movement side is so hard to do that. And AI is just bright, neon light, screaming or saying to us pay attention and get ready to learn.
Pamela Isom:Exactly. I believe that one of my other guests or he said to me and where they're going right, so help us reach and connect, and then I have a follow-up. You started to talk Mars and AI, so tell me more about this Mars and AI.
Renee Wynn:Yeah, so AI has been around for decades. So AI has been around for decades. What we're seeing right now is AI at anyone's fingertips with a computer or a mobile phone, because there's more people that use mobile phones every day than they do computers. I'm a both person, and so NASA put AI on Mars in 2003. So the rovers that are up there as much as we might want to imagine there's somebody down here, just like they do with AI and this learning and relearning on being on Mars, learning about the terrain and how to adjust the terrain, just in simple movement. Recently, in 2020, nasa put the first ever helicopter on Mars. Ingenuity was the name and it went on the Perseverance launch in 2020. Perseverance was the rover.
Pamela Isom:You know I had a little bit to do with that, right, you did tell me about that, because of Spectrum, because of Spectrum, yep, I had no idea, yeah.
Renee Wynn:So yes, spectrum management is a huge issue in the IT world. When you're flying, whether you're flying in an airplane or in a spaceship, whether you're communicating, our whole communications, radio systems and all of that all have this spectrum and that's just a whole other topic in and of itself. So AI are on these vehicles to help them learn, help them advance science and help them operate autonomously. And it's a great thing because it's a long communication timeframe from Earth to Mars and then a lot of the it depends upon solar access and stuff, and so Mars does have dark periods. So communication is a challenge, if at all, if there isn't any communication, and so AI is really helpful to advance missions in space.
Renee Wynn:In particular, we've been talking about Mars, and since 2003, it's been there. It just happens to be exploding across Earth. I like to remind people it's Earth 1.0, and there is no 2.0 yet designated. So I would also advocate that we should probably be treating current Earth 1.0, mother Earth, a little bit better, because we still don't have an option for all of us billions of people to go somewhere else just yet.
Pamela Isom:Not yet. Huh, no, not yet. What's your take on generative AI? Since you're mentioning Mars and AI, what's generative AI? Is it being updated or what's happening.
Renee Wynn:Absolutely. You know, generative AI has the name generative, right. It's like you and I, sometimes right, if I touch something hot and it's red, then the next time I see something red, I'm not going to go reach right into it and do that, and so it's very much a requirement, I think, if you're operating in harsh environments and environments that are incredibly remote and Mars is a harsh environment and it is incredibly remote. It doesn't seem like it because you can look at images from it every day, but it's a long way away from this planet. But one of the things I really like about generative AI and the application of AI is in the health field. So one remote medicine and there's remote medicine going on ever since we had International Space Station, because the astronauts' health is monitored on a regular basis. So you've got remote health care on Space Station.
Renee Wynn:We're seeing now across the globe, and especially after COVID, a lot more tools at the ready for doctors and nurses to work with us you and I, frankly, from home, and so this is where I think AI is going to be incredibly helpful, as ethical adoption of it happens in the medical field.
Renee Wynn:So if you're a rural doctor and you're accustomed to a spectrum of illnesses because of your population and somebody walks in with an illness and you've done all the testing and you've concluded it's none of the things that you thought about. Well, this is where the AI tools in healthcare can really begin to help you, because you could go to them with the symptoms. I wouldn't put the patient's name in there, but you might have to put information about gender and age and ethnicity, because those do have a factor in disease susceptibility. So for Black women, there is a type of breast cancer that is incredibly aggressive that you're not seeing in general in white women. So here in this diagnosis case, right, understanding ethnicity and race matters, because it could be pretty potent what you have, and so this is where the AI part can help with these diagnosis, when you're hitting something that's particularly unusual or you're just not accustomed to seeing it, and so I think it's got some real benefits.
Pamela Isom:I was hearing the relationship between space and remote monitoring of our astronauts and for monitoring their health, and then how that also parlayed into using AI to diagnose and for remote care on earth. Is that did I? Okay, so, absolutely? Is that right? Okay, so, absolutely? Did it start there? Did it start with the space? And then we started to model that remote health care and all that from what happened with our astronauts. Is that you?
Renee Wynn:know that's an interesting question, Pam. I've never looked into this. Space technology is part of it. Space technology is part of it. There's several elements of a mobile phone that is technology that was developed for space use that is now used across the planet in carrying a mobile phone, and there's all sorts of technology in our daily lives that become that. But I never actually looked at the correlation associated with astronaut remote medical monitoring and how that may have helped inform and lay the groundwork for across Earth remote medical providing. I haven't looked into that, so that'll be something for your listeners and it'll certainly be some homework for me.
Pamela Isom:Okay, what I'd like to do is maybe have a follow-up conversation with you, because I'd like to talk some more about AI readiness and we kind of did throughout this whole discussion, but there were a few more things that we probably could go into. I know you had mentioned the quantum resistant encryption, so what I'd like to do is give you time to share your words of wisdom and any other experiences with us before we have to depart here. And then I wanted to remind us that we want to pay attention to technical debt. So, in the traditional digital transformation system, lifecycle AI development work you want to think about that in the governance realm and in the AI world in particular, because you cannot let the technical debt mount up. You need to address that debt as soon as possible.
Pamela Isom:And then sometimes it's not called technical debt. You know it's called ML debt or AI debt, but let's deal with that technical debt, let's understand what it is and let's start to get our arms around. What's going to remain. That's the debt. What's going to remain when it comes to an issue that we're going to set aside and deal with later, or residual risk, right, what is that? And are we going back and making sure that we are addressing these matters and that we are monitoring things. So, from a governance perspective, we want to be sure that we pay attention to that and reflect back on the fact that that has always been a part of our system development and product development life cycle, and let's not let that go by the wayside, especially today. So now I'm going to give over to you.
Renee Wynn:So, pamela, let's for our next conversation. Let's talk about modernization, emerging tech which is quantum, and AI can continue and advancing the mission. Yeah, you know, those are all all intertwined in all of that. As we close out, I do want to thank you for the invitation to talk to you. It's always a treat to see you, my friend, and as we leave it and we're going to talk about emerging tech and we've talked today about AI and the governance, all in all, in advancing anything within an organization and within society, it comes down to people. Are the people ready to adapt and adopt, and are the people ready to listen and be part of it? And what are you doing about helping people get ready for change?
Pamela Isom:And I think that pretty much sums up this conversation. I'll add one more thing, and that is be brave, get out of yourself, just get out of yourself, don't overthink it and take the risk. I think I want us to do more of that. I want us to do exactly like what you said. What you said is remember the people, so remember the human side, and I think that that's so important because that's missing, so you can be brave and take care of people Absolutely. Yeah, yeah, all right. Well, it's great to talk to you.