E320 - The Human Factors Of Letting AI Cook

(A) E320 - The Human Factors Of Letting AI Cook

===

[00:00:00]

[00:00:00] Nick Roome: All right. Hey, everybody. Welcome back to episode 320 of Human Factors Cast. We're recording this episode live on May 15th, 2026. A rare Friday episode for y'all. Like I said, this is Human Factors Cast.

[00:00:13] Nick Roome: I'm your host, Nick Rome. I'm joined today by Mr. Barry Kirby.

[00:00:18] Barry Kirby: Hello there. And you can hear me and everything.

[00:00:20] Nick Roome: Yeah. Yeah. We can hear you. You can hear me now, apparently. And, yeah, we're doing great. We have a great show lined up for you, and you know what? Actually, we're, we're talking it.

[00:00:29] Nick Roome: We're doing it. We're talking about AI. You know why? 'Cause there's some reports out. We're gonna talk about it. It'll be great, and it won't feel too samey to some of the stuff that we've done in the past. But first, hey, we got some programming notes. We have a really exciting news.

[00:00:42] Nick Roome: We teased some exciting news last time. And so we've been approved for an HFES panel this year. It's gonna be a live recording of Human Factors Cast. You'll, if you come to that event and, attend, you'll get to see some of the, behind-the-scenes stuff, that we have done over the last 10 [00:01:00] years to make this podcast a reality, so super exciting.

[00:01:04] Nick Roome: There's other news that we're bound to release at some point, soon. But, again, if you wanna hear it early, come to Patreon. You'll get that news first. And, it... you'll get some other benefit too. Barry, since we last met, you went to EHF.

[00:01:21] Barry Kirby: I did, and it was really cool.

[00:01:24] Barry Kirby: Tell me about it. So three days, so the EHF conference is a three-day conference from, Monday through Wednesday. This year we also had the... Of the weekend before that, we had the IEA, executive and council, having their meeting, in the same place. So on the Sunday night, where we got there a bit early, we were able to have a dinner with the IEA executive as well.

[00:01:45] Barry Kirby: So it was really nice to see and meet some of them people who we've spoken to and engaged with but never actually met face to face. But we also had, throughout the week, we had barbecues, and we had the annual dinner and the awards ceremony. But for me, this w- this was the first year in a while where I haven't been [00:02:00] doing or stressing about doing sort of big talks, introductions, and that type of thing.

[00:02:04] Barry Kirby: It was nice to be, just, ju- just a person, going to listen and to engage with the talks that I wanted to engage with. And, the keynotes were fabulous. We had, the British bobsleigh team, Natalie, Dunman and, Amelia Coleman. Natalie was the, performance director. Amelia, the, an actual athlete, talk, giving us a talk and a behind-the-scenes view about how you get them You know, minuscule performance increases that mean gold, over years and how they've grown up.

[00:02:33] Barry Kirby: And they were really candid with some of the questions that, that they answered. We also s- listened to, Lucy Easthope, Dr. Lucy Easthope, who's a post-disaster management specialist. So when you have ... So she was involved in things like the Twin Towers and things like that. So when the actual event is over, how do they then manage that disaster people get the right ar- right artifacts, so the right things are preserved, and things like that.

[00:02:56] Barry Kirby: Some of the insights that she came up with were absolutely stunning a- and a fabulous person [00:03:00] to listen to. And then the irony of the ... I mentioned we had the annual dinner. The next morning, obviously everyone's feeling a bit, tender, shall we say. I went to bed at 2:00, 2:30 in the morning, and I wasn't the last person to go to bed.

[00:03:15] Barry Kirby: To have this keynote on work-related fatigue was just completely ironic. So then we have the keynotes, and then we had some really cool, lots and lots of good talks. But b- particularly stand out, hair dr- we had a talk on hairdressing and the human factors of hairdressing. Which was brilliant.

[00:03:30] Barry Kirby: They'd never been there before, and so they were testing the water, but showing how they'd apply to human factors. Lot ... Did digital human modeling, something that I'm really into at the moment, use of mannequins and things like that in, in, anthropometric design. And what really caught my attention, was a talk about us- an alternative design process proce- using what I could only describe as subversive ambush.

[00:03:52] Barry Kirby: So this idea that you have a red team and a blue team. The blue team is designing the product in the way you design it. The red team is designing the produ- product but [00:04:00] trying to get it to fail, but nobody realizing that it's them causing it to fail. And by the ... Then doing a compare and contrast between the two, you get a really robust product.

[00:04:10] Barry Kirby: Really fascinating, and completely blew my mind. So yeah, really good, really good three days. Lots to, learn, and really good to catch up with everybody. So

[00:04:20] Nick Roome: yeah, really cool. That's awesome. I'm ha- I'm glad you had a great time, and I'm glad you're back. And I'm glad you're here to talk about human factors through all the lenses that you've heard recently.

[00:04:31] Nick Roome: So what do you say? Should we get into the news? Guess we should. Let's do it.

[00:04:36] Nick Roome: That's right, this is the part of the show all about human factors news. Barry, what is our story this week?

[00:04:44] Barry Kirby: Our story this week is about AI getting weirdly better. Stanford's 2026 AI Index report shows rapid gains in AI capability, adoption, coding, performance, and agentic task completion, whilst also pointing out that the jagged [00:05:00] frontier problem, systems can crush advanced ben- advanced bench-benchmarks and still fail at surprisingly basic tasks.

[00:05:08] Barry Kirby: At the same time, another report from MIT Tech Review is looking at how agentic AI could help reshape software engineering, with AI agents moving from helpful coding assistance into broader roles across design, development, testing, and product lifecycle management. For us in the AI, for us in the HF domain, the big question centers around whether humans can still understand, supervise, verify, and recover from what these AI systems are doing.

[00:05:35] Barry Kirby: As AI becomes a teammate in the technical work, factors like trust c- calibration, cognitive workload, automation surprise, accountability, and the design of workflows is going to be a challenge to overcome. So Nick, are you willing to take on this challenge, or do you think it's insurmountable?

[00:05:52] Nick Roome: It's not insurmountable.

[00:05:55] Nick Roome: It will be important that we get this right. So let's talk about it, because I- [00:06:00] we, we talked a little bit in the pre-show about some of this stuff, but I think the interesting pieces to me is that across both of these articles, you're really starting to look at what AI is doing, not just on an individual, UI level, like how does an individual interact with AI, but what does AI look like at scale?

[00:06:21] Nick Roome: What does it look like as it's taking this systems approach? What does it look like when you have human-AI teaming, not just with, one individual, but having an AI be a member of a team, and having multiple AI members on a team that coordinate together? So the whole thing is becoming this process-based approach of how do you integrate these AI agents with your, with your processes versus how do you design an interface, specifically for one person?

[00:06:53] Nick Roome: And I think it's not that the interface problem is going away, it's I think that you're now having [00:07:00] both. There's going to be a larger focus on that. And especially, the other thing that strikes me here is that the human in the loop piece of this is going to be really important here. And this is part of the piece that I kinda talked a little bit about, before, before we went live.

[00:07:16] Nick Roome: I kinda showed off this, This agentic AI interface, it's called cto.new. It's not an advertisement for it or anything, but, I've been playing around with this recently and the thing that struck me about it was the, was the interface. And, you can go and watch the pre-show if you want to see the interface.

[00:07:35] Nick Roome: If you're listening, you can go back and watch it. But, the interface itself actually has visual representation of all the different agents that are building this thing. And so what it'll actually do is it will visualize the communication patterns between them, it will visualize sort of the decision-making process between them, and I think all this to say that the human in the loop, [00:08:00] and especially as it comes to permissions thing, super important because at least with that platform, we saw it in the pre-show, it was asking me about almost every single decision.

[00:08:09] Nick Roome: It wasn't very, autonomous on its own. It required me to s- approve even messages between the different agents. It required me to make decisions about hiring different agents. And, th- on sort of the flip side, I've been using Codex as well on PC, and it has different levels of security where last night I tried to do something and it was asking me every step of the way, "Hey, can I run something on System32?"

[00:08:36] Nick Roome: Oh, w- what are you doing there in System32? And then once I realized what it was doing, I upped the permissions because it was a specific pathway within System32, and I said, "Okay, anything that you're doing in this pathway, okay, you don't have to ask me anymore." And it was cool to see that, I can trust you to work in that little thing.

[00:08:52] Nick Roome: Ask me if you go higher type of thing. And so I think, they both ask these permissions to d- to, do these [00:09:00] basic things dozens of times. It takes longer for things to get done, and from a user perspective, user has to know what they're trying to do. What the system needs to know is when it's okay or not okay to proceed with some of these tasks.

[00:09:12] Nick Roome: All that to say, I'm really thankful for reports like these because it gives us a pulse on the state of the world and where we're at. I am curious though, Barry, what are some of your sort of initial thoughts based on what you were reading in these reports?

[00:09:28] Barry Kirby: So I think it's... So I guess an overriding view is, the way...

[00:09:34] Barry Kirby: As you talk, as you say, if we have to understand what sort of permissions that we need to give and how we're engaging with it, I almost worry or think actually, is it just gonna get too easy to- rely on this stuff? Is it too easy to engage with? And the reason I say this is it's already easy to rely on technology, isn't it?

[00:09:53] Barry Kirby: How many of us do, long, long multiplication and long division anymore? We don't. We just use a calculator. Reading a map, [00:10:00] we don't do that anymore for travel generally. Google Maps or other maps are available. You use them, to do that. And if you wanna do any logarithmic equations, again, who- nobody would even know what a slide rule is now, never mind, engage with it.

[00:10:14] Barry Kirby: There's, and I th- I... There's an element here about how we use some of this stuff means w- it's kinda gonna go the same way, I think, 'cause, you, the, in your example, you kinda highlighted you were quite diligent about the way you queried what it was doing and how that came up. I think there are possibly some other people out there who might not be s- quite so diligent, and just going, yes, yes."

[00:10:37] Barry Kirby: Let it happen, and you inadvertently give it way m- way more p- permission or way, way more authority than perhaps you're recognizing what you're giving. So how do we get around that? How do we work with that? And I think this, again, the index report, and, does nail, does highlight some of this, as well as how we talk about some of the, the way that people are using this for software engineering.

[00:10:58] Barry Kirby: I mentioned, obviously I [00:11:00] talked a bit about AI, EHF '26 earlier. W- quite a few of the topics that, that were coming up there was around human-AI teaming, so the HAT issue. And I kinda got led into a question, in one of the talks when, a, an esteemed professor was giving us his insights, and I kinda led myself to thinking, actually, we're spending a lot of time working out how we are going to team with AI, as opposed to it being subservient to us, 'cause we think that's the thing.

[00:11:29] Barry Kirby: And I got led to asking the question, actually, do we want to team with AI? Do we ever want, truly want to move out of that subservient role? Does it ever want to be more than a tool, rather than, or do we want to bring it up to the air? And I'm, whereas I was absolutely convinced, and I've, I've seen a lot of the research, and things like that about how we do, have human-AI teaming, or even having, AI group leaders, team leaders, section leaders, and how we deal with some of them principles.

[00:11:58] Barry Kirby: I wonder whether we [00:12:00] need to go back to, back to the start to a certain extent, and ask the fundamental question of, is it, do we want it to be more than a tool? And then I was k- I was intrigued by your comment as well, where you highlighted, we need to get, we need to get this right.

[00:12:12] Barry Kirby: But with any other tool, any other, software and things like that, if we don't get it right, we'll start again. We'll do it some- somewhere again. But we have this inevitability that once the AI is out of the box, there's no getting it back. That we're never gonna gain control of it again.

[00:12:26] Barry Kirby: Get... And I wonder, to a certain extent, whether we're scaring ourselves. And we are people who are working in this domain and kind of know what's going on. There's other people who just think AI is just AI, and it's a black, ugly thing waiting to, to bite out of there. And, I don't know.

[00:12:42] Barry Kirby: I wonder what is the risk of getting it wrong and then, and redoing it.

[00:12:45] Barry Kirby: Could I gi-

[00:12:46] Nick Roome: can I give just a little example there? I feel- yeah ... I feel like right now AI is a toddler. And if you let that toddler go without, some specific directions, they're gonna open the back door and go into the pool.

[00:12:57] Nick Roome: Or- ... they're gonna, go out into the [00:13:00] street and cause mayhem. They're gonna grab a lighter and do some damage somewhere. Seriously. That's- Yeah ... that's the s- that's the analogy I

[00:13:09] Barry Kirby: would use. But I think the, w- the, we have a natural fear of it from that respect. A- and I think it is a respect of where it is.

[00:13:17] Barry Kirby: I think other people think that, it's already at a stage that if we just let it go wrong, then it's gonna burn down the world, and it's not quite... I don't think it's, I don't think it's necessarily there yet. But anyway, to go on with the toddler example, I- ... I quite like you, you highlighted, the idea of software vibe coding, and I think that's a really powerful thing.

[00:13:36] Barry Kirby: And I think we, this is definitely something I think we need to talk about in a bit more depth, tonight 'cause the, the way that people can use, software, through an AI, interface opens up software to so many more people. The ability to produce ️‍️which were the, the domain of big or even small software houses.

[00:13:57] Barry Kirby: It was an exclusive club, of the right languages, [00:14:00] of the right thing you're talking about. And I come from a software background myself. And so knowing, there was almost a bit of a thing where there was a, how many languages have you got on your CV? A, te- And, whereas now y- you take away that exclusivity.

[00:14:13] Barry Kirby: And it's down to how do you describe what it is that you want to do. And we've talked about in the past, I've been, playing around with it a bit like you have. But I know of a, of at least one software house that I've got a lot of respect for. They had a large number of software engineers And they really seriously investigated this whole, use of AI to support their coding thing.

[00:14:34] Barry Kirby: And actually what they ended up doing was not only downsizing the number of software engineers they've got, but upping their output and get- and eliminating their backlog in, in a significant amount of- a sig- significantly short amount of time. And so they're using it as a... they're a professional software company, and they're using, they're using it in a way that, that I think is really clever.

[00:14:56] Barry Kirby: I guess overall my biggest fear with this is w- and I [00:15:00] meant... I'll use the Google Maps example, is that we start taking this for granted. Because Google Maps, and, and I apologize to long-term listeners of, of the podcast, I've used this analogy before. Google Maps, when we first started doing it, you would check...

[00:15:12] Barry Kirby: It would suggest a route and you, and it would be different from what you were doing. You would go, "Why is it doing that? I'm not doing that, because that's not the way I go. You can't tell me what to do." Whereas now, you... The route comes in and you're like, "Yeah, okay. No worries. I'm not even gonna read what the route is.

[00:15:26] Barry Kirby: I'm just gonna know that you're taking me in the right direction." And, and it will, and it does it. I think we'll get there with AI, and I think the, that's the biggest risk, that we will sit there and you'll just say yes to stuff. And we lose the knowledge of exactly what it's doing and how we get there.

[00:15:42] Barry Kirby: So then I don't... I think a lot of people talk about this, how are we gonna lose AI, and therefore how are you going to do that type of thing. I think we'll end up just not realizing what it's doing on our behalf. Yeah. So I think there's... i've probably gone on far too much now. But the, We'll see how...

[00:15:58] Barry Kirby: I think we'll, we need to get, dig into [00:16:00] a bit about how this is gonna grow in the shorter term.

[00:16:04] Nick Roome: Yeah. It's ironic that you're saying just approve things as I'm hitting the lizard button over here trying to get through these, ... Lizard. Liz- So let's talk through these, these reports here.

[00:16:13] Nick Roome: So the first one here was by MIT Tech Review, and what they had come up with in this report, I'm gonna go through the key findings and then maybe we can talk through some of the themes here. Yeah. So key findings for MIT Tech was that adoption momentum is building for agentic AI. So it seems like it's a top investment priority, in software engineering.

[00:16:36] Nick Roome: You have the, another key finding here that some of these early gains will be incremental and that, basically these, these early investments are going to get the most, sort or the, these improvements to y- be slight here in the beginning, but then, towards the end they'll be larger and bigger.

[00:16:56] Nick Roome: Agents will accelerate the time to market. So- [00:17:00] there, this figure here, 98% of respondents that were, part of this report expect their team's delivery of software projects, to go from pilot to production to accelerate. Yeah. Number four here, the goal for most is fully agentic lifestyle, life cycle management.

[00:17:17] Nick Roome: Again, large numbers. 41% of organizations' teams aim to achieve this for most products within 18 months. And that, by the way, is for aiming for AI agents to be managing product development and software development. That's kinda crazy. Yeah. The last one here for MIT Tech Review, compute costs and integration, pose key early challenges.

[00:17:40] Nick Roome: Having those compute costs and the integration, being the largest barrier to them being able to conduct, or to implement some of this a- agentic AI in software engineering. Yeah. Okay. Contrast that with the Stanford AI report. This one's more interesting because it's more of a global [00:18:00] approach to AI and there's some attitudes about AI in here.

[00:18:04] Nick Roome: There's some of these key findings that I think are more relevant for us in our field. But, here, let's just go through the top 10. So number one, AI capability is not plateauing, it's accelerating and reaching more people than ever. Number two is the US-China AI model performance gap has effectively closed.

[00:18:24] Nick Roome: Number three, the United States hosts the most AI data centers, with the majority of their chips fabricated by one Taiwanese foundry. AI models can win a gold medal at the International Mathematical Olympiad but cannot reliably tell time, an example of what researchers call the jagged frontier of AI, what you referred to earlier.

[00:18:46] Nick Roome: Responsible AI is not keeping pace with AI capability, with safety benchmarks lagging and incidents rising sharply. The United States leads AI investment, but its ability to attract global talent is [00:19:00] declining. And then these last four here, seven, eight, nine, and 10, I think are more relevant to us as, as well as the first one that I read here.

[00:19:08] Nick Roome: So number seven, AI adoption is spreading at historic speed, and consumers are deriving substantial value from tools they often access for free. So I think that one's an important one. Number eight here, formal education is lagging behind AI, but people are learning AI skills at every stage of life.

[00:19:28] Nick Roome: Number nine, AI sovereignty is becoming a defining feature of national policy, but capabilities remain uneven. And then the last one here AI experts and the public have very different perspectives on the technology's future. Yep ... global trust and, in institutions to manage AI is fragmented, and this is something that we've seen.

[00:19:51] Nick Roome: So Barry, of those 15 key findings across both the reports, which ones stick out to you as some of the [00:20:00] more compelling ones to talk about?

[00:20:02] Barry Kirby: I think the, to go to the software type of stuff first, I think the, that adoption momentum bu- building, it absolutely is. I think as much as we, we think in the public domain people are skeptical, people are nervous, talking to software engineers who are using this stuff, s- I will qualify that.

[00:20:22] Barry Kirby: Talking to, senior and principal software engineers and, and CTOs who are using this stuff love it. They ... The way that they described it to me was every software agent that you use, treat them, treat it like a junior software engineer, and you can replace your junior software engineers with agentic AI.

[00:20:41] Barry Kirby: Bec- because you're t- you're giving them tasks. You bound it. You tell them what it is that you want them to do. And then as they, as you develop your infrastructure, you're building your own internal knowledge base as much as the taken in from the wider, platform you're using.

[00:20:58] Barry Kirby: And so you develop the [00:21:00] style that you want to do. You s- and it will work in the way that you want to do it. So them costs that you're seeing, I can really see why. Yes, your early gains are, are gonna be bit by bit as you implement, the idea. It replaces some of them elements. And then you sit there and go, "Oh, I didn't realize I could do that.

[00:21:18] Barry Kirby: I can now do this." You're not only replacing capability that you've got, you're enhancing capability. With the example I mentioned earlier, the team was then able to go much further and faster than it thought it could in the, ... And so then it was g- it wasn't just standing still and maintaining the capability with less staff.

[00:21:35] Barry Kirby: It was actually improving the capability and reducing its backlo- backlog. So you can see that if you've got that incremental improvement, if you've got that ability to improve your capability, that then leads you into that time to market piece, that you will be able to sit there and say, "I've had an idea," random idea.

[00:21:53] Barry Kirby: And you press go. Simplifying mi- wildly. You can press go and have a [00:22:00] product that is fit for market. It might not be perfect, but also we are used to seeing, already software co- coming out now that is a, a s- the, the basic version of what we're doing and then you will incrementally improve it.

[00:22:14] Barry Kirby: We're used to that as a consumer, so to see a, a b- a basic thing come out, something that is maybe a bit buggy is not, is not unknown. So you- something you, you can get something to market really quickly now. So I'm, from that perspective, from an engineering perspective, I'm quite excited. The bit that I find I think is the biggest surprise, it is we're not just talking about the software engineering side of things, which to me makes a lot of sense.

[00:22:39] Barry Kirby: As a, as an ex-software engineer, really all we're doing is changing the in- the interpretation interface. It's the programmatic stuff that the AI will also now handle. Managing the actual project itself, we're also getting the AI to do, or can have the cap, cap- capability to do that product development piece.

[00:22:58] Barry Kirby: I think that's quite [00:23:00] fascinating. What do you think on that?

[00:23:02] Nick Roome: Yeah, I think you're right. I think the, y- you're absolutely spot on with sort of this, the speed to market because even as we've been sitting here, we started one of these projects in the pre-show. Whoops. There g- there goes my camera.

[00:23:16] Nick Roome: We started one of these projects in the pre-show and, w- we'll presumably have something here in the post-show by the time we're done today with a product that is ready for some sort of consumption, whether or not it's, fully integrated and fully, taken care of. But I think it'll be interesting to, to see how far we get in just one hour given a fairly broad goal and me just clicking the lizard button here behind the scenes, which is not why my camera died, but...

[00:23:43] Nick Roome: Sorry, I'm gonna be disembodied voice here for a second while I figure that out. But I think the other, the other piece of it is that the... so that time to market piece is great. I think the really interesting thing to me, and I've mentioned this, a couple times before on the show, is this fragmentation about the technology's [00:24:00] future, and how there are some in the public who really don't see AI as, the future or they are resistant towards AI.

[00:24:09] Nick Roome: And, I think there, it's good to be cautious about where we implement AI in our processes, procedures, into our life, and I think the big problem that a lot of people have with AI is using the creative piece, or using it in creative tools because- A lot of people say, "I want AI to do my dishes for me.

[00:24:32] Nick Roome: I don't want AI to make artwork for me." Or s- I want myself to be the creative and then AI, you do the dishes." And so I think it's interesting to see that fragmentation because from a software development perspective, sometimes doing the dishes is making software. And if you have an agent doing those things, it's, it is going to be like, "Hey, now I can spend more of my free [00:25:00] time, exploring some of these other things that might be kinda cool with the product that I didn't have before."

[00:25:05] Nick Roome: And hopefully this actually will result in, something better long-term for everybody. The other things that stuck out to me is the, the piece about it accelerating. Both of these articles cite this as their number one. You have the, MIT Tech Review saying that adoption momentum is building, and then you have the, Stanford one saying that it's accelerating and reaching more people than ever.

[00:25:34] Nick Roome: And that's crazy that both of them came up with the same finding from two different reports. So that one has very strong evidence to me that, the pervasiveness of AI is not only perceived but is measured.

[00:25:48] Barry Kirby: Yeah. It's... I do wonder with a lot of this whether we are truly measuring apples with apples, because how many people truly understand [00:26:00] how much AI is in their lives now? And the... So even if you're just doing a straight Google search now, that is utilizing an LLM in the background and other, other models.

[00:26:13] Barry Kirby: So there, there's a lot of background stuff there that, That you're probably, you're using it and you don't realize you're using it. The other bit that you highlighted there in, I think it was number seven, where the AI a- adoption, this idea that, that there's so many free assets out there to use, there's so many free capabilities, which is just brilliant.

[00:26:33] Barry Kirby: It's a weird one where you think, where you've, you hear news about a lot of AI companies losing money hand over fist, yet they're still giving out free versions 'cause they know that if people get bought into it, they will then buy credits and they will, then they will, they will buy into it.

[00:26:48] Barry Kirby: So I think there's two elements to it where, where we have that where it's not being used, where we, or we're using it without realizing it, and actually a lot of the user-friendliness of some of these things is really good. And then [00:27:00] that there is that, the education piece that is highlighted because we s- the thing says that formal education is lagging behind AI.

[00:27:07] Barry Kirby: That's because, a lot of people, most people when they're talking about AI don't actually know what AI, AI is. They throw the term around, thinking that's, they, it's not everything, without really driving into the nuance of, what type of AI, what, Are we talking about where an LLM is just a very clever statisti- statistical model, as opposed to, so- something a bit more in depth.

[00:27:26] Barry Kirby: And so if we don't, the day- day-to-day public don't understand it, the, our educators, our day-to-day educators of primary, secondary education, the vast majority of them don't get it. Yet people are going onto the web and using this stuff. Certainly, younger generations are using this stuff in a way that is tantamount to the Wild West.

[00:27:45] Barry Kirby: So it's gonna be, I think there's an interesting... I used to say decade, but I think there's probably an interesting three years ahead of us. I've shortened that right down. There's gonna be an interesting three to five years ahead of us where the, there's going to be a, not, I think the revolution [00:28:00] has already started, but there's going to be an evolution within this revolution about how AI is used on a day-to-day basis.

[00:28:05] Barry Kirby: And it's Pandora's box, so we're not going back.

[00:28:10] Nick Roome: Yeah. It really is Pandora's box. I'm sitting here looking at an interface from when we... We have an interface now, Barry. Throughout the course- Yeah ... of our, us talking, we have an interface. Isn't that crazy? We'll test it out in the post-show. But yeah, you're right.

[00:28:25] Nick Roome: I think the, the key takeaways here is that AI's here to stay. That, that to me is- Yep ... is what I'm seeing, and that, we are moving towards a model where AI is going to be part of a larger team, where humans are but a cog in that machine rather than the leaders of that machine. And from a human factors, human-AI robot teaming perspective, gotta get it right Gotta get it right.

[00:28:52] Nick Roome: Any other closing thoughts on these two articles here, Barry?

[00:28:58] Barry Kirby: I think they're both actually [00:29:00] really good articles and giving real practical... Certainly the software one I think has given a really practical one about where we are going to see massive acceleration, and maybe give, gives us a hint about how the software field is perhaps something that maybe 10 years ago we weren't expecting to be so heavily hit, by this.

[00:29:19] Barry Kirby: 'Cause you think, oh, software engineers, they're gonna love AI, they're gonna be all over it. But actually, this is, it is a, it's a bit of a cliche, it's a game changer in this domain because it opens up and democratizes in many ways, something that was a comparatively closed shop. So yeah, I think the future's interesting.

[00:29:39] Barry Kirby: What about you?

[00:29:40] Nick Roome: Yeah, my, my final thought is, let's go to work. Let's get it right. Let's... Every human factors practitioner needs to have some sort of knowledge- ... about how this stuff works because it is going to... It's already everywhere. It's in, integrated into almost every single product now [00:30:00] in some way, shape, or form, and if it's not already, it will be someday.

[00:30:04] Nick Roome: Yeah ... and so we need to get it right, so let's go to work, everyone. We'll, yeah, we'll get it. We'll get it. We got this, right? He says confidently. Let's hope. Yeah, that's it for me. Thank you to our friends over at Stanford and MIT Tech Review for our news stories this week.

[00:30:19] Nick Roome: If you wanna follow along, we do post the links to all the original articles in our Discord where you can come and join us for more discussion on these stories and much more. We're gonna take a quick break, and then we'll be back to see what's going on in the aerospace sector right after this. .

[00:30:35] Nick Roome: That's right. Huge thank you as always to our patrons. Listeners like you keep the show running. If you wanna hear more about more Human Factors Cast in your feeds, in your ear holes, go support us over there because, y- you really support the podcast. But beyond that, you support the mission of science communication in the human factors field, and I need you to know that.

[00:30:55] Nick Roome: And thank you from the bottom of my heart. Thank you. [00:31:00] But we got, we, we got an exciting one this week here. This Week In Aerospace, is a little bit lengthier this week. We have a great, guest. Without further ado, I'm gonna pass it over to you, Phil, Elena, over to you.

[00:31:11] Elena Zheng: Thanks Nick and Barry. This is Elena and Phil from Aerospace System TG at HFBS. In the previous episodes, we discussed the ongoing shift towards increasingly autonomous operations in aviation, including the growing role of uncrewed aircraft. Today, we're joined by Andy Thurling to help us better understand how these operations are being structured and evaluated in practice.

[00:31:33] Elena Zheng: Andy, to start, can you introduce yourself and tell us a bit about your current role?

[00:31:38] Andy Thurling: Yeah, sure. Hi, I'm, Andy Thurling. I've, a veteran of the unmanned industry now for jeez, going on 20 years. Uh, my, my background is military flight test, of both manned and unmanned aircraft, and then, proceeded, after retiring from the military, went, fully over to the dark side, a- as we say, and went to embrace the unmanned world, because it [00:32:00] was, it seemed to be at the time the next big thing.

[00:32:02] Andy Thurling: And, it's one of those, be careful what you wish for, because, it, it's definitely turned out to be that way. Fast-moving industry and, and it's a fascinating place to be. So I started at AeroVironment, went to the New York UAS test site, was, chief technology officer there.

[00:32:19] Andy Thurling: Now I'm, vice president of airspace innovation at DroneUp. And throughout that, I have been working on trying to figure out how to do this, multi-aircraft operations, we call it now the routine autonomous multi-aircraft operations, group at NASA. But it started off as the M to N, where it's M, small group of humans, small group of M humans managing, a large fleet of N vehicles.

[00:32:48] Andy Thurling: And so it could be air vehicles, it could be sea vehicles, could be ground vehicles and... But the concept remains the same, that you have a small team of humans managing a large fleet of highly automated [00:33:00] vehicles. And the fascinating thing is that, uh, this is important to industry because, I've...

[00:33:05] Andy Thurling: none of these new entrants, we call them, really scale well without having a small team of humans involved with large number of autonomous vehicles, be it, drones or advanced air mobility, eVTOL type, air taxis, or high altitude platform systems. So I've been involved in all of those different, i-industries and still am in drones and also in high altitude platform systems, uh, under, my, my consulting, work that I do on the side.

[00:33:35] Andy Thurling: So still involved with that, still fascinated by that stratospheric, high altitude systems. And they all need this, they all need this technology of the small team of humans managing a large fleet of, vehicles. That's kinda like my background and why I'm here.

[00:33:52] Phil Doyon: All right.

[00:33:52] Phil Doyon: That's very exciting, to learn about all of this. And you already kinda touched about it, so you mentioned the reason why we invited you, because we wanted to hear [00:34:00] more about a working group that NASA is hosting, and it's called the Routine Autonomous Multi-Aircraft Operations, so RAM AO, and it was previously, known as the N2M Working Group.

[00:34:12] Phil Doyon: And so I guess what would be interesting is that could you describe to our listeners what are the main overall objectives of the RAM AO, and what gap is it trying to address in the current airspace operation?

[00:34:24] Andy Thurling: I'll go back to the gap. And the gap right now is that in traditional aviation we are used to having a human managing a vehicle, or more than one human managing a vehicle as you see in commercial airliners.

[00:34:37] Andy Thurling: There could be two, three, or, or four depending on the length of the mission, humans involved in actually flying that aircraft. We need to flip that on its head, and we need to have a small team of humans managing a large fleet of delivery drones or eVTOL air taxis or high altitude platform systems.

[00:34:55] Andy Thurling: And so that's the objective of RAM AO, is routine [00:35:00] day-to-day this is a normal thing. It's not a waiver. It's not an exemption. It is part of day-to-day life. Autonomous, having highly automated, systems on board these aircraft is really a prerequisite for any kind of this multi-vehicle operation because the vehicles need to be pretty much self-managed.

[00:35:18] Andy Thurling: And the humans are really only dealing with the exceptions. And humans are brought into the human machine team to handle the exceptions and intervene When necessary. And so we'll talk about interventions and exceptions, I'm sure in a little bit. That's one of the working groups, that is in the RAM AO.

[00:35:40] Andy Thurling: But, but the gap is that we just-- we don't have in the regulations, a path towards having that small team of humans managing a large fleet of systems. The drone industry right now is doing it to some extent, but it's all under waivers and exemptions. It is not part of [00:36:00] the... It's not part of the rule.

[00:36:01] Andy Thurling: Now, the Part 108 rule that the FAA is promulgating for, drones at low altitude will involve, some mention of multi-vehicle operations. But the requirement from that part is that the opera- the, original equipment manufacturer or the OEM will declare that they have a maximum ratio of human to vehicles based on what?

[00:36:27] Andy Thurling: Based on some kind of industry consensus standard, is what the, FAA is foreseeing. And guess what? We don't have one. So this, I would love to see this NASA, project provide enough, material to, to hand off to a standards development organization so that they could create such a, an industry consensus standard that then the drone industry for one could start to, to base their declarations, of compliance on this industry consensus standard.

[00:36:54] Andy Thurling: So that, that would plug a big gap. Right now we have a big regulatory gap in the sense [00:37:00] of, uh, getting approvals. Technology is really not the question. We see, we see people like Zipline and Wing and, Drone Up, all having waivers for multi-vehicle operations. So it's not such a technical issue.

[00:37:16] Andy Thurling: It is a, it's a regulatory and approvals issue. But the regulators are just not going to approve, operations that they don't find are safe. So how do you show that they're safe? How do you prove that they're safe without spending... A- and Zipline and Wing both were, fortunate or smart enough or both to achieve thousands of hours in environments where, the rules were more permissive so that they were able to show the FAA, "Look, here's our data.

[00:37:48] Andy Thurling: Here, the-- we can show you that our operations are safe because we have this empirical data." But not everybody's got that opportunity to go and collect that empirical, that data [00:38:00] empirically. So it would be nice to have an analytical framework to go about, showing that the, That the operation could be safely managed by the team of humans that you've got in place.

[00:38:09] Andy Thurling: And that's really one of the major focuses of, at least my working group. We're working on a small part of that. And some of the other working groups are also working on, on, on the broader picture of how that can be done. Because although you might be able to do that, and I think we- we've, shown that Wing and Zipline have been able to empirically achieve that, you're not gonna be able to do that for eVTOLs, and you're not gonna be able to do it for the high altitude platform systems.

[00:38:36] Andy Thurling: So they're just stuck until we have this consensus process for finding that the, uh, the human machine team is appropriately sized, ... and is safely able to manage all those, eventualities. So that's really what RAM AO is shooting for and trying to fill that gap.

[00:38:57] Elena Zheng: Yeah. It's a very insightful breakdown of the [00:39:00] current industry standards, regulations, and the gaps, so thank you for explaining that to the listeners. And then your instinct was right. We are gonna ask you more about your specific, working group inside of RAM AO. So can you clarify what is meant by intervention and exception in this context, and why this area is particularly critical for routine autonomous operations?

[00:39:21] Andy Thurling: Yeah, sure. This is, so as I previously alluded to, that you really need to be managing highly automated vehicles. The vehicles need to pretty much know how to handle the mission themselves and really are only reaching back to the humans, for kind of strategic decisions. E- the humans shouldn't be flying any aircraft individually.

[00:39:44] Andy Thurling: They should be managing, the aircraft. W- ideally, we would like, automation in the ground station managing the overall kind of strategic direction of the fleet's flight paths and that, only in the event where something is [00:40:00] out of the ordinary to the extent that the automation can't handle it, that exception then gets, provided to the humans to handle it and intervene.

[00:40:08] Andy Thurling: So there's the exceptions. The automation says, "I have an issue. I can't solve this." It may be because it was, one of the unknown unknowns or it's outside of my operational design domain. But but I can't handle it. So here, Mr. Human, y- you, you work on it. And now the human has to intervene, and so we're making the assumption that the ground station human factors is architected such that the human is able to see that this exception has happened and be able to intervene, effectively.

[00:40:40] Andy Thurling: So the interventions and exceptions group is looking at the structure of these interventions, trying to create a taxonomy that we can go through and, and describe in some standard schema the intervention. Like how long it takes to solve, how bad would it be if it was not [00:41:00] solved quickly, how quickly does it...

[00:41:02] Andy Thurling: does the things go bad? You can have some interventions which you can delay for a little while because the, emergency is not that grave. If I was in a, if I was in a large high altitude aircraft and one of my battery packs was, ... had gone offline and it could no longer recharge, I may not have an issue if it's the summer and there's plenty of sunlight.

[00:41:26] Andy Thurling: That may be... it may be months away before I need to figure out what to do with that vehicle. On the contrary, if I'm a small drone at low altitude and I lose, s- some of my batteries, this could be, immediately an issue. So how quickly do things go bad? Who needs to solve the problem?

[00:41:44] Andy Thurling: How quickly does it take to, to solve, in the intervention? And describe these interventions in a standard taxonomy that we can then ingest into a, modeling and simulation framework that we'll be able to go ahead and do some, Monte [00:42:00] Carlo simulations to, to show that, the...

[00:42:04] Andy Thurling: We're making the assumption also that we're starting this process understanding that the, uh, the hazards that I'm trying to mitigate as a human and the decisions that I might need to make strategically as a human- that we understand the likelihood of having to do that and the severity. So we're starting with a, an operational risk assessment.

[00:42:23] Andy Thurling: We're starting with a, a functional hazard assessment, and we're analyzing those hazards through, a process that allows us to identify the role the human has in doing that, and then taxonomizing that, that role to, to use in this modeling and simulation environment. So hopefully in the end we'll have a modeling and simulation framework that will be able to show that the task loading of your M humans, is never, is never greater than a certain percentage.

[00:42:54] Andy Thurling: W- we might say 80 or 85%. No one human is overloaded, and the team of [00:43:00] humans is not overloaded as, as well. That's making the argument for the safety case, and we'll be able to take that, somewhat quantitative data now to the regulators and say, given the likelihood and severity of these things that the humans are expected to do, we have the ratio set correctly.

[00:43:17] Andy Thurling: So the ratio's not important for the safety case. You have to make the safety case, one way or the other. But where the ratio becomes really important is the business case because, some businesses are going to be, scalable only at higher ratios of humans to vehicles where, or vehicles to humans such that, that we have a very few humans managing large fleets.

[00:43:43] Andy Thurling: It depends on the business case. So that's, that's- ... that's why getting the ratio is important. It's safety, but also, but also the business case.

[00:43:52] Elena Zheng: Yeah. That's very fascinating to me personally also because my research project has been kinda discussing that ratio. And then as [00:44:00] part of the effort to address that taxonomy that you were talking about, I understand some working groups at RMEO are developing whitepapers.

[00:44:07] Elena Zheng: Is that something also what your working group is looking at, and what would be the key dimensions of that whitepaper? Can you tell us a bit more about that?

[00:44:15] Andy Thurling: Yeah. So that's, that is the... that's the core of our whitepaper, is explaining, the process, explaining the framework for how to move from concept of operations, hazard assessments, operational risk assessments through an analysis of the human and, and autonomy teaming, the breakdown of tasks that the human, and the autonomy's gonna be responsible for, and then identifying those tasks which the human will need to be responsible for in those exceptions.

[00:44:48] Andy Thurling: Then taking that, the, those exceptions- and creating that taxonomy of the intervention, that we can then pull into this, Monte Carlo simulation, w- so one, one [00:45:00] of the things that we were challenged with for a long time was there's a lot of it depends in this analysis.

[00:45:06] Andy Thurling: For example, if I'm flying an aircraft and I lose my attitude indicator, what is the severity of that hazard? It depends. If I'm in the clouds, it's pretty bad. If I'm not in the clouds, it's not a big deal. So there's so much of this it depends that we decided that what we would want to do is approach this more from a Monte Carlo simulation, and really simplify that schema for, of taxonomy for these interventions, and then use a Monte Carlo, analysis technique to show that, e- even though I've assumed some simplistic...

[00:45:40] Andy Thurling: i've, I've made some simplifying assumptions about these interventions, the Monte Carlo i- analysis will, in a sense, because I try so many different combinations, that those complexities get addressed with the number of the repeated, scenarios in the Monte Carlo. So it comes out in the wash, at least that's... at [00:46:00] least that is our, thesis, and we're working with, NASA Langley and their, their, multi-aircraft simulation and operations capability, at the range in NASA Langley. And we're hoping that we will be able to get some opportunity to go and test this framework for evaluation on, and the actual, on the actual range that we could first do in simulation and then go ahead and do in, in, in even live flight.

[00:46:27] Andy Thurling: So we're, we're trying to put together this framework, this modeling and simulation, technique, and then validate it in simulation and live flight tests with the folks at NASA Langley. So that white paper would draw that arc, from the, motivation, the framework description, the modeling and simulation, and then to the flight test or the simulation and flight test.

[00:46:48] Andy Thurling: And then hopefully in the end, the conclusion would be that we have, validated that this approach is sound. And then we can press on from there and perhaps turn that material over to a standards development [00:47:00] organization, and they can take that and make it into more of a industry consensus standard as a means of compliance for the, the Part 108 and other regulatory requirements for, for having some way of saying that your operation is safe at the ratio that you're proposing

[00:47:18] Phil Doyon: All right.

[00:47:18] Phil Doyon: That's very exciting, to hear all about the, analytical and solution portion of that work, which really tries to go beyond what we can get from empirical data, that we can, get that data earlier, based on the, system analysis. For closing, for listeners who are interested in following or contributing to RAMEO, are there public resources or opportunities that they should be aware of?

[00:47:43] Phil Doyon: I understand sometimes you have in-person meeting at NASA.

[00:47:47] Andy Thurling: Yes, we just had one, just a couple weeks ago up at NASA Ames. Sometime, we tend to ping-pong between Ames, Langley, and one other industry, location. But we were just at Ames a [00:48:00] couple weeks ago and really the, the best resource I think would be the Routine Autonomous Multi-Aircraft Operations website, that NASA hosts.

[00:48:09] Andy Thurling: And so if you just Google, NASA Routine Autonomous Multi-Aircraft Operations, you'll get to that website, and then will be able to, to look at y- the description of the working group, the various subgroups, in more detail. Even the whitepapers that have been published, and we have published a couple in the past, those are linked as well on that website.

[00:48:32] Andy Thurling: And, and events that, uh, upcoming and both upcoming and previous events are also linked on that website. So there's plenty of opportunity to go and, and figure out where you want to fit into this work.

[00:48:46] Elena Zheng: Awesome. This has been such a informative and exciting conversation. That's all we have for the podcast segment this week.

[00:48:54] Elena Zheng: Andy, thank you for joining us and sharing your insights. Thank you for having me. Back to the main show.

[00:48:59] Nick Roome: You know [00:49:00] what I'm gonna say? Oh, go then. You know what I'm gonna say? We never plan these things for it to- Yeah ... it just works. It just works. Huge thank you, to the Aerospace Systems Technical Group. That, that segment, continues to be a wonderful gift every other week. We are just about at the end of the show here, so Barry, it is time for one more thing.

[00:49:21] Nick Roome: What's your one more thing this week?

[00:49:24] Barry Kirby: So this week as many people will know, I'm into the, into this pottery thing now, and have been for about a year. I've got a sale tomorrow morning, and I'm equally excited and nervous. It feels like I've got... I'm taking a bunch of stuff to go and see if people will hand over their hard-earned cash in exchange for my basically baked earth offerings.

[00:49:46] Barry Kirby: And, yeah, it's a really novel- feeling, I think, 'cause the sort of work we get involved with is quite large. This is just something that we put together. It's there, it's a product, and you can either buy it or not. It feels like I'm gonna get m- my [00:50:00] children judged.

[00:50:01] Nick Roome: Well- Yeah. That's it ... hopefully it'll be a largely positive thing. I think that- ... that maybe it will be a positive experience. I don't know. Maybe I'm hopeful. Yes. What about you? W- what's your one more thing? I teased it in the pre-show, but I have been building... So you had asked the question, what, how long has it been since we've talked about topic?

[00:50:23] Nick Roome: Yeah. And so this thing that I've been building in Codex has been, it, it's called the News Triage Tool, and what I'm attempting to do is filter down, like many different RSS feeds, that have different news sources and different topics of news, and filter them for a human factors lens to say, "This is human factors news," identify it, post it to our Discord, and, and maybe flag it for a potential episode.

[00:50:57] Nick Roome: And do all that in a dashboard-like view where I [00:51:00] can say, "This is not human factors," or, "This is not something that, that would be..."

[00:51:04] Nick Roome: Oh, look,

[00:51:04] Nick Roome: the lights went out. Or this is not human factors, or this is core human factors, or actually cluster these three stories together and that makes a great episode.

[00:51:13] Nick Roome: And it's really cool 'cause I took some of the work that we did on the paper and I put that in there and said, "Okay, now make this into a piece of technology that we can actually use." So that's been a ton of fun. Whenever that ends up being done, I will make that freely available for anyone, so that way you can t- tweak it to your own heart's desire.

[00:51:30] Nick Roome: You can go into a subspecialty or, or maybe even tweak it for something else. But, Nice ... anyway, that was my one more thing. It's just been a ton of fun to watch it, go and be, Be a cool, useful tool. Anyway, that's it for today, everyone. If you like the, episode and enjoy some of the discussion about AI, we have a ton of AI episodes out there.

[00:51:50] Nick Roome: Go check out any of them. Comment wherever you're listening with what you think of the story this week. Did you like those reports? I thought they were pretty nifty. For more in-depth discussion, you can always join us on our Discord [00:52:00] community, where I'm apparently spamming, news stories now. And visit our official website, sign up for our newsletter.

[00:52:05] Nick Roome: Stay up to date with all the latest Human Factors news. If you like what you hear, you wanna support the show and the mission of science communication, you can leave us a five-star review wherever you're watching or listening right now. You can tell your friends about us. That helps the show grow organically.

[00:52:18] Nick Roome: Or if you have the financial means to, you can support us on Patreon or make a direct donation. As always, links to all of our socials and our website will be in the description of this episode. Mr. Barry Kirby, thank you for being on the show today. Where can our listeners go and find you if they wanna talk about conveniently placing two different topics in the same episode that happen to overlap very nice and neatly?

[00:52:39] Barry Kirby: Yeah. If you wanna come and, chat to me and engage, then find me on LinkedIn, Facebook. If you want to be, find more about the pottery side of what I do, then find me on Instagram. If you wanna chat with or listen to me having chats with other people in industry in the Human Factors domain, find me on TwelveOTwo: The Human Factors podcast, which is at twelveotwopod.com.

[00:52:57] Nick Roome: As for me, I've been your host, Nick Rome. You can [00:53:00] find me on our Discord and across social media at Nick_Rome. If you're tuning in live, stay tuned for the post-show. Until next time, it depends.