Welcome to Entanglements. In this episode, hosts Brooke Borel and Anna Rothschild ask: Should we allow AI technology to be put in our brains? It sounds like something out of science fiction, but researchers are already exploring using machine learning in neural implants.
Their guests this week are Judy Illes, a professor of neurology and neuroethics at the University of British Columbia, and Joseph J. Fins, a physician scientist, and medical ethicist at Weill Cornell Medicine in New York.
Below is the full transcript of the podcast, lightly edited for clarity. You can also subscribe to Entanglements on Apple Podcasts and Spotify.
Brooke Borel: Hi, Anna.
Anna Rothschild: Hi, Brooke.
Brooke Borel: I have a question for you. Would you ever give AI access to your mind?
Anna Rothschild: Oof.
Brooke Borel: I get it, I get it. But this is already happening, at least in other people, not in your mind specifically, and there are actually some interesting reasons scientists are considering it.
Brooke Borel: You’re listening to Entanglements, the show where we dig into the thorniest scientific debates — and try to find common ground. I’m Brooke Borel, articles editor at Undark magazine.
Anna Rothschild: And I’m science journalist Anna Rothschild. OK, Brooke. You have my attention. Tell me more.
Brooke Borel: Alright well, first let me back up a little bit. Are you familiar with neural implants?
Anna Rothschild: Yeah, like those electrodes that can go in the brain to help treat seizures? That sort of thing?
Brooke Borel: Yeah, that’s one example. Neural implants are basically hardware that you put into the brain.
Anna Rothschild: Pretty crazy that that’s even a thing, I have to say. But yes, go on.
Brooke Borel: I know. And this kind of tech has tons of applications these days. It may help stimulate parts of the brain to help relieve otherwise untreatable depression, it can give us a way to communicate with people who are totally locked in — meaning they’re conscious but can’t communicate because they’re paralyzed. I mean, some really amazing stuff.
Anna Rothschild: Incredible.
Brooke Borel: Yeah. And some researchers think that this technology could be exponentially more powerful if it used AI.
Anna Rothschild: Why am I not surprised.
Brooke Borel: I know.
Anna Rothschild: Everything could benefit from AI.
Brooke Borel: Allegedly, yes.
Anna Rothschild: Allegedly, yes. But what does that actually mean in practice? How would the neural implants use AI? And what sort of AI are we talking about, are we talking about large language models?
Brooke Borel: Yeah, good question. So in some cases, yes. Maybe an LLM is helping analyze a certain brain signal and predicting whether it’s intended for a specific physical movement, or a specific thought. But there are other uses of AI, too — maybe AI implants could treat visual impairments by, say, reducing noise to make a clearer image.
Anna Rothschild: Oh, interesting. I mean, this sounds amazing, frankly. But if we connect AI directly with our thoughts could it hijack our brains? Or am I just paranoid here?
Brooke Borel: It sounds amazing, but dangerous. Well, you’re definitely not the only one worried about that — or about what other ways this technology could get out of our control. And that’s exactly what we’re going to talk about today. I have two great guests with some really fascinating ideas about what this new era of medicine has to offer — and what the danger is. One is a neurologist, the other is a physician, and both of them have thought a lot about the ethics of their work.
Brooke Borel: Should we put AI-enabled tech in our brains?
Judy Illes: Yes. When the intention is to improve the human condition from a place where the human condition has become suboptimal due to a disease, either a neurologic or psychiatric disease.
Brooke Borel: This is Judy Illes. She’s a professor of neurology and neuroethics at the University of British Columbia. Her training is in neuropsychology but she mainly focuses on the ethics side of things now, pulling together disciplines like philosophy and law and bioethics and applying it to neuroscience. She spends a lot of time thinking about neural implants and other technologies and how they can relieve suffering — without introducing unnecessary risks.
Judy Illes: My focus is alway, well, it will always be in the health arena, attributing these interventions to improve people’s lives when they are suffering.
Anna Rothschild: What kind of suffering is Judy talking about?
Brooke Borel: Well, there are so many potential uses for this tech, at least hypothetically, so we could take this conversation in a bunch of different directions. But one application Judy sees as kind of low-hanging fruit is pediatric epilepsy — so, young kids who are having seizures regularly.
Anna Rothschild: I mean, I know about implants to treat seizures. Where does the AI part come in? How does that help?
Brooke Borel: Yeah, good question. So, let me back up from that for a sec and explain how this tech generally works. So, a seizure is a sort of electrical surge in the brain that basically makes your nerve cells fire kind of willy nilly. And neural implants anticipate those surges before they turn into seizures and counter them with other small electrical pulses into the brain.
Anna Rothschild: Cool. So kind of like a pacemaker?
Brooke Borel: Kind of like a pacemaker. And in theory, AI could help fine-tune these neural implants so they’re more responsive to a particular person’s brain and needs.
Anna Rothschild: OK, that seems like a potentially good use of the tech.
Brooke Borel: Yeah, and in terms of the AI specifically, this is pretty low-risk, because you’re not asking AI to read your mind; you’re just focusing on specific types of electric signals.
Anna Rothschild: Yeah, I feel like I could get behind that, honestly.
Brooke Borel: Right. But some researchers are already eyeing some more ambitious use cases — and ones that come with substantially more risk.
Judy Illes: There is no question that using imaging technology now, we can detect in about one in 20 people who are clinically unresponsive, that they’re actually more conscious of external stimuli in the world than we could otherwise detect. So the question then is, what do we do with that information?
And how can AI help us better understand the signal that these individuals are producing in response to very specific stimuli? And could we use AI technology, maybe in a modulatory way — that is, putting signals into the brain to help people communicate, come out of their disordered state of consciousness? So, you know, the power of AI is there, but the ethical questions are abundant.
Anna Rothschild: Right, I mean AI is certainly known for its hallucinations, right? Like totally making stuff up? And also getting things just wrong?
Brooke Borel: Yeah, and that’s exactly the kind of thing Judy and people in this field think about a lot.
Judy Illes: So one super ethical question is, let’s say we could use AI-enabled neurotechnology to really reliably and reproducibly enable somebody with a bad brain injury to communicate. Let’s say they communicated to us their preference for music. Well, that’s not so hard. We could play Mozart or we could play Michael Jackson or Taylor Swift.
But what if somebody in a disordered state of consciousness with an AI-enabled technology communicated that they wanted to die? How many times would we need to have that signal produced reliably take their word for it? I mean, we’re talking about a huge proxy, mathematically processed signal. You know, the better AI gets, the better the brain signals are going to be. There’s no question about that. But how we respond to that information is a tremendous ethical challenge.
Anna Rothschild: She’s not wrong.
Brooke Borel: Yeah. But Judy still thinks it’s worth exploring this tech, as long as it can somehow be beneficial to people who are suffering.
Brooke Borel: How do we determine when it’s going to be beneficial?
Judy Illes: Well, this is the basis of good neuroscience. We have to understand the brain, we have to understand the human condition, both in health and in disease. And, we need to understand how the technology is operating, if it’s operating according to the hypotheses that we’ve set forth.
Brooke Borel: In other words, Judy’s not just running on vibes here.
Anna Rothschild: Ah, how Gen-Z of you.
Brooke Borel: Oh I’m pretending, I’m pretending. I’m definitely not Gen-Z, but I am young at heart. Anyway. So Judy wants to see good science being done before any of this tech makes it anywhere near a hospital: That means clinical trials, controlled experiments, and good solid evidence that the AI-enabled tech actually works.
And I think it’s also worth stressing that Judy thinks strictly medically necessary uses of this tech should take priority. So there is also a lot of interest from Silicon Valley to use neural implants on any person who is interested in using it.
Anna Rothschild: For what?
Brooke Borel: Human enhancement.
Anna Rothschild: Of course.
Brooke Borel: Of course. Yeah, and Judy has no illusions about the fact that Silicon Valley is going to do what Silicon Valley does, right? It’s just not the direction she’s looking in.
Judy Illes: All the power to, to the people who want to experiment. One of my big ethical concerns is that resources are not diverted from medical applications when there’s so much need that still exists, to treat people who are suffering from real diseases.
Brooke Borel: And even though Judy thinks about the risks of this tech really seriously, she also doesn’t truly think we’re anywhere close to having our brains hijacked by it.
Judy Illes: Maybe I’m naive, maybe I’m old school. I actually don’t believe that any intervention is really ever going to read thoughts in the sense of thought complexity, or cause us to behave in a way where we’ve completely deferred to the effect of a device. But there’s a lot of concern about mind reading and thought hijacking, and I am just not there yet.
Brooke Borel: So you don’t think AI is going to be able to implant thoughts in our head or extract thoughts from our head?
Judy Illes: I know there are people worried about that. Freedom of thought, we’ve been talking about that since Hippocrates. And as long as we remain human, these are just instruments. And I always say if you can throw it in the garbage — and we can throw an electrode in the garbage, we can throw neurotechnology in the garbage — it is not a human. And it will never have humanness.
Anna Rothschild: OK. But I do get the concern. AI is moving really fast, and giving it direct access to our minds does feel sort of scary, even if you can just throw it in the garbage.
Brooke Borel: Yeah, and even though Judy doesn’t see AI reading our original, unique thoughts or hijacking our minds anytime soon, if ever, she still thinks we need some strict precautions.
Judy Illes: So we always need to have human override in anything that AI does or gives us. And I think that’s the most important in ensuring that AI is used beneficially and doesn’t sort of run away without us.
Anna Rothschild: OK, I mean, I still feel a little uneasy about the whole thing, but it sounds like Judy is pretty clear-eyed about how to move forward safely.
Brooke Borel: Yes. But not everyone thinks it’s actually possible to do that right now, despite our best intentions.
Anna Rothschild: Including our next guest?
Brooke Borel: Exactly. My next guest is a clinician who actually uses neural implants in his work, so this is really close to home for him. And he sees what we stand to gain by hooking up these implants to AI. But he’s just not so sure there’s a way to do it ethically. At least not yet.
Brooke Borel: So the question of the day is: Should we put AI-enabled tech in our brains?
Joseph J. Fins: I would say categorically no, not yet. But I think it’s probably an inevitability that some version of AI technology is going to be introduced into the human brain. But it has to be done with anticipatory governance and a real prudential ethic, and we haven’t figured that out yet.
Brooke Borel: This is Joseph J. Fins. He’s a physician-scientist and medical ethicist at Weill Cornell Medicine in New York. He and his colleagues have run studies on neural implants during his career, and he knows about some of the astonishing things they can do, in particular, for people in minimally conscious states.
Joseph J. Fins: These are people that look like they’re vegetative, they’re unresponsive, but every once in a while they demonstrate awareness of self, others, or the environment. And unlike vegetative patients, people with a minimally conscious state have intact neural networks inside the brain.
Anna Rothschild: Is this like locked-in syndrome?
Brooke Borel: I wondered the same thing, but no. So, locked-in syndrome is when someone is cognitively intact, they know what’s going on, but they can’t physically move. Their whole body is paralyzed and that’s why they can’t speak. In a minimally conscious state, their cognition is not fully intact, but they may have moments where they are responsive.
Anna Rothschild: OK gotcha.
Brooke Borel: And Joe and his colleagues have conducted a small number of studies with people in a minimally conscious state, where the researchers have used deep brain stimulation, which involves a neural implant. He even wrote a book about it. And it can have totally life-changing results.
Joseph J. Fins: We had a patient who was in a minimally conscious state and who could not communicate, could sometimes move their eyes in response to a command or something, but they couldn’t talk, they couldn’t eat on their own. They needed a feeding tube and had really poor postural tone.
With the stimulation, this subject who could only communicate sometimes moving their eyes was able to say sixteen or seventeen-word sentences, say the first six or seven words of the Pledge of Allegiance, and tell his mother he loved her. He was also able to go to Old Navy, the clothing store, and choose clothing with his mother. So what we did was we gave him — we restored agency.
Brooke Borel: But even though AI could potentially help us interpret the signals coming from the brains of people like this, Joe is wary about the idea of hooking neural implants up to AI.
Brooke Borel: Do you think of it as being more of a privacy concern? These might be private companies that you might be signing rights away to. Or is it more over the fact that you are outsourcing some of this to a machine, so some of that autonomy goes away?
Joseph J. Fins: There’s certainly privacy concerns, and there’s certainly proprietary concerns and intellectual property concerns and things like that, which are certainly real. But I think they’re subsidiary concerns, and the bigger concern is that you’re outsourcing yourself to this entity that may not represent you. People who have CMD —
Brooke Borel: That’s cognitive motor dissociation.
Joseph J. Fins: — don’t have language output. And wouldn’t it be good if we give them voice? But here’s the caveat: We don’t want to put words in their mouth. And if AI is doing a probabilistic culling and scraping of the internet based on what would a person like this be saying, or what do I think a person like this would be saying, who is severely “disabled,” and ableism is running rampant on the internet — you know, the person might say, “I want to…” And the final sentence, “I want to die,” versus, “I want you to scratch the itch on the back of my left thigh.”
Brooke Borel: Joe gave a real-life example of a patient who had been in a minimally conscious state and who had then been somewhat brought back with deep brain stimulation. But this person got really sick with pneumonia and ended up in the hospital.
Joseph J. Fins: The doctors asked him if he wanted to be with his mother. His mother had died a few years ago, but one of his problems as a person with brain injury was he had a problem with temporality. And he didn’t always remember what time it was. And he loved his mother. His mother was devoted to him. So when the doctor said, “do you want to be with your mother?” and he said, “yes,” the doctors interpreted that as he wanted to die because his mother had passed.
His sister reached out to me and Nico Schiff to try to help him. And long story short, he didn’t want to die. He just wanted to be with his mother, and he didn’t quite understand that his mother had died. And there’s a situation where in real life, what people were saying was misconstrued because of a profound disability bias. So I think what can happen in real life can happen in artificial life. And I think we have to be very careful about introducing biases and ableism into the discourse.
Brooke Borel: So some people might argue that there’s so much potential for good, and then of course, the benefits have to outweigh the risks in all of this. So is there any way to put safeguards on the technology so it can be used, or is that just
Joseph J. Fins: I think so —
Brooke Borel: — something to think about in the future? Tell me more.
Joseph J. Fins: Yeah, I do. I do. And I think the word I like to use is curation. Or editors in the journalism business. You know, reporters used to write things and editors or fact-checkers, they would review what was written and make sure it was accurate and sensible and cogent and readable and understandable. And so I think curation is a really important notion here, or collaboration, right? So working with AI together.
Anna Rothschild: So in a way, it sounds like Joe is coming around to the same point Judy made — that basically their challenge is figuring out how to work with AI so that it can’t just take over for someone, even if that person fully relies on it to express themselves.
Brooke Borel: Yes, exactly. And Joe’s idea of curation is really interesting. But I do wonder how well our society will be able to actually do that. I mean figuring out how to work with AI without letting it take over for us is one of the big questions of our time.
Anna Rothschild: So true. And it sounds like our two guests have different levels of optimism about how well — and I guess how soon — we’ll be able to do that.
Brooke Borel: Absolutely. But when I brought the two of them together, they agreed on a lot more than I expected them to.
Brooke Borel: You’re both coming off of travel, right? How was Munich?
Joseph J. Fins: It was great. It was great
Brooke Borel: By the way, right before we spoke, these two were together at a neurotechnology seminar in Germany.
Judy Illes: It was an extraordinary workshop on neuro AI.
Brooke Borel: The biggest point that Joe and Judy disagreed on was just how promising or risky AI is in general.
Judy Illes: Perhaps I am naive, but I just don’t think that these devices, no matter how sophisticated or AI-fueled they are, will really ever be able to hijack our thoughts, capture our spirits.
Joseph J. Fins: So at this point, I’m not sure about, you know, taking control and wresting control of your brain and turning you into an automaton. That I think is much more hypothetical. But in the real world right now, you get into a car and the driver, instead of making their own judgment in how to proceed, they tap their thing into the Uber thing, and they’re not using their brain. Or the college kid who instead of learning something decides to outsource his or her or their learning to AI and they don’t ever experience the joy of creativity.
And the other problem with AI, it only knows what we currently know. It doesn’t know what we’re going to discover, I don’t think. And so I’m concerned about humans being creative and remaining creative and ceding that wonderful space that we can occupy uniquely to AI.
Judy Illes: But I wonder if AI, just to be a bit provocative, Joe, isn’t just another layer of human evolution. You know, I think in the moment it feels a bit weird and tricky and maybe a bit scary to us, but I think AI is just another cog in the wheel of human innovation, like fire, like the wheel, like the car. Right? We’re constantly evolving and AI seems to be faster and deeper and more widespread as an enabler. And I, I just think that’s just part of the evolution of the world.
Joseph J. Fins: I think it’s — I agree with you, Judy. We’ve always kind of progressed and, you know, fire and wheels and radios and TVs and all that. But I think this is, this is a logarithmic leap that is very different and impenetrable and not understandable. And I think that makes it, to me, you know, very concerning. As a physician, if I don’t understand why somebody is sick, I’m much more concerned than when I understand, OK, what is the pathology? What is the etiology of their illness? This is kind of a mystery illness, and I don’t really — no one seems to understand how a bunch of tokens get together and produce thoughts and an occasional hallucination.
Brooke Borel: So Judy and Joe brought up different perspectives about the promise and the risks of AI in general — that is, AI as a tech more broadly, not specifically as it relates to neural implants. But it turns out that when it comes to the latter, they’re not all that far apart.
Brooke Borel: When I spoke with each of you individually, I asked both of you this question: Should we put AI-enabled tech in our brains?
Joe, you said that for now you would say categorically, “no, not yet.” that it was inevitable at some point but you wanted there to be some anticipatory governance and a lot of other things sort of set up in anticipation of that.
And then Judy, you said “yes” to this question. You said, “With the intention to improve human condition from a place where the human condition has gone — and has become suboptimal due to maybe a disease or a neurologic or a psychiatric disease.” So my question for you both is: What do you think is at the root of the different answers that you gave?
Judy Illes: Well, I’ll jump in, Brooke, and say that I don’t really think Joe and I are giving you different answers. There’s no question that whatever we do has to be safe and effective and reliable and oriented towards improving the lives of people who might need them or use them.
So some of neuro AI I think is here today. We’re seeing it, for example, in the kinds of iterative processes that are embedded in neuroadaptive feedback for implanted devices for pediatric epilepsy, for example. So that’s an early form of AI, early-ish form of AI, that is very powerful.
Joseph J. Fins: I think we don’t fundamentally disagree, but I think using it to detect and abort seizures is probably one of the simplest kinds of technology where it reads a signal, it makes a decision, and it feeds it back. What I’m concerned about is AI that robs agency from the individual and begins to substitute itself for the self. And I think that is truly possible. You know, we already see this where AI is being used to write text outside of the brain, and people have lost their voice as authors to AI. And college teachers can’t tell who wrote it, the student or the LLM.
So I think, you know, it’s a matter of time before devices get into the brain and rob you of your agency. Now there’s a tremendous potential for therapeutics that I don’t want to deny, and I want to promote, but I think that we need to understand the boundaries. And right now, it’s a kind of an ungovernable space. There’s no regulation, and I think it’s going to be sort of the Wild West until that gets regulated.
Brooke Borel: Joe is especially worried about this when it comes to treating people with brain injuries who aren’t fully conscious.
Joseph J. Fins: I mean, basically, we’ve been using deep brain stimulation in an investigational way to help people who are in the minimally conscious state and people who have had severe brain injury function at a better level, a higher level. And one example with the minimally conscious state, a patient who was unable to speak regained voice through deep brain stimulation, through an integrated process that happened through stimulation of the thalamus.
And my concern, Judy, that I mentioned earlier was that we want to give people voice, but we don’t want to put words in their mouth. And if it’s a probabilistic kind of token counting phenomenon where somebody says: “I want…” You know, it might be: “I want to die,” because there’s a kind of an ableism and disability bias against people in this kind of brain state, when in fact the person says: “I want to go outside.”
You know, we have a real challenge of the probabilities. The probabilities on the internet that’s being scraped for the evidence of what comes next in this probabilistic train is full of disability bias and ableism that could be problematic, and that was my concern. However, it’d be great if we could help facilitate communication as long the people are speaking for themselves.
Judy Illes: You know, I could not agree more. And I think that pertains to just about everything that we do in the neuro innovation space. We want to provide enabling technology but not overstep what an individual might wish to express, be able to express, and put our desires from the outside into others.
And I think it comes back to the humility that we have to envelop around all this technology at this time. And it’s not only about transparency, which so many of us in the neuroethics world are calling for. It’s also good methodological approaches, good reporting, both of positive results and of failures, and we learn from both of those. And, you know, with people who have suffered brain injuries who may be able to exit their disordered state of consciousness at some point naturally through sort of the natural biologic process of recovery or enabled by deep brain stimulation, the respect for dignity here is absolutely essential, both in terms of who these people were before they were injured and who they might be when they come out the other side.
Brooke Borel: I’m curious if in this specific scenario we’re talking about, are there some situations with the right guardrails in place where you could see it being useful, being — it being AI-enabled tech for someone who’s in one of these cognitively minimal- minimally conscious states?
Joseph J. Fins: You know, I think it’s all pretty hypothetical at this point.
Brooke Borel: Of course.
Joseph J. Fins: And the additional complexity here is for that population, because they can’t give their own consent, we’d be looking to surrogates to consent. So there’s an additional wrinkle of complexity and delegation of authority because we want to have people speak for themselves. But if we turn to surrogates for authorization for this kind of thing, we have another sort of gap in authorization that could be problematic. I do think it can be very helpful. I mean, if someone’s using an eye tracking device, for example, it is already somewhat predictive. People start spelling out the word C-A- and then it says T for cat. So that’s a kind of artificial intelligence at a very simple level. But I do want to have the opportunity for the individual to have override, so they can say, “No, I don’t mean that. I meant to say ‘can’ versus ‘cat.’”
But I think it’s going to facilitate things because these technologies are so cumbersome right now and in a way clunky and time-consuming. And the idea here of this notion, the word ironically is embodiment, right? So we want somebody, if they’re getting a prosthetic arm or they’re getting a speech assistive device, we want something that’s mobile that allows them to integrate in society, to go out and have a job, work and all that. So if AI can facilitate engagement and social engagement and that kind of thing, it would be a wonderful thing. But there has to be the opportunity for human override, so that as they go out, they’re not hijacked in a way they don’t want to be.
Judy Illes: Yeah, I’m going to layer in a couple of other considerations there if, if I may. One is, in giving voice to these individuals, which will be ever improved with neuro AI and neural implants, is to what extent will we listen to the voices if the voice does not tell us a message we wish to hear. For example, somebody seeking to end their life, somebody signaling that they don’t want to continue in this type of experimental situation. And so there may be a surrogate, but to what extent can the surrogate override these expressed signals from these individuals? How good are these signals? Can they be reproduced often enough and emphatically enough that we ought to listen to these individuals, particularly when they may be expressing something that is an irreversible condition like death?
Brooke Borel: If you do have that situation, at what point do you feel comfortable trusting this technology or not? Like, at what point do we as a society say, “Yes, this person has said they want to die this many times through this technology.” At what point do you believe them or not?
Joseph J. Fins: Right. So, I don’t think we need to reinvent clinical ethics here in how we assess people’s decision-making capacity for complicated choices. And I think one of the challenges here at this point, a lot of this is going to be receptive. Do you want this? Do you want that? People are going to say yes and no. But there was a framework that James Drane, a bioethicist, wrote about in the ’80s called the sliding scale of competence, today we say capacity, where people have to evidence understanding that’s commensurate with the seriousness of the choice.
So if I’m going into a patient’s room, and I ask you to draw blood in the morning and say, “Can we draw your blood?” And they say, “Sure,” that’s enough of an exchange for me to have the moral authority to draw blood. If we’re talking about wanting to die or having a major surgery, they have to evidence understanding. And they have to be able to ask us questions as well. It’s not just to be passively responsive to what we say to them, but they’re able to ask us questions, and then they have to repeat back evidencing understanding of what they want.
Now, I think somebody who has a severe brain injury who wants to die, you also want to try to mitigate the reasons why they want to die. For example: Are they in pain? Are we understanding their needs? Are they uncomfortable? Are they lonely? Are they isolated? And, and so those are, those are a whole bunch of other considerations. But I think as far as the fidelity of their choice, we make those determinations in clinical medical ethics all the time when people refuse treatments in the hospital to understand whether they truly understand.
And people have a right to refuse life-sustaining therapy, and in some jurisdictions, they have a right to physician-assisted suicide. But I think here, the output is going to be of much narrower band, and it may not be as capacious as a normal conversation. So we have to try to understand what crosses the threshold of capacity, decision-making capacity, that would be respectful of their right to make decisions and at the same time not mistaking what they’re saying for something else.
Brooke Borel: It seems that no matter what, when we’re talking about neural implants and the potential for using AI within those technologies, there are a lot of potential risks and a lot of potential benefits. And moving forward is this balance that changes depending on what condition we’re talking about. I would like to hear each of you talk a little bit about how you personally as researchers, as ethicists, balance that risk and benefit with this kind of technology. Joe, do you want to go first?
Joseph J. Fins: Well, you know, we call that in ethics land proportionality, and we try to maximize the benefits and minimize the burdens. The problem is I really don’t know what the risks are. I can imagine them. I don’t know them. So it’s really hard to regulate them ahead of time.
Judy Illes: So I’m going to go forward from the word regulation, Joe, and add here, which is that we are looking at a world in which trust in science is at an all-time low. And I think one of the biggest risks is that mistakes are made, and then it takes technology that has enormous potential and almost brings it to a screeching halt because people don’t understand it. And the disposition towards science and scientists is so fractured at this time. So what’s really important is that we anticipate risks. We have some sense of what risks could exist. We have to understand that we don’t necessarily understand the risk-benefit equation yet.
And we have to think about good ways of regulating in a way that actually supports innovation, supports the natural process of human curiosity advancing at an incredible pace while putting guardrails in that are both adaptive to the pace of innovation. And I would say in a way that is also culturally acceptable, right? We have to accept cultural values into this story. And I think we have to self-regulate as scientists and ethicists and innovators to really prevent regulation from external sources that would potentially criminalize certain kinds of innovation and avert those by actually being smart and thoughtful from within the professions where we actually understand deeply what’s going on, and regulate ourselves, regulate each other as professionals.
Joseph J. Fins: I would just add to this point that Judy’s making about risk, I think there — different people can conceive a risk in different ways. And that means we need to have an inclusive conversation to determine what is the risk. So there’s a saying in the disability rights community: Nothing about us without us. You need to include people who are going to be stakeholders and who are going to be exposed to the risk to have a say in what is an acceptable risk.
Brooke Borel: Joe, are you more excited or optimistic about the possibilities for the benefits of this technology? Or do you feel more worried about possible risks? Where do you land there?
Joseph J. Fins: I think people like me, we get paid to worry.
Brooke Borel: OK.
Joseph J. Fins: We’re society’s worriers. But I’m not a philistine, I’m a technology hawk. I believe in technology and I think it can be managed. But we need to do the ancillary work around it in addition to the scientific work. But I’m concerned about this technological imperative getting ahead of ourselves and not doing the broader social work around a new technology that — Judy’s right, I mean, we always do it in retrospect. We never do it prospectively. And again, it’s probably too far gone for us to do it prospectively because the process is already well on its way. But I’d like to be around in 100 years to see how this went, you know?
Judy Illes: You know, one of the things I think we’re also very good at, being worriers, is also being warriors. In our way of being warriors we anticipate some of these issues of where they are going, some of the issues that are already at hand, and some of the issues that we try to anticipate just by feeling out what’s coming down the pipeline in the space of neurotechnology. So being worriers and being warriors is really a very powerful way of thinking where the world is going with AI, with neural implants, being careful not to fall into a space of technological colonialism where technical science once again hijacks people’s rights and keeps them from having access to good technology. So there is a lot here, and I think we are going in a good place.
[MUSIC]
Brooke Borel: All right, Anna, after hearing all of that, do you change your mind? Would you put AI in your brain?
Anna Rothschild: Honestly, I have to say I kind of would.
Brooke Borel: You would? Tell me more.
Anna Rothschild: Not today
Brooke Borel: OK.
Anna Rothschild: Not today. But if I were in a position where I had very severe issues with communication, I think I would want AI to help me communicate. That being said, I don’t know that I would want to do it if I were in a minimally conscious state like Joe was talking about. I’d want there to be some way for me to confirm that the AI got what I intended correct.
Brooke Borel: Yeah, so there’s still a line there for you.
Anna Rothschild: Yeah, absolutely. And I mean, I’m very sensitive to Joe’s concerns here. Like, certainly for patients where you can’t confirm that the AI is understanding you properly, that’s scary.
Brooke Borel: That’s scary.
Anna Rothschild: That’s really terrifying and could have really horrible impacts on a person’s life. Like catastrophic impact. So I get why we should proceed with a lot of caution, particularly in cases like that.
Brooke Borel: I think so too. And I think there’s also something to the point that Joe made, about even now when we’re just using like ChatGPT or whatever, people are in some cases really leaning on these products to think for them, right? Like they’re having them write homework assignments and messages, like text messages to their friends and family and all sorts of stuff.
So there is part of me too that feels very squeamish about kind of offloading so much of our autonomy, like giving up so much of our autonomy to these devices potentially in the future, whether it’s helpful or not,.
Anna Rothschild: Oh, 100 percent. But I think that autonomy is actually a really key word here, right? Because we actually — you and I, Brooke — are privileged enough to have a lot of autonomy in our lives. So we get to make the choice every day whether we use AI or not, and I agree with you. I don’t want to outsource any more of my brain power to AI than I already am. But I think if I were in a very different place where I had a lot less autonomy in my life and this were a means to get it, I don’t know. I think I’d say sign me up.
Brooke Borel: Yeah. All right, listeners, we want to hear what you think. Do you want a neural implant? Do you want it with AI? What do you think?
Anna Rothschild: Send us an email at [email protected]. We would love to hear from you.
Brooke Borel: And that’s it for this episode of “Entanglements,” brought to you by Undark Magazine, which is published by the Knight Science Journalism Program at MIT. For this episode, we had a big assist on research from senior editor Nora Belblidia. Our amazing producer and editor is Samia Bouzid. The show is fact-checked by Undark deputy editor Jane Reza. Our production editor is Amanda Grennell, and Adriana Lacy is our audience engagement editor. Special thanks to our editor-in-chief, Tom Zeller Jr. I’m Brooke Borel.
Anna Rothschild: And I’m Anna Rothschild. Thanks for listening. See you next time.