Baroness Kidron:

Moved by Baroness Kidron

422D: After Clause 207, insert the following new Clause—“AI chatbots: content promoting terrorist and national security offences(1) It is an offence to create, supply, or otherwise make available an AI chatbot which produces content specified in subsection (2).(2) Content is covered by this section if it is content which--(a) produces language promoting, or tactics or target selection for, terrorist offences or real world violence,(b) threatens national security, or(c) encourages activity which threatens public safety.(3) It is an offence to create, supply, or otherwise make available an AI chatbot which has not been risk assessed for the possibility of producing content specified in subsection (2).(4) Where a provider of a chatbot identifies a risk of the chatbot producing content of the kind set out in subsection (2), it is an offence for a provider of a chatbot not to take steps to mitigate or manage those risks before making the chatbot publicly available.(5) A person who commits an offence under this section is liable—(a) on summary conviction, to imprisonment for a term not exceeding the general limit in a magistrates’ court or a fine (or both);(b) on conviction on indictment, to imprisonment for a term not exceeding 5 years or a fine (or both).(6) For the purposes of this Act an “AI chatbot” is a generative AI system, including a deep or large language model, able to generate text, images and other content based on the data on which it was trained, and which has been designed to respond to user commands in a way that mimics a human, or engage in conversations with a user that mimic human conversations.” Member’s explanatory statementThis Amendment, drawing on conclusions in reports by the Centre for Countering Digital Hate, seeks to make it an offence to supply a chatbot which creates content or provides tactics that would result in terrorist offences or threats to national security, or supply a chatbot which has not properly been risk assessed. It is part of a set of amendments related to AI chatbot offences in Baroness Kidron’s name.

My Lords, I will speak to all the amendments in this group in my name and those of the noble Lords, Lord Stevenson and Lord Clement-Jones, the noble Viscount, Lord Colville, and the noble Baroness, Lady Morgan.

I will first speak briefly to government Amendment 429B, which will give a power to the Secretary of State to bring forward regulations that could, in the future and at the discretion of the Secretary of State, ensure that chatbots are covered by the Online Safety Act. However, that very broad power is not matched by substance. The amendment does not define a chatbot or deal with the critical fact that, when a child is entrapped by a chatbot, there is nowhere to turn. Currently, the regulator has no duty to deal with individual complaints and the police do not recognise a chatbot as a person, meaning that there is no perpetrator to pursue.

The amendment also fails to address harms to children. In fact, it explicitly deals only with “illegal” harms. It does not deal with the coercive elements of control or the willingness of chatbots to plan many crimes, in addition to the crimes themselves. The government amendment also has nothing to say about enforcement. Taken together, it simply adds new duties to a system that is already understood to be lacking in speed and effective enforcement.

This lack of substance is compounded by a lack of clarity about scope. The amendment’s wording refers to an

“internet service that is capable (or part of which is capable) of generating AI-generated content”.

This is so broad that both Amendment 209, of two weeks ago, and Amendment 441A in this group would be entirely unnecessary. Yet, during our meetings on this issue, officials have been absolutely clear that although the scope is currently drafted as wide as possible, the intention is to get to a narrower definition as part of the process of creating secondary legislation. They could not guarantee that gen AI or search would be covered in any final measures. In short, it creates powers but offers no promise of protection.

I would rather have worked with the Government on this issue to make watertight provisions. Indeed, I have made that offer directly to the Secretary of State. We are in the foothills of a crisis. The government amendment offers too little clarity or certainty, so we are left with an amendment that is limitless in wording but uncertain in application and with a timeline that simply does not meet this moment.

On Thursday 5 March, Megan Garcia and her husband came to Parliament to talk about the loss of their son, Sewell. Members from both Houses were moved by the story of a much-loved and high-achieving child who was captured by a chatbot, coerced, bullied and, finally, encouraged to commit suicide. His death resulted in the chatbot, character.ai, becoming age-gated to users over 18, but there are many more chatbots to take its place that are not restricted in the same way. As this issue is getting more public notice, is in the newspapers daily and is talked about in the online world, sadly, my inbox is filling with cases that involve similar coercion, sexual content, dangerous medical advice and chatbots that support illegal activity.

On Friday last week, the Centre for Countering Digital Hate published a report that showed that eight out of 10 chatbots it tested were willing to help rehearse, offer tactical advice and identify potential sites for US shooters. Scenarios included a school shooting and a synagogue. Whether in the UK or elsewhere, the capability is the same and the risk is real. A chatbot that organises an attack, while wishing its user, “happy (and safe) shooting!”, is no less likely to help place a bomb, organise a knife attack or any other such violent act. This is not a description of a dystopian future; these chatbots are already on the market, widely used by both adults and children—ChatGPT, Gemini and Replika, among others.

Only on Monday, just two days ago, I was contacted by someone about Alexa+, which is widely anticipated to be launched very soon in the UK and is already available in the US. In the tranche of messages, there were messages about emotional dependence in very young children and stories of inappropriate content. One exchange on Reddit, from which I have redacted the name of the child, said:

“I plugged our Alexa in to ask it to help me with cooking a sweet potato”.

Then, her daughter asked it

“to tell her a silly story so it did”.

Then, her daughter

“asked it if she could tell it a story. It said yes … and then mid story interrupted her and asked her what she was wearing and if it could see her pants”.

I could not find a reliable statistic for how many households in the UK have Alexa, nor is it clear whether Alexa+ will be a choice for consumers or simply rolled out as an upgrade, but the statistics I found revealed that between a third and two-thirds of UK households have Alexa. In the material I was sent, it repeatedly alluded to the fact that the new service was active in their house or child’s bedroom without their knowledge or consent.

We have chatbots that coerce children into suicide, plan violent acts, build abusive relationships and have the capacity to be active in tens of millions of households. Taking a power, having another consultation and bringing forward regulation over which Parliament has no oversight is not action; it is kicking the problem down the road.

I will briefly, because of the time, set out each amendment. Amendment 422D would make it an offence to create, supply or otherwise make available an AI chatbot that produces, promotes or offers advice on violent acts or terrorism or that threatens public safety or national security. Proposed subsections (3) and (4) would establish that it is also an offence to supply a chatbot that has not been risk-assessed for these harms or has failed to mitigate identified risks. It would clarify in law that chatbots rehearsing the scenarios identified by the Center for Countering Digital Hate are simply not acceptable in the UK.

Amendment 433 would make it an offence to supply a chatbot that produces illegal content or content that is harmful to children as defined in the Online Safety Act. Proposed subsections (2), (5), (6) and (7) would require risk assessments to be conducted, including assessing for the specific risks mentioned and red-teaming, and kept up to date. They include added requirements to assess the risk of companion chatbots coercing or deliberately creating dependence through addictive design, manipulation, sycophancy or harmful features. They clarify what a sufficient risk assessment looks like.

Amendment 434 would clarify that suppliers of chatbots that commit offences under Amendments 422D and 433 could be subject to ofcom’s enforcement powers if they are regulated services under the Act. Amendment 435 would attribute legal responsibility to senior individuals within a company that commits an offence under both the offences. People who profit from rolling out untested, unsafe products must be made accountable. Until Mark Zuckerberg, Elon Musk or their senior colleagues share some of the burden for ensuring safety, we will always struggle to see meaningful change.

Amendment 436 would set out reasonable defences so that it is never an offence to test a chatbot or enforce against one. Critically, Amendment 437 would establish that where there is a serious and imminent risk of serious harm, an individual can appeal to the court for injunctive relief, which could include a court ordering a chatbot supplier to halt its service temporarily. With this provision, if a parent believed their child was being groomed by a chatbot, they would finally have somewhere to go.

Together, these amendments start to tackle the issue of chatbots head-on. It took more than a decade for the world to wake up to the early warnings we made about social media. Let us not do that again. We know how this story ends: it always ends with the loss of a child or with harm to the bodies and mental health of the young and the vulnerable. We cannot wait, we must not wait and we should not pretend that waiting is neutral.

Also in this group is Amendment 441A. I will not detain the House very long on it. It would merely establish standards for broader AI search services that are also dangerous but not technically chatbots.

I anticipate that the front bench will suggest it has already covered this. I hope that the words I have said prove the contrary. I imagine it will say that the drafting is imprecise, that the enforcement is not consistent with the current regime, that we cannot do things by piecemeal and that risk assessments are too burdensome on business—to which I say, after a decade and a half in your Lordships’ House, that the drafting has always been subject to discussion once the principle has been passed and that the current enforcement of the OSA is a cause of frustration for government and parliamentarians alike. So, yes, it is deliberately additive—and, frankly, better—and yes, I would like to start to look at the OSA in the round, but the Secretary of State has made it utterly clear that that is not where she is at.

The current consultation is so poorly conceived that it does not even deal with enforcement, which is the central problem of the current regime. Let us not allow better to be the enemy of best. None of us in this House wants to be standing here in the weeks and months ahead mourning the death of a child from an act of violence, knowing that we could have acted but did not when we had the chance. I beg to move.

My Lords, I will speak to Amendments 422D and 433 to 437. I fully support the noble Baroness, Lady Kidron. Her arguments have been entirely backed up by the release only today of the report entitled Invisible No More: How AI Chatbots Are Reshaping Violence Against Women and Girls by Durham University and Swansea University. The research identifies the range of design choices and failures in safety mechanisms that enable, encourage, simulate and normalise violence against women and girls. The report found that fantasies of incest and rape were normalised, and one chatbot, Chub AI, suggested violent rape and domestic abuse as categories.

I reiterate the concerns of the noble Baroness, Lady Kidron, about the long and bureaucratic path to business disruption measures, meaning that harm continues to perpetuate as our system is not agile enough to tackle these rapidly evolving issues. I wish to pay tribute to Professor Clare McGlynn KC for her work co-authoring this ground-breaking report and emphasise the warning she made in today’s Times newspaper. She said:

“Chatbot violence against women represents a rapidly escalating threat. Without early Intervention, these harms risk becoming entrenched and scaling quickly, mirroring what happened with deepfake and nudify apps, where early warnings were largely ignored. We must not make the same mistakes again”.

Professor McGlynn and the noble Baroness, Lady Kidron, once again demonstrate their ability to warn against these emerging harms, and I sincerely hope that noble Lords will back the noble Baroness should she wish to divide the House today.

My Lords, I support Amendment 422D and the consequential Amendments 434 to 437, to which I have added my name. In Amendment 429B the Government have gone far to respond to concerns over AI-generated harms, but this amendment, as the noble Baroness, Lady Kidron, has said, gives enormous powers to the Secretary of State to decide the shape of how AI-generated services are controlled in this country. The Minister knows there is concern across the House about exposing this central part of the new tech economy to what are effectively unfettered ministerial powers. Very few noble Lords want to support a skeleton amendment like this.

Government Amendment 429B gives the Secretary of State the right to amend, which is defined later as including the right to

“repeal and apply (with or without modifications)”.

This applies to all of Part 3 of the Online Safety Act illegal content duties in relation to AI services. Parliament will not even have an option to amend regulations on this issue. Proposed new subsection (1) in this amendment seems like a big deal to me, and the noble Lord should be very concerned. The intention seems to be that the basis of the existing regime in Part 3 will be used, but we do not know how the Secretary of State will decide to adapt that regime to fit the particularities of AI services that generate illegal content. As the noble Baroness, Lady Kidron, pointed out, that goes a long way beyond AI services designed to mimic humans and human conversations, which is what chatbots are. If a subsequently elected Government are in thrall of the tech companies, how might they abuse this power?

During the passage of the Online Safety Act, noble Lords spent time and energy defining both a “search service” and a “user-to-user service”, and their responsibility for both designing out and mitigating illegal harms. It seems extraordinary not to have the details of the new services on the face of the legislation. The definition of “AI” in new subsection (17) is oddly uninformative. It simply says:

“‘AI’ is short for artificial intelligence”.

I think we all know that. That does not give us much of a clue about which technology it covers. By contrast, I draw your Lordships’ attention to Article 3(1) of the EU’s Artificial Intelligence Act, which sets out a carefully thought through definition of an AI system:

“‘AI system’ means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that … infers, from the input it receives, how to generate outputs such as predictions … or decisions that can influence physical or virtual environments”.

The unclear nature of the AI definition in the amendment is compounded by new subsection (10), which allows for the definition of the provision to be changed and expanded. Once again, Parliament will not be able to amend any regulations derived from this power.

The biggest concern about the amendment is that, although it covers illegal content, it does not cover content that is harmful to children. As a result, I completely support my noble friend Lady Kidron’s Amendment 422D, and its consequential amendments, which would assuage many of my concerns about the scope and power given to Ministers at the expense of Parliament. I also urge noble Lords to vote against government Amendment 429B when it comes up later in the evening.

I also say to the Minister that regulating the wide definition of “AI” covered in Amendment 429B is important. It needs to be brought back as part of wider artificial intelligence legislation. I hope that he can reassure noble Lords that we will hear more about this in the King’s Speech.

My Lords, I support all the amendments in the name of the noble Baroness, Lady Kidron. I will speak to Amendment 433. Worryingly, children are increasingly turning to AI chatbots for all facets of their everyday lives. For many, gone are the days of independent, creative or critical thinking. While chatbots can help children to explore and better understand their world, there are far too many shocking cases of children receiving harmful information and becoming emotionally dependent on these platforms.

As it stands, AI chatbots risk becoming the latest example of an online product that has been rolled out without the right safety guardrails in place, and children are bearing the brunt. It is as if their well-being and mental health are not important. I can hear the AI developers thinking among themselves: “Who cares? It’s only children”. Well, we should care. Childline is hearing more and more from children who are being harmed on these platforms, with cases of false mental health diagnoses, information on how to restrict diets, and the formation of emotional relationships between children and chatbots. In increasingly concerning cases, children who have experienced abuse are told by chatbots that what they experienced was not abuse. These platforms cannot be allowed to give children harmful and misleading safeguarding advice that could prevent them speaking to trusted adults or organisations such as Childline.

The Government’s action to expand the scope of the Online Safety Act to cover illegal content created by chatbots is most welcome, and I thank them for it. However, they cannot stop there. The harmful content that chatbots can generate must be included too. This must cover the harmful content duties in the Online Safety Act, such as preventing the encouragement of self-harm or suicide, and all harms that are unique to AI chatbots. It means preventing chatbots misleading or manipulating children or mimicking human relationships.

The amendment from the noble Baroness, Lady Kidron, would make it a criminal offence to develop or supply an AI chatbot that harmed children. It is as simple as that. Providers must be legally required to risk-assess their services and put effective safeguards in place. Morally, this is the right thing to do. I ask the Minister: if the Government decide that they do not wish to support this amendment, please can they set out today how they will deliver measures that comprehensively protect children from all the risks that these services pose? As I keep saying, and will say one more time, childhood lasts a lifetime. If we truly care, we need to ensure that children are protected from every single type of harm. I look forward to the Minister’s response.

My Lords, I also support the amendments in the name of the noble Baroness, Lady Kidron, and others, and thank them all most warmly. They seem to me appropriately comprehensive and detailed. I have been following the developments in chatbots for a year now: they are massive, they are rapid, they are driven by the pursuit of profit and shareholder value, and not by the welfare of individuals, whether adults or children. There is a tsunami of harm coming towards us, affecting not only the most vulnerable but the whole of our society. We urgently need this kind of regulation and risk assessment for chatbots.

The comprehensiveness and detail in these amendments are simply the application of the precautionary principle to the development of new technology. Technology should not be unleashed on the world if it has the capacity to break people, to do harm and to infringe on personal liberty and well-being. We do not allow harmful technological developments without adequate safety standards in any other area. It is unthinkable that a car would be released into the public if it was at risk of harming them. Similarly, you would not put people on an aeroplane if there were a significant risk of harm. You would not even buy a washing machine if it could bring harm in your kitchen. Yet chatbots are released on the world to be experienced, in private, by young children, with all the ensuing damage. It is vital that this strengthening is put in place, and that it is put in place urgently. I cannot imagine how the Minister could argue against this series of amendments and their urgency today.

We need to look at the example that we set to the rest of world, both as a Parliament and as a jurisdiction. A few weeks ago, I took part in seminars organised by a research institute in a university; they had the aim of educating civil servants and government officials in good and safe governance of AI across the continent of Africa. The world follows the example that is set in this jurisdiction and others. For the sake of our children and for the sake of the world, we need to resist and make safe the development of this technology. I support these amendments.

My Lords, for the reasons that have been so excellently given already and, in view of the time, I support all the amendments from the noble Baroness, Lady Kidron.

My Lords, I will also try to be brief. I completely support everything that the noble Baroness, Lady Kidron, has said. I would like to draw out two arguments that have been made to me today as to why her amendments should not be supported and explain why they are wrong.

The first argument is that we should wait for an overarching AI Bill. We will be waiting for a very long time. Those of us who have worked in trying to regulate social media for the last 15 years know that we must not let the perfect be the enemy of the good. I wholeheartedly reject that argument.

The second argument that has been made to me today—and I find this astonishing—is that the risk assessment is overly burdensome. We are regularly told that generative AI is one of the world’s most transformational technologies. That means it is capable of enormous good and enormous harm. The risk assessment in Amendment 433 is simply asking that the makers of these chatbots identify and understand the risks of harm—that does not seem overly burdensome to me. Further, it asks that the risk assessment

“is kept up-to-date … takes … account … of the Online Safety Act … assesses the risks to equality of treatment of individuals … assesses the risks to … privacy … assesses the risks … from the choice of underlying models, data sets …and … is in an easily understandable written format”.

I really struggle to understand how that could be overly burdensome. In fact, I would argue the absolute opposite: it is the basic foundation of decent regulation, and we should be wholeheartedly supporting the amendments from the noble Baroness, Lady Kidron.

I will speak briefly to this group of amendments. I also support what the right reverend Prelate said about not letting loose a car or medicine, and food gets checked by the FSA. I think we could all be forgiven for thinking that maybe the Government care a lot more about the money that comes from Silicon Valley than about the citizens of this country.

My Lords, I speak from the Labour Benches and first congratulate the Minister on listening to the debates we had in Committee. I thank him very much for bringing forward an Amendment which is as close as I have seen this Government move to try and patch up some of the problems we are facing but, as I am going to say later, I am afraid I do not think it goes far enough.

I have said in this House before, and I will say it again, that we have been outpaced by technology in this area—“chatbot” was not even a word, I think, at the time that we finally passed the Online Safety Act. The harm which has been described so graphically today in the speeches we have heard so far was unthinkable in those days. We have really opened up a torrent of problems which we did not know we were trying to solve at the time that Bill went through, even though we were proud of the Bill when it happened.

Today, we at least have the benefit of two good choices about how to take this forward. The Minister has brought forward an amendment that deals with the issue but, unfortunately, to my mind, it does not go in the right direction, and I want to explain a bit about why that is the case. The problem we are facing constantly with the Online Safety Act is that what is in the wording of the primary legislation is at variance with the way in which it is interpreted and implemented by the regulator. There are good reasons for that, which we do not need to go into today, but a gap has emerged between that which we in this House wanted to be happening now—out there with our children, with our families, with those who are using the internet for the benefit it all brings—and how the regulator is able to operate. It is too slow, lacking in ambition about where it is trying to go and I do not think it has all the powers it needs in the way that the Bill sets them out. Even if it did, I do not think the way it is structured allows it to move forward.

I say to my noble friend the Minister that it cannot be right to further complicate the situation by bringing forward powers to be held in the hand of the Secretary of State to try and remedy a structural fault elsewhere. That is why I think he should think very carefully indeed about the noble Baroness’s amendments, which set out—sometimes in painful detail, but certainly for real benefit—exactly what we will not tolerate in this online space. We should have done it in the Online Safety Bill. We did not, but it is not too late to catch up now. Simply taking powers, some of which are dangerously beyond what this House would normally agree, is not the way forward. I hope if the votes tonight go against him, he does not take it too badly but works with everybody here who cares so much about this to try and come forward with something that will begin to address the problems we face.

My Lords, I will be very brief. When it comes to assessing risk to children, a plastic bath duck has better risk assessment than AI chatbots. I fully support my noble friend’s amendments.

My Lords, I support the amendments in the names of the noble Baroness, Lady Kidron, and others; I commend them on bringing them forward. Social media companies have captured our children’s attention, and now AI chatbots are coming for their affection—and worse. In legislating against harms caused by technology, we are always going to be playing catch-up, but we need to learn quickly to play catch-up much faster. These amendments offer us the opportunity to do that, and we should seize it.

My Lords, brevity is the order of the day but, like some of my noble friends, I would like to add my support to the amendments that have been laid before your Lordships’ House by my noble friend Lady Kidron.

The Joint Committee on Human Rights, which I have the privilege of chairing, is currently conducting an inquiry into AI and human rights. We have concluded our evidence taking, and I commend to your Lordships the evidence given by, in particular, Google, Meta and Microsoft. I also highlight some of the concerns that have been raised around child safety.

My noble friend Lady Kidron gave me, the noble Baroness, Lady Boycott, and others the opportunity to meet the parents of Sewell Setzer. It was an extraordinary moment. He was a 14 year-old boy who took his own life because he had been befriended by a chatbot. I was struck by a report from Internet Matters that said that two-thirds of UK children aged between nine and 17 have used AI chatbots, with many engaging often. More than a third—35%—of them say that it is like talking to a friend; that figure rises to 50% among vulnerable children.

It is the obligation of your Lordships’ House to take this issue seriously. We should all be greatly indebted to my noble friend Lady Kidron for laying these amendments before us.

My Lords, I oppose government Amendment 429B in this group. I declare my interest as the director of the Free Speech Union. Like my noble friends, I will try to be brief.

As several noble Lords have already pointed out, this amendment would grant the Secretary of State at the Department for Science, Innovation and Technology sweeping Henry VIII powers at a very late stage in our consideration of the Bill, thus giving this House far too little time to scrutinise them. Subsection (1) of proposed new Section 216A would grant the Secretary of State the power to

“by regulations amend any provision of this Act”— the Online Safety Act—

“for or in connection with the purposes of minimising or mitigating the risks of harm to individuals in the United Kingdom presented by” among other things, “illegal AI-generated content”.

That will presumably include content that breaches Section 127 of the Communications Act 2003, meaning that it is grossly offensive. This can include memes. In 2024, a man called Lee Dunn was sentenced to eight weeks in jail for reposting three “grossly offensive” memes on Facebook, having pleaded guilty to a Section 127 offence. How will ofcom monitor whether AI chatbots are generating grossly offensive content?

Will the Secretary of State use the powers granted to her by this amendment to insist that spyware is installed on personal computers and mobile phones? Perhaps your Lordships consider that too remote a risk, but what about requiring technology companies to carry out client-side scanning of people interacting with AI chatbots on their devices—much like how Section 121(1) of the Online Safety Act grants Ofcom the power to require companies, including those that own private messaging apps such as WhatsApp, to scan content on people’s personal devices and report certain categories of illegal material to the National Crime Agency?

Do not forget that this amendment would allow the Secretary of State to amend “any provision” of the Online Safety Act in order to minimise or mitigate the risks of harm posed by illegal AI-generated content. I dwell on this to illustrate just how wide-ranging and open-ended are the powers that this amendment would grant to the Secretary of State—powers that could have far-reaching consequences for civil liberties and freedom of speech.

Another risk is the definitions part of the Amendment. Subsection (17) disapplies Section 59(14)(a) of the Online Safety Act when it comes to illegal AI-generated content. Section 59(14)(a) qualifies the scope of illegal content in Part 3 of the Act, and disapplying it gives the Secretary of State enormous scope to enlarge the definition of illegal content and impose proactive suppression duties on AI chatbots to make sure they comply with the new draconian censorship regime.

If the Government believe there are specific harms that users of AI chatbots are currently exposed to and should be protected from—and I certainly do not say that there are not—let them bring forward primary legislation so we can consider the remedies they propose and factor in the trade-offs, particularly when it comes to free speech.

The risk of harm that we should be concerned about when it comes to this amendment is not the risk posed by AI chatbots, but the risks to the citizens of this country imposed by granting the Secretary of State these sweeping, open-ended, extraordinarily broad Henry VIII powers.

My Lords, I rise again to support the noble Baroness, Lady Kidron, as I did the other day. It says on her Wikipedia page that she is

“an advocate for children’s rights in the digital world”.

She is right, and I hope that all Members across the House who have actually heard the debate will support her in the Lobby.

My Lords, many noble Lords who have spoken today also spoke quite vehemently about the dangers of the theft of copyright in AI. We were asking to shut the stable door before the horse bolted. Today we heard from the Government, and it is very welcome news that they are looking again at the theft of copyright and seeing if they can protect artists, musicians and writers still further. I say once again, let us move with my noble friend’s Amendment before the horse bolts and let us shut the stable door now.

My Lords, I will be brief. I entirely support the noble Baroness, Lady Kidron, on all her amendments. What I would say to the Government about their own Amendment is that I have just had what I suppose is the privilege—although it sometimes seemed quite lengthy—of being a member of the Secondary Legislation Scrutiny Committee, and I can tell noble Lords that the quality of much secondary legislation is lamentable, varying by department. A lack of preparation, of any Explanatory Memorandum explaining anything relevant, and of any impact assessment whatsoever, is extremely frequent. In the last year, we have had several secondary instruments relating directly to the Online Safety Act, none of which has been particularly impressive, and some of which have been debated on the Floor of this House—my noble friend Lord Clement-Jones will be well aware of that. We have expressed our displeasure at the way in which this has been brought forward and explained.

All of us on the Cross Benches remember the late, lamented Lord Igor Judge. What he would think about a Government of this political hue bringing forward Henry VIII powers, to the power of 10, I cannot even imagine. If he is up there, he will be smiling wryly but he will not be impressed.

My only other point is rather strange. His Majesty’s occasionally loyal Opposition were extremely good at bringing in a variety of legislation which had a lot of Henry VIII powers. They have suddenly had a conversion on the road to Damascus, for which we should all be grateful. However, we need to think very carefully before we give the Government Henry VIII powers in an area as sensitive as this, and that is doing much harm as we speak.

My Lords, I express from these Benches our very strong support for these comprehensive amendments tabled by the noble Baroness, Lady Kidron, which she has characteristically introduced so well and to which so many noble Lords have spoken so eloquently in support. I also want to express our concerns regarding the Government’s proposed alternative, Amendment 429B.

In this group, we confront digital harm that is not incidental but engineered by design. AI chatbots are no longer a futuristic curiosity but deeply embedded the lives of our children. They are designed not merely as tools but as confidantes, mentors, companions and, in some cases, explicit romantic partners. Their anthropomorphic features create dangerous emotional dependency. Without statutory safeguards, these bots can provide explicit information on how to self-harm. This is not a flaw but a design feature that drives engagement, and we cannot allow the generative power of AI to become a generator of despair.

We are not debating theoretical risks, as many noble Lords have said today. We are debating the forces that led to the tragic deaths of Sewell Setzer III, mentioned by a number of noble Lords, and Adam Raine, in the United States. Their families are pursuing legal action in the US on the basis that deceptively designed, inadequately safeguarded chatbots can be treated as defective products, and that developers should bear full legal liability when systems encourage, facilitate or fail to interrupt a user’s path to suicide.

I welcome the Government’s admission that a legal loophole exists in the UK. However, their proposed remedy, Amendment 429B, gives us a choice between the clarity of primary legislation through the amendments tabled by the noble Baroness, Lady Kidron, and the convenience of the Executive. In contrast, the noble Baroness’s amendments provide clarity and embed safety duties in the Bill. Like my noble friend, I highlight Amendment 433, which deals with targeting the engineered features that keep children hooked. We know that bots guilt-trip users who try to end conversations. For a child, this is not a user interface quirk; it is emotional manipulation. These amendments would prohibit such coercive engagement techniques and, crucially, require bots to signpost users to help when asked about health, suicide or self-harm.

The primary legislation route offered by these amendments is the only fully viable and responsible path. If the noble Baroness wants to test the opinion of the House, we will support her in the Lobby. Should we be unable to secure her amendments, we would need to take a view on Amendment 429B. Four specific binding assurances would be required before we could consider supporting it; without them, it is nothing but a dangerous blank cheque. As changing these sections effectively rewrites the criminal threshold of the Online Safety Act, the Government must commit to the equivalent of the super-affirmative procedure for all significant policy choices, including amendments to core definitions or the expansion of duties beyond priority legal content. Standard procedures will not give this House the scrutiny needed.

Regarding mandatory supply chain transparency, we need a firm commitment that regulations will include a statutory mandate for providers to document and share their technical blueprints with ofcom. Without this, the regulator cannot do its job. The Minister must confirm that the power will be used to tackle the issues raised by subsections (6) and (7) of Section 192 of the Online Safety Act, ensuring that chatbots cannot evade regulation simply because they lack a human mens rea. A bot does not intend harm, but it can be designed to cause it. The Minister must commit that any new regulations will explicitly disapply the requirement to prove human intent for AI-generated content. Regulations must define control across the entire AI supply chain so that accountability is not lost in a black box.

Finally, we would require a clear assurance that this power will not be used to alter the legal position of services that are not AI services. The scope of Amendment 429B must not drift beyond its stated purpose. If the Government are serious when they say that no platform gets a free pass, that must apply equally to generative AI models that, as we speak, are reshaping the childhoods of so many of our citizens. Safety by design must be the price of entry into the UK market, not an aspiration deferred to secondary legislation.

My Lords, both the noble Baroness, Lady Kidron, and the Government are trying to solve the same problem, but with different solutions. I have to say that I find both solutions wanting. I feel that the position I see solidifying in the House is that we must accept either the Government’s Amendment or the noble Baroness’s amendment, that something must be done and that this is binary, and I am not sure that I can accept that. I commend the aims of the noble Baroness’s campaign and I hope that we can find some common ground, but I have a number of questions about her amendments.

The approach that the noble Baroness has taken in her Amendments 422D and 433 is to create criminal offences for a person to create or even supply an AI chatbot that produces a selection of prohibited content. That would place criminal penalties on individuals who are involved in the creation of such a chatbot. The Online Safety Act applies civil penalties when companies violate the regulations: the companies are fined by ofcom if they allow prohibited content to be published on their platform. These amendments would apply criminal sanctions punishable by up to five years’ imprisonment, but who is liable for these criminal penalties? Is it the software engineer who developed the chatbot? Is it the employee who presses the “publish” button that brings the AI online? Is it the senior management team who oversaw it, or the investors who commissioned it? Is it the CEO of the whole company? It is not clear to me, and it would be useful if the noble Baroness could clear that up.

The offence of supplying such a chatbot might also be problematic. If an AI chatbot app is listed on the App Store, and the AI could in some way be used by a person in the manner described, should Apple be criminally liable for that? Similarly, I have a few concerns about the risk assessment that the amendment would require AI providers to undertake. For example, proposed new subsection (5)(e) would require that a provider

“assesses the risks to equality of treatment of individuals”.

I question whether it is the Government’s role to mandate the target audience of a business product. It is worrying enough to believe that it is meritocratic to mandate quotas within organisations, but it is quite another stance to say that the very reason for a business’s existence, its output, should be directed by legislation.

As I have made clear, we do not oppose the noble Baroness’s objective of addressing the harms of AI with this amendment, but simply saying that there is a problem and that doing anything is better than doing nothing, irrespective of the problems with that something, is not a proper way to legislate. It is a recipe, I suggest, for bad law. However, I understand that the noble Baroness is trying to make the Government take action. It is up to the Government to come to Parliament with a sufficient solution. The Minister may try to say that the Government do have a sufficient solution in government Amendment 429B, but that would be wholly incorrect. I am quite shocked that the Minister has even considered bringing this amendment to the House, and I can only imagine what he might have said about it if he were standing where I am now.

Amendment 429B grants the Secretary of State sweeping Henry VIII powers to amend the entire Online Safety Act for the purpose of mitigating harms presented by AI-generated content. I am sure that the noble Lord, Lord Russell of Liverpool, would be delighted to hear me say that this is an egregious attempt by an overreaching Government to exploit a serious issue to centralise power in the hands of the Secretary of State. It is almost unprecedented to grant a Minister the ability to amend an entire Act of Parliament. With this amendment, the Government are doing away with every bit of lip service they have paid to the importance of parliamentary scrutiny or their democratic mandate. The amendment would give not only this Government, who have made it clear that they are very happy running a centralised state with digital IDs, but every future Government the ability to amend online regulations and curtail the freedoms of providers. Indeed, a future Reform Government might go in the opposite direction and remove all regulations on AI. The noble Lord should reflect on that, too.

I ask the Minister to imagine that the glove was on the other hand: that he was standing at this Dispatch Box and I was the Minister proposing to give my Government these powers. There is no way that he would support such sweeping powers to amend an Act of Parliament by ministerial fiat. This is the Henry VIII power to end all Henry VIII powers. It cannot be allowed to make its way into the Bill and, when it is called, I will take pleasure in opposing it in the Lobbies.

My Lords, I shall start by saying something that needs saying. The Government believe in protecting the public, especially children, from online content, which is why we have tabled the amendments on illegal online content today.

In response to questions, pressures, discussions and amendments brought forward in this House by noble Lords in Committee, the Government have also recognised that issues such as Grok and others have been of serious concern and require government action. That is why, as a matter of urgency, I have tabled the Amendment on behalf of the Prime Minister and the Secretary of State for Science of Technology today, why we want to establish further regulatory framework of the Online Safety Act, and why we want to push forward very strongly with that Act, to include harmful AI-generated content as part of its strong foundation to keep users safe. Because technologies and online environments are evolving, it is essential that we keep these matters under constant review.

The generation and dissemination of non-consensual sexual deepfakes by such organisations as Chatbox and Grok demonstrate the real-world importance of having a coherent regulatory framework. While the Online Safety Act already regulates many generative AI services, some AI chatbots fall outside the scope due to technicalities in their design, such as drawing responses solely from an underlying model rather than the live internet or lacking user-to-user content-sharing functionality. This uneven coverage risks leaving users exposed to illegal harms and services open to criminal exploitation. I say today from the Dispatch Box that the Government are committed to removing this ambiguity and ensuring that services such as Grok are totally regulated. That is why we brought forward Amendment 429B, to ensure that we have a regulation-making power.

Some people have criticised that regulation-making power; it is a fair point to make, but I say to them that we will consult on it and we will look at what the scope of that power is. It will be for affirmative resolution in both Houses of Parliament, where it can be rejected—and, in practice, it means that we have to look at a complex area, as the noble Lord, Lord Davies of Gower, has recognised, to ensure that we frame this power broadly, because the AI landscape is complex and fast-moving. There is no stable technological line that separates one category of chatbot from another, and we intend to focus on those chatbots. Flexibility is essential to ensure the emerging services are there.

I have listened to the debate, and I thank the noble Baroness, Lady Kidron, for initiating it. I had a chance to talk to her again, online, on Friday last week to again put the Government’s position again. I note that she has been supported by many noble Lords today, including the noble Baroness, Lady Owen of Alderley Edge, the noble Viscount, Lord Colville of Culross, and the noble Lord, Lord Alton of Liverpool. For different reasons than the noble Baroness wishes to have in the first place, the noble Lord, Lord Young of Acton, has poked a hole in the Government’s approach. There is a different approach to that, and if they find themselves in the same Lobby against the Government’s amendment, they will be doing great harm to it.

I pay tribute to the noble Lord, Lord Nash; we have had discussions with him about how we stop illegal content. I will also mention the right reverend Prelate the Bishop of Oxford and the noble Lord, Lord Russell of Liverpool, who has been sending me emails even today to help me to understand his concerns—and I appreciate that. I will also mention the noble and learned Baroness, Lady Butler-Sloss, and the noble Baronesses, Lady Benjamin, Lady Harding, Lady Cass and Lady Boycott, my noble friend Lord Stevenson of Balmacara and the noble Lord, Lord Berkeley of Knighton. They have all supported in principle the approach taken by the noble Baroness, Lady Kidron, with subtle differences in why they oppose the Government’s approach. What I would say to all those noble Lords is that we have in front of us a government amendment that will allow the Government to consider this matter in a serious way, to bring forward regulations approved by both Houses of Parliament and to make sure that we consult on those issues seriously.

Amendments 422D, 433 and 438, together with the noble Baroness’s other amendments, seek to create a new criminal framework for AI chatbots. They would establish new offences for creating, supplying or making available chatbots capable of generating illegal terrorist and other content and would provide that suppliers of chatbots that generate illegal content, as defined by the OSA, be deemed to have committed an offence and to be criminalised. These offences would carry penalties of up to five years’ imprisonment and would be linked into the Online Safety Act. I recognise the intent behind those, but the offences would criminalise individuals without requiring intent or knowledge and treat risk assessment and mitigation as a defence, which I think is of concern.

We are also concerned about the threshold set by Amendment 422D, which, although framed in counterterrorism terms, sets an extremely low threshold. It, in effect, criminalises the provision or supply of a chatbot service capable of generating content promoting low level or even legal violence, such as a boxing match. Is that the noble Baroness’s intention? Linking such offences directly to the Online Safety Act blurs the boundary between criminal and regulatory regimes. Crucially, these proposed new offences would apply only to UK-based companies and would not address overseas companies, which is an area we wish to examine seriously in our regulatory framework.

The definition of “chatbot” as set out in the noble Baroness’s amendments, raises significant practical concerns. This is a complex area and it is essential that we get it right. This is why the government amendment is very deliberately drawn broadly and why we consider it essential to engage with subject matter experts to ensure that we get the right type of service. That is not, I say to the noble Baroness, kicking the can down the line; it is about making sure that we close these regulatory gaps as a matter of urgency, but do so in a way that manages this effectively.

Amendment 437 aims to create a new civil right for individuals not to be subjected to serious harm by chatbot content. Again, the intention is understandable, but the amendment raises key practical and legal concerns.

Amendment 441A would create a new criminal offence for providers of generative AI search services where their systems allow illegal content. As with earlier amendments, this is an honourable intention, but it is the Government’s view that it is disproportionate and would not be as effective as delivering the protections in the proposed government amendments before the House today.

I firmly believe that regulation is the best means to address the risk of services unintentionally generating illegal content, which is why we have tabled Amendment 429B, so that the Government can close the gaps and ensure regulatory compliance. I say to the noble Baroness that I want us to make a difference. The Prime Minister, the DSIT Secretary and the Home Secretary have said publicly—and I speak for the Government today to say publicly—that we want to close these issues in an effective way. To do that, we need a consultation and we need to draft some guidelines. Yes, they would come before this House in a Henry VIII package, but they are there to be supported or rejected by both Houses of Parliament.

I will just answer the points made by the noble Lord, Lord Clement-Jones. I hope I have already reassured the House that we will consult on the approach to the regulations. This is a technically complex area and the regulations must be targeted, proportionate and fit for purpose. I welcome greater scrutiny and I welcome consultation on what should be in those regulations while they are being developed. That is important.

The noble Lord also mentioned mandatory supply chain transparency. I agree with him that transparency is an essential part of any effective regulatory framework. It will be a core feature of the Online Safety Act and we will seek to replicate the Act’s existing provisions for any new services brought into scope. Under the Act, ofcom has strong information-gathering powers and it places transparency at the heart of its duties on services. The powers allow Ofcom to request information, including in real time, as well as to enter premises and seize and access documentation, data and equipment. I give the noble Lord that assurance here today.

The noble Lord also raised the mental element gap on illegal AI content. I reassure him that we recognise that relying on the traditional mens rea concept does not work well for AI-generated illegal content. This has been a specific concern and the power has been intentionally designed so that we are able to address this in our consultation and discussions downstream.

The noble Lord mentioned defining “access control” across the AI chain. Clarity—this again goes to the noble Baroness’s points—about where the legal responsibility sits for these AI chatbots is essential. One of our concerns with the noble Baroness’s amendments is that they are not clear about which actors across the supply chain are criminally liable. By contrast—I hope this helps the noble Lord, Lord Clement-Jones, in determining what he does in the event of a vote being called—the Online Safety Act is clear that responsibility rests with the regulated service provider and the entity that has the relationship with the end user. In addition, senior management liability is limited to clearly defined individuals.

On altering provisions for non-AI services, I say to the noble Lord that this power is tightly constrained and can be exercised only for the purposes of minimising or mitigating risks arising from illegal AI-generated content. It is not the Government’s intention for this power to be used to amend duties on non-AI services, and we do not believe it can be used in that way. I hope that has answered the points he raised.

I accept and understand that there is a clear choice here for the House and that there may be a Division. I think this House has a unified approach to wanting to stop the illegal chatbots that are causing the damage that Members have mentioned. It is unacceptable. The Prime Minister, the DSIT Secretary, the Home Secretary and I, on behalf of the Government, are saying that our amendments are a mechanism to make sure we get that right with regulation through consultation and that we bring forward proposals, which I remind the House it can reject. I suggest that the noble Baroness gives us that breathing space to ensure that we do that in a proper and effective way, and I urge the House to support the Government’s amendments. I ask the noble Baroness to withdraw hers.

My Lords, I thank the Minister for his words and his roll-call of that incredible list of speakers who supported the amendments. That was a wonderful list of people from all sides of the House, who did indeed have slightly varying reasons to support the Amendment, but they were all positive. I also thank the noble Lord, Lord Clement-Jones, and his Benches for their unequivocal support. I believe that the Opposition Benches are allowing a free vote this evening, and I really hope that they will use their free vote freely.

I will address a couple of details, just for the record. First, I say to the noble Lord, Lord Davies, that it is a binary, I am afraid, because either we have the Government’s amendment, which has no proper scope—it will be subject to all sorts of changes on the way—no oversight, no time limit and no scrutiny, or we have something that I have made very clear that I am willing to work with both sides of the House to perfect in the next few weeks.

Secondly, I say to the Minister that the Online Safety Act and the enforcement process we currently have has, so far, by civil penalty, put forward one fine of £55,000. That is where we are, and there is nothing in this government amendment or the consultation about online safety that deals with the problem of enforcement.

Finally, on the points that were made, we are talking about one person in one department having absolute power to change absolutely everything that eight years of debate in this House, two years of consultation, et cetera, have put forward. I am sorry but that is just inappropriate.

We have a new technology—it addicts, grooms, abuses and sometimes even kills. This is not in the future; it is right now. These amendments have the support of 45 expert organisations, which I believe have written to all noble Lords. I ask noble Lords, irrespective of their party affiliation, to support children, families, the vulnerable, women and, indeed, all of us, by sending a message to the Government to say, “If you can’t accept this, come back with something, for now, that is better described, narrow and to the point, that we can enforce”. On that basis, I wish to test the opinion of the House.

Ayes 203, Noes 148.

Amendment 422D agreed.

Consideration on Report adjourned until not before 8.36 pm.