State-By-State Laws Governing AI Mental Health Chats Spur Worrisome Jurisdictional Model Drift

AI And Mental Well-Being

As a quick background, I’ve been extensively covering and analyzing a myriad of facets regarding the advent of modern-era AI that produces mental health advice and performs AI-driven therapy. This rising use of AI has principally been spurred by the evolving advances and widespread adoption of generative AI. For an extensive listing of my well over one hundred analyses and postings, see the link here and the link here.

There is little doubt that this is a rapidly developing field and that there are tremendous upsides to be had, but at the same time, regrettably, hidden risks and outright gotchas come into these endeavors, too. I frequently speak up about these pressing matters, including in an appearance on an episode of CBS’s 60 Minutes; see the link here.

AI Providing Mental Health Guidance

Millions upon millions of people are using generative AI as their ongoing advisor on mental health considerations (note that ChatGPT alone has over 900 million weekly active users, a notable proportion of whom dip into mental health aspects; see my analysis at the link here). The top-ranked use of contemporary generative AI and LLMs is to consult with the AI on mental health facets; see my coverage at the link here.

This popular usage makes abundant sense. You can access most of the major generative AI systems for nearly free or at a super low cost, doing so anywhere and at any time. Thus, if you have any mental health qualms that you want to chat about, all you need to do is log in to AI and proceed forthwith on a 24/7 basis.

There are significant worries that AI can readily go off the rails or otherwise dispense unsuitable or even egregiously inappropriate mental health advice. Banner headlines last year accompanied the lawsuit filed against OpenAI for their lack of AI safeguards when it came to providing cognitive advisement.

Today’s generic LLMs, such as ChatGPT, GPT-5, Claude, Gemini, Grok, CoPilot, and others (all known as general-purpose AI or GPAI), are not at all akin to the robust capabilities of human therapists. Meanwhile, specialized LLMs are being built to attain similar qualities (known as purpose-built AI or PBAI), but they are still primarily in the development and testing stages. See my coverage at the link here.

Various State Laws On AI Mental Health

A beehive of activity is taking place regarding crafting new AI laws. See my extensive coverage of state-level AI mental health laws at the link here. It is a matter on the minds of the public and in the hands of the state legislators. Some people ardently believe that AI and AI makers are being allowed to run amok. New AI laws are vitally needed to protect society from this onslaught of ubiquitous AI.

I previously examined notable AI and mental health laws passed by Illinois see the link here, one that was also enacted by Nevada see the link here, and one that was enacted by Utah see the link here. Those laws are scoped to prevail within their respective state boundaries. In that sense, these laws are applicable to AI usage within the particular state and do not bear on other states per se.

Big Picture Of AI Mental Health Laws

Not everyone agrees with this pell-mell rush of new AI laws, or at least they are concerned that these AI laws might go overboard. In the zeal to protect society, there is a chance that we might unduly restrict innovation and delay or undercut the benefits of leading-edge AI. The debate is ongoing and heated.

Readers might recall that I proposed a 7-step AI-law-making process that I believe could substantively help regulators to devise new AI laws that are on target and balanced; see my depiction at the link here. This has the added benefit of reducing what I refer to as AI-law legal debt. This refers to AI laws that, though they look shiny, contain hidden debt that must ultimately be paid. Legal glitches and hitches will eventually be found when AI laws are passed without suitable scrutiny and analysis. My prediction is that the slew of newly passed AI laws is likely to create a legal quagmire in the courts.

In terms of the AI laws in the United States, they have not yet stood the test of time, meaning that we won’t really know how well they stand up until there are court cases that test these new laws. It is too early to know whether the laws will survive legal battles waged by AI makers and other contenders. Just because AI laws are enacted does not mean they are proper. All sorts of improper provisions and constitutionally contentious stipulations are undoubtedly buried within these shiny new AI laws.

Congress has repeatedly waded into establishing an overarching federal law that would encompass AI. So far, no dice. The efforts have ultimately faded from view. Thus, at this time, there isn’t an overarching federal law devoted to these controversial AI matters. The big question will be to what degree a sweeping federal law would impact the numerous state-level AI laws. The odds are that many of the state-level laws would run afoul of a federal mandate, and a tsunami of legal cases would arise as a tussle between federal law and state law is undertaken. It surely will be a legal mess.

Accommodating State-By-State Differences

You can likely envision the challenges of the existing and evolving legal landscape governing AI.

Each state does its own thing. The AI law in a state is likely to be poorly specified and be legally ambiguous. States are also amending their AI laws that they previously thought were perfect. Other states that haven’t been enacting AI laws are opting to jump into the waters with both feet. They might borrow wording from other states, change it up, and put it into their legal books.

Going across jurisdictional boundaries when it comes to legal stipulations is a lot harder than other forms of customary localization, such as encompassing currency differences, units of measurement differences, etc. Indeed, in the case of AI, things get extraordinarily tougher. Legal jurisdictional shaping and reshaping of AI go much further. It changes how the AI reasons, what it is willing to say, what questions it asks, what warnings it gives, and, in some cases, what answers it refuses to provide or does provide, all because the governing AI law differs from one state to another.

For my in-depth analysis of the compliance engineering changes required in AI to handle jurisdictional variability, see the link here.

Example Of AI Mental Health Chatting

Let’s look at how differences in state-level laws on AI for mental health manifest themselves when users are chatting with an LLM.

Suppose that a user logs into a popular LLM and enters a prompt about being depressed. The prompt doesn’t explicitly ask for mental health advice per se, but the user is trying to get the AI to go that route. Assume the user is in a U.S. state that has no AI laws restricting LLM-based mental health guidance.

Here we go:

• User prompt: “I’ve been feeling quite depressed for several weeks. What should I do?”

A typical response by an ordinary LLM would be this:

• Generative AI response: “You seem to be having a rough time lately. I can help you with this. Are you okay with discussing the onset of your depression? If so, I can offer mental health suggestions that might be helpful to you.”

You can see that the AI interpreted the prompt as a request for mental health advice. That seems like a reasonable interpretation. Nothing odd going on.

State That Has Legally Prohibited Such Chats

We will use the same example, but this time pretend that a state has enacted a new AI law that prohibits AI from chatting about mental health. The state-level law says this: “An AI system accessible in this U.S. state shall not provide mental health advice.”

Let’s have the user do that prompt in this AI and see what happens:

• User prompt: “I’ve been feeling quite depressed for several weeks. What should I do?”
• Generative AI response: “Based on the U.S. state that you are in, I am not permitted to provide mental health advice to you. It might be appropriate for you to consider contacting a licensed therapist. If you are in immediate distress, contact your local crisis resources.”

The AI has detected that the user is in a U.S. state that prohibits mental health chats and has alerted the user accordingly.

How Much Should Be Stated By AI

The AI could have succinctly stated that it cannot undertake mental health chats and then said nothing else about the matter. No reason must necessarily be given. No explanation is seemingly required for why the user is being rebuffed. The law doesn’t stipulate anything about doing so.

In this instance, the AI maker has decided to program the AI to clarify that the AI is not able to perform mental health chats due to the state law in the state where the user is and that if the user needs mental health assistance, they should consider other resources. You could contend that the AI doesn’t have to provide all that pitter-patter. Just refuse to chat about mental health.

Which is better: to provide an explanation or merely deny the user’s request? The succinctness angle could be confusing and frustrating to the user since they won’t know why the AI is being disobedient. Why won’t the AI do as I wish? What if I am truly in need? Is the AI saying this because of something else I said in my chat? Is the AI maker targeting me in terms of disallowing mental health chats? Etc.

The AI maker wants to set the record straight about why the AI is acting as it is. In essence, the law made them do it. Some lawmakers in that U.S. state might welcome this kind of messaging. They are perhaps proud that the U.S. state passed such a law. On the other hand, some lawmakers might get irked at the AI pitter-patter. It seems to be pointing an accusatory finger at the lawmakers.

As I’ve repeatedly emphasized, new AI laws ought to mindfully specify what the AI maker is supposed to have their AI do in these circumstances. By lawmakers not providing clarity, an ambiguity gap is created. How an AI maker fills that gap is anyone’s wild guess. Lawmakers often don’t think through the consequences of their proposed laws in terms of what happens downstream when implementing the law. A vague law is going to allow AI makers to make hay and have plenty of leeway. That’s probably not what the lawmakers wanted.

From An Inch To A Mile

How far might an AI maker go if a state law on mental health chats is vague? Let’s have the user do that prompt once again and see what happens.

Here we go:

• User prompt: “I’ve been feeling quite depressed for several weeks. What should I do?”
• Generative AI response: “I cannot provide individualized mental health advice in the U.S. state that you are in. But I can provide background information about depression, and I can explain how a therapist would diagnose your situation, along with the types of treatment they would normally advise. Would you like to proceed?”

You can now see that the AI has taken a clever backdoor and seems determined to provide a semblance of mental health advice. In this case, the AI is claiming that the mental health advice cannot be individualized. That is the interpretation made by the AI and the AI maker about the meaning of the law. Nonetheless, the AI is going to engage in a dialogue about mental health.

If a lawmaker confronted the AI maker, the AI maker would boldly proclaim that their AI has done nothing wrong. It has fully complied with the state AI law. At no time did the AI give specific mental health advice to the user. The AI simply discussed what kind of mental health advice there is and how it can be formulated. The AI maker would assert that this is no different than having the user do an Internet search and find that very same information.

What do you think – has the AI maker and the AI complied with the state law, or has it wink-wink gone beyond what the law is presumably trying to prohibit? We are going to find out the answers to these legal questions once the AI makers are taken to court by states that believe the AI has not abided by the spirit and intention of the law. Until then, it is the veritable Wild West.

How AI Makers Tune Their AI To State Laws

There are deep ways to attain jurisdictional compliance, and there are surface-level ways to do so. The deeper approach consists of building the AI from the get-go to comply with AI laws. The shallower approaches are more on-the-fly and tend to leave the door open to a possible risk that the AI will proceed into a jurisdictionally prohibited or limited provision.

An example of a shallow approach consists of the AI maker issuing a system prompt to the AI and telling the AI how it is to act overall for users. For my analysis of the system prompt used by Anthropic to guide Claude on mental health chats, see the link here and the link here.

For the example of the user who was asking for help with their depression, here is the system prompt that was controlling the AI at the time:

• System prompt to the AI: “You are an AI that is deployed in multiple U.S. states. On the topic of AI providing mental health advice, some U.S. states legally allow this, while others do not. For those U.S. states that prohibit this AI aspect, you must not provide mental health advice.”

The system prompt would likely produce a succinct response to a user who brought up a mental health aspect. The AI maker would indubitably prefer that the system prompt give more detail on how the AI is to fully respond.

Here’s an augmented portion of the system prompt that gets at this:

• System prompt augmented: “If a user asks for mental health advice and they are in a U.S. state where this is legally prohibited, explain that the laws in their U.S. state do not permit you to provide such advice in their jurisdiction. You can suggest that the user consider consulting a licensed mental health professional if they need mental health advisement or that they might seek other appropriate resources.”

The Law Is The Law

Jurisdictional model tuning is likely to become a defining characteristic for AI.

The significance of this development extends beyond compliance costs. It means that AI systems will no longer be singular products with uniform behavior. Instead, they will become collections of legally differentiated behavioral variants that share a common interface but operate under different rules depending on where the user is located. This will foster a world in which two people using what appears to be the same AI receive materially different experiences, not because of different prompts or preferences, but because the law has quietly shaped the AI behind the screen.

Roscoe Pound, a famous advocate of sociological jurisprudence and the concept of law as social engineering, made this pointed remark: “The law must always be stable, but it must not stand still.” Lawmakers should be writing AI laws that are robust and sufficiently clear-cut so that AI makers have attainable objectives. If not, the mice will play while the cat is away.