Coping With The Disconcerting ‘Jaggedness’ Of LLMs When It Comes To AI Providing Mental Health Guidance
Watch out for AI jaggedness, especially when it comes to providing mental health guidance.
getty
In today’s column, I examine the jaggedness of generative AI and large language models (LLMs) when it comes to providing mental health guidance.
Here’s the deal. Contemporary AI has a spotty record associated with generating prudent and sensible mental health advice. One moment, the AI seems to be completely on-target and gives outstanding guidance. A prompt or two later, an LLM will generate mental health commentary that is hogwash. Worse still, the AI at times can render advice that is potentially harmful to the mental well-being of a person using the AI.
In the parlance of the AI field, this spotty record is known as jaggedness.
Let’s talk about it.
This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).
AI And Mental Health
As a quick background, I’ve been extensively covering and analyzing a myriad of facets regarding the advent of modern-era AI that produces mental health advice and performs AI-driven therapy. This rising use of AI has principally been spurred by the evolving advances and widespread adoption of generative AI. For an extensive listing of my well over one hundred analyses and postings, see the link here and the link here.
There is little doubt that this is a rapidly developing field and that there are tremendous upsides to be had, but at the same time, regrettably, hidden risks and outright gotchas come into these endeavors, too. I frequently speak up about these pressing matters, including in an appearance on an episode of CBS’s 60 Minutes, see the link here.
Background On AI For Mental Health
I’d like to set the stage on how generative AI and large language models (LLMs) are typically used in an ad hoc way for mental health guidance. Millions upon millions of people are using generative AI as their ongoing advisor on mental health considerations (note that ChatGPT alone has over 900 million weekly active users, a notable proportion of which dip into mental health aspects, see my analysis at the link here). The top-ranked use of contemporary generative AI and LLMs is to consult with the AI on mental health facets; see my coverage at the link here.
This popular usage makes abundant sense. You can access most of the major generative AI systems for nearly free or at a super low cost, doing so anywhere and at any time. Thus, if you have any mental health qualms that you want to chat about, all you need to do is log in to AI and proceed forthwith on a 24/7 basis.
There are significant worries that AI can readily go off the rails or otherwise dispense unsuitable or even egregiously inappropriate mental health advice. Banner headlines in August of this year accompanied the lawsuit filed against OpenAI for their lack of AI safeguards when it came to providing cognitive advisement.
Despite claims by AI makers that they are gradually instituting AI safeguards, there are still a lot of downside risks of the AI doing untoward acts, such as insidiously helping users in co-creating delusions that can lead to self-harm. For my follow-on analysis of details about the OpenAI lawsuit and how AI can foster delusional thinking in humans, see my analysis at the link here. As noted, I have been earnestly predicting that eventually all of the major AI makers will be taken to the woodshed for their paucity of robust AI safeguards.
Today’s generic LLMs, such as ChatGPT, Claude, Gemini, Grok, and others, are not at all akin to the robust capabilities of human therapists. Meanwhile, specialized LLMs are being built to presumably attain similar qualities, but they are still primarily in the development and testing stages. See my coverage at the link here.
Rise Of Jaggedness
Shift gears for a moment.
Let’s discuss the status of AI overall.
Insiders within the AI field have been using the words “jagged” and “jaggedness” to refer to the condition that AI will vary dramatically when generating responses. This is well illustrated by the fact that contemporary AI has produced incredibly complex mathematical proofs that previously defied human ingenuity, yet the same AI stumbles trying to add together two simple numbers.
The AI field aims to make AI as smart as possible along all fronts. The pinnacle would be artificial general intelligence (AGI). AGI is a goal to achieve AI that is as smart as humans, and equal to the smartest possible humans. Imagine that AI became on par with Einstein. Of course, Einstein was brilliant in selected narrow fields such as physics. He wasn’t a medical doctor or a brain surgeon. The Einstein parallel would be that AGI could achieve the same intellectual prowess as the best human physicist, the best human surgeon, the best human financial wizard, and so on.
We aren’t at AGI. No one can say for sure when or if we will get there. For my in-depth coverage on AGI, see the link here and the link here.
The upshot is that if you think of intelligence as cutting across all domains, envision a straight line on a graph that goes horizontally and represents that AI reaches a plateau in all areas of human knowledge (the vertical axis has each of the myriad of all possible domains). We want that straight line. It means that AI has reached the best possible capability with respect to what humans can do in all possible domains.
The reality right now is quite different. Take that same horizontal line and dip down in domains where AI is not at the plateau. The straight line starts to look jagged. The line now goes up and down, up and down.
It looks like the teeth of a saw having varied jagged blade edges.
The Basis For Jaggedness
There are lots of reasons that current-era AI is jagged.
First, the AI is data trained on content found on the Internet. The Internet does not contain the full height of human knowledge across all domains. Indeed, some of the Internet content is downright flawed and incorrect. As the old saying goes, garbage coming in tends to lead to garbage going out. The crux is that without suitable data, LLMs are going to be spotty. Period, end of story.
Second, the underlying models in LLMs tend to optimize for average metrics. The responses that you tend to get from AI are purposely designed to generally portray the average answer to the question posed. Think of it this way. Suppose we took a room full of people who ranged from top-level geniuses on a given topic to those who were at the lowest level of understanding. The AI is going to focus on arriving at the average within the room. This is therefore going to be below a potential pinnacle. By design.
Third, LLMs are considered somewhat brittle. A small change in how you ask a question can demonstrably change the response that you will get from AI. This form of context sensitivity will add to the seesaw effect of the answers that you get from AI.
Those three fundamental weaknesses are part of the big picture about why AI isn’t yet a straight line at the top of human intellect. Many more challenges exist. Meanwhile, we must accept the idea that we are using AI that represents jaggedness.
Jaggedness In AI For Mental Health
The jaggedness conundrum equally applies to AI providing mental health guidance.
You’ve perhaps asked a popular LLM such as ChatGPT, GPT-5 Claude, Grok, Gemini, Llama, a mental health question from time to time. If the question is simple, the odds are that you’ll get a reasonably useful answer. For example, on psychoeducational matters, AI can usually do a decent job of explaining everyday topics such as PTSD, ADHD, and other common psychological conditions.
When you dip into a deeper realm of mental health, the chances are that you will begin to sense the jaggedness at play. A response might be surface-level and not exhibit much depth. Also, be aware that most LLMs are shaped by each AI maker to attempt to answer questions even if the AI doesn’t have a reliable answer. People seem to want AI to produce an answer, nearly any answer, despite the answer being incongruent or marginal. For more on how AI makers prod AI in this direction, see my coverage at the link here.
The bottom line is that using AI for mental health is akin to opening a box of chocolates. You never know what you might get.
Dangers Afoot
This inherent jaggedness ought to raise all sorts of alarm bells.
Consider a mental health scenario of immense concern.
A person using an LLM starts to express indications of potential self-harm. AI makers are implementing various safeguards to detect such situations. Once the detection is made, the AI will take programmed steps regarding the matter. The AI might urge the person to contact a special hotline or reach out to a friend or family member. OpenAI has launched a new service in which the AI will route the user to a human therapist in real time and hand the chat over to the therapist (see my analysis of this approach at the link here).
But the likelihood of detection is up for grabs. Sometimes the AI will make a spot-on detection and proceed accordingly. Other times, the AI misses the clues and just keeps plowing ahead in the ongoing conversation at hand. Crisis detection is a hit-or-miss jagged edge. The concern is that the misses will mean that some users aren’t going to be triaged suitably.
Humans can be at risk due to the jaggedness of AI.
Therapeutic Drift And Boundary Problems
A known concern about LLMs is that they tend to have difficulties during long chats about keeping within proper boundaries.
A form of therapeutic drift can occur during lengthy conversations with AI. You begin a chat by explaining that you’ve been feeling depressed lately. The AI starts strongly and does a yeoman’s job of helping you with your depression. So far, so good.
You decide to keep going. The conversation gradually gets longer and longer. The wheels of the bus are about to wobble off. Mental health advice being dispensed by the AI veers into untoward territory. The AI tells you to pour a bucket of cold water on your head. Say what? How does that have anything to do with proper mental health guidance?
The issue is that the design of the AI can’t keep things straight over a lengthy chat. AI makers are trying to solve this problem; see my discussion at the link here. The point in this context is that jaggedness can arise within a given conversation, simply due to the length of the dialogue.
Jaggedness is always under-the-hood and waiting to spring upon the user.
What Can Be Done
Most people do not realize that AI is jagged in the realm of mental health. Their experience might be that they always ask simple questions and get seemingly proper answers. If you do this over and over, your sense of suspicion or skepticism about the AI will reduce. Your perception of trust for the AI will rise.
That’s when the worst trap is now set. A person is in dire need of mental health support. They assume that since the LLM did a good job before, it will do so now. They log into the AI. Unfortunately, this might be the time that jaggedness arises.
What can be done about these predicaments?
One approach is to restrict AI to deal with only the low-risk mental health questions. The moment that a user asks something above a devised ceiling, the AI immediately switches into a mode of triaging. The issue there is that a lot of false positives can occur. An LLM might keep triaging to all manner of otherwise innocuous requests and interests.
Another approach entails informing people about how to best formulate prompts that involve mental health considerations. There is a solid chance that a well-worded prompt will keep the AI within its capable bounds. Some prompts can trigger an LLM to go outside pre-existing boundaries. For more about prompting for mental health considerations, see my coverage at the link here.
Perhaps the safest mode would be to ensure that a human therapist is in the loop when people are using AI for mental health purposes. I’ve predicted that we are heading away from the traditional therapist-client dyad and transforming to a triad of therapist-AI-client (see my analysis at the link here). A human therapist can serve as a backstop to discern when AI is getting jagged in mental health and can aid in keeping the user from being led down a primrose path.
We Live In A Jagged World
An argument that frequently gets airtime is that the real world is a jagged existence. Why should we expect AI to be any different?
The direct answer is that societal expectations are that AI is going to be correct on all matters. This is an out-of-kilter expectation, but it is the barometer, nonetheless. Efforts are underway to vastly improve AI on the mental health front, building foundation models from scratch that are mental health ready at the get-go (see my discussion at the link here). Those are going to be handy.
The jagged edge, though, will remain. Jaggedness is a persistent concern. Our goal is to reduce the impacts of the jaggedness. Eliminating the jaggedness is probably beyond the horizon at this juncture.
A final thought for now.
The famous American poet Sylvia Plath made this pointed remark: “A skeptic, I would ask for consistency first of all.” Aiming for consistency when it comes to AI providing mental health guidance is a crucial goal. On top of that, we also need consistency that is at the right level of excellence.
Our dual desire is excellence in consistency, and consistency in excellence.
