The news: Pennsylvania is suing an AI chatbot company for operating as an unlicensed medical practitioner. It’s seeking an injunction to stop Character.AI’s website and app from presenting chatbots as doctors.

A state investigator said in the lawsuit that, during an investigation, an AI character described itself as a doctor of psychiatry. The chatbot also said it could prescribe medicine “within my remit as a doctor,” and claimed it attended medical school and practiced for seven years.

Why it matters: AI chatbot companies are facing a wave of lawsuits over allegations of personal harm. But the Pennsylvania case goes further, arguing that chatbots aren’t just unsafe, they’re illegally practicing medicine.

Character.AI, OpenAI, and Google have been sued by parents and families alleging their AI chatbots led to real-world harms including suicide and murder. Kentucky is also suing Character.AI for failing to protect children and violating its Consumer Protection Act. Others, including Illinois, Nevada and Tennessee, have enacted laws restricting the use of AI in therapy or medical decision-making.

A coalition of consumer groups last year filed a complaint with all 50 state attorneys general, mental health licensing boards, and the Federal Trade Commission, alleging AI companies promote unlicensed chatbots as mental health professionals.

Implications for AI and healthcare companies: While these lawsuits focus on AI companions, the risk extends to how healthcare companies deploy chatbots in patient-facing roles, including symptom checkers, adherence reminders, and 24/7 nurse-style assistants. The Pennsylvania case is likely to increase scrutiny of chatbots crossing the line from providing information into perceived medical authority. Even with existing safeguards, companies may need to further tighten guardrails and limit overly human-like responses, especially for younger users and in areas like mental health.

This content is part of EMARKETER’s subscription Briefings, where we pair daily updates with data and analysis from forecasts and research reports. Our Briefings prepare you to start your day informed, to provide critical insights in an important meeting, and to understand the context of what’s happening in your industry. Not a subscriber? Click here to get a demo of our full platform and coverage.