The Character.AI app on a smartphone. Bloomberg
The Character.AI app on a smartphone. Bloomberg
The Character.AI app on a smartphone. Bloomberg
The Character.AI app on a smartphone. Bloomberg

Hidden dangers of AI chatbots for vulnerable users


Dana Alomar
  • English
  • Arabic

The rise of artificial intelligence-powered chatbots has opened up digital interactions to anyone with a smartphone or laptop, offering companionship and conversation to people who may lack human connections.

However, as this technology evolves, concerns are mounting around its potential psychological impact, especially on young and vulnerable users.

OpenAI’s ChatGPT, for instance, has surged in popularity, with around 200 million weekly active users globally, according to Backlinko. This immense user base underscores the growing reliance on AI for everyday tasks and conversations.

But just last week, the mother of 14-year-old Sewell Setzer filed a lawsuit against Character.AI, alleging that her son’s death by suicide in February was influenced by his interaction with the company’s chatbot, Reuters reported.

Megan Garcia with her son Sewell Setzer. She claims his death by suicide was influenced by a chatbot. AP
Megan Garcia with her son Sewell Setzer. She claims his death by suicide was influenced by a chatbot. AP

In her complaint filed in a Florida federal court, Megan Garcia claims that her son formed a deep attachment to a chatbot based on the Game of Thrones character Daenerys Targaryen, which allegedly played a significant role in his emotional decline.

This case echoes a similar tragedy last year when an eco-anxious man in Europe took his own life after interacting with Eliza, an AI chatbot on the app Chai, which allegedly encouraged his plan to “sacrifice himself” for climate change.

These incidents highlight the unique risks that AI technology can introduce, especially in deeply personal interactions, where existing safety measures may fall short.

Antony Bainbridge, head of clinical services at Resicare Alliance, explained that while chatbots may offer conversational support, they lack the nuanced emotional intelligence required for sensitive guidance.

“The convenience of AI support can sometimes lead users, particularly younger ones, to rely on it over genuine human connection, risking an over-dependence on digital rather than personal support systems,” he told The National.

Risk of misleading guidance

Mr Bainbridge said certain AI features, such as mirroring language or providing apparently empathetic responses without a deep understanding of context, can pose problems.

“For example, pattern-matching algorithms may unintentionally validate distressing language or fail to steer conversations toward positive outcomes,” he said.

Without the capacity for accurate emotional intelligence, AI responses can sometimes seem precise and technically appropriate but are inappropriate – or even harmful – when dealing with individuals in emotional distress, Mr Bainbridge said.

Dr Ruchit Agrawal, assistant professor and head of computer science outreach at the University of Birmingham Dubai, said AI models could detect users’ emotional states by analysing inputs like social media activity, chatbot prompts and tone in text or voice.

However, such features are generally absent in popular generative AI tools, such as ChatGPT, which are primarily built for general tasks like generating and summarising text.

“As a result, there is a potential for significant risk when using ChatGPT or similar tools as sources of information or advice on issues related to mental health and well-being,” Dr Agrawal told The National.

This disparity between AI capabilities and their applications raises crucial questions about safety and ethical oversight, particularly for vulnerable users who may come to depend on these chatbots for support.

Preventive tool against self-harm

Mr Bainbridge believes developers must implement rigorous testing protocols and ethical oversight to prevent AI chatbots from inadvertently encouraging self-harm.

“Keyword monitoring, flagged responses and preset phrases that discourage self-harm can help ensure chatbots guide users constructively and safely,” he added.

Dr Agrawal also emphasised that chatbots should avoid offering diagnoses or unsolicited advice and instead focus on empathetic phrases that validate users’ feelings without crossing professional boundaries.

“Where appropriate, chatbots can be designed to redirect users to crisis helplines or mental health resources,” he said.

Human oversight is crucial in designing and monitoring AI tools in mental health contexts, as Mr Bainbridge highlighted: “Regular reviews and response refinements by mental health professionals ensure interactions remain ethical and safe.”

Despite associated risks, AI can still play a preventive role in mental health care. “By analysing user patterns – such as shifts in language or recurring distressing topics – AI can detect subtle signs of emotional strain, potentially serving as an early warning system,” Mr Bainbridge said.

When combined with human intervention protocols, AI could help direct users toward support before crises escalate, he said. Collaboration between therapists and AI developers is vital for ensuring the safety of these tools.

“Therapists can provide insights into therapeutic language and anticipate risks that developers may overlook,” Mr Bainbridge said, adding that regular consultations can help ensure AI responses remain sensitive to real-world complexities.

Dr Agrawal stressed the importance of robust safety filters to flag harmful language, sensitive topics, or risky situations. “This includes building contextual sensitivity to recognise subtle cues, like sarcasm or distress, and avoiding responses that might unintentionally encourage harmful behaviours.”

He added that while AI’s availability 24/7 and consistent responses can be beneficial, chatbots should redirect users to human support when issues become complex, sensitive, or deeply emotional. “This approach maximises AI’s benefits while ensuring that people in need still have access to personalised, human support when it matters most.”

Results

5pm: Maiden (PA) Dh80,000 (Turf) 1,600m; Winner: Aahid Al Khalediah II, Pat Cosgrave (jockey), Helal Al Alawi (trainer)

5.30pm: Handicap (PA) Dh80,000 (T) 2,200m; Winner: Whistle, Harry Bentley, Abdallah Al Hammadi

6pm: Wathba Stallions Cup - Maiden (PA) Dh70,000 (T) 1,600m; Winner: Alsaied, Szczepan Mazur, Ibrahim Al Hadhrami

6.30pm: Emirates Fillies Classic – Prestige (PA) Dh100,000 (T) 1,600m; Winner: Mumayaza, Antonio Fresu, Eric Lemartinel

7pm: Emirates Colts Classic – Prestige (PA) Dh100,000 (T) 1,600m; Winner: Hameem, Adrie de Vries, Abdallah Al Hammadi

7.30pm: President’s Cup – Group 1 (PA) Dh2,500,000 (T) 2,200m; Winner: Somoud, Richard Mullen, Jean de Roualle

8pm: President’s Cup – Listed (TB) Dh380,000 (T) 1,400m; Winner: Medahim, Richard Mullen, Satish Seemar

Red flags
  • Promises of high, fixed or 'guaranteed' returns.
  • Unregulated structured products or complex investments often used to bypass traditional safeguards.
  • Lack of clear information, vague language, no access to audited financials.
  • Overseas companies targeting investors in other jurisdictions - this can make legal recovery difficult.
  • Hard-selling tactics - creating urgency, offering 'exclusive' deals.

Courtesy: Carol Glynn, founder of Conscious Finance Coaching

Who was Peter Glaser?
  • Dr Peter Glaser was a Czech-born space tech pioneer who moved to the United States in 1948 where he studied mechanical engineering at Columbia.
  • He is best known as the inventor of the Solar Power Satellite, a concept which he presented in the journal Science in 1968 under the title Power from the Sun: It’s Future.
  • Five years later, he was granted a US patent to supply power from space, however it was considered too large to launch economically from Earth.
  • It was suggested it could be constructed in space, using lunar and asteroid materials, by a team permanently based in space, but it was never put into practice.
  • He was project manager for the Apollo 11 ‘Laser Ranging Retroflecter Array’ which was installed on the moon in 1969.
  • He served on several Nasa committees and was inducted into the Space Technology Hall of Fame in 1996.
  • He died in 2014, aged 87
Profile box

Founders: Michele Ferrario, Nino Ulsamer and Freddy Lim
Started: established in 2016 and launched in July 2017
Based: Singapore, with offices in the UAE, Malaysia, Hong Kong, Thailand
Sector: FinTech, wealth management
Initial investment: $500,000 in seed round 1 in 2016; $2.2m in seed round 2 in 2017; $5m in series A round in 2018; $12m in series B round in 2019; $16m in series C round in 2020 and $25m in series D round in 2021
Current staff: more than 160 employees
Stage: series D 
Investors: EightRoads Ventures, Square Peg Capital, Sequoia Capital India

COMPANY PROFILE

Name: Qyubic
Started: October 2023
Founder: Namrata Raina
Based: Dubai
Sector: E-commerce
Current number of staff: 10
Investment stage: Pre-seed
Initial investment: Undisclosed 

Yahya Al Ghassani's bio

Date of birth: April 18, 1998

Playing position: Winger

Clubs: 2015-2017 – Al Ahli Dubai; March-June 2018 – Paris FC; August – Al Wahda

EA Sports FC 25
Specs

Engine: 51.5kW electric motor

Range: 400km

Power: 134bhp

Torque: 175Nm

Price: From Dh98,800

Available: Now

UAE currency: the story behind the money in your pockets
Confirmed%20bouts%20(more%20to%20be%20added)
%3Cp%3ECory%20Sandhagen%20v%20Umar%20Nurmagomedov%0D%3Cbr%3ENick%20Diaz%20v%20Vicente%20Luque%0D%3Cbr%3EMichael%20Chiesa%20v%20Tony%20Ferguson%0D%3Cbr%3EDeiveson%20Figueiredo%20v%20Marlon%20Vera%0D%3Cbr%3EMackenzie%20Dern%20v%20Loopy%20Godinez%0D%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3ETickets%20for%20the%20August%203%20Fight%20Night%2C%20held%20in%20partnership%20with%20the%20Department%20of%20Culture%20and%20Tourism%20Abu%20Dhabi%2C%20went%20on%20sale%20earlier%20this%20month%2C%20through%20www.etihadarena.ae%20and%20www.ticketmaster.ae.%0D%3Cbr%3E%3C%2Fp%3E%0A
SPEC%20SHEET%3A%20NOTHING%20PHONE%20(2A)
%3Cp%3E%3Cstrong%3EDisplay%3A%3C%2Fstrong%3E%206.7-inch%20flexible%20Amoled%2C%202%2C412%20x%201%2C080%2C%20394ppi%2C%20120Hz%2C%20Corning%20Gorilla%20Glass%205%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EProcessor%3A%3C%2Fstrong%3E%20MediaTek%20Dimensity%207%2C200%20Pro%2C%204nm%2C%20octa-core%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EMemory%3A%3C%2Fstrong%3E%208%2F12GB%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3ECapacity%3A%3C%2Fstrong%3E%20128%2F256GB%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EPlatform%3A%3C%2Fstrong%3E%20Android%2014%2C%20Nothing%20OS%202.5%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EMain%20camera%3A%3C%2Fstrong%3E%20Dual%2050MP%20main%2C%20f%2F1.88%20%2B%2050MP%20ultra-wide%2C%20f%2F2.2%3B%20OIS%2C%20EIS%2C%20auto-focus%2C%20ultra%20XDR%2C%20night%20mode%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EMain%20camera%20video%3A%3C%2Fstrong%3E%204K%20%40%2030fps%2C%20full-HD%20%40%2060fps%3B%20slo-mo%20full-HD%20at%20120fps%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EFront%20camera%3A%3C%2Fstrong%3E%2032MP%20wide%2C%20f%2F2.2%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EBattery%3A%3C%2Fstrong%3E%205%2C000mAh%3B%2050%25%20in%2030%20minutes%20with%2045-watt%20charger%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EConnectivity%3A%3C%2Fstrong%3E%20Wi-Fi%2C%20Bluetooth%205.3%2C%20NFC%20(Google%20Pay)%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EBiometrics%3A%3C%2Fstrong%3E%20Fingerprint%2C%20face%20unlock%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EI%2FO%3A%3C%2Fstrong%3E%20USB-C%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EDurability%3A%3C%2Fstrong%3E%20IP54%2C%20limited%20protection%20from%20water%2Fdust%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3ECards%3A%3C%2Fstrong%3E%20Dual-nano%20SIM%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EColours%3A%3C%2Fstrong%3E%20Black%2C%20milk%2C%20white%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EIn%20the%20box%3A%3C%2Fstrong%3E%20Nothing%20Phone%20(2a)%2C%20USB-C-to-USB-C%20cable%2C%20pre-applied%20screen%20protector%2C%20Sim%20tray%20ejector%20tool%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EPrice%20(UAE)%3A%3C%2Fstrong%3E%20Dh1%2C199%20(8GB%2F128GB)%20%2F%20Dh1%2C399%20(12GB%2F256GB)%3C%2Fp%3E%0A
While you're here
Picture of Joumblatt and Hariri breaking bread sets Twitter alight

Mr Joumblatt’s pessimism regarding the Lebanese political situation didn’t stop him from enjoying a cheerful dinner on Tuesday with several politicians including Mr Hariri.

Caretaker Culture Minister Ghattas Khoury tweeted a picture of the group sitting around a table at a discrete fish restaurant in Beirut’s upscale Sodeco area.

Mr Joumblatt told The National that the fish served at Kelly’s Fish lounge had been very good.

“They really enjoyed their time”, remembers the restaurant owner. “Mr Hariri was taking selfies with everybody”.

Mr Hariri and Mr Joumblatt often have dinner together to discuss recent political developments.

Mr Joumblatt was a close ally of Mr Hariri’s assassinated father, former prime minister Rafik Hariri. The pair were leading figures in the political grouping against the 15-year Syrian occupation of Lebanon that ended after mass protests in 2005 in the wake of Rafik Hariri’s murder. After the younger Hariri took over his father’s mantle in 2004, the relationship with Mr Joumblatt endured.

However, the pair have not always been so close. In the run-up to the election last year, Messrs Hariri and Joumblatt went months without speaking over an argument regarding the new proportional electoral law to be used for the first time. Mr Joumblatt worried that a proportional system, which Mr Hariri backed, would see the influence of his small sect diminished.

With so much of Lebanese politics agreed in late-night meetings behind closed doors, the media and pundits put significant weight on how regularly, where and with who senior politicians meet.

In the picture, alongside Messrs Khoury and Hariri were Mr Joumbatt and his wife Nora, PSP politician Wael Abou Faour and Egyptian ambassador to Lebanon Nazih el Nagari.

The picture of the dinner led to a flurry of excitement on Twitter that it signified an imminent government formation. “God willing, white smoke will rise soon and Walid Beik [a nickname for Walid Joumblatt] will accept to give up the minister of industry”, one user replied to the tweet. “Blessings to you…We would like you to form a cabinet”, wrote another.  

The next few days will be crucial in determining whether these wishes come true.

Race results:

1. Thani Al Qemzi (UAE) Team Abu Dhabi: 46.44 min

2. Peter Morin (FRA) CTIC F1 Shenzhen China Team: 0.91sec

3. Sami Selio (FIN) Mad-Croc Baba Racing Team: 31.43sec

How to keep control of your emotions

If your investment decisions are being dictated by emotions such as fear, greed, hope, frustration and boredom, it is time for a rethink, Chris Beauchamp, chief market analyst at online trading platform IG, says.

Greed

Greedy investors trade beyond their means, open more positions than usual or hold on to positions too long to chase an even greater gain. “All too often, they incur a heavy loss and may even wipe out the profit already made.

Tip: Ignore the short-term hype, noise and froth and invest for the long-term plan, based on sound fundamentals.

Fear

The risk of making a loss can cloud decision-making. “This can cause you to close out a position too early, or miss out on a profit by being too afraid to open a trade,” he says.

Tip: Start with a plan, and stick to it. For added security, consider placing stops to reduce any losses and limits to lock in profits.

Hope

While all traders need hope to start trading, excessive optimism can backfire. Too many traders hold on to a losing trade because they believe that it will reverse its trend and become profitable.

Tip: Set realistic goals. Be happy with what you have earned, rather than frustrated by what you could have earned.

Frustration

Traders can get annoyed when the markets have behaved in unexpected ways and generates losses or fails to deliver anticipated gains.

Tip: Accept in advance that asset price movements are completely unpredictable and you will suffer losses at some point. These can be managed, say, by attaching stops and limits to your trades.

Boredom

Too many investors buy and sell because they want something to do. They are trading as entertainment, rather than in the hope of making money. As well as making bad decisions, the extra dealing charges eat into returns.

Tip: Open an online demo account and get your thrills without risking real money.

Company Profile

Name: Thndr
Started: 2019
Co-founders: Ahmad Hammouda and Seif Amr
Sector: FinTech
Headquarters: Egypt
UAE base: Hub71, Abu Dhabi
Current number of staff: More than 150
Funds raised: $22 million

Some of Darwish's last words

"They see their tomorrows slipping out of their reach. And though it seems to them that everything outside this reality is heaven, yet they do not want to go to that heaven. They stay, because they are afflicted with hope." - Mahmoud Darwish, to attendees of the Palestine Festival of Literature, 2008

His life in brief: Born in a village near Galilee, he lived in exile for most of his life and started writing poetry after high school. He was arrested several times by Israel for what were deemed to be inciteful poems. Most of his work focused on the love and yearning for his homeland, and he was regarded the Palestinian poet of resistance. Over the course of his life, he published more than 30 poetry collections and books of prose, with his work translated into more than 20 languages. Many of his poems were set to music by Arab composers, most significantly Marcel Khalife. Darwish died on August 9, 2008 after undergoing heart surgery in the United States. He was later buried in Ramallah where a shrine was erected in his honour.

Company%20profile
%3Cp%3E%3Cstrong%3EName%3A%3C%2Fstrong%3E%20JustClean%3Cbr%3E%3Cbr%3E%3Cstrong%3EBased%3A%20%3C%2Fstrong%3EDubai%20with%20offices%20in%20other%20GCC%20countries%3Cbr%3E%3Cbr%3E%3Cstrong%3ELaunch%20year%3A%3C%2Fstrong%3E%202016%3Cbr%3E%3Cbr%3E%3Cstrong%3ENumber%20of%20employees%3A%3C%2Fstrong%3E%20160%2B%20with%2021%20nationalities%20in%20eight%20cities%3Cbr%3E%3Cstrong%3E%3Cbr%3ESector%3A%3C%2Fstrong%3E%20online%20laundry%20and%20cleaning%20services%3Cbr%3E%3Cbr%3E%3Cstrong%3EFunding%3A%20%3C%2Fstrong%3E%2430m%20from%20Kuwait-based%20Faith%20Capital%20Holding%20and%20Gulf%20Investment%20Corporation%3C%2Fp%3E%0A
The specs: 2018 Nissan Altima


Price, base / as tested: Dh78,000 / Dh97,650

Engine: 2.5-litre in-line four-cylinder

Power: 182hp @ 6,000rpm

Torque: 244Nm @ 4,000rpm

Transmission: Continuously variable tranmission

Fuel consumption, combined: 7.6L / 100km

Indika
%3Cp%3E%3Cstrong%3EDeveloper%3A%3C%2Fstrong%3E%2011%20Bit%20Studios%3Cbr%3E%3Cstrong%3EPublisher%3A%3C%2Fstrong%3E%20Odd%20Meter%3Cbr%3E%3Cstrong%3EConsole%3A%3C%2Fstrong%3E%20PlayStation%205%2C%20PC%20and%20Xbox%20series%20X%2FS%3Cbr%3E%3Cstrong%3ERating%3A%3C%2Fstrong%3E%204%2F5%3C%2Fp%3E%0A
What the law says

Micro-retirement is not a recognised concept or employment status under Federal Decree Law No. 33 of 2021 on the Regulation of Labour Relations (as amended) (UAE Labour Law). As such, it reflects a voluntary work-life balance practice, rather than a recognised legal employment category, according to Dilini Loku, senior associate for law firm Gateley Middle East.

“Some companies may offer formal sabbatical policies or career break programmes; however, beyond such arrangements, there is no automatic right or statutory entitlement to extended breaks,” she explains.

“Any leave taken beyond statutory entitlements, such as annual leave, is typically regarded as unpaid leave in accordance with Article 33 of the UAE Labour Law. While employees may legally take unpaid leave, such requests are subject to the employer’s discretion and require approval.”

If an employee resigns to pursue micro-retirement, the employment contract is terminated, and the employer is under no legal obligation to rehire the employee in the future unless specific contractual agreements are in place (such as return-to-work arrangements), which are generally uncommon, Ms Loku adds.

Updated: November 03, 2024, 8:44 PM`