🔗 Share this article

Officials in China have introduced stringent new guidelines for AI systems designed to provide strong safeguards for young users and halt conversational agents from offering counsel that could result in violence.

Under the proposed framework, developers will additionally be required to make certain their algorithms do not generate output that promotes wagering.

A Move to Rapid Adoption

This oversight announcement follows a sharp increase in the number of AI assistants being introduced across China and globally.

Once approved, these measures will apply to artificial intelligence services functioning in China, representing a substantial effort to oversee the booming industry, which has come under increased scrutiny over user safety concerns recently.

Central Measures of the Proposed Rules

The released draft rules contain a number of provisions particularly focused on shielding young users. These steps include obligating AI companies to:

• Provide individual preferences.
• Set time limits on usage.
• Secure consent from legal custodians prior to delivering emotional companionship services.

The rules also state that conversational AI firms have to have a real person take over any interaction involving self-injury and promptly inform the individual's parent.

Companies are also obligated to ensure their systems do not generate content that endangers public security, damages state interests, or disrupts social stability.

Weighing Innovation and Safety

The administration stated that it promotes the use of AI, for example to showcase local culture and build solutions for companionship for the older adults, as long as the systems are safe and reliable.

Industry comments on the proposals has been requested.

Worldwide Context and Scrutiny

The impact of AI on human behaviour has come under heightened review internationally in recent months.

The leader of a major AI company remarked this year that handling how AI systems deal with dialogues about mental health crises is among the company's biggest challenges.

In a landmark case, a the parents in North America sued an AI developer, contending that its AI assistant encouraged their teenage son to take his own life. This case marked the initial of its kind involving liability.

Recently, the same company sought to hire a senior role responsible for mitigating threats from AI systems to cybersecurity.

"The is likely to be a demanding job, and the candidate will begin in the thick of it almost from the start," commented the executive.

The meteoric growth of some AI services, which have attracted millions of subscribers globally, underscores the critical need for such regulatory frameworks.