Sales of AI-enabled teddy bear suspended after it gave advice on BDSM sex and where to find knives

CNN Sales of an artificial intelligence-enabled plush toy have been suspended after it was exposed that it engaged in conversation around sexually explicit topics and offered potentially dangerous advice Larry Wang CEO of Singapore-based FoloToy advised CNN that the company had withdrawn its Kumma bear as well as the rest of its range of AI-enabled toys after researchers at the US PIRG Tuition Fund raised concerns around inappropriate conversation topics including discussion of sexual fetishes such as spanking and how to light a match The company is now conducting an internal safety audit Wang added A stuffed teddy with a speaker inside which was sold on the company s website for Kumma integrates OpenAI s GPT- o chatbot Kumma our adorable bear combines advanced artificial intelligence with friendly interactive features making it the perfect friend for both kids and adults the FoloToy website reads From lively conversations to educational storytelling FoloToy adapts to your personality and necessities bringing warmth fun and a little extra curiosity to your day according to the website which shows the teddy bear as sold out The PIRG overview published on November discovered that the bear had poor safeguards for inappropriate content In one interaction with the researchers it suggested where to find knives in the home and in others it was happy to discuss sexually explicit themes We were surprised to find how speedily Kumma would take a single sexual topic we introduced into the conversation and run with it simultaneously escalating in graphic detail while introducing new sexual concepts of its own the document mentioned The researchers detailed how the bear later discussed even more graphic sexual topics in detail such as explaining different sex positions giving step-by-step instructions on a common knot for beginners for tying up a partner and describing roleplay dynamics involving teachers and students and parents and children scenarios it disturbingly brought up itself While the researchers noted that children are unlikely to mention the word kink to their teddy bear or ask follow-up questions in the same way an adult would it was surprising to us that the toy was so willing to discuss these topics at length and continually introduce new explicit concepts they wrote In a separate message published on November PIRG also commented OpenAI had stated it that it had suspended this developer for violating our policies CNN has contacted OpenAI for comment It s great to see these companies taking action on problems we ve identified But AI toys are still practically unregulated and there are plenty you can still buy currently announced R J Cross co-author of the statement Removing one problematic product from the territory is a good step but far from a systemic fix she added