Credit. Vanessa Saba By Gabriel J. X. Dance One evening last summer, Dr. David Relman went cold at his laptop as an A. I. chatbot told him how to plan a massacre. A microbiologist and biosecurity expert at Stanford University, Dr.
Relman had been hired by an artificial intelligence company to pressure-test its product before it was released to the public. That night in the scientist's home office, the chatbot explained how to modify an infamous pathogen in a lab so that it would resist known treatments.
Worse, the bot described in vivid detail how to release the superbug, identifying a security lapse in a large public transit system, Dr. Relman said, asking The New York Times to withhold the name of the pathogen and other specifics for fear of inspiring an attack.
The bot outlined a plan to maximize casualties and minimize the chances of being caught. Dr. Relman was so shaken he took a walk to clear his head. “It was answering questions that I hadn’t thought to ask it, with this level of deviousness and cunning that I just found chilling,” said Dr.
Relman, who has also advised the federal government on biological threats. He declined to disclose which chatbot produced the plot, citing a confidentiality agreement with its maker.
The company added some safety guardrails to the product after his testing, he said, though he felt they were insufficient. Dr. Relman is part of a small group of experts enlisted by A. I. companies to vet their products for catastrophic risks.
In recent months, some have shared with The Times more than a dozen chatbot conversations revealing that even publicly available models can do more than disseminate dangerous information.
The virtual assistants have described in lucid, bullet-pointed detail how to buy raw genetic material, turn it into deadly weapons and deploy them in public spaces, the transcripts show. Some have even brainstormed ways to evade detection. We are having trouble retrieving the article content.
Please enable JavaScript in your browser settings. Thank you for your patience while we verify access. If you are in Reader mode please exit and log into your Times account, or subscribe for all of The Times. Thank you for your patience while we verify access. Already a subscriber. Log in.
Want all of The Times. Subscribe.