Medical AI Program Creates 40,000 Chemical Weapons in Six Hours

An artificial intelligence (AI) program created by computer scientists to help drug companies find new medications was slightly modified by programmers recently with frightening results. The AI program created 40,000 new chemical weapon compounds in only six hours of operating time.

Collaboration Pharmaceuticals Inc. is a North Carolina research company that recently published the results of a study in the industry journal Nature Machine Intelligence showing how AI technology designed to discover helpful medications can be manipulated to create new biochemical weapons.

They describe how a “thought experiment” using existing technology turned into a sobering “wake-up call.” The company has developed an AI machine learning system named MegaSyn, which is designed to analyze and filter out toxic chemical compounds from potential medically beneficial substances.

The research scientists decided to test the program by reversing its algorithm to isolate and develop toxic compounds rather than eliminating them. They used an open-source database to test their idea, instructing MegaSyn to look for compounds with properties similar to the known nerve agent VX, perhaps the most dangerous chemical weapon ever created.

VX attacks the human nervous system, paralyzing muscles and shutting down respiration. The tasteless and odorless chemical was used to assassinate Kim Jong-Nam, half-brother of North Korean dictator Kim Jong-Un.

MegaSyn was able to generate a copy of VX along with 40,000 new molecules consistent with known chemical weapons or potential weapons. The program rated some of the new compounds as more toxic to humans than any existing weapon.

The authors of the study wrote that by inverting the beneficial machine learning models they were able to turn the system into an extremely efficient creator of unbelievably deadly weapons. They pointed out that the better the original system is at screening out toxic materials, the better the modified system can isolate and intentionally create toxic compounds. They warned that their research could be easily replicated by researchers with evil intentions.

Senior scientist with Collaboration Pharmaceuticals Fabio Urbina said that anyone with scientific knowledge of chemistry and internet access to databases could accomplish the same results as their study. He said that combining simple generative models with open-source datasets could produce similar results.

Urbina added that discovering a model of a new compound does not translate to the capacity to develop it in the real world, which would present significant obstacles. Even so, he said that the results disturbed the group enough that they were hesitant about publishing the study. He said they decided to go forward in order to “get ahead” of the potential risks, and by raising awareness of the technology others will work on efficient ways to contain harmful uses.