Q&A: Risks Associated With

AI-Powered Autonomous Weapons

The Evolution of Military Technology

For decades, the military has used self-guided weapons, such as heat-guided missiles, torpedoes, and mines. These devices operate using basic reactive feedback, without requiring human intervention. However, the introduction of artificial intelligence (AI) has significantly transformed the field of weapon design.

Kanaka Rajan, an associate professor of neuroscience at Harvard Medical School, argues that AI-powered autonomous weapons mark a dangerous turning point in modern warfare. Along with her team, she warns that such weapons could directly threaten scientific progress and global security.

Increasing Use and Concerns

According to Rajan, AI-powered weapons are already being developed and deployed. These weapons often take the form of robots or drones, and their use is expanding rapidly. She fears that this trend could have wide-reaching consequences, particularly for nonmilitary AI research in academic and commercial settings.

In a position paper presented at the 2024 International Conference on Machine Learning, Rajan and her collaborators, Shayne Longpre, Riley Simmons-Edler, and Ryan Badman, outlined their concerns and proposed steps forward.

Why Neuroscientists Are Studying AI Weaponry

Rajan explained that her team became interested in this issue after witnessing widespread speculation about artificial general intelligence in 2023. They questioned whether these scenarios were exaggerated and began analyzing the military’s use of AI. Their research revealed major investments in autonomous weapons with potential global consequences.

The team also realized that military reliance on academic and commercial AI experts raises ethical challenges. Much like corporate-funded university research, military-sponsored AI projects create conflicts for researchers and institutional leaders.

Major Risks of AI in Warfare

Rajan identifies three primary risks associated with AI-powered weapons:

1. Increased Likelihood of War: By removing soldiers from direct danger, AI weapons may make it politically easier for nations to initiate conflict. The reduced human cost can lead to more frequent wars with higher overall casualties.
2. Suppression of Nonmilitary Research: As AI becomes a cornerstone of national defense, governments may impose restrictions on academic research. This could limit innovation in healthcare, science, and other civilian applications.
3. Shift of Responsibility: Military use of AI may blur accountability. As machines make critical decisions, it becomes harder to hold humans responsible for wartime actions.

Historical Parallels and Future Implications

Rajan draws comparisons to fields like rocketry and nuclear physics, which faced similar restrictions during the Cold War. If AI research follows this pattern, scholars may encounter censorship, travel restrictions, and the need for security clearances, severely hampering progress.

She warns that if AI becomes central to national defense, experts might be coerced into classified work. This could lead to intellectual stagnation and reduced collaboration across international research communities.

Ignored Threats and Misconceptions

Many people underestimate the dangers of AI in weapons design. Since 2023, global powers have rapidly adopted these technologies, often without adequate oversight. The lack of transparency from tech companies further complicates understanding and regulation.

Even when humans appear to have control over AI systems, their role may be minimal. In fast-paced conflict scenarios, reliance on opaque machine decisions becomes the norm. Rajan cautions that a mere appearance of “human-in-the-loop” oversight may mislead policymakers and researchers.

Urgent Research Questions

Although many AI algorithms used in weapons were initially created for civilian applications like self-driving cars, they now serve military purposes. Researchers must take responsibility for the ethical use of these tools and resist inappropriate military applications.

As AI becomes essential for defense, academic and corporate experts will face growing pressure to collaborate. Institutions must define their role in this shift and implement safeguards to maintain academic freedom and ethical standards.

Path Forward: Ethical Collaboration

Rajan suggests that universities adopt oversight measures similar to those for private sector funding. This includes seminars, internal policies, and ethical review processes for projects funded by defense agencies.

She emphasizes that while some partnerships with government and military are beneficial, others, like those with the tobacco or fossil fuel industry, have led to scientific compromise. Clear ethical frameworks are essential to avoid repeating these mistakes.

The Bottom Line

While some advocate for a total ban on military AI, Rajan acknowledges that this is unrealistic. AI’s utility in defense makes an international consensus on a ban unlikely.

Instead, she advocates for weapons that support human soldiers rather than replacing them. Maintaining human oversight is critical to preventing the most dangerous outcomes.

Finally, Rajan urges governments and institutions to regulate the most dangerous forms of AI weaponry as soon as possible. With strong ethical standards, societies can harness AI’s potential without compromising peace, safety, and scientific progress.