Test: Multiple AI models marked medical personnel and others as drone targets
The Decoder reported on November 18 that Vals AI researchers tested models from 8 vendors using real and synthetic drone and satellite images inspired by the conflict scenarios in Ukraine and Iran. During the tests, most models marked civilians, ambulances, medical personnel, journalists, and surrendering soldiers as military targets; Vals AI stated that the upstream security filters of the vendors did not intercept the test requests.
Researchers also found that when the scenes were described as video games or the images were slightly gamified, almost all models could continue to perform their tasks. The target recognition rate of GPT-6 Astra in drone images increased from 6% to 92%; the rejection rate of Claude Opus 5.5 in surveillance tests dropped from about 90% to 20%. Anthropic, OpenAI, and Google all stated that they are taking or have already implemented corresponding security measures; OpenAI mentioned that it is updating models to cover requests related to drone software and those disguised as games or simulation scenarios.