Monitoring and control

The standardization of AI for monitoring and control tasks is currently in an active phase of development. In Russia, notable standards include a series of GOST regulations in the field of intelligent transport infrastructure management [1, 2, 3], situational video analytics [4], human behavior monitoring and intent prediction [5, 6], and recognition and classification of road transport objects [7, 8]. There's no doubt that in the coming years, both in Russia and abroad, a regulatory framework for the security of AI monitoring and control systems will be fully established.

The first stage of the AI system lifecycle involves preparing data for AI model training. The data must meet standard requirements, including accuracy, completeness, security, and unbiasedness [9-11]. Any deviation from these requirements can result in AI models with low accuracy, poor reliability, and weak generalization properties [10].

For AI systems that monitor people, requirements for confidentiality, ethics, and personal data protection are critically important and mandatory. The European Union holds the strictest position on this issue. Under the EU Artificial Intelligence Act (EU AI Act) [12], several AI applications related directly or indirectly to human monitoring are classified as unacceptable risks to citizens' rights and freedoms and are therefore banned in the EU [13]. Prohibited AI practices include:

The next stage in the AI system lifecycle is model creation and training, which should ideally take place in secure software environments using trusted software. It's worth noting the widespread use of open-source YOLO (You Only Look Once) models in video analytics [14, 15]. These are deep neural networks for object detection and recognition. It's essential to understand that "neural networks downloaded from the internet"—even with additional training or modification of the model—may contain personal data theft modules, triggers that induce undeclared behavior in the model, or other malicious functionalities. Only a thorough analysis of the topology and internal logic of such models can ensure their security.

During the training stage, AI models must be protected from input data attacks [16], including so-called adversarial attacks. For video analytics tasks, this latter type of attack is especially relevant. Most adversarial attacks aim to poison input images in order to alter the model's output while keeping the modified images visually indistinguishable from the originals. The sheer volume of successful attacks on input data, their diversity, and the ability to target specific types of neural networks present a major challenge for AI security methods—primarily adversarial training techniques [17, 18]

Once the model is created, it must undergo rigorous testing following a predefined methodology [10].For models whose vulnerabilities pose the greatest threats, external testing by AI security experts may also be required.

The key to ensuring the security of your AI solution is understanding its specific vulnerabilities and strictly following all AI security recommendations and methodologies.