Learning and Assessing Risks for Enhancing Dependability of Autonomous Socio- Technical Systems

  • Troubitsyna, Elena (Ansvarig forskare)
  • Vistbakka, Inna (CoI)
  • Majd, Amin (CoI)



Autonomous systems are increasingly integrated in our everyday life.

Autonomous vehicles of all kinds, variety of servo-, industrial and medical robots perform a wide spectrum of tasks alongside humans. It raises a serious concern regarding the risks associated with blending the autonomous technologies into the safety-critical activities and dependability of the resulting sociotechnical systems.

Currently, risk assessment and mitigation are typically performed at the design time. They focus on identifying hazards associated with the system and creating and verifying system design to ensure that the means for hazard prevention are correctly implemented. We argue that such an approach is inadequate for ensuring dependability of future autonomous sociotechnical systems because it is unable to cope with the inherent uncertainty, i.e., recognize and mitigate unpredictable dynamically emerging risks.

In this project, we aim at developing and validating a prototype of a platform for assessing and mitigating the risks of autonomous socio-technical systems. The platform, called a dynamic risk manager, will mimic two complementary self-preservation control mechanisms of humans ? the automatic immediate response (reflexes) triggered when the danger is present and long-term learning to predict and avoid danger. Our dynamic risk manager will enhance situation-awareness system capabilities and ensure that the system can guarantee safety in presence of uncertainty, continuous changes and evolution.

The dynamic risk manager will combine a formally verified emergency response functionality ? a component that should be activated to stop the hazard occurrence ? and a risk assessor. The risk assessor will combine learning and reasoning to analyse the emerging situations and at the run-time synthesise the strategy for preventing and mitigating the dynamically appearing hazards. Learning and reasoning implemented over the streams of heterogeneous data will enable run-time safety strategy synthesis. Formally verified models will be used at run-time to guarantee that the generated strategies preserve the safety constraints. The reinforced learning loop augmented with the run-time verification of safety constraints will ensure that during its functioning the autonomous system will continuously improve its risk assessment and mitigation capabilities.

The dynamic risk manager will be built on a heterogeneous hardware platform that will contain a critical kernel ? a highly available and reliable component to implement the emergency response functionality and a combination of components with high computing power, such as GPUs to analyse the data streams. The platform will be reconfigurable to ensure that it is capable to adapt to a variety of emerging situations, providing seamless real-time operation in every situation.

The proposed platform will be validated in two use cases: moving assistant robot and a heterogeneous swarm of robots cooperating with humans at an industrial production environment. The project pursues the ambitious scientific goals that have immediate practical application.

We believe that a strong expertise of the complementary consortium in the areas of safety-critical systems, machine learning, artificial intelligence and hardware design as well as wide academic and industrial collaboration network ensures the feasibility of the project plan.
Gällande start-/slutdatum01/01/1831/12/19
  • Multi-Layered Safety Architecture of Autonomous Systems: Formalising Coordination Perspective

    Vistbakka, I., Troubitsyna, E. & Majd, A., 2019, Proceedings of 2019 IEEE 19th International Symposium on High Assurance Systems Engineering (HASE). Dongjin, Y., Nguyen, V. & Jiang, C. (red.). IEEE, s. 58–65

    Forskningsoutput: Kapitel i bok/konferenshandlingKonferensbidragVetenskapligPeer review

    11 Citeringar (Scopus)
  • Deriving and Formalising Safety and Security Requirements for Control Systems

    Troubitsyna, E. & Vistbakka, I., 2018, Computer Safety, Reliability, and Security 37th International Conference, SAFECOMP 2018, Västerås, Sweden, September 19-21, 2018, Proceedings. Gallina, B., Skavhaug, A. & Bitsch, F. (red.). Springer, s. 107–122

    Forskningsoutput: Kapitel i bok/konferenshandlingKonferensbidragVetenskapligPeer review

    9 Citeringar (Scopus)
  • Deriving Mode Logic for Autonomous Resilient Systems

    Vistbakka, I., Majd, A. & Troubitsyna, E., 2018, Formal Methods and Software Engineering - 20th International Conference on Formal Engineering Methods, ICFEM 2018, Gold Coast, QLD, Australia, November 12-16, 2018, Proceedings. Sun, J. & Sun, M. (red.). Springer, s. 320–336

    Forskningsoutput: Kapitel i bok/konferenshandlingKonferensbidragVetenskapligPeer review

    4 Citeringar (Scopus)