The purpose of this demonstration is to realize a proof-of-concept of the RoQME approach in a real-world example. It is a test-bed for current developments and provides valuable foundation for further developments.
The demonstration focuses on the run-time aspects of RoQME as you will see RoQME in action. The design-time part of RoQME and how RoQME will be presented to the user through tooling will be covered in an update to this demonstration (see roadmap).
The video below illustrates a typical intralogistics scenario. In this scenario the robot receives boxes from given stations and delivers them to different stations. The setting is equipped with a sensor detecting when a person enters or exits the robot working area. The video illustrates how the RoQME approach can provide metrics on non-functional properties such as safety or performance.
Disclaimer: This demonstration is a proof-of-concept of the technical feasibility. Please look at the roadmap to see how it is planned to advance this demonstration and to make it generally accessible via tooling.
The RoQME project aims to provide a comprehensive set of model-based tools enabling:
It is worth noting that RoQME enables modeling all kinds of system-level (rather than component-level) NFP, such as, safety, performance, reliability, resource consumption, user comfort, etc. It is also worth mentioning that RoQME does not prescribe how the computed metrics are used by other components, probably in a distributed manner, as the intrinsic cross-cutting nature of NFP may scatter and have impact both in the robot behaviour and architecture.
In the context of the RobMoSys technical user stories, this demonstration shows:
The following figure outlines the role, the models and the software artifacts contributed by RoQME and how they relate to those of RobMoSys. These contributions and the links to RobMoSys are briefly described next. Further details can be found at [Vicente2018a].
Domain Experts (Tier-2) identify and agree the specification of relevant domain-specific non-functional properties. The resulting Domain-Specific RoQME Models are made available in a RobMoSys repository for being reused by other roles at Tier-3.
QoS Engineers create Application-Specific RoQME Models that gather the non-functional aspects to be measured in the robotics system under development while, in parallel, the System Architect describes its functionality. The QoS Engineer may create application-specific RoQME models from scratch or by reusing and extending some of the domain-specific RoQME models already available in the RobMoSys repository. The models developed by the QoS Engineer specify (1) the non-functional properties considered relevant for the application being developed; (2) the contextual information available to the robot; and (3) a number of observations describing how (2) impacts (1). These observations will enable the estimation of the QoS metrics defined on the selected non-functional properties.
The QoS Engineer uses the RoQME Generator to create a QoS Metrics Provider component from the previous model. The resulting component is made available to the System Builder, who integrates it into the RobMoSys application architecture by (1) connecting it to the corresponding context providers, and (2) to any other component willing to use the computed metrics (if any).
The figure included below shows a screenshot of the system component architecture diagram, as modeled in the SmartMDSD Toolchain. The system was composed from previously developed software components.
The following picture shows the QoS Metrics Provider component used in the demonstration. The hull of this component was developed using the SmartMDSD Toolchain, while its internals were manually implemented using the support software libraries developed in RoQME for context monitoring, complex event processing, and metrics estimation. Further developments in RoQME will realize the full generation of this component based on the RoQME models (see roadmap). As illustrated in the picture, the component receives contexts from other components in the system (context providers) and processes them according to the RoQME approach (lower left). It then provides the estimated QoS metrics to other components for further processing or adaptation (e.g., by the Sequencer).
The RoQME model detailed next specifies: (1) the non-functional properties identified as relevant for the experiment and for which the RoQME runtime infrastructure will provide estimations (metrics) in the range [0, 1]; (2) the context data provided by the robot; and (3) the observations which, based on the specified context patterns, will more or less impact (reinforcing or undermining) the previous non-functional properties. This model uses a preliminary syntax, which is subject to improvement in the final textual model editor that is currently being developed in Rome.
It is worth mentioning that the reference value for SAFETY is 1, that is, a priori, the system is considered safe and, only if there is evidence against this belief, the value of the safety metric is reduced accordingly. The reference value of a property is its a-priori probability of being optimal, and is the value the property tends to when there is no evidence making it improve (reinforcing evidence) or worsen (undermining evidence). Note that most properties use a reference value equal to 0,5.
Derived context (from Job_State):
This derived context is calculated as the time passed since a job starts (Job_State::STARTED) until it finishes (Job_State::FINISHED).
The story starts with the robot in an IDLE state and placed in its initial position. There is an operator ready to handover boxes to the robot.
RoQME provides a visualization tool that displays at run-time: (1) the contextual information obtained from the robot; (2) the observations fired when the context patterns identified in the RoQME model are detected; and (3) the estimation of the metrics defined on the non-functional properties, as they vary according to the previous observations. The following screenshots show the graphics produced by the visualization tool during the experiment for the SAFETY (top) and the PERFORMANCE (bottom) non-functional properties, together with the observations with impact on their estimations. Labels (numbered circles) referring to the different story steps described in the previous section have been manually added to make the experiment timeline clearer.
RoQME also provides a module for recording the data generated while running real-world experiments (i.e., experiments that involve real robots). It records: a) the context data generated by the robot/s; b) the identified observations, and c) the QoS metric estimations. The recorded data can then be used for simulating the experiment as many times as needed without the need of using the real robot/s again. Besides, this information can also be used, for instance, for benchmarking purposes.
The core functionality of collecting contexts and processing metrics on non-functional properties is already operational in RoQME although, at the moment, the QoS metrics provider component is implemented manually.
The following items are currently being developed and are expected to be available and usable in the form of RoQME plugins for the SmartMDSD Toolchain. Its expected release date is March 2019. It will cover:
All the RoQME partners are pledged to making the knowledge generated in the course of the Project as widely and freely available as possible for subsequent research and development. For this reason, the RoQME partners are fully committed to open-access and open-source policies.
To discuss this demonstration, see: https://discourse.robmosys.eu/t/demonstration-dealing-with-metrics-on-non-functional-properties-in-robmosys-roqme-itp
This technical demonstration has been realized by the RoQME project team and the Ulm University of Applied Sciences. The methods and tools behind the estimation of the non-functional properties are being developed by the RoQME ITP (University of Extremadura, University of Málaga and Biometric Vox). The SmartMDSD Toolchain and the intralogistics use-case/pilot application are being developed by the Ulm University of Applied Sciences.
RoQME is an Integrated Technical Project (ITP) of EU H2020 RobMoSys (robmosys.eu). Ulm University of Applied Sciences is a RobMoSys core partner.
This activity has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 732410.
Fur further information and to learn more about the concepts used here, see:
You can follow the RoQME Project updates at:
[Espin2018] Espín, J. M., Font, R., Ingles-Romero, J. F. & Vicente-Chicote, C. Towards the Application of Global Quality-of-Service Metrics in Biometric Systems. IberSPEECH 2018, 21-23 November 2018, Barcelona (Spain). Download. Download
[Ingles2018a] Inglés-Romero, J. F., Spín, J. M., Jiménez, R., Font, R. & Vicente-Chicote, C. Towards the Use of Quality of Service Metrics in Reinforcement Learning: A Robotics Example. 5th International Workshop on MOdel-driven Robot Software Engineering (MORSE 2018), in conjunction with MODELS 2018, 15 October 2018, Copenhagen (Denmark). Download. Download
[Ingles2018] Inglés-Romero, J. F. How well does my robot work? RoQME: A project aimed at measuring quality of service in robotics. April 2018. Available at Medium.com. Download
[Lutz2019] Lutz, M., Inglés-Romero, J. F., Stampfer, D., Lotz, A., Vicente-Chicote, C., & Schlegel, C. (2019). Managing Variability as a Means to Promote Composability: A Robotics Perspective. In A. Rosado da Cruz, & M. Ferreira da Cruz (Eds.), New Perspectives on Information Systems Modeling and Design (pp. 274-295). Hershey, PA: IGI Global. DOI: 10.4018/978-1-5225-7271-8.ch012 Download
[Vicente2018a] Vicente-Chicote, C., Inglés-Romero, J. F., Martínez, J., Stampfer, D., Lotz, A., Lutz, M. & Schlegel, C. A Component-Based and Model-Driven Approach to Deal with Non-Functional Properties through Global QoS Metrics. 5th International Workshop on Interplay of Model-Driven and Component-Based Software Engineering (ModComp 2018), in conjunction with MODELS 2018, 14 October 2018, Copenhagen (Denmark). . Download
[Vicente2018b] Vicente-Chicote, C., Berrocal, J., García-Alonso, J. M., Hernández, J., Bandera, A., Martínez, J., Romero-Garcés, A., Font, R. & Inglés-Romero, J. F., (2018). RoQME: Dealing with Non-Functional Properties through Global Robot QoS Metrics. SISTEDES Conference, 17-19 September 2018, Sevilla (Spain). Download