Towards automatic evaluation of the Quality-in-Use in context-aware software systems

Abstract

Context-aware systems adapt their services to the user's intentions and environment to improve the user experience. However, how to evaluate the quality of these systems in terms of user perception and context recognition is still an open problem. Our goal in this work is to evaluate the Quality-in-Use (QinU) for context-aware software systems according to the ISO/IEC 25010 standard and in an automated manner. This evaluation is oriented to be model-based, with domain specification and log data as input, while quality metrics and representations of users' behavior as output. In this process, we use probabilistic models to discover user patterns, heuristic metrics as QinU estimation, clustering techniques to obtain user profiles according to their QinU, and feature selection to identify relevant factors of context. We propose a framework for assessing the QinU in context-aware software systems called Framework for Assessing Quality-in-use of Software (FAQuiS). FAQuiS includes a set of models to represent all dimensions of context, a methodology to apply the quality analysis to any system, and a set of tools and metrics to support and automate the process. We seek to test the impact and ease of integration in the industry for this framework. A case study in a company allows us to validate the applicability in a real environment. We analyze the mechanisms that support the QinU evaluation in context-aware systems, the feasibility of the QinU quantification, and the suitability of the integration in companies. Compared to previous works, our proposal offers a novel data-driven approach with general-purpose and industrial viability. FAQuiS can be used as a solution to assess the QinU based on the ISO 25010 standard and the models of user behaviors in different contexts. This solution analyzes the context changes in the user interaction, can quantify the quality loss in these contexts, and does not require big efforts to be integrated into a software development process.