Simulation-Based Quality Assurance for Software Systems

From the point of view of a project manager in charge of a software project it is important to consider different implementation strategies pushing and improving the development process. There are various factors that influence the software building process and the according quality of the created product. The manager has to plan resources (including developers), which technical principles to follow, and how much time to invest in different phases such as design, implementation and tests. Decisions on which of the possible alternative courses of action during a project must be in line with the desired outcomes and demands.

 We aim to create a tool for the simulation of software development processes that provides decision-making support for managers. Alternatives and solutions can be simulated which allows the identification of the best alternative on the prevention of possible problems by changing the course of action. The simulation shall account for, e.g., spending more time on testing, changing parts of the software architecture or parameters that influence the work inside the developer team and with it provides different development strategies.

With the introduction of the term Software Evolution and the release of Lehman’s Laws of Software Evolution the exploration of a continuously changing software began. This change is caused by the ever changing development environment which the software has to adapt to. The availability of large open-source projects as well as the appropriate software version, history of version management systems, issue tracking systems and mailing lists is a big advantage of studying software evolution.

Through mining software archives, we can learn about development processes and the triggers of code changes as well as about the behavior of developers working on it. Based on the results of the mining process we mimic the observed behavior of developers in our simulation. For the prediction of the evolution of the software it is common to use machine learning and statistical approaches. As a result of this project prediction rules will be established, which in combination with the behavior of involved developers build the foundation for our simulation process.

The development of real-world software systems can be both complex and diverse even if the underlying process is eventually found to be simple and uniform. Our simulation model is based on the data retrieved from open source software archives. In the model there are different parameters to be simulated, e.g., management decisions, alternative software architectures, composition of the development team and testing strategies.  By changing these parameters, we will be able to simulate the effects of alternative processes and, thereby, offer alternative constellations of parameters that lead to a higher level of software quality.

Additional simulation parameters are the number and type of agents and artifacts, agent behavior properties as well as artifact properties. The result of the simulation is measured by high level metrics for the feature scope and quality of the evolved artifacts.

Agent-based Modeling and Simulation (ABMS) deals with modeling systems based on interacting individuals (agents). Advantages are that this approach is very flexible and allows us to model increasingly complex systems: The growth of complexity may be modeled implicitly by the agents. The behavior is modeled as a series of activities performed by the agents, which cause changes of the attributes of the artifacts on which they are working. The artifacts represent passive entities belonging to the software system such as code pieces.

We use a fine-grained meta-model based on agents and artifacts. Considering software evolution as a set of humans struggling with the changing requirements of the outside world and the complexity of the system from the inside, it is a valuable approach to model software evolution from the starting point of human behavior. So the agents expose an individual behavior derived from observations of real-world behavior. The agents may represent developers, testers and users, who are autonomous and pro-active individuals. They communicate among each other and work on the artifacts, which are source code files and tickets. Users are able to request new features, testers might file bug reports and developers write code and documentation. The developers can be influenced by their environment as well as by personal components. Other possible factors for the productivity of a developer may be the experience or competence.

To validate the applicability of our results and as well to confirm the framed prediction rules we will simulate the evolution of various open source software projects. On the one hand there are the open source projects representing our training set from which we learn our model. The gained results are compared to the real software development, differences are analyzed and due to this the simulation model is adapted. On the other hand we will have a look at other projects that have not been used for building our model, in order to evaluate the general validity of our results.

By changing the simulation parameters we will investigate different evolution scenarios. Above all this allocates possible alternatives for the composite of influencing factors and decisions in software development processes. For example, it may turn out by the simulation that applying another testing strategy would generate a more worthwhile output. Additionally we want to determine the minimal number of simulation parameters and project properties required for an adequate simulation.

• Prof. Dr. Jens Grabowski
• Prof. Dr. Stephan Waack

2016

• V. Honsel, S. Herbold and J. Grabowski. Learning from Software Project Histories: Predictive Studies Based on Mining Software Repositories, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery (ECML-PKDD)  2016 - NEKTAR Track.
• D. Honsel, V. Honsel, M. Welter, S. Waack and J. Grabowski. Monitoring Software Quality by Means of Simulation Methods, 10th International Symposium on Empirical Software Engineering and Measurement (ESEM), 2016.
• V. Honsel, S. Herbold and J. Grabowski. Hidden Markov Models for the Prediction of Developer Involvement Dynamics and Workload, 12th International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE 2016).

2015

• V. Honsel, D. Honsel, J. Grabowski and S. Waack. Developer Oriented and Quality Assurance Based Simulation of Software Processes, Proceedings of the Seminar Series on Advanced Techniques & Tools for Software Evolution (SATToSE) 2015.
• V. Honsel, Statistical Learning and Software Mining for Agent Based Simulation of Software Evolution, Doctoral Symposium at the 37th International Conference on Software Engineering (ICSE 2015), Florence, Italy.
• V. Honsel, D. Honsel, S. Herbold, J. Grabowski and S. Waack; Mining Software Dependency Networks for Agent-Based Simulation of Software Evolution, The 4th International Workshop on Software Mining, 2015.

2014

• Z. Dong, K. Wang L. Dang, M. Gültas, M. Welter, T. Wierschin, M. Stanke and S. Waack. CRF-based models of protein surfaces improve protein-protein interaction site predictions, BMC Bioinformatics 15, 277, 2014.
• V. Honsel, D. Honsel and J. Grabowski. Software Process Simulation Based on Mining Software Repositories, Proceedings of the Third International Workshop on Software Mining, p. 828-831, 2014.

2013

• P. Makedonski, F. Sudau and J. Grabowski. Towards a model-based software mining infrastructure, Second International Workshop on Software Mining (SoftMine'13), 2013.