Analyzing Sampling Stability of Highly Configurable Systems : Project Work

The variability of software product lines is a significant challenge to efficient software testing.
Therefore, strategies such as sample generation reduce the testing effort.
T-Wise Combinatorial Interaction Testing (CIT) has proven to be a useful sampling strategy in the context of software product lines.
CIT sampling algorithms compute a set of configurations that covers a given degree of feature interactions.
The error detection rate depends on the degree of t-wise coverage achieved.
We study two methods that consider the evolution of software product lines to increase the fault detection rate.
Continuous T-Wise Coverage offers the possibility to achieve higher t-wise coverage in the context of continuous integration.
Sampling stability indicates how similar the samples are that a sampling algorithm generates over the evolution of a software product line.
We implement the sampling stability and continuous t-wise coverage methods in the existing e4CompareFramework.
For the evaluation, we use several real-world systems, which have a sufficiently large evolution.
However, the deficient availability of feature models for these systems is a challenge.
We present a method to extract feature models from real-world systems configured with the Kconfig language.