Testing the white box

3r33816. The development of high quality programs implies that the program and its parts are being tested. Classic unit testing involves breaking a large program into small blocks suitable for testing. Or, if the development of tests occurs in parallel with the development of code or tests are developed before the program (TDD - test driven development), then the program is initially developed in small blocks that meet the requirements of tests. 3r33838. 3r33814.  3r33825. 3r33816. One of the varieties of unit testing can be considered as propery-based testing (this approach is implemented, for example, in the libraries 3-3339. QuickCheck 3r3-3818., 3r3-311. ScalaCheck
). This approach is based on finding universal properties that should be valid for any input data. For example, 3r3155. serialization followed by deserialization should produce the same object 3r33416. . Or,
re-sorting should not change the order of the elements in the list 3r33416. . To test such universal properties in the above libraries, a random input data generation mechanism is supported. This approach works especially well for programs based on mathematical laws that serve as universal properties valid for a wide class of programs. There is even a library of ready-made mathematical properties - 3r317. discipline 3r33818. - allowing you to check the performance of these properties in new programs (a good example of test reuse). 3r33838. 3r33814.  3r33825. 3r33816. Sometimes it turns out that it is necessary to test a complex program without being able to disassemble it into independently verifiable parts. In this case, the program under test is
black
white box (white - because we have the opportunity to study the internal structure of the program). 3r33838. 3r33814.  3r33825. 3r33816. Under the cat described several approaches to testing complex programs with a single entrance with different degrees of complexity (involvement) and different degrees of coverage. 3r33838. 3r330. 3r33818. 3r33814.  3r33825. 3r33816. * 3r3154. In this article, we assume that the program under test can be represented as a pure function without an internal state. (Some of the considerations given below can also be applied if the internal state is present, but there is a possibility of resetting this state to a fixed value.) 3r31616. 3r33838. 3r33814.  3r33825. 3r340. Test bench (test bench)
3r33814.  3r33825. 3r33816. First of all, since only one function is being tested, the calling code of which is always the same, we do not need to create separate unit tests. All such tests would be the same with the accuracy of the input data and checks. It is quite enough to transfer the initial data in a cycle (3r3754. Input ) And check the results (3r3754. ExpectedOutput 3r3-3755.). In order for an error to be identified, it is possible to identify the problematic set of test data, all test data must be labeled (3r3754. Label 3r3755.). Thus, one set of test data can be represented as a triple:
3r33814.  3r33825. 3r33737. 3r33712. case class TestCase[A, B](label: String, input: A, expectedOutput: B) 3r33737. 3r33814.  3r33825. 3r33816. The result of a single run can be represented as TestCaseResult : 3r33814.  3r33825. 3r33737. 3r33712. case class TestCaseResult[A, B](testCase: TestCase[A, B], actualOutput: Try) 3r33737. 3r33814.  3r33825. 3r33816. (We present the result of the launch using 3r3754. Try To capture possible exceptions.) 3r33838. 3r33814.  3r33825. 3r33816. To simplify the run of all test data through the program under test, you can use an auxiliary function that will call the program for each input value: 3r31919. 3r33814.  3r33825. 3r33737. 3r33712. def runTestCases[A, B](cases: Seq[TestCase[A, B]) (f: A => B): Seq[TestCaseResult[A, B]]= 3r33825. cases 3r33825. .map {testCase =>
TestCaseResult (testCase, 3r33825. Try {f (testCase.input)}
) 3r33825.}
.filter (r => r.actualOutput! = Success (r.testCase.expectedOutput)) 3r33737. 3r33814.  3r33825. 3r33816. This helper function will return problem data and results that differ from those expected. 3r33838. 3r33814.  3r33825. 3r33816. For convenience, you can format test results 3r33814.  3r33825. 3r33737. 3r33712. def report (results: Seq[TestCaseResult[_, _]]): String =
s "Failed $ {results.length}: n" +
results
.map (r => r.testCase.label + ": expected" + r.testCase.expectedOutput + ", but got" + r.actualOutput)
.mkString ("n") 3r33737. 3r33814.  3r33825. 3r33816. and display the report only in case of errors: 3r33838. 3r33814.  3r33825. 3r33737. 3r33712. val testCases = Seq (
TestCase ("1", ? 0)
) 3r33825. 3r33825. test ("all test cases") {3r33825. val testBench = runTestCases (testCases) _
val results = testBench (f)
assert (results.isEmpty, report (results)) 3-333825.} 3r33737. 3r33814.  3r33825.

Preparing the input

3r33814.  3r33825. 3r33816. In the simplest case, you can manually create test data to test the program, write it directly in the test code, and use it, as demonstrated above. It often turns out that interesting cases of test data have much in common and can be represented as some basic instance, with a few changes. 3r33838. 3r33814.  3r33825. 3r33737. 3r33712. val baseline = MyObject () //input object can be created manually or generated
val testCases = Seq (
TestCase ("baseline", baseline, ???),
TestCase ("baseline + (field1 = 123)", baseline.copy (field1 = "123"), ???) 3r33838 .) 3r3755. 3r33737. 3r33814.  3r33825. 3r33816. When working with nested immutable data structures, lenses are a great help, for example, from the 3-3-3150 library. Monocle : 3r33814.  3r33825. 3r33737. 3r33712. val baseline = ??? 3r33825. val testObject1 = (field1 composeLens field2) .set ("123") (baseline)
//which is equivalent to the following line:
val testObject1 = baseline.copy (field1 = baseline.field1.copy (field2 = "123")) 3r33737. 3r33814.  3r33825. 3r33816. Lenses allow you to elegantly "modify" deeply nested parts of data structures: Each lens is a getter and setter for one property. Lenses can be connected and get lenses "focusing" on the next level. 3r33838. 3r33814.  3r33825. 3r3168. Use DSL to submit changes to r3r3813. 3r33814.  3r33825. 3r33816. Next, we will consider the formation of test data by making changes to some initial input object. Usually a few changes are required to get the test object we need. It is very useful to include a list of changes in the textual description of the TestCase: 3r-3819. 3r33814.  3r33825. 3r33737. 3r33712. val testCases = Seq (
TestCase ("baseline", baseline, ???),
TestCase ("baseline +" +
"(field1 = 123) +" + //description of the 1st change 3r33825. " (field2 = 456) + "+ //of the 2nd 3r3-3825." (field3 = 789) ", 3rd of the 3rd 3r3255. baseline 3r3-3825. .copy (field1 =" 123 ") //1st change of 3r325. .copy (field2 = "456") //2nd change
.copy (field3 = "789"), //3rd change
???) 3r32525.) 3r3755. 3r33737. 3r33814.  3r33825. 3r33816. Then we will always know for which test data the test is performed. 3r33838. 3r33814.  3r33825. 3r33816. So that the text list of changes does not disagree with the actual changes, it is necessary to follow the principle of the "unified version of the truth." (If the same information is required /used at several points, then you should have a single primary source of unique information, and all other points of use should be distributed automatically, with the necessary conversions. If this principle is violated and the information is copied manually, then inevitably discrepancy between the versions of information at different points. That is, in the description of the test data we will see one thing, and in the test data - another. For example, by copying the change to 3r3754. field2 = "456" and correcting it in 3r???. field3 = "789" We may accidentally forget to correct the description. As a result, the description will reflect only two changes out of three.) 3r33814.  3r33825. 3r33816. In our case, the primary source of information is the changes themselves, or rather, the source code of the program that makes the changes. We would like to derive from them the text describing the changes. Offhand, as a first option, you can suggest using a macro that will capture the source code of the changes, and use the source code as documentation. This seems to be a good and relatively uncomplicated way to document actual changes, and it may well be applied in some cases. Unfortunately, if we submit changes in plain text, we lose the ability to perform meaningful transformations of the list of changes. For example, to detect and eliminate duplicate or overlapping changes, draw up a list of changes in a convenient way for the end user. 3r33838. 3r33814.  3r33825. 3r33816. To be able to operate with changes, it is necessary to have their structured model. The model should be expressive enough to describe all the changes we are interested in. Part of this model, for example, will be addressing the fields of objects, constants, assignment operations. 3r33838. 3r33814.  3r33825. 3r33816. The change model should allow to solve the following tasks: 3r33814.  3r33825. 3r33775.  3r33825. Generating instances of the change model. (That is, in fact, the creation of a specific list of changes.) 3r3787.  3r33825. Formation of the text description of changes. 3r3r7787.  3r33825. Apply changes to domain objects. 3r3r7787.  3r33825. Perform optimization transformations on the model. 3r3r7787.  3r33825. 3r33814.  3r33825. 3r33816. If a universal programming language is used to make changes, it can be difficult to represent these changes in the model. In the source code of the program can be used complex constructions that are not supported by the model. Such a program to change the fields of an object can use secondary patterns, like lenses or the 3r3754 method. copy which are lower-level abstractions relative to the level of the model of change. As a result, additional analysis of such patterns may be required to display instances of changes. Thus, initially a good option using a macro is not very convenient. 3r33838. 3r33814.  3r33825. 3r33816. Another way to create instances of a change model can be a specialized language (DSL), which creates change model objects using a set of extension methods and auxiliary operators. Well, in the simplest cases, instances of the change model can be created directly, via constructors. 3r33838. 3r33814.  3r33825. 3r33424. 3r33434. Details language changes
3r33427. 3r33816. The language of change is a rather complex construction that includes several components, which are also, in turn, non-trivial. 3r33838. 3r33814.  3r33825. 3r33775.  3r33825. Data structure model 3r3r7787.  3r33825. Model of change. 3r3r7787.  3r33825. Actually Embedded (?) DSL - auxiliary constructions, extension-methods, for convenient design of changes. 3r3r7787.  3r33825. A change interpreter that allows you to actually "modify" the object (in fact, of course, create a modified copy). 3r3r7787.  3r33825. 3r33814.  3r33825. 3r33816. Here is an example of a program recorded using DSL: 3r33819. 3r33814.  3r33825. 3r33737. 3r33712. val target: Entity[Target]//object being modified
val updateField1 = target field1: = "123"
val updateField2 = target subobject field2: = "456"
//or, without using DSL:
val updateField1 = SetProperty (PropertyAccess (target, Property (field? typeTag[String])), LiftedString ("123"))
val updateField2 = SetProperty (PropertyAccess (PropertyAccess (target, 3r33825. Property (subobject, typeTag[SubObject])), 3r33825. Property (field? typeTag[String])), LiftedString ("456")) 3r33737. 3r33814.  3r33825. 3r33816. That is, using the extension-methods 3r3755. and 3r3754. : = objects are formed. PropertyAccess , 3r3754. SetProperty from previously created objects 3r3754. target , 3r3754. field1 3r3755. , 3r3754. subobject 3r3755. , 3r3754. field2 3r3755. . Also due to the (dangerous) implicit conversions, the string "123" is packaged in 3r3754. LiftedString (you can do without implicit conversions and call the corresponding method explicitly: lift ("123") ). 3r33838. 3r33814.  3r33825. 3r33816. A typed ontology can be used as a data model (see [url]Https://habr.com/post/229035/3r33818.[/url] And [url]Https://habr.com/post/222553/[/url] ). (In brief: objects-names are declared that represent properties of a particular domain type: 3r3754. Val field1: Property[Target, String]3r3755.) In this case, the actual data can be stored, for example, in the form of JSON. The convenience of a typed ontology in our case is that the change model usually operates with separate properties of objects, and the ontology just gives a suitable tool for addressing properties. 3r33838. 3r33814.  3r33825. 3r33816. To present the changes, a set of classes of the same plan as the above class 3r3754 is needed. SetProperty : 3r33814.  3r33825.  3r33825. 3r3754. Modify - the use of the function, 3r3787.  3r33825. 3r3754. Changes - applying several changes afterwardspreferably 3r3787.  3r33825. 3r3754. ForEach - Apply changes to each element of the collection,  3r33825. etc. 3r3r7787.  3r33825. 3r33333. 3r33814.  3r33825. 3r33816. The change language interpreter is a regular recursive expression evaluator based on PatternMatching. Something like: 3r33814.  3r33825. 3r33737. 3r33712. def eval (expression: DslExpression, gamma: Map[String, Any]): Any = expression match {
case LiftedString (str) => 3r33825. str 3r33825. case PropertyAccess (obj, prop) => 3r32525. Getter (prop) (gamma) .get (obj)
}
def change[T](expression: DslChangeExpression, gamma: Map[String, Any], target: T): T = expression match {
case SetProperty (path, valueExpr) => 3r33838. val value = eval (valueExpr, gamma)
Setter (path) (gamma) .set (value) (target)
} 3r33737. 3r33814.  3r33825. 3r33816. To directly operate the properties of objects, it is necessary for each property used in the change model, set getter and setter. This can be achieved by filling in the mapping ( Map ) Between the ontological properties and the corresponding lenses. 3r33838.
3r33814.  3r33825. 3r33816. This approach generally works, and really allows you to describe the changes once, but gradually there is a need to introduce more and more complex changes and the model of changes is growing somewhat. For example, if you need to change a property using the value of another property of the same object (for example, 3r3754. Field1 = field2 + 1 3r3755.), Then you need to support variables at the DSL level. And if the property change is nontrivial, then at the DSL level, support of arithmetic expressions and functions will be required. 3r33838. 3r33814.  3r33825. 3r33383. Testing branches 3r33814.  3r33825. 3r33816. The code under test can be linear, and then by and large we only need one set of test data to see if it works. In the case of a branch (3r3754. If-then-else ), You must run the white box at least twice with different input data so that both branches are executed. The number of sets of input data sufficient to cover all the branches seems to be numerically equal to the cyclomatic complexity of the code with branches. 3r33838. 3r33814.  3r33825. 3r33816. How to generate all the input data sets? Since we are dealing with a white box, we can isolate the branch conditions and double-modify the input object so that in one case one branch is fulfilled, in the other case the other. Consider an example: 3r33838. 3r33814.  3r33825. 3r33737. 3r33712. if (object.field1 == "123") A else B 3r33737. 3r33814.  3r33825. 3r33816. Having such a condition, we can form two test cases: 3r33814.  3r33825. 3r33737. 3r33712. val testCase1 = TestCase ("A", field1.set ("123") (baseline), /* result of A * /) 3r32525. val testCase2 = TestCase ("B", field1.set (/* not "123", for example, * /"123" + "1">) (baseline), /* result of B * /) 3r3755. 3r33737. 3r33814.  3r33825. 3r33816. 3r3155. (If one of the test scenarios cannot be created, then we can assume that a dead code has been detected, and the condition along with the corresponding branch can be safely removed.) 3r31616. 3r33838. 3r33814.  3r33825. 3r33816. If independent properties of an object are checked in several branches, then it is quite easy to form an exhaustive set of modified test objects that completely covers all possible combinations. 3r33838. 3r33814.  3r33825. 3r33424. 3r33434. DSL to form all combinations of changes [/b] 3r33427. 3r33816. Let us consider in more detail the mechanism that allows you to create all possible lists of changes that provide full coverage of all branches. In order to use the list of changes when testing, we need to combine all the changes into one object, which we give to the input of the code being tested, that is, support for the composition is required. To do this, you can either use the above DSL to simulate changes, and then a simple list of changes is enough, or you can present one change as a function of the modification 3r3754. T => T : 3r33814.  3r33825. 3r33737. 3r33712. val change1: T => T = field1.set ("123") (_) 3r33825. //val change1: T => T = _.copy (field1 = "123") 3r32525. val change2: T => T = field2.set ("456") 3r3755. 3r33737. 3r33814.  3r33825. 3r33816. then the chain of changes will simply be a composition of functions: 3r33819. 3r33814.  3r33825. 3r33737. 3r33712. val changes = change1 compose change2 3r33737. 3r33814.  3r33825. 3r33816. or, for a list of changes: 3r33814.  3r33825. 3r33737. 3r33712. val rawChangesList: Seq[T => T]= Seq (change? change2)
val allChanges: T => T = rawChangesList.foldLeft (identity) (_ compose _) 3r33737. 3r33814.  3r33825. 3r33816. To compactly record all changes that correspond to all possible branches, you can use DSL of the next level of abstraction, which simulates the structure of the white box under test: 3r31919. 3r33814.  3r33825. 3r33737. 3r33712. val tests: Seq[(String, T => T)]= 3r33825. IF ("field1 == '123'") //the name of the condition we are modeling 3r33825. THEN (field1.set ("123")) (//or target field1: = "123" 3r32525. IF ("field2 == '456') 3r32525. THEN (field2.set (" 456 ")) (TERMINATE)
ELSE (field2.set ("456" + "1")) (TERMINATE)
) 3r3-3825. ELSE (field1.set ("123" + "1")) (TERMINATE) 3r3755. 3r3163716. 3r3-33814.  3r33825. 3r33816. Here is a collection of 3r3754. tests contains aggregated changes corresponding to all possible combinations of branches. Parameter type 3r3754. String 3r3755. will contain all the names of the conditions and all the descriptions of the changes that form the aggregated function of the changes. And the second element of the pair is type T => T - just the aggregated function of changes resulting from the composition of individual changes. 3r33838. 3r33814.  3r33825. 3r33816. To get the changed objects, you need to apply all the aggregated functions of the changes to the baseline object: 3r33838. 3r33814.  3r33825. 3r33737. 3r33712. val tests2: Seq[(String, T)]= tests.map (_. map_2 (_ (baseline))) 3r33737. 3r33814.  3r33825. 3r33816. As a result, we get a collection of pairs, and the string will describe the changes applied, and the second element of the pair will be the object in which all these changes are combined. 3r33838. 3r33814.  3r33825. 3r33816. Based on the structure of the model of the tested code in the form of a tree, the lists of changes will be paths from the root to the sheets of this tree. Thus, a significant part of the changes will be duplicated. You can get rid of this duplication by using the DSL option, in which the changes are directly applied to the baseline object as you move through the branches. In this case, slightly less unnecessary calculations will be made. 3r33838. 3r33814.  3r33825. 3r33510. Automatic generation of test data 3r33814.  3r33825. 3r33816. Since we are dealing with a white box, we can see all the branches. This makes it possible to build a model of logic contained in a white box and use the model to generate test data. In case the code under test is written in Scala, you can, for example, use 3r3515. scalameta to read the code, and then convert it to a logic model. Again, as in the previously discussed issue of modeling the logic of change, it is difficult for us to simulate all the capabilities of a universal language. Further we will assume that the code under test is implemented using a limited subset of the language, either in another language or DSL, which is initially limited. This allows you to focus on those aspects of the language that are of interest to us. 3r33838. 3r33814.  3r33825. 3r33816. Consider an example of code containing a single branch: 3r33814.  3r33825. 3r33737. 3r33712. if (object.field1 == "123") A else B 3r33737. 3r33814.  3r33825. 3r33816. The condition splits the set of field values ​​for field1 3r3755. into two equivalence classes: == "123" and 3r3754. ! = "123" . Thus, the entire set of input data is also divided into two equivalence classes with respect to this condition - 3r3754. ClassCondition1IsTrue and 3r3754. ClassCondition1IsFalse . From the point of view of completeness of coverage, it is enough for us to take at least one example from these two classes in order to cover both branches of 3r3754. A and 3r3754. B . For the first class, we can build an example, in a certain sense, uniquely: take a random object, but change the field 3r3754. field1 3r3755. on 3r3754. "123" . In this case, the object must be in the equivalence class ClassCondition1IsTrue and the calculations will go along the branch. A . For the second class there are more examples. One way to generate some kind of second-class example is to generate arbitrary input objects and discard those that have 3r3754. field1 == "123" 3r3755. . Another way: take a random object, but change the field field1 3r3755. on 3r3754. "123" + "*" (for modification, you can use any change in the control line, ensuring that the new line is not equal to the control line). 3r33838. 3r33814.  3r33825. 3r33816. 3r3-3563 is quite suitable as random data generators. 3r3754. Arbitrary generator mechanism. and 3r3754. Gen 3r3755. from the ScalaCheck library . 3r33838. 3r33814.  3r33825. 3r33816. In essence, we do appeal 3r3r744. The Boolean function used in the operator 3r3754. if . That is, we find all the values ​​of the input object for which this boolean function takes the value 3r3754. true - 3r3754. ClassCondition1IsTrue , and all values ​​of the input object for which it takes the value false - 3r3754. ClassCondition1IsFalse . 3r33838. 3r33814.  3r33825. 3r33816. Similarly, it is possible to generate data that is suitable for constraints generated by simple conditional operators with constants (more /less constants, is included in the set, begins with a constant). Such conditions are not difficult to reverse. Even if simple functions are called in the code under test, we can replace their call with their definition (inline) and, nevertheless, reverse the conditional expressions. 3r33838. 3r33814.  3r33825. 3r? 3592. Difficult reversible functions 3r33814.  3r33825. 3r33816. The situation is different when the condition uses a function that is difficult to reverse. For example, if a hash function is used, then it is not possible to automatically generate an example that gives the desired hash code value. 3r33838. 3r33814.  3r33825. 3r33816. In this case, you can add an additional parameter to the input object that represents the result of the function calculation, replace the function call with a call to this parameter, and update this parameter, regardless of the violation of the functional connection: 3r31919. 3r33814.  3r33825. 3r33737. 3r33712. if (sha (object.field1) == "a9403 ")
//after the introduction of an additional parameter ==>
if (object.sha_field1 == "a9403 ")
3r33737. 3r33814.  3r33825. 3r33816. An additional parameter allows to ensure the execution of code inside a branch, but, obviously, it can lead to virtually incorrect results. That is, the program being tested will produce results that can never be observed in reality. Nevertheless, verification of a part of the code that is otherwise unavailable to us is still useful and can be considered as a type of unit testing. After all, during unit testing, the subfunction is called with such arguments that may never be used in the program. 3r33838. 3r33814.  3r33825. 3r33816. With such manipulations we replace (replace) the object of testing. However, in a sense, the newly built program includes the logic of the old program. Indeed, if as the values ​​of the new artificial parameters we take the results of calculating the functions that we replaced with the parameters, then the program will produce the same results. Apparently, testing a modified program may still be of interest. We just need to remember under what conditions the changed program will behave in the same way as the original one. 3r33838. 3r33814.  3r33825. 3r3620. Dependent conditions 3r33814.  3r33825. 3r33816. If the branches in the code are based on the independent fields of the object, then the problem of full coverage is solved in a straight line. It is enough to write down all the subsets for each field under test, and then, using the appropriate generators, generate new values ​​for all the fields. If the subsequent conditions specify the set of field values, then it is necessary to take the intersection of the subsets. (For example, the first condition, X> 0 , And the following - X <= 1 Each of the conditions could give two subsets, but due to the intersections, only three subsets will be obtained - (-∞, 0] , (? 1] , (? + ∞) , Examples of which will need to be generated.) 3r3r1919.
 3r33825. 3r33816. If, during the formation of refinement sets, we find that one of the subsets is empty, then this means that the condition will always assume a fixed value of 3r3754. true or false regardless of input values. Therefore, the corresponding branch, which is never called, is a "dead code" and can be removed from the code along with the condition. 3r33838. 3r33814.  3r33825.

Related options

3r33814.  3r33825. 3r33816. Consider the case when the branch condition is based on two fields of the object, also bound by the conditions: 3r33819. 3r33814.  3r33825. 3r33737. 3r33712. if (x> 0)
if (y> 0)
if (y> x) 3r33737. 3r33814.  3r33825. 3r33816. (Here, each of the fields is limited to 3r3r.3754.> 0 , And both fields together are also limited to Y> x )
 3r33825. If a the condition is “free”, as in this example, it is sufficient to generate examples of field values, check all conditions, and discard unsuitable property values. Since the range of permissible values ​​is quite large compared to the range of unsuitable values, in a large number of cases we will generate suitable combinations of properties and the "efficiency" of this method will be quite large. 3r33814.  3r33825. If the condition is “hard”, functional ( Y == x + 1 ), Then you can build a function that calculates the value of the second field based on the generated first. 3r33814.  3r33825. If the condition is “semi-rigid” ( Y> x + 1 && y < x + 2 ), Then you can build a dependent generator, the range of values ​​of which is determined by the generated value of the first field. 3r33838. 3r33814.  3r33825. 3r38080. Symbolic execution 3r33814.  3r33825. 3r33816. In order to collect all the conditions that generate the result for any of the branches, you can use the "symbolic execution" ( Symbolic Execution , Symbolic execution of programs 3r3r1818.), The essence of which is as follows. Input data is taken as equal to some character values ​​(3r3754. Field1 = field1_initial_value 3r3755.). Then, all the manipulations described in the code under test are performed on the character values. All manipulations are performed in the same symbolic form: 3r33814.  3r33825. 3r33737. 3r33712. val a = field1 + 10
//add to the context a = field_initial_value + 10
val b = a * 3
//add to the context b = 3 * field_initial_value + 30 3r33737. 3r33814.  3r33825. 3r33816. For branch processing, both possible conditions are accepted - 3r3754. true and 3r3754. false . And the calculation is made immediately for two branches. At the same time, opposing restrictions are imposed on the character values ​​in each branch. For example, 3r33819. 3r33814.  3r33825. 3r33737. 3r33712. if (a> 0) A else B
//in branch A we assume that field_initial_value + 10> 0
//in branch B we assume that field_initial_value + 10 <= 0 3r33737. 3r33814.  3r33825. 3r33816. The constraints accumulated in symbolic form can be used either to generate a generator generating values ​​that satisfy these restrictions, or to test random values ​​generated by a less accurate generator. In any case, it is possible to generate random data leading to the execution of a previously known path (and, possibly, to a known result). 3r33838. 3r33814.  3r33825.

Testing cycles and recursive functions

3r33814.  3r33825. 3r33816. So far, we have bypassed the issue of cycles. This is due to the fact that the state changes in the cycle, that is, the cycle necessarily uses a variable variable. We have designated the boundaries of our consideration by pure functions, which implies the use of only immutable data structures. Also, if there are cycles, there is a risk of forming conditions under which the result will not be obtained in a reasonable time. 3r33838. 3r33814.  3r33825. 3r33816. It is known that any cycle can be replaced by recursion. This can be difficult for complex cycles. But, let us assume that in our case such an operation was performed. Thus, in the code under test, recursive calls will occur, and we can continue our reasoning, retaining the original assumption of considering only pure functions. How could we test such a white box, given the fact that recursive calls, like cycles, may not complete in a reasonable time? 3r33838. 3r33814.  3r33825. 3r33816. We use this construction as the Y-combinator (3r3736. "Fixed point combinator" , Stackoverflow: What is a Y-combinator? (2nd answer) , 3rr3740. Habr: Getting the Y-combinator in 7 simple steps 3r33818.). The combinator allows you to implement recursion in languages ​​that do not support recursion in its pure form. (The combinator itself is recursive, therefore it must be implemented in a language that supports recursion.) It works as follows. All recursive calls are removed from the recursive function and replaced by calls to the function that is passed as an additional argument. Such a revised function will no longer be recursive, but serves only as a “stock” for obtaining the objective function. The Y-combinator turns such a “procurement of a recursive function” into a full-fledged recursive function (passing its own continuation as an argument). 3r33838. 3r33814.  3r33825. 3r33816. Consider first an important special case of tail recursion ( Tail recursion ). We rewrite the code under test, replacing the recursive calls with the calls of the auxiliary function. For testing purposes, we will pass our own implementation of an auxiliary function that will not generate recursion. Such an auxiliary function can generate a return result corresponding to the type of returned values ​​of the white box. For example, if strings are returned, the auxiliary function will also form a string, which we can check within 3r3754. TestCase 3r3755. 'but. If the return type is inconvenient for us, we can throw an exception (3r3754. Throw Has type 3r3754. Nothing Or Bottom , Which is a subtype of everyone else). Thus, we will get the opportunity to test the code without cycles and the risk of freezing. 3r33838. 3r33814.  3r33825. 3r33816. In the case of general recursion, a recursive call returns the result, which is then used. In this case, the above approach does not work directly. You can try to apply an approach similar to what we used to call difficult reversible functions. Namely, we replace each recursive call with a new parameter. The value of these parameters can be generated as usual, based on the branching conditions in which these parameters are used. As in the case of the replacement of function calls with parameters, the results that we will receive may differ from the results that we can get in reality. Coincidence will be achieved if the value of the parameter coincides with the value of the recursive function. This approach allows us to test the steps performed after a recursive call. 3r33838. 3r33814.  3r33825. 3r33737. The meaning of testing white box
3r33814.  3r33825. 3r33816. With a certain zeal, you can ensure that tests written by hand or generated automatically will cover all the branches of the code under test, that is, they will provide 100% coverage. Thus, we can say with confidence that the white box does what it does. Hm Wait a second. And what, in fact, the meaning of such testing, the attentive reader will ask? After all, for any content of the white box will be built tests that only confirm that the white box works in some certain way. 3r33838. 3r33814.  3r33825. 3r33816. In some cases, such a test suite may still make sense: 3r33838. 3r33814.  3r33825. 3r33775.  3r33825.
If the white box is a prototype and there is a non-zero probability of error for the actual implementation in another environment. 3r3r7787.  3r33825.
If there are several implementations of the same logic (in different languages ​​or in different environments). 3r3r7787.  3r33825.
The white box code is evolving or refactoring, and we would like to detect differences from the reference behavior. 3r3r7787.  3r33825.
We want to discover subsets of input data that provide the results we need. 3r3r7787.  3r33825.
3r33814.  3r33825. 3r33816. Some features of implementation-based testing should be kept in mind, as opposed to specification-based testing. First, if the initial implementation did not support some of the functionality that one would expect based on the specification, then our tests would not notice its absence. Secondly, if such functionality was present, but worked differently than indicated in the specification (that is, with errors), then our tests will not just not detect these errors, but, on the contrary, errors will be "codified" in tests. And if subsequent /alternative implementations try to correct errors, then such tests will not allow it to just do so. 3r33838. 3r33814.  3r33825.
Conclusion 3r33813. 3r33814.  3r33825. 3r33816. Testing the white box shifts the focus from the question “what should the code do” to “what does the code actually do”. In other words, instead of using a higher level of abstraction, the formation of tests based on the specification uses exactly the same level of abstraction as in the implementation of the code. We can get good results in terms of code coverage, but at the same time such testing makes sense in a limited set of cases. 3r33838. 3r33814.  3r33825. 3r33816. If you are faced with a case in which testing a white box is justified, then the considerations above may be useful. First, it makes sense to concentrate the main efforts on the formation of test datasets, since the entrance to the white box is one (function call), and I would like to test all branches. Secondly, it seems to make sense to build a model of the code under test. For this, a specialized DSL can be used that is expressive enough to represent the logic being tested. Thirdly, using the model of the tested logic, you can try to automatically generate test data covering all branches. Fourth, the code under test may be subjected to automatic transformations, which make it more convenient for testing (excluding calls to hard-to-reversible functions, moving from cycles to recursion, excluding recursive calls). Using these approaches, you can get good results in terms of code coverage. 3r33838. 3r33814.  3r33825. 3r33816. Thus, in favorable conditions and with the implementation of some of the above approaches, it becomes possible to automatically generate meaningful tests. Perhaps interested readers will suggest other areas where white box testing or any of the approaches considered could be applied. 3r33838. 3r33814.  3r33825. 3r33812. Thanks
3r33814.  3r33825. 3r33816. I would like to thank r3r3817. @mneychev
for patience and invaluable assistance in the preparation of the article. 3r33838.
3r33825. 3r33825.
! function (e) {function t (t, n) {if (! (n in e)) {for (var r, a = e.document, i = a.scripts, o = i.length; o-- ;) if (-1! == i[o].src.indexOf (t)) {r = i[o]; break} if (! r) {r = a.createElement ("script"), r.type = "text /jаvascript", r.async =! ? r.defer =! ? r.src = t, r.charset = "UTF-8"; var d = function () {var e = a.getElementsByTagName ("script")[0]; e.parentNode.insertBefore (r, e)}; "[object Opera]" == e.opera? a.addEventListener? a.addEventListener ("DOMContentLoaded", d,! 1): e.attachEvent ("onload", d ): d ()}}} t ("//mediator.mail.ru/script/2820404/"""_mediator") () (); 3r33824. 3r33825.
+ 0 -

Add comment