285:(FSM) which represents all possible usage scenario of the tested system and the Operational Profiles (OP) which qualify the FSM to represent how the system is or will be used statistically. The first (FSM) helps to know what can be or has been tested and the second (OP) helps to derive operational test cases. Usage/Statistical Model-based Testing starts from the facts that is not possible to exhaustively test a system and that failure can appear with a very low rate. This approach offers a pragmatic way to statically derive test cases which are focused on improving the reliability of the system under test. Usage/Statistical Model Based Testing was recently extended to be applicable to embedded software systems.
269:
proofing, if this property is valid in the model, the model checker detects witnesses and counterexamples. A witness is a path where the property is satisfied, whereas a counterexample is a path in the execution of the model where the property is violated. These paths can again be used as test cases.
204:
Depending on the complexity of the system under test and the corresponding model the number of paths can be very large, because of the huge amount of possible configurations of the system. To find test cases that can cover an appropriate, but finite, number of paths, test criteria are needed to guide
224:
over the valid interpretation of the set of the predicates describing the system under test. Each class describes a certain system behavior, and, therefore, can serve as a test case. The simplest partitioning is with the disjunctive normal form approach wherein the logical expressions describing the
268:
can also be used for test case generation. Originally model checking was developed as a technique to check if a property of a specification is valid in a model. When used for testing, a model of the system under test, and a property to test is provided to the model checker. Within the procedure of
55:
needs to be derived from a corresponding abstract test suite. The executable test suite can communicate directly with the system under test. This is achieved by mapping the abstract test cases to concrete test cases suitable for execution. In some model-based testing environments, models contain
205:
the selection. This technique was first proposed by Offutt and
Abdurazik in the paper that started model-based testing. Multiple techniques for test case generation have been developed and are surveyed by Rushby. Test criteria are described in terms of general graphs in the testing textbook.
256:
Constraint programming can be combined with symbolic execution. In this approach a system model is executed symbolically, i.e. collecting data constraints over different control paths, and then using the constraint programming method for solving the constraints and producing test cases.
67:
Tests can be derived from models in different ways. Because testing is usually experimental and based on heuristics, there is no known single best approach for test derivation. It is common to consolidate all test derivation related parameters into a package that is often known as "test
280:
are an efficient way to handle Model-based
Testing. Test models realized with Markov chains can be understood as a usage model: it is referred to as Usage/Statistical Model Based Testing. Usage models, so Markov chains, are mainly constructed of 2 artifacts : the
240:
can be used to select test cases satisfying specific constraints by solving a set of constraints over a set of variables. The system is described by the means of constraints. Solving the set of constraints can be done by
Boolean solvers (e.g. SAT-solvers based on the
180:
The effectiveness of model-based testing is primarily due to the potential for automation it offers. If a model is machine-readable and formal to the extent that it has a well-defined behavioral interpretation, test cases can in principle be derived mechanically.
171:
Offline generation of manually deployable tests means that a model-based testing tool generates test cases as human-readable assets that can later assist in manual testing; for instance, a PDF document in a human language describing the generated test steps.
46:
A model describing a SUT is usually an abstract, partial presentation of the SUT's desired behavior. Test cases derived from such a model are functional tests on the same level of abstraction as the model. These test cases are collectively known as an
197:. This automaton represents the possible configurations of the system under test. To find test cases, the automaton is searched for executable paths. A possible execution path can serve as a test case. This method works if the model is
68:
requirements", "test purpose" or even "use case(s)". This package can contain information about those parts of a model that should be focused on, or the conditions for finishing testing (test stopping criteria).
388:
John Rushby. Automated Test
Generation and Verified Software. Verified Software: Theories, Tools, Experiments: First IFIP TC 2/WG 2.3 Conference, VSTTE 2005, Zurich, Switzerland, October 10–13. pp. 161-172,
379:
Jeff Offutt and Aynur
Abdurazik. Generating Tests from UML Specifications. Second International Conference on the Unified Modeling Language (UML ’99), pages 416-429, Fort Collins, CO, October 1999.
164:
Offline generation of executable tests means that a model-based testing tool generates test cases as computer-readable assets that can be later run automatically; for example, a collection of
456:
Antti Huima. Implementing
Conformiq Qtronic. Testing of Software and Communicating Systems, Lecture Notes in Computer Science, 2007, Volume 4581/2007, 1-12, DOI: 10.1007/978-3-540-73066-8_1
142:
and refers to information needed to convert an abstract test suite into an executable one. Typically, IXIT contains information on the test harness, data mappings and SUT configuration.
135:
98:, models are built before or parallel with the corresponding systems. Models can also be constructed from completed systems. Typical modeling languages for test generation include
17:
64:. This is called solving the "mapping problem". In the case of online testing (see below), abstract test suites exist only conceptually but not as explicit artifacts.
465:
Gordon Fraser, Franz Wotawa, and Paul E. Ammann. Testing with model checkers: a survey. Software
Testing, Verification and Reliability, 19(3):215–261, 2009. URL:
61:
632:
532:
Model-Based
Statistical Testing of Embedded Real-Time Software with Continuous and Discrete Signals in a Concurrent Environment: The Usage Net Approach
201:
or can be transformed into a deterministic one. Valuable off-nominal test cases may be obtained by leveraging unspecified transitions in these models.
476:
Helene Le Guen. Validation d'un logiciel par le test statistique d'usage : de la modelisation de la decision Ă la livraison, 2005. URL:
790:
466:
216:
was originally used for automated proving of logical formulas. For model-based testing approaches, the system is modeled by a set of
51:. An abstract test suite cannot be directly executed against an SUT because the suite is on the wrong level of abstraction. An
447:
Jefferson Offutt. Constraint-Based
Automatic Test Data Generation. IEEE Transactions on Software Engineering, 17:900-910, 1991
751:
691:
660:
642:
626:
610:
591:
506:
367:
314:
71:
Because test suites are derived from models and not from source code, model-based testing is usually seen as one form of
670:
Santos-Neto, P.; Resende, R.; Pádua, C. (2007). "Proceedings of the 2007 ACM symposium on
Applied computing - SAC '07".
43:(SUT), or to represent testing strategies and a test environment. The picture on the right depicts the former approach.
600:
539:
355:
198:
253:. A solution found by solving the set of constraints formulas can serve as a test cases for the corresponding system.
564:
Bringmann, E.; Krämer, A. (2008). "2008 International Conference on Software Testing, Verification, and Validation".
400:
1040:
783:
760:
675:
242:
119:
477:
341:
Paul Ammann and Jeff Offutt. Introduction to Software Testing, 2nd edition. Cambridge University Press, 2016.
165:
1065:
1004:
161:
Online testing means that a model-based testing tool connects directly to an SUT and tests it dynamically.
616:
568:. International Conference on Software Testing, Verification, and Validation (ICST). pp. 485–493.
1096:
1009:
776:
304:
99:
95:
574:
491:
2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops
417:
1050:
999:
969:
911:
309:
299:
294:
87:
725:
106:, mainstream programming languages, finite machine notations, and mathematical formalisms such as
226:
194:
91:
569:
412:
319:
237:
190:
742:. Computational Analysis, Synthesis, and Design of Dynamic Systems. Vol. 13. Boca Raton:
929:
924:
52:
702:
138:
An example of a model-based testing workflow (offline test case generation). IXIT refers to
974:
964:
282:
250:
56:
enough information to generate executable test suites directly. In others, elements in the
8:
959:
954:
944:
888:
823:
221:
57:
48:
220:, specifying the system's behavior. To derive test cases, the model is partitioned into
1014:
989:
949:
939:
883:
512:
430:
246:
28:
1024:
850:
818:
813:
747:
714:
687:
656:
652:
AST '08: Proceedings of the 3rd International Workshop on Automation of Software Test
638:
622:
606:
587:
559:
535:
502:
363:
217:
72:
40:
994:
979:
934:
860:
845:
838:
799:
735:
730:, Muhammad Shafique, Yvan Labiche, Carleton University, Technical Report, May 2010.
679:
579:
516:
494:
434:
422:
32:
60:
must be mapped to specific statements or method calls in the software to create a
1019:
650:
359:
213:
1070:
1045:
984:
893:
865:
566:
2008 International Conference on Software Testing, Verification, and Validation
265:
123:
36:
426:
1090:
718:
277:
78:
Model-based testing for complex software systems is still an evolving field.
683:
350:
1060:
1055:
898:
828:
489:
Böhr, Frank (2011). "Model Based Statistical Testing of Embedded Systems".
498:
272:
146:
There are various known ways to deploy model-based testing, which include
1075:
919:
855:
583:
107:
621:, Jonathan Jacky, Margus Veanes, Colin Campbell, and Wolfram Schulte,
232:
134:
743:
672:
Proceedings of the 2007 ACM symposium on Applied computing - SAC '07
768:
115:
111:
31:
for designing and optionally also executing artifacts to perform
762:
2011/2012 Model-based Testing User Survey: Results and Analysis.
764:
Robert V. Binder. System Verification Associates, February 2012
16:
733:
103:
39:. Models can be used to represent the desired behavior of a
637:
Advanced Lecture Series, LNCS 3472, Springer-Verlag, 2005.
727:
A Systematic Review of Model Based Testing Tool Support
669:
273:
Test case generation by using a Markov chain test model
478:
ftp://ftp.irisa.fr/techreports/theses/2005/leguen.pdf
189:
Often the model is translated to or interpreted as a
233:
Constraint logic programming and symbolic execution
168:classes that embodies the generated testing logic.
337:
335:
618:Model-Based Software Testing and Analysis with C#
175:
1088:
129:
602:Practical Model-Based Testing: A Tools Approach
563:
352:Practical Model-Based Testing: A Tools Approach
332:
156:offline generation of manually deployable tests
784:
184:
398:
399:Brucker, Achim D.; Wolff, Burkhart (2012).
225:system's behavior are transformed into the
791:
777:
700:
573:
416:
740:Model-Based Testing for Embedded Systems
648:
133:
15:
634:Model-Based Testing of Reactive Systems
1089:
734:Zander, Justyna; Schieferdecker, Ina;
152:offline generation of executable tests
772:
798:
529:
488:
392:
344:
315:Object-oriented analysis and design
20:General model-based testing setting
13:
629:, Cambridge University Press 2008.
552:
208:
90:or in Object Management Group's (
14:
1108:
605:, Mark Utting and Bruno Legeard,
401:"On Theorem Prover-based Testing"
362:, Mark Utting and Bruno Legeard,
260:
1041:Graphical user interface testing
703:"Model-Based Testing Adds Value"
140:implementation extra information
523:
482:
701:Roodenrijs, E. (Spring 2010).
676:Symposium on Applied Computing
649:Hong Zhu; et al. (2008).
470:
459:
450:
441:
382:
373:
243:Boolean satisfiability problem
176:Deriving tests algorithmically
1:
325:
130:Deploying model-based testing
1066:Software reliability testing
1005:Software performance testing
7:
558:OMG UML 2 Testing Profile;
405:Formal Aspects of Computing
288:
10:
1113:
908:Testing types, techniques,
185:From finite state machines
1033:
907:
876:
806:
427:10.1007/s00165-012-0222-y
305:Model-driven architecture
96:model-driven architecture
81:
1051:Orthogonal array testing
1000:Smoke testing (software)
970:Dynamic program analysis
310:Model-driven engineering
300:Domain-specific modeling
295:Domain-specific language
88:Model Driven Engineering
684:10.1145/1244002.1244306
613:, Morgan-Kaufmann 2007.
227:disjunctive normal form
195:state transition system
678:. pp. 1409–1415.
370:, Morgan-Kaufmann 2007
320:Time partition testing
238:Constraint programming
191:finite state automaton
143:
21:
930:Compatibility testing
499:10.1109/ICSTW.2011.11
137:
53:executable test suite
27:is an application of
19:
975:Installation testing
965:Differential testing
736:Mosterman, Pieter J.
584:10.1109/ICST.2008.45
530:Böhr, Frank (2012).
283:Finite State Machine
251:Gaussian elimination
960:Development testing
955:Destructive testing
945:Conformance testing
889:Integration testing
834:Model-based testing
824:Exploratory testing
707:Methods & Tools
222:equivalence classes
62:concrete test suite
58:abstract test suite
49:abstract test suite
25:Model-based testing
1015:Symbolic execution
990:Regression testing
950:Continuous testing
940:Concurrent testing
884:Acceptance testing
807:The "box" approach
534:. Verlag Dr. Hut.
493:. pp. 18–25.
358:2012-08-25 at the
247:numerical analysis
144:
29:model-based design
22:
1084:
1083:
1025:Usability testing
851:White-box testing
819:All-pairs testing
814:Black-box testing
753:978-1-4398-1845-9
693:978-1-59593-480-2
662:978-1-60558-030-2
643:978-3-540-26278-7
627:978-0-521-68761-4
611:978-0-12-372501-1
593:978-0-7695-3127-4
508:978-1-4577-0019-4
368:978-0-12-372501-1
73:black-box testing
41:system under test
1104:
1097:Software testing
995:Security testing
980:Negative testing
935:Concolic testing
861:Mutation testing
846:Grey-box testing
839:Scenario testing
800:Software testing
793:
786:
779:
770:
769:
757:
722:
697:
666:
597:
577:
546:
545:
527:
521:
520:
486:
480:
474:
468:
463:
457:
454:
448:
445:
439:
438:
420:
396:
390:
386:
380:
377:
371:
348:
342:
339:
33:software testing
1112:
1111:
1107:
1106:
1105:
1103:
1102:
1101:
1087:
1086:
1085:
1080:
1029:
1020:Test automation
909:
903:
872:
802:
797:
754:
738:, eds. (2011).
694:
663:
594:
575:10.1.1.729.8107
555:
553:Further reading
550:
549:
542:
528:
524:
509:
487:
483:
475:
471:
464:
460:
455:
451:
446:
442:
418:10.1.1.208.3135
397:
393:
389:Springer-Verlag
387:
383:
378:
374:
360:Wayback Machine
349:
345:
340:
333:
328:
291:
275:
263:
235:
214:Theorem proving
211:
209:Theorem proving
187:
178:
132:
84:
12:
11:
5:
1110:
1100:
1099:
1082:
1081:
1079:
1078:
1073:
1071:Stress testing
1068:
1063:
1058:
1053:
1048:
1046:Manual testing
1043:
1037:
1035:
1031:
1030:
1028:
1027:
1022:
1017:
1012:
1010:Stress testing
1007:
1002:
997:
992:
987:
985:Random testing
982:
977:
972:
967:
962:
957:
952:
947:
942:
937:
932:
927:
922:
916:
914:
905:
904:
902:
901:
896:
894:System testing
891:
886:
880:
878:
877:Testing levels
874:
873:
871:
870:
869:
868:
866:Static testing
863:
858:
848:
843:
842:
841:
836:
831:
826:
821:
810:
808:
804:
803:
796:
795:
788:
781:
773:
767:
766:
758:
752:
731:
723:
698:
692:
667:
661:
646:
630:
614:
598:
592:
561:
554:
551:
548:
547:
541:978-3843903486
540:
522:
507:
481:
469:
458:
449:
440:
411:(5): 683–721.
391:
381:
372:
343:
330:
329:
327:
324:
323:
322:
317:
312:
307:
302:
297:
290:
287:
274:
271:
266:Model checkers
262:
261:Model checking
259:
234:
231:
210:
207:
186:
183:
177:
174:
148:online testing
131:
128:
86:Especially in
83:
80:
37:system testing
9:
6:
4:
3:
2:
1109:
1098:
1095:
1094:
1092:
1077:
1074:
1072:
1069:
1067:
1064:
1062:
1059:
1057:
1054:
1052:
1049:
1047:
1044:
1042:
1039:
1038:
1036:
1032:
1026:
1023:
1021:
1018:
1016:
1013:
1011:
1008:
1006:
1003:
1001:
998:
996:
993:
991:
988:
986:
983:
981:
978:
976:
973:
971:
968:
966:
963:
961:
958:
956:
953:
951:
948:
946:
943:
941:
938:
936:
933:
931:
928:
926:
923:
921:
918:
917:
915:
913:
906:
900:
897:
895:
892:
890:
887:
885:
882:
881:
879:
875:
867:
864:
862:
859:
857:
854:
853:
852:
849:
847:
844:
840:
837:
835:
832:
830:
827:
825:
822:
820:
817:
816:
815:
812:
811:
809:
805:
801:
794:
789:
787:
782:
780:
775:
774:
771:
765:
763:
759:
755:
749:
745:
741:
737:
732:
729:
728:
724:
720:
716:
712:
708:
704:
699:
695:
689:
685:
681:
677:
673:
668:
664:
658:
655:. ACM Press.
654:
653:
647:
644:
640:
636:
635:
631:
628:
624:
620:
619:
615:
612:
608:
604:
603:
599:
595:
589:
585:
581:
576:
571:
567:
562:
560:
557:
556:
543:
537:
533:
526:
518:
514:
510:
504:
500:
496:
492:
485:
479:
473:
467:
462:
453:
444:
436:
432:
428:
424:
419:
414:
410:
406:
402:
395:
385:
376:
369:
365:
361:
357:
354:
353:
347:
338:
336:
331:
321:
318:
316:
313:
311:
308:
306:
303:
301:
298:
296:
293:
292:
286:
284:
279:
278:Markov chains
270:
267:
258:
254:
252:
248:
244:
239:
230:
228:
223:
219:
215:
206:
202:
200:
199:deterministic
196:
192:
182:
173:
169:
167:
162:
159:
157:
153:
149:
141:
136:
127:
125:
121:
117:
113:
109:
105:
101:
97:
93:
89:
79:
76:
74:
69:
65:
63:
59:
54:
50:
44:
42:
38:
34:
30:
26:
18:
1061:Soak testing
1056:Pair testing
899:Unit testing
833:
829:Fuzz testing
761:
739:
726:
713:(1): 33–39.
710:
706:
671:
651:
633:
617:
601:
565:
531:
525:
490:
484:
472:
461:
452:
443:
408:
404:
394:
384:
375:
351:
346:
276:
264:
255:
236:
212:
203:
188:
179:
170:
163:
160:
155:
151:
147:
145:
139:
85:
77:
70:
66:
45:
24:
23:
1076:Web testing
920:A/B testing
856:API testing
249:, like the
326:References
218:predicates
925:Benchmark
744:CRC Press
719:1661-402X
570:CiteSeerX
413:CiteSeerX
1091:Category
1034:See also
356:Archived
289:See also
245:) or by
912:tactics
517:9582606
435:5774837
116:Event-B
750:
717:
690:
659:
641:
625:
609:
590:
572:
538:
515:
505:
433:
415:
366:
166:Python
154:, and
82:Models
513:S2CID
431:S2CID
193:or a
120:Alloy
104:SysML
910:and
748:ISBN
715:ISSN
688:ISBN
657:ISBN
639:ISBN
623:ISBN
607:ISBN
588:ISBN
536:ISBN
503:ISBN
364:ISBN
94:'s)
680:doi
580:doi
495:doi
423:doi
124:Coq
122:or
118:),
100:UML
92:OMG
35:or
1093::
746:.
711:18
709:.
705:.
686:.
674:.
586:.
578:.
511:.
501:.
429:.
421:.
409:25
407:.
403:.
334:^
229:.
158:.
150:,
126:.
110:,
102:,
75:.
792:e
785:t
778:v
756:.
721:.
696:.
682::
665:.
645:.
596:.
582::
544:.
519:.
497::
437:.
425::
114:(
112:B
108:Z
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
