-
Precedent case
-
Zhongke Yong Lian advanced technology training center
A test case is a set of test inputs, execution conditions and expected results written for a specific goal to test the program path or verify whether it meets specific requirements.
At present, there is no classic definition of test cases. Generally speaking, it refers to the description of a specific software product testing task, which embodies the testing scheme, method, technology and strategy. The contents include test objectives, test environment, input data, test steps, expected results, test scripts, etc. , and form a document.
Different types of software have different test cases. Different from systems, tools, control and game software, the user requirements of management software are more inconsistent and change more quickly. The author is mainly engaged in the testing of enterprise management software. Therefore, our method is to separate test data and test scripts from test cases. Test cases are often test schemes designed for the functions, business rules and business processing of software products. The test of each specific function or operation path of the software constitutes a test case.
With the development of software industry in China, software testing is also developing. From the initial part-time testing of software programmers to the establishment of independent full-time testing departments in software companies. Testing has also developed from simple testing to formal testing, including: preparing test plans, writing test cases, preparing test data, writing test scripts, implementing tests, testing evaluation and so on. The testing method has developed from simple manual testing to both manual and automatic testing, and there is a trend to the third-party professional testing company.
To make the end users satisfied with the software, the most powerful measure is to clarify the expectations of the end users, so as to verify these expectations and confirm their effectiveness. Test cases reflect the requirements that need to be verified. However, verifying these requirements may be realized by different testers in different ways. For example, executing software to verify its function and performance can be realized by a tester using automatic test technology; The shutdown step of the computer system can be completed by manual testing and observation; However, market share and sales data (and product demand) can only be achieved by evaluating products and competitive sales data.
Because it may not be possible (or necessary) to verify all requirements, whether the most suitable or critical requirements can be selected for testing is related to the success or failure of the project. Choosing the requirements to be verified will be the result of weighing the cost, risk and the necessity of verifying the requirements.
Identifying test cases is important for the following reasons.
Test cases are the basis of designing and formulating the test process.
The "depth" of the test is directly proportional to the number of test cases. Since each test case reflects a different scenario, condition or event flow in the product, with the increase of the number of test cases, you will have more confidence in the product quality and testing process.
One of the main evaluation methods to judge whether a test is complete is requirement-based coverage, which is based on the number of test cases determined, implemented and/or executed. A statement like this: "95% of the key test cases have been executed and verified" is far more meaningful than "we have completed 95% of the tests".
The test workload is proportional to the number of test cases. According to the comprehensive and detailed test cases, the time progress of each successive stage in the test cycle can be estimated more accurately.
The types of test design and development and the resources needed are mainly controlled by test cases.
Test cases are usually classified according to their associated test types or test requirements, and will change accordingly with the changes of types and requirements. The best solution is to compile at least two test cases for each test requirement:
A test case is used to prove that the requirements have been met, which is usually called a positive test case;
Another test case reflects unacceptable, abnormal or unexpected conditions or data to prove that requirements can only be met under the required conditions. This test case is called a negative test case.
First of all, test cases are the core of software testing.
There is no doubt about the importance of software testing. However, how to complete the test in the shortest time with the least manpower and material resources, find the defects of the software system and ensure the excellent quality of the software is the goal that software companies explore and pursue. Every software product or software development project needs an excellent test scheme and method.
There are many factors that affect software testing, such as the complexity of the software itself, the quality of developers (including those who analyze, design, program and test), the application of testing methods and technologies, and so on. Because some factors exist objectively and cannot be avoided. Some factors are fluctuating and unstable, for example, the development team is mobile, experienced people have left, and new people are constantly joining; A specific person's work will also be affected by emotions and so on. How to ensure the stability of software testing quality? With test cases, no matter who tests, the quality of the test can be guaranteed by quoting test cases. The influence of human factors can be minimized. Even if the initial test case is not well thought out, it will be improved with the progress of the test and the update of the software version.
Therefore, the design and writing of test cases is the most important in software testing activities. Test case is the guide of testing work and the criterion that software testing must follow. This is also the fundamental guarantee for the stability of software testing quality.
Second, prepare test cases.
This paper mainly introduces some specific methods of writing test cases.
1, test case document
To write test case documents, there must be a document template, which must meet the requirements of internal specifications. The test case documents will be bound by the test case management software.
Test cases of software products or software development projects generally form test case documents in units of software modules or subsystems of products, but they are not absolute.
The test case document consists of introduction and test case. In the introduction part, the test purpose, test scope, definition terms, reference documents and overview are written. The test case section lists each test case one by one. Each specific test case will include the following detailed information: use case number, use case name, test level, entry criteria, verification steps, expected results (including judgment criteria), exit criteria, comments, etc. The above contents cover the basic elements of test cases: test indicators, test environment, test input, test operation, expected results and evaluation criteria.
2. Settings of test cases
Our early test cases were set by function. Later, the path analysis method was introduced to set use cases according to the path. At present, it has evolved to set use cases according to the mixed mode of function and path.
Testing by function is the simplest, and each function is tested by use case specification.
For program modules with complex operations, the realization of their functions is interactive, closely related and interlocking, which can evolve into a lot of changes. Without rigorous logical analysis, omissions are inevitable. Path analysis is a good method, and its biggest advantage is that it can avoid missing measurement.
However, the path analysis method also has limitations. There are more than ten paths in a very simple dictionary maintenance module. It is not surprising that a complex module has dozens to hundreds of paths. The author thinks this is a suitable scale for path analysis. If a subsystem has ten or more modules, these modules are interrelated. If the path analysis method is used again, the number of paths will increase geometrically, reaching more than 5 digits, and it will not be used. Then the test path or test case between subsystem modules still needs to be solved by traditional methods. This is the origin of setting use cases according to the mixed mode of function and path.
3. Design test cases
Test cases can be divided into basic events, alternative events and abnormal events. When designing use cases of basic events, we should refer to use case specifications (or design specifications), and design test cases according to related functions and operations through path analysis. For isolated functions, test cases are designed directly according to the functions. The test cases of basic events should contain all required functions, and the coverage rate is 100%.
It is much more complicated and difficult to design use cases for substitute events and exception events. For example, the code of a dictionary is unique and cannot be repeated. The test needs to be verified: there is already a constraint on the dictionary code in the dictionary adder. If there is code duplication, an error must be reported and the error text is correct. The documents often formed in the design and coding stage do not have detailed analysis and description of alternative events and abnormal events. The test itself requires verifying all non-basic events and finding software defects as much as possible.
Basic methods commonly used in software testing can be used to design test cases, such as equivalence class division method, boundary value analysis method, error inference method, causality chart method and logical covering method. According to the different attributes of software, different methods are adopted. How to flexibly use various basic methods to design complete test cases and finally expose hidden defects depends on the rich experience and careful design of test designers.
Third, the role of test cases in software testing.
1, to guide the implementation of the test.
Test cases are mainly suitable for integration testing, system testing and regression testing. When testing, the test case is taken as the test standard, and the tester must implement the test one by one in strict accordance with the test case and test steps. And record the test situation in the test case management software, so as to automatically generate the test result document.
According to the test level of test cases, integration test should test those cases, system test and regression test should test those cases, which have been made clear when designing test cases, and testers cannot change them at will when implementing tests.
2, planning the preparation of test data
In our practice, test data is separated from test cases. Prepare one or several sets of original test data and standard test results according to test cases. Especially for the correctness of data sets such as test reports, it is very necessary to prepare test data according to test case planning.
In addition to normal data, a large number of edge data and error data must be designed according to test cases.
3. Write the test script Design Specification.
In order to improve the efficiency of testing, software testing has vigorously developed automated testing. The central task of automated testing is to write test scripts. If software programming in software engineering must have design specifications, then the design specifications of test scripts are test cases.
4. Evaluate the measurement benchmark of test results
After the test is completed, it is necessary to evaluate the test results and prepare the test report. Some quantitative results are needed to judge whether the software test is completed and measure the test quality. For example: what is the test coverage, pass rate, pass rate of important tests, etc. The previous statistical benchmark was a software module or function point, which was too rough. It is more accurate and effective to use test cases as measurement benchmarks.
5, the standard of defect analysis
By collecting defects and comparing test cases with defect database, analyze them to confirm whether defects are missed or reproduced. The missing tests reflect the imperfection of test cases, and the corresponding test cases should be supplemented immediately, so as to gradually improve the software quality. There are already corresponding test cases, which reflect that there are problems in the implementation of testing or change processing.
Fourth, related issues.
1, test case review
Test case is the criterion of software testing, but once it is written, it does not become the criterion. In the process of designing and writing test cases, peer review should be organized. After the preparation is completed, an expert review shall be organized, and it can only be used after passing. The review committee may be composed of project leaders, testers, programmers, analysts and designers, and may also invite customer representatives to participate.
2. Modification and update of test cases
Test cases need to be improved after being recorded. There are three main reasons: first, it is found that the design of test cases is not well considered and needs to be improved during the testing process; Second, the software defects fed back after the software is delivered and used, which are caused by loopholes in test cases; Third, the new functions of the software itself and the update of the software version, and the test cases must also be modified and updated.
Generally speaking, minor modifications and improvements can be made in the original test case document, but the document should have a change record. When the software version is updated, the test cases should usually be updated.
3. Test case management software
Using test cases also requires test case management software. It has three main functions: first, it can automatically import the key contents of the test case document, such as number and name, into the management database to form a record completely corresponding to the test case document; Second, it can be used to input the test situation in time when the test is implemented; Third, automatically generate test result documents, including each test metric value, test coverage table and test case list that passed or failed the test.
With management software, testers can easily write daily test logs or generate software test reports.
Design of test cases for verbs (abbreviation of verb)
(a) white box technology
White-box testing is a kind of structured testing, so the object of testing is basically the source program, and the test cases are designed based on the internal logic of the program.
1, logical override
The degree of logical coverage within the program, when there is a loop in the program, it is impossible to cover every path. It is necessary to design a test case to cover the most representative path with high coverage. According to the program shown in Figure 7- 1, several common stacking techniques are discussed respectively.
(1) statement override.
In order to improve the possibility of finding errors, every statement in the program should be executed during the test. Statement coverage refers to designing enough test cases so that each statement in the tested program can be executed at least once.
As shown in Figure 7- 1, it is the flow chart of a tested program:
(2) Determine the coverage.
Decision coverage refers to designing enough test cases so that each decision expression in the tested program can get a "true" value and a "false" value at least once, and each branch of the program can pass at least once. Therefore, decision coverage is also called branch coverage.
(3) Conditional coverage.
Conditional coverage refers to designing enough test cases so that all possible values of each condition in the judgment expression appear at least once.
(4) Judgment/condition test.
Coverage criteria refers to designing enough test cases so that all possible values of each condition of a decision expression appear at least once, and all possible results of each decision expression also appear at least once.
(5) Conditional combination coverage.
Conditional combination coverage is a strong coverage standard, which means designing enough test cases to make the combination of various possible values of conditions in each decision expression appear at least once.
(6) Path coverage.
Path coverage refers to designing enough test cases to cover all possible paths in the tested program.
In the actual logical coverage test, test cases are generally designed based on conditional combination coverage, and then some test cases are supplemented to reach the path coverage test standard.
2. Cyclic coverage
3. Basic path test
(b) black box technology
1. equivalence class division
(1) partition equivalence classes.
① If the input condition specifies the range of values or the number of values. Then a reasonable equivalence class (the input value or number is within the range) and two unreasonable equivalence classes (the input value or number is less than the minimum value of the range or greater than the maximum value of the range) can be determined.
(2) If a set of values of input data is specified and the program treats different input values differently, then each allowed input value is a reasonable equivalent class, and there are also unreasonable equivalent classes (any disallowed input value).
(3) If the rules that input data must follow are specified, a reasonable equivalence class (conforming to the rules) and several unreasonable equivalence classes (violating the rules from all angles) can be determined.
④ If the elements in the divided equivalence class are treated differently in the program, then the equivalence class should be further divided into smaller equivalence classes.
(2) Determine the test case.
① Number each equivalent class.
② Design a test case, covering as many reasonable equivalent classes as possible. Repeat this step until all reasonable equivalent classes are covered by test cases.
③ Design a test case that only covers an unreasonable equivalence class.
2. Boundary value analysis
When designing test cases, boundary value analysis is generally combined with equivalence class division. However, it does not choose an example from the equivalence class as a representative, but takes the test boundary as the key object, and selects the test data just equal to, just greater than or just less than the boundary value.
(1) If the input condition specifies the range of values, you can choose the data just equal to the boundary value as a reasonable test case, or you can choose the data just beyond the boundary value as an unreasonable test case. If the input range is [1, 100], the values of 0, 1, 100,1can be used as test data.
(2) If the input condition indicates the quantity of input data, the test case is designed according to the maximum quantity, the minimum quantity, 1 less than the minimum quantity and 1 greater than the maximum quantity. For example, an input file can contain 1-255 records, so test cases of input files containing 1 records, 255 records and 0 records are designed respectively.
(3) For each output condition, determine the boundary of output value according to the above principle (1) or (2) respectively. For example, a student achievement management system stipulates that only 95-98 grades of college students can be queried, and test cases can be designed, so it is necessary to design one or four grades of students within the query range (unreasonable output equivalence class).
Because the boundary of the output value does not correspond to the boundary of the input value, it may not be possible to check the boundary of the output value or produce a result beyond the output value, but it is necessary to try again if necessary.
(4) If the input or output fields given in the program specification are ordered sets (such as sequential files, linear tables, linked lists, etc.). ), you should choose the first and last elements of the collection as test cases.
3. Wrong speculation
When testing a program, people may infer various errors that may exist in the program according to experience or intuition, and then write targeted test cases to check these errors, which is the error inference method.
4. Causality diagram
Equivalence class division and boundary value method only consider the test function of each input data in isolation, and do not consider the error caused by the combination of multiple input data.
5. Comprehensive strategy
Each method can design a set of useful examples, and it is easy to find one type of error, but it may not be easy to find another type of error. Therefore, in the actual test, various test methods are combined to form a comprehensive strategy. Usually, the basic test cases are designed by black-box method first, and then some necessary test cases are supplemented by white-box method.
Sixth, the misunderstanding of test case design
(Source: Guanhe Examination Network)
Use cases that can find defects that have not been found so far are good use cases;
First of all, I must declare that this sentence is actually very reasonable, but I find that many people misinterpret the original intention of this sentence, are bent on designing and discovering "difficult-to-find defects", fall into blind one-sidedness and forget the purpose of testing, which is very terrible. I tend to think of test cases as a set, and its evaluation can only be carried out on the set of test cases. The exam itself is a kind of "V &；; AMpv ",the test needs to ensure the following two points:
The program did what it should do.
The program didn't do anything it shouldn't.
Therefore, as the basis of test realization, test cases must be able to completely cover the test requirements, and should not be judged for a single test case.
The test case should record all the operation information in detail, so that a person who has not been exposed to the system can test it;
I don't know if there is any company in China that can really do this, or I don't know if there is any company in China that can write every test case in such detail. In my testing experience, I have many questions about the details and complexity of test case description. It's so simple that no one can carry it out except yourself. It's written in too much detail, and the time spent in maintaining test cases (don't forget, test cases are dynamic, and once the test environment, requirements, design and reality change, test cases need to change with them) is really amazing, which is probably difficult to do in the current situation that most domestic software companies lack test resources. But I happen to meet some such bosses or project leaders, even the test engineers themselves. Regardless of the actual resources, I must write a use case that "people who have never been exposed to the system can test".
Before discussing this problem, we can consider the purpose of the test. The purpose of testing is to find the defects in the program as much as possible. The testing activity itself can also be regarded as a project, which needs to achieve the goal as much as possible under the given resource conditions. According to my personal experience, most domestic software companies are not equipped with enough resources in testing, so we must make clear the testing objectives in the testing planning stage, and everything should be carried out around the testing objectives.
In addition to resource constraints, the level of detail of test cases also needs to be determined as needed. If the executors, designers and people involved in the test activities have a deep understanding of the system, there is no need to describe the test cases in detail. The function of the document is communication, as long as it can achieve the purpose of communication, it is OK. In the project where I am the test manager, in the test planning stage, the test design is usually given about 30%-40% time. The test design engineer can determine the details of the test case according to the needs of the project, and the relevant personnel involved in the test case review will check it.
Test case design is once and for all;
I don't think anyone here will recognize this sentence, but in practice, we can often find the shadow of this idea. I once participated in a project, and the software requirements and design have been changed many times, but the test cases have not been modified at all. The direct result is that the new test engineer is at a loss when executing the test case, and the indirect result is that the test case becomes a pile of waste paper. After being troubled by invalid defect reports many times, developers are dismissive of testers.
This example may be extreme, but in the actual development process, it is not uncommon for test cases to be out of sync with requirements and design. Test case documents are "living" documents that test engineers should keep in mind.
Test cases should not contain actual data;
A test case is "a set of inputs, execution conditions and expected results", which undoubtedly should include clear input data and expected output. A test case without test data is only instructive at best and not executable. Of course, including input data in test cases will bring problems such as maintenance and synchronization with test environment. In this regard, the book Effective Software Testing provides detailed test cases and test data maintenance methods for your reference.
There is no need for obvious verification means in the test case;
I have seen many test cases written by test engineers, in which "expected output" is only described as the behavior visible to the program. In fact, the meaning of "expected result" is not only the behavior visible to the program. For example, for the ordering system, if you enter the ordering data and click OK, the system will prompt "Ordered successfully". Is this a complete use case? Should the "successful order" output by the system be our only means of verification? Obviously not. The success of the order depends on whether the corresponding data records are updated. Therefore, in such a use case, we should also include an explicit means to verify the test results: execute a query statement in the database to see if the query results are consistent with expectations.
Seven, generate test cases from the use cases.
The test cases used for functional testing come from the test target's use cases. Test cases should be compiled for each use case scenario. The use case scenario should be determined by describing the path of the use case, and it should traverse all the basic processes and optional processes from the beginning to the end of the use case.
For example, each different path of the use case in the figure below reflects the basic process and the optional process, both of which are represented by arrows. The basic process is represented by a black straight line, which is the simplest path in the use case. Each alternative process starts with the basic process and then executes the alternative process under certain conditions. Alternate flows can rejoin the basic flow (alternate flows 1 and 3), or can originate from another alternate flow (alternate flow 2), or terminate the use case without rejoining a certain flow (alternate flows 2 and 4).
Example of event flow for use cases
Following the possible path of each use case in the above figure, different use case scenarios can be determined. Starting with the basic process, and then combining the basic process with the alternative process, the following use case scenarios can be determined:
Scenario 1 basic stream
Scenario 2 Basic Stream Alternative Stream 1
Scenario 3 Basic Stream Alternative Stream 1 Alternative Stream 2
Scenario 4 Basic Stream Alternative Stream 3
Scenario 5 Basic Stream Alternative Stream 3 Alternative Stream 1
Scenario 6 Basic Stream Alternative Stream 3 Alternative Stream 1 Alternative Stream 2
Scenario 7 Basic Stream Alternative Stream 4
Scenario 8 Basic Stream Alternative Stream 3 Alternative Stream 4
Note: For convenience, Scenes 5, 6 and 8 only describe the situation where the loop shown in Flow 3 is executed once instead.
Generating test cases for each scenario is accomplished by determining specific conditions, which will lead to the execution of specific use case scenarios.
For example, suppose the use case described in the above figure defines alternative process 3 as follows:
"This event flow will occur if the dollar amount entered in step 2' Enter the withdrawal amount' above exceeds the current account balance. A warning message will be displayed, and then the basic process will be rejoined, and the above step 2 "Enter the withdrawal amount" will be executed again. At this time, bank customers can enter a new withdrawal amount. "
Based on this, you can start to determine the test cases that need to be used to execute alternative process 3:
Test Case ID Scenario Condition Expected Result
TC x scenario 4 step 2-withdraw amount > account balance rejoins the basic stream in step 2.
TC y scheme 4 step 2-withdrawal amount
TC z Scenario 4, Step 2- Withdrawal Amount = Account Balance No alternative process 3 is executed, and the basic process is executed.
Note: Because no other information is provided, the test case shown above is very simple. Test cases are rarely that simple.
The following is a more realistic example of generating test cases from use cases.
Example:
The protagonist and use case of an ATM machine.
The following table contains the basic process and some spare processes of the withdrawal case in the above figure:
The start of this use case is that ATM is ready.
Ready to withdraw money-the customer inserts the bank card into the card reader of the ATM machine.
Verify the bank card-ATM reads the account code from the magnetic stripe of the bank card to check whether it is an acceptable bank card.
Entering PIN-ATM requires customers to enter PIN code (4 digits).
Verify Account Code and PIN- Verify the account code and PIN to determine whether the account is valid and the entered PIN is correct for the account. For this event flow, the account is valid and the PIN of this account is correct.
ATM Options-The ATM displays various options available on this machine. In this event flow, bank customers usually choose "withdrawal".
Input Amount-The amount withdrawn from the ATM. For this event flow, the customer needs to select a preset amount (10 USD, 20 USD, 50 USD or 100 USD).
Authorization -ATM initiates the verification process by sending the card ID, PIN, amount and account information to the banking system as a transaction. For this event flow, the banking system is online, and gives a reply to the authorization request, approves the completion of the withdrawal process, and updates the account balance accordingly.
Issue currency-provide cash.
Return the bank card-the bank card has been returned.
Receipt-Print the receipt and provide it to the customer. ATM will also update the internal records accordingly.
At the end of the use case, ATM is ready again.
Alternative process 1- Basic process of bank card invalidation Step 2-Verify the bank card. If the card is invalid, it will be returned and relevant information will be notified.
Alternative process 2-ATM has no cash-ATM option is in the fifth step of the basic process. If ATM has no cash, the "Withdraw" option will not be available.
The cash shortage in the alternative process ATM belongs to the basic process step 6- enter the amount. If the amount in the ATM is less than the amount of the withdrawal request, an appropriate message will be displayed, and the basic flow will be rejoined at step 6- Enter the amount.
Alternate Process 4- Wrong PIN in Basic Process Step 4-Verify the account and PIN, and the customer has three chances to enter the PIN.
If the PIN is entered incorrectly, the ATM will display appropriate information; If there is still an input opportunity, this event stream will rejoin the basic stream in step 3- Input PIN.
If the PIN code entered in the last attempt is still wrong, the card will be reserved by ATM, ATM will return to the ready state, and this use case will be terminated.
Alternative process 5- Account does not exist in basic process Step 4- Verify account and PIN. If the code returned by the banking system indicates that the account cannot be found or it is forbidden to withdraw money from the account, the ATM displays an appropriate message and returns to the bank card to rejoin the basic process in step 9.
Optional process 6- basic process step 7- authorization, the book amount is insufficient, and the banking system returns a code indicating that the account balance is less than the amount entered in basic process step 6- enter the amount, and then the ATM displays an appropriate message and re-enters the basic process in step 6- enter the amount.
Alternative Process 7- Reaching the Maximum Daily Withdrawal Amount In Step 7- Authorization of the basic process, the code returned by the banking system indicates that the customer has or will exceed the maximum amount allowed to withdraw within 24 hours, and then the ATM displays appropriate information, and the basic process is re-added in Step 6- Input the amount.
Alternate process x- recording error If the record cannot be updated in the basic process step 10- receipt, ATM will enter "safe mode", in which all functions will be suspended. At the same time, an appropriate alarm message is sent to the banking system, indicating that the ATM has been suspended.
The customer can decide to terminate the transaction (exit) at any time. When the transaction is terminated, the bank card will be withdrawn.
The optional Flow Z-"Tilt-Up" ATM contains a large number of sensors to monitor various functions, such as power detector, manometers at different doors and entrances, and motion detector. At any time, if someone