添加到百度搜藏最新更新
下载排行
推荐下载
百度搜索
软件测试-Manage Software Testing
-
软件简介:
- 上论坛找商业破解网站程序
- 源码使用如需帮助，请到论坛发帖！
Contents
List of Figures ..................................................................................................................................... xi
List of Tables .................................................................................................................................... xvii
Preface ................................................................................................................................................xix
1
Introduction..........................................................................................................................................1
1.1 Conclusions............................................................................................................................................................ 1
1.2 What Has This Got to Do With Software? ........................................................................................................... 1
1.3 If It Isn’t Planned, It Won’t Happen ..................................................................................................................... 2
1.4 Let’s Not Test.......................................................................................................................................................... 2
1.5 Something to Do When You Really Hit Opposition............................................................................................ 5
1.6 Conventions ........................................................................................................................................................... 7
1.7 Apologies ................................................................................................................................................................ 8
1.8 Some Basics............................................................................................................................................................ 8
1.9 The Players ........................................................................................................................................................... 11
1.10 Quality Assurance and Quality Control ............................................................................................................. 14
1.11 Processes, Tools, Products, and Standards .......................................................................................................... 15
2
The Big Questions You Need Answers To .........................................................................................17
2.1 Why Do We Test?................................................................................................................................................. 17
2.2 How Do We Test? ................................................................................................................................................ 18
2.3 When Do We Start Testing? ................................................................................................................................ 18
2.4 When Do We Stop Testing? ................................................................................................................................ 20
2.5 What Do We Test it Against? ............................................................................................................................... 21
2.6 What Do We Test it With? .................................................................................................................................. 22
2.7 Are We Getting Better or Worse at Finding Bugs?............................................................................................. 22
2.8 How Do We Measure Our Test Coverage? ......................................................................................................... 26
2.9 Are There Any More Embarrassing Questions? ................................................................................................. 28
3
Risk Management ...............................................................................................................................29
3.1 How to Manage Risk ........................................................................................................................................... 30
3.2 A Worked Example.............................................................................................................................................. 35
3.3 Hazard Analyses and Testing ............................................................................................................................... 37
4
Test Planning and Management.........................................................................................................39
4.1 Testing Principles................................................................................................................................................. 39
4.2 Laws ...................................................................................................................................................................... 41
4.3 Test Management Principles ............................................................................................................................... 42
4.4 Life-Cycles and Four-Wheel-Drive Karmas........................................................................................................ 43
4.5 Starting from Where You Are .............................................................................................................................. 56
4.6 Arguments You Need to Win.............................................................................................................................. 59
5
Testing and the Web............................................................................................................................65
5.1 How Websites Work ............................................................................................................................................ 66
5.2 Website Risks and Problems ............................................................................................................................... 67
5.3 Web Test Planning ............................................................................................................................................... 69
5.4 Web Performance................................................................................................................................................. 71
5.5 Web Tests.............................................................................................................................................................. 72
5.6 Monitoring Issues ................................................................................................................................................ 82
6
The Overall Test Life-Cycle.................................................................................................................83
6.1 Initiation Phase.................................................................................................................................................... 83
6.2 Requirements Analysis Phase.............................................................................................................................. 85
6.3 Architectural Definition Phase ............................................................................................................................ 87
6.4 Coding and Unit Testing..................................................................................................................................... 88
6.5 Software/Hardware Integration ........................................................................................................................... 88
6.6 Test Readiness Review ......................................................................................................................................... 89
6.7 System-Testing Phase........................................................................................................................................... 89
6.8 Beta Testing .......................................................................................................................................................... 90
6.9 Certification- or Acceptance-Testing Phase ........................................................................................................ 90
6.10 Documentation, Help, and Training Phase ........................................................................................................ 91
6.11 Install, Coexist, Cutover, Support, and Evolution Phase ................................................................................... 91
7
Testing Processes and Infrastructure..................................................................................................93
7.1 Initiation............................................................................................................................................................... 93
7.2 Keeping the Configuration Management System in Order............................................................................... 94
7.3 Test Environment................................................................................................................................................. 96
7.4 Test Automation................................................................................................................................................... 97
7.5 Monitoring Test Progress .................................................................................................................................. 101
7.6 Bug Management Process and Tool .................................................................................................................. 101
7.7 Operational Profiles........................................................................................................................................... 109
7.8 Static Analyzers .................................................................................................................................................. 115
7.9 Dynamic Analyzers ............................................................................................................................................ 119
8
Test Documents ................................................................................................................................125
8.1 General ............................................................................................................................................................... 125
8.2 Requirements Specification............................................................................................................................... 125
8.3 Test Strategy Document .................................................................................................................................... 130
8.4 Interface Specification ....................................................................................................................................... 132
8.5 Test Plan ............................................................................................................................................................. 133
8.6 Test Monitoring Document ............................................................................................................................... 142
8.7 Test Design Specification .................................................................................................................................. 148
8.8 Test Case Specification ...................................................................................................................................... 149
8.9 Test Procedure Specification .............................................................................................................................. 151
8.10 Release Note ....................................................................................................................................................... 152
8.11 Test Log .............................................................................................................................................................. 153
8.12 Bug Report ......................................................................................................................................................... 154
8.13 Test Summary .................................................................................................................................................... 156
8.14 Usability Test Summary .................................................................................................................................... 157
8.15 Acceptance Certificate ....................................................................................................................................... 158
8.16 Security Risk Analysis Report ............................................................................................................................ 158
8.17 Risk Log.............................................................................................................................................................. 160
8.18 Daily Test Report ............................................................................................................................................... 160
8.19 Reporting Using TestDirector? (now Quality CentER) ................................................................................. 162
8.20 Choosing a Set of Test Documents ................................................................................................................... 166
8.21 Quality Plan ....................................................................................................................................................... 167
9
The Test Team and Its Context.........................................................................................................171
9.1 Recruiting the Test Team................................................................................................................................... 171
9.2 Test Qualifications ............................................................................................................................................. 175
9.3 Test Team Structures ......................................................................................................................................... 175
9.4 Assessing the Test Team .................................................................................................................................... 176
9.5 Motivating the Test Team.................................................................................................................................. 177
9.6 Career Progression............................................................................................................................................. 177
9.7 Relations ............................................................................................................................................................. 178
9.8 The Politics of Testing ....................................................................................................................................... 185
10
Outsourcing ......................................................................................................................................187
10.1 Dangers of Outsourcing.................................................................................................................................... 187
10.2 Process of Outsourcing ..................................................................................................................................... 187
10.3 Backsourcing Considerations............................................................................................................................ 202
10.4 Testing Proprietary Software............................................................................................................................. 203
11
Test Techniques: Functional (Black-Box) ........................................................................................205
11.1 Equivalence Partitioning ................................................................................................................................... 205
11.2 Boundary-Value Analysis .................................................................................................................................. 205
11.3 Cause–Effect Graphing...................................................................................................................................... 206
11.4 Random Data Selection..................................................................................................................................... 207
11.5 Feature Test Process........................................................................................................................................... 208
12
Test Techniques: Structural (White-Box) ........................................................................................209
12.1 Statement Coverage ........................................................................................................................................... 209
12.2 Decision (Branch) Coverage............................................................................................................................. 209
12.3 Condition Coverage........................................................................................................................................... 210
12.4 Path Analysis ...................................................................................................................................................... 211
12.5 All-DU-Paths Coverage ..................................................................................................................................... 213
12.6 Execution-Time and Resource-Use Analysis .................................................................................................... 213
12.7 Algorithm Analysis ............................................................................................................................................ 213
12.8 Linear Code Sequence and Jump (LCSAJ) ....................................................................................................... 214
12.9 Dynamic Analysis Techniques ........................................................................................................................... 215
12.10 Derivative Techniques ....................................................................................................................................... 217
13
Test Techniques: Static Analysis........................................................................................................219
13.1 Static Dataflow Analysis .................................................................................................................................... 220
13.2 Dataflow Bugs .................................................................................................................................................... 221
13.3 Control-Flow Analysis ....................................................................................................................................... 221
13.4 Function Value Analysis .................................................................................................................................... 222
13.5 Symbolic Execution ........................................................................................................................................... 223
13.6 Mutation Testing................................................................................................................................................ 222
13.7 Mutation Testing Problems............................................................................................................................... 223
13.8 Fault Injection.................................................................................................................................................... 224
13.9 Test Data Generators ......................................................................................................................................... 228
13.10 Failure Mode Effects and Criticality Analysis................................................................................................... 229
14
Unit Test Process...............................................................................................................................231
14.1 Managing Unit Testing ...................................................................................................................................... 231
14.2 Purpose of Unit Testing .................................................................................................................................... 231
14.3 Identify the Test Baseline .................................................................................................................................. 232
14.4 Identify Unit Characteristics............................................................................................................................. 233
14.5 Unit Test Development ..................................................................................................................................... 235
14.6 Why the System Test Group Should (Not) Unit Test ...................................................................................... 237
14.7 What We Have to Do to Unit Test .................................................................................................................... 238
14.8 Unit Test Evaluation .......................................................................................................................................... 239
14.9 Unit-Test Tool Criteria ...................................................................................................................................... 240
15
System and Acceptance Testing ........................................................................................................243
15.1 System Test Process (Figure 15.1) ..................................................................................................................... 243
15.2 Plan the System Tests ........................................................................................................................................ 246
15.3 Design and Code the Tests................................................................................................................................ 250
15.4 System Test Coverage ........................................................................................................................................ 254
15.5 Is It Worth All This Work?................................................................................................................................ 256
15.6 Test Support Software and Tools....................................................................................................................... 260
15.7 Regression Testing.............................................................................................................................................. 260
15.8 Localization Testing ........................................................................................................................................... 261
15.9 System Integration and Operations Testing ..................................................................................................... 262
15.10 Non-Functional Testing .................................................................................................................................... 262
15.11 Conduct a Test Readiness Review ..................................................................................................................... 272
15.12 Beta Testing Process........................................................................................................................................... 273
15.13 Acceptance Testing............................................................................................................................................. 274
15.14 Test a Build......................................................................................................................................................... 275
15.15 Monitor the Test Process................................................................................................................................... 277
15.16 Report ................................................................................................................................................................. 280
15.17 Agree That the Release is Ready ........................................................................................................................ 280
16
Performance and Stress Testing........................................................................................................281
16.1 General ............................................................................................................................................................... 281
16.2 Service Level Agreements.................................................................................................................................. 281
16.3 Performance Testing: Tool Use .......................................................................................................................... 282
16.4 Performance Testing: Requirements.................................................................................................................. 283
16.5 What is the Baseline for Performance Tests? .................................................................................................... 283
16.6 Performance Testing: Process............................................................................................................................ 284
16.7 Stress Testing ...................................................................................................................................................... 285
16.8 Reliability Testing............................................................................................................................................... 287
17
Usability Test Process ........................................................................................................................289
17.1 Why We Should Usability Test........................................................................................................................... 289
17.2 When Should We Usability Test?....................................................................................................................... 289
17.3 What is the Baseline for UI Tests?..................................................................................................................... 290
17.4 How to Usability Test ........................................................................................................................................ 290
17.5 Readability Indices............................................................................................................................................. 296
17.6 Task Analysis ...................................................................................................................................................... 297
17.7 Who Is Involved in Usability Testing?............................................................................................................... 298
17.8 How Will We Know if We’ve Usability-Tested Successfully? ........................................................................... 299
17.9 What It’ll Cost Us to Usability Test................................................................................................................... 300
17.10 Why We Shouldn’t Usability Test ...................................................................................................................... 301
17.11 Usability-Testing Deliverables........................................................................................................................... 301
17.12 Usability Test Specification ............................................................................................................................... 302
17.13 Usability Test Warnings..................................................................................................................................... 304
18
Metrics...............................................................................................................................................305
18.1 Functionality Metrics ........................................................................................................................................ 305
18.2 Testability............................................................................................................................................................ 309
18.3 Performance Metrics ......................................................................................................................................... 312
18.4 Resilience (Robustness) Metrics ........................................................................................................................ 316
18.5 Usability Metrics ................................................................................................................................................ 320
18.6 Reliability............................................................................................................................................................ 324
18.7 Availability.......................................................................................................................................................... 327
18.8 Maintainability Metrics ..................................................................................................................................... 329
18.9 Process Metrics .................................................................................................................................................. 331
18.10 Estimates of Bugs in the System........................................................................................................................ 338
18.11 Function Point Metric....................................................................................................................................... 351
18.12 Test Case Estimation ......................................................................................................................................... 354
18.13 Cost of Finding and Fixing a Coding Bug ....................................................................................................... 358
18.14 Some Metrics from the “Real” World ............................................................................................................... 360
18.15 Multiple Regression Analysis ............................................................................................................................. 361
A
Examples ...........................................................................................................................................365
A.1 Case Study: Getting Test Objectives Out of a Use Case .................................................................................. 365
A.2 Case Study: Withdraw Cash Use Case .............................................................................................................. 365
A.3 Case Study: Getting Test Objectives Out of Business Rules............................................................................ 370
A.4 Case Study: Test Coverage................................................................................................................................. 373
A.5 Example: Requirements Traceability Using TestDirector?............................................................................. 379
A.6 Example: How to Get a Report Out of TestDirector ....................................................................................... 379
A.7 Case Study: Performance Testing (Throughput) ............................................................................................. 386
A.8 Case Study: Performance Testing (Bottlenecks and Critical Variables).......................................................... 389
A.9 Example: End-of-System-Test Report ............................................................................................................... 399
A.10 Example: Test Function Report ......................................................................................................................... 404
A.11 Case Study: Testing a Formal Requirements Specification (TransitCard Ticketing System)........................ 406
B
Checklists ..........................................................................................................................................417
B.1 Test Manager’s Terms of Reference ................................................................................................................... 417
B.2 What Test Managers Do Checklist .................................................................................................................... 420
B.3 Communication, Team Compatibility, and Motivation Checklist.................................................................. 424
B.4 Bid Review Checklist ......................................................................................................................................... 424
B.5 Requirements Analysis Checklist...................................................................................................................... 424
B.6 Risk Management Checklist.............................................................................................................................. 426
B.7 Test Planning Checklists.................................................................................................................................... 427
B.8 Quality Review Checklist .................................................................................................................................. 429
B.9 Localization Checklist ....................................................................................................................................... 429
B.10 Safety Testing Checklist..................................................................................................................................... 430
B.11 Security Testing Checklist ................................................................................................................................. 433
B.12 Usability Checklist ............................................................................................................................................. 437
B.13 Web Issues Checklist.......................................................................................................................................... 440
B.14 Performance Checklists ..................................................................................................................................... 442
B.15 Coding and Unit Testing Review Checklist ...................................................................................................... 446
B.16 Software–Software Integration Test Review Checklist..................................................................................... 447
B.17 System Test Review Checklist ............................................................................................................................ 448
B.18 Operations Acceptance Checklist ...................................................................................................................... 448
B.19 Metrics Checklist ............................................................................................................................................... 451
B.20 Very Wonderful New Approaches Checklist..................................................................................................... 451
C
Category and Priority Classifications for Bug Reporting...............................................................453
C.1 Purpose............................................................................................................................................................... 453
C.2 Classification by Category................................................................................................................................. 454
C.3 Classification by Severity Approach 1............................................................................................................... 454
C.4 Classification by Severity Approach 2............................................................................................................... 455
C.5 Beizer’s Taxonomy ............................................................................................................................................. 455
C.6 Bug Severity Calculation Matrix ....................................................................................................................... 456
D
Tools ..................................................................................................................................................457
D.1 Automated Testing Tools................................................................................................................................... 457
D.2 Dynamic Analysis Tools .................................................................................................................................... 457
D.3 Executable
UML
Tools ...................................................................................................................................... 458
D.4 Formal Methods’ Tools...................................................................................................................................... 458
D.5 Localization Links.............................................................................................................................................. 458
D.6 Metrics Tools...................................................................................................................................................... 459
D.7 Performance-Testing Tools................................................................................................................................ 459
D.8 Risk Tools ........................................................................................................................................................... 460
D.9 Security Tools..................................................................................................................................................... 461
D.10 Source Code Comprehension and Analysis Tools............................................................................................ 461
D.11 Static Analysis Tools .......................................................................................................................................... 461
D.12 Unit Testing........................................................................................................................................................ 462
D.13 Web Tools ........................................................................................................................................................... 462
D.14 Web Usability Test Tools and Facilities ............................................................................................................. 463
D.15 Web Security ...................................................................................................................................................... 463
D.16 Other Tools......................................................................................................................................................... 463
Glossary.............................................................................................................................................467
Abbreviations Used ........................................................................................................................................................ 467
Terms Used ..................................................................................................................................................................... 470
References..........................................................................................................................................513
Books and Papers ........................................................................................................................................................... 513
Related Standards........................................................................................................................................................... 520
Index..................................................................................................................................................527
投一票:
(0)
(0)
