User:Fhcnkdjm/sandbox

software Chapter 8: Software Testing�L1-L2-L3

PROGRAM TESTING: Testing is intended to show that a program does what it is intended to do and to discover program defects before it is put into use. When you test software, you execute a program using artificial data. You check the results of the test run for errors, anomalies or information about the program’s non-functional attributes. Can reveal the presence of errors NOT their absence. Testing is part of a more general verification and validation process, which also includes static validation techniques.

GOALS: To demonstrate to the developer and the customer that the software meets its requirements. - For custom software, this means that there should be at least one test for every requirement in the requirements document. For generic software products, it means that there should be tests for all of the system features, plus combinations of these features, that will be incorporated in the product release.

To discover situations in which the behavior of the software is incorrect, undesirable or does not conform to its specification. - Defect testing is concerned with rooting out undesirable system behavior such as system crashes, unwanted interactions with other systems, incorrect computations and data corruption.

VALIDATION AND DEFECT TESTING:

The first goal leads to validation testing You expect the system to perform correctly using a given set of test cases that reflect the system’s expected use. The second goal leads to defect testing - The test cases are designed to expose defects. The test cases in defect testing can be deliberately obscure and need not reflect how the system is normally used.

TESTING PROCESS GOALS:

Validation testing - To demonstrate to the developer and the system customer that the software meets its requirements A successful test shows that the system operates as intended. Defect testing - To discover faults or defects in the software where its behaviour is incorrect or not in conformance with its specification - A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system.

INSPECTIONS AND TESTING:

Software inspections Concerned with analysis of  the static system representation to discover problems  (static verification) May be supplement by tool-based document and code analysis. Software testing Concerned with exercising and observing product behavior (dynamic verification) - The system is executed with test data and its operational behavior is observed. Inspections and testing are complementary and not opposing verification techniques. Both should be used during the V & V process.

ADVANTAGES OF INSPECTION: During testing, errors can mask (hide) other errors. Because inspection is a static process, you don’t have to be concerned with interactions between errors.

STAGES OF TESTING: Development testing, where the system is tested during development to discover bugs and defects. Release testing, where a separate testing team test a complete version of the system before it is released to users. User testing, where users or potential users of a system test the system in their own environment.

DEVELOPMENT TESTING: Development testing includes all testing activities that are carried out by the team developing the system. Unit testing, where individual program units or object classes are tested. Unit testing should focus on testing the functionality of objects or methods. Component testing, where several individual units are integrated to create composite components. Component testing should focus on testing component interfaces. System testing, where some or all of the components in a system are integrated and the system is tested as a whole. System testing should focus on testing component interactions.

UNIT TESTING: Unit testing is the process of testing individual components in isolation. It is a defect testing process. Units may be: Individual functions or methods within an object. Object classes with several attributes and methods Composite components with defined interfaces used to access their functionality.

AUTOMATED TEST COMPONENTS: A setup part, where you initialize the system with the test case, namely the inputs and expected outputs. A call part, where you call the object or method to be tested. An assertion part where you compare the result of the call with the expected result. If the assertion evaluates to true, the test has been successful  if false, then it has failed.

TESTING STRATAGIES: Partition testing, where you identify groups of inputs that have common characteristics and should be processed in the same way. You should choose tests from within each of these groups. Guideline-based testing, where you use testing guidelines to choose test cases. - These guidelines reflect previous experience of the kinds of errors that programmers often make when developing components.

PARTITION TESTING: Input data and output results often fall into different classes where all members of a class are related. Each of these classes is an equivalence partition or domain where the program behaves in an equivalent way for each class member. Test cases should be chosen from each partition.

TEST DRIVEN DEVELOPMENT (TDD): Test-driven development (TDD) is an approach to program development in which you inter-leave testing and code development. Tests are written before code and ‘passing’ the tests is the critical driver of development. You develop code incrementally, along with a test for that increment. You don’t move on to the next increment until the code that you have developed passes its test. TDD was introduced as part of agile methods such as Extreme Programming. However, it can also be used in plan-driven development processes.

TDD PROCESS ACTIVITIES: Start by identifying the increment of functionality that is required. This should normally be small and implementable in a few lines of code. Write a test for this functionality and implement this as an automated test. Run the test, along with all other tests that have been implemented. Initially, you have not implemented the functionality so the new test will fail. Implement the functionality and re-run the test. Once all tests run successfully, you move on to implementing the next chunk of functionality.

REGRESSION TESTING: Regression testing is testing the system to check that changes have not ‘broken’ previously working code. In a manual testing process, regression testing is expensive but, with automated testing, it is simple and straightforward. All tests are rerun every time a change is made to the program. Tests must run ‘successfully’ before the change is committed.�

Chapter 8 : Software Testing L4-L5:

RELEASE TESTING: Release testing is the process of testing a particular release of a system that is intended for use outside of the development team. The primary goal of the release testing process is to convince the supplier of the system that it is good enough for use. Release testing, therefore, has to show that the system delivers its specified functionality, performance and dependability, and that it does not fail during normal use. Release testing is usually a black-box testing process where tests are only derived from the system specification.

RELEASE TESTING AND SYSTEM TESTING: Release testing is a form of system testing. Important differences: A separate team that has not been involved in the system development, should be responsible for release testing. System testing by the development team should focus on discovering bugs in the system (defect testing). The objective of release testing is to check that the system meets its requirements and is good enough for external use (validation testing).

PERFORMANCE TESTING: Part of release testing may involve testing the emergent properties of a system, such as performance and reliability. Tests should reflect the profile of use of the system. Performance tests usually involve planning a series of tests where the load is steadily increased until the system performance becomes unacceptable. Stress testing is a form of performance testing where the system is deliberately overloaded to test its failure behaviour.

USER TESTING: User or customer testing is a stage in the testing process in which users or customers provide input and advice on system testing. User testing is essential, even when comprehensive system and release testing have been carried out. - The reason for this is that influences from the user’s working environment have a major effect on the reliability, performance, usability and robustness of a system. These cannot be replicated in a testing environment.

TYPES OF USER TESTING: Alpha testing : Users of the software work with the development team to test the software at the developer’s site. Beta testing: A release of the software is made available to users to allow them to experiment and to raise problems that they discover with the system developers. Acceptance testing: Customers test a system to decide whether or not it is ready to be accepted from the system developers and deployed in the customer environment.Primarily for custom systems.

STAGES IN THE ACCEPTANCE TESTING PROCESS: Define acceptance criteria. Plan acceptance testing. Derive acceptance tests. Run acceptance tests. Negotiate test results. Reject/accept system.

Chapter 9: Software Evolution�L1-L2�

A spiral model of development and evolution� 1)SPECIFICATION 2)IMPLEMENTATION 3) OPERATION 4) VALIDATION

Evolution: The stage in a software system’s life cycle where it is in operational use and is evolving as new requirements are proposed and implemented in the system. Servicing: At this stage, the software remains useful but the only changes made are those required to keep it operational i.e. bug fixes and changes to reflect changes in the software’s environment. No new functionality is added. Phase-out: The software may still be used but no further changes are made to it.

CHANGE IN IMPLEMTATION: Iteration of the development process where the revisions to the system are designed, implemented and tested. A critical difference is that the first stage of change implementation may involve program understanding, especially if the original system developers are not responsible for  the change implementation. During the program understanding phase, you have to understand how the program is structured, how it delivers functionality and how the proposed change might affect the program.

URGENT CHANGE REQUEST: Urgent changes may have to be implemented without going through all stages of the software engineering process. If a serious system fault has to be repaired to allow normal operation to continue; If changes to the system’s environment (e.g. an OS upgrade) have unexpected effects; If there are business changes that require a very rapid response (e.g. the release of a competing product).

AGILE METHOD AND EVOLUTION: Agile methods are based on incremental development so the transition from development to evolution is a seamless one. Evolution is simply a continuation of the development process based on frequent system releases. Automated regression testing is particularly valuable when changes are made to a system. Changes may be expressed as additional user stories.

Chapter 9 : Legacy system L3-L4

LEGACY SYSTEMS: Legacy systems are older systems that rely on languages and technology that are no longer used for new systems development. Legacy software may be dependent on older hardware, such as mainframe computers and may have associated legacy processes and procedures. Legacy systems are not just software systems but are broader socio-technical systems that include hardware, software, libraries and other supporting software and business processes.

LEGACY SYSTEM COMPONENTS: System hardware Legacy systems may have been written for hardware that is no longer available. Support software The legacy system may rely on a range of support software. Application software The application system that provides the business services is usually made up of a number of application programs. Application data These are data that are processed by the application system. They may be inconsistent, duplicated or held in different databases.

Business processes These are processes that are used in the business to achieve some business objective. Business processes may be designed around a legacy system and constrained by the functionality that it provides. Business policies and rules These are definitions of how the business should be carried out and constraints on the business. Use of the legacy application system may be embedded in these policies and rules.

LEGACY SYSTEM LAYERS:

1) BUSINESS PROCESS 2) APPLICATION SOFTWARE 3) PLATFORM AND INFRASTRUCTURE SOFTWARE 4) HARDWARE

Low quality, low business value : These systems should be scrapped. Low-quality, high-business value: These make an important business contribution but are expensive to maintain. Should be re-engineered or replaced if a suitable system is available. High-quality, low-business value : Replace with COTS, scrap completely or maintain. High-quality, high business value: Continue in operation using normal system maintenance.

The use of the system If systems are only used occasionally or by a small number of people, they may have a low business value. The business processes that are supported A system may have a low business value if it forces the use of inefficient business processes. System dependability If a system is not dependable and the problems directly affect business customers, the system has a low business value. The system outputs - If the business depends on system outputs, then the system has a high business value.�

Business process assessment How well does the business process support the current goals of the business? Environment assessment How effective is the system’s environment and how expensive is it to maintain? Application assessment - What is the quality of the application software system?

TYPES OF MAINTENANCE: Fault repairs : Changing a system to fix bugs/vulnerabilities and correct deficiencies in the way meets its requirements. Environmental adaptation : Maintenance to adapt software to a different operating environment Changing a system so that it operates in a different environment (computer, OS, etc.) from its initial implementation. Functionality addition and modification : - Modifying the system to satisfy new requirements.

MAINTENANCE COST: Usually greater than development costs. Affected by both technical and non-technical factors. Increases as software is maintained. Maintenance corrupts the software structure so makes further maintenance more difficult. Ageing software can have high support costs (e.g. old languages, compilers etc.).

Reduced risk: There is a high risk in new software development. There may be development problems, staffing problems and specification problems. Reduced cost : The cost of re-engineering is often significantly less than the costs of developing new software.

REFACTORING: Refactoring is the process of making improvements to a program to slow down degradation through change. You can think of refactoring as ‘preventative maintenance’ that reduces the problems of future change. Refactoring involves modifying a program to improve its structure, reduce its complexity or make it easier to understand. When you refactor a program, you should not add functionality but rather concentrate on program improvement.

Chapter 10: Dependable systems L1-L2-L3

SYSTEM DEPENDABILITY: Dependability covers the related systems attributes of reliability, availability and security. These are all inter-dependent.

CAUSES OF FAILURE: Hardware failure : Hardware fails because of design and manufacturing errors or because components have reached the end of their natural life. Software failure: Software fails due to errors in its specification, design or implementation. Operational failure: - Human operators make mistakes. Now perhaps the largest single cause of system failures in socio-technical systems.

PRINCIPAL PROPERTIES: Availability : The probability that the system will be up and running and able to deliver useful services to users. Reliability : The probability that the system will correctly deliver services as expected by users. Safety: A judgment of how likely it is that the system will cause damage to people or its environment. Security: A judgment of how likely it is that the system can resist accidental or deliberate intrusions. Resilience: - A judgment of how well a system can maintain the continuity of its critical services in the presence of disruptive events such as equipment failure and cyberattacks.

OTHER DEPENDABILITY PROPERTIES: Repairability: Reflects the extent to which the system can be repaired in the event of a failure. Maintainability: Reflects the extent to which the system can be adapted to new requirements; Error tolerance: - Reflects the extent to which user input errors can be avoided and tolerated.

�The sociotechnical systems stack�: SOCIETY ORGANIZATION BUSINESS PROCESS APPLICATION SYSTEM COMMUNICATIONS AND DATA MANAGEMENT OPERATING SYSTEM EQUIPMENT

LAYERS IN THE STS STACK: Equipment : Hardware devices, some of which may be computers. Most devices will include an embedded system of some kind. Operating system: Provides a set of common facilities for higher levels in the system. Communications and data management: Middleware that provides access to remote systems and databases. Application systems: Specific functionality to meet some organization requirements. Business processes: A set of processes involving people and computer systems that support the activities of the business. Organizations: Higher level strategic business activities that affect the operation of the system. Society: Laws, regulation and culture that affect the operation of the system.

REDUNDANCY AND DIVERSITY:

Redundancy: Keep more than a single version of critical components so that if one fails then a backup is available.

Diversity: Provide the same functionality in different ways in different components so that they will not fail in the same way.

Redundant and diverse components should be independent so that they will not suffer from ‘common-mode’ failures - For example, components implemented in different programming languages means that a compiler fault will not affect all of them.

Redundancy: Where availability is critical (e.g. in e-commerce systems), companies normally keep backup servers and switch to these automatically if failure occurs. Diversity: To provide resilience against external attacks, different servers may be implemented using different operating systems (e.g. Windows and Linux)

Explicitly defined: A process that has a defined process model that is used to drive the software production process. Data must be collected during the process that proves that the development team has followed the process as defined in the process model. Repeatable: A process that does not rely on individual interpretation and judgment. The process can be repeated across projects and with different team members, irrespective of who is involved in the development.

CLASSES OF WRROR: Specification and design errors and omissions: - Developing and analysing a formal model of the software may reveal errors and omissions in the software requirements. If the model is generated automatically or systematically from source code, analysis using model checking can find undesirable states that may occur such as deadlock in a concurrent system.

Inconsistences between a specification and a program: - If a refinement method is used, mistakes made by developers that make the software inconsistent with the specification are avoided. Program proving discovers inconsistencies between a program and its specification.