User:Hyad/SE/Test4

=Code Inspections=

Motivation
expensive testing costs almost 100 times more to fix than if it had been found and fixed at requirements itself all other deliverables produced from it: change the design, change the code, rerun tests! cheaper they are to fix
 * Detecting errors late in the development cycle is
 * E.g. A requirements defect that is found only at
 * Need to rework not only the requirements doc, but
 * The earlier in the lifecycle we find problems, the

Multiple stages of defect removal

 * If we inspect each deliverable (requirements, design, code), and then do multiple stages of testing (unit tests, integration tests, system tests), then we get many chances to find defects
 * Like filtering multiple times: the result is much cleaner!

Defect Prevention

 * In addition to removing defects through inspections, we can eliminate defects using
 * Checklists: common mistakes, concerns to address, activities to do
 * Templates: standard document formats that list the different aspects to be covered
 * Reduce work and avoid incompleteness

Inspections
and identify opportunities for improvement development finding problems and possible improvements identify the problems that need fixing
 * A group of people review an artifact (code, documents) to find defects
 * Can be used for any document or code produced during the
 * Preferably, all major development artifacts should be inspected
 * Each reviewer spends several hours going through the artifact and
 * Hold a review meeting to discuss the inputs from each reviewer and
 * Fix problems, re-review if necessary
 * A team of 5 can review about 200 lines of code per hour

Other benefits of Inspections

 * Team members get familiar with the code
 * Backup if someone is unavailable
 * More uniform design and coding practices across team
 * Knowledge sharing
 * Shared understanding & improved communication
 * Identify miscommunication and misperceptions
 * More perspective on how everything comes together

Need for inspection processes
something is really a problem or not solutions, wastes everyone else’s time
 * Some reviewers may not come prepared to the meeting
 * Waste everyone’s time while they read artifact!
 * Maybe because they did not have the artifact in advance
 * Meeting becomes chaotic as people debate whether
 * Meeting tries to solve the problems found
 * Only one or two people may be discussing and figuring out
 * Problems get found, but later on, get forgotten about
 * All that effort put in is wasted!


 * The inspection meeting is about identifying problems & potential problems and recording them - it's not about fixing the problems
 * You can schedule another meeting where you fix problems after inspection meeting

Formal inspection process
Defined roles: Author, Reader, Moderator, Scribe, Inspectors


 * Author distributes the artifact ahead of time, arranges time for meeting
 * Reader interprets the code for the inspectors - you don't want Author to be the Reader
 * If reader is different from author, reduces possibility of author propagating their own misunderstandings
 * Inspectors prepare comments before meeting, provide their inputs and contribute to discussions during meeting - usually 3 or so inspectors
 * Moderator keeps the discussions on track, also responsible for checking later that the problems found have been fixed
 * Scribe ensures that problems found get recorded


 * You can keep Reader & Moderatoe & Scribe together
 * Have more than 1 Author for the meeting - so one person isn't critized for the whole meeting

Inspection Meeting Preparation Process
• Author prepares the artifact to be inspected and makes it available to the moderator • Preferably at least 2 days prior to meeting • Give inspectors time to prepare • Moderator obtains the inspection checklist and other support material • Moderator distributes materials to all inspectors • Inspectors inspect the artifact prior to coming to the meeting and make their own notes state (Requirements documents, other design documents, library documentation, interface specs, etc.)
 * State of materials
 * All materials to be inspected must be in a “ready for inspection”
 * Design documents must be in their final format
 * Code must be line-numbered, and available in hard-copy
 * Checklists must be complete
 * All support materials must be consistent with the artifact

Inspection Meeting Process
artifact being inspected reading it line by line
 * The meeting is called to order by the moderator
 * The reader interprets for the team as he/she understands what is in the
 * In code inspections, the reader paraphrases the code as opposed to

Avoid personal attacks on the producer
issues
 * Avoid personal attacks on the producer
 * In code reviews avoid discussions of style, concentrate on important
 * Inspection meetings must not last more than two hours

=Testing=
 * http://www.se.rit.edu/~se361/slides/TestingPhases.pdf

Testing Phases

 * Unit Testing
 * Developer tests individual modules
 * Usually glass box
 * Integration testing
 * Put modules together, try to get them working together
 * Integration testing is complete when the different pieces are able to work together
 * System testing
 * Black-box testing of entire deliverable against specs
 * Acceptance testing
 * Testing against user needs, often by the user

System Testing
delivery, based on requirements specification
 * Thorough black-box testing of entire software before
 * Final phase of testing before delivery
 * Requirements coverage is a must, equivalence class coverage preferred
 * For most commercial software, reasonably good coverage of combinations of equivalence classes is the norm
 * Can automate system testing by creating “test harnesses” that feed test inputs and check outputs

Integration Testing
of the problem test case will run through!
 * Integration: Putting together the code created by different developers and trying to run an end-to-end test case
 * Can be a major challenge, because every small misunderstanding/miscommunication between developers results in mismatches that need to be found and removed
 * Each time it doesn’t work, need to trace which code is the source
 * Not uncommon to need to resolve dozens of bugs before that first
 * Integration testing becomes easier if
 * Unit testing is thorough
 * Interfaces between modules are well-defined

1. Big bang integration
working together
 * Put all the code together in one go, and try to get it all
 * -you push the "Run" button
 * Works only for relatively small projects

2. Incremental integration
-takes longer appparently, but shorter in the long run Software Engineering I – SE361
 * Test each individual module in isolation
 * Start with one module, add one module at a time, work out problems among them before adding next
 * More manageable, but more work
 * 2 ways to do incremental integration
 * 1) “Horizontal” integration if there are multiple subsystems
 * 2) “Vertical” integration if there are multiple layers

Vertical integration
you either build stubs or drivers

Top down integration
-you have to write tons of stubs

layers. the stubs.
 * Start by testing just the user interface.
 * The underlying functionality are simulated by stubs.
 * Pieces of code that have the same interface as the lower level functionality.
 * Do not perform any real computations or manipulate any real data.
 * Then you work downwards, integrating lower and lower
 * Advantage: Easy to see the behavior at each stage
 * The big drawback to top down testing is the cost of writing

Bottom-up integration
make calls to these lower layers. not be easy to visualize'''
 * Start by testing the very lowest levels of the software.
 * You needs drivers to test these lower layers of software.
 * Drivers are simple programs designed specifically for testing that
 * Drivers in bottom-up testing have a similar role to stubs in top-down testing, and are time-consuming to write.
 * Advantage: Each piece being tested is complete in itself
 * Problem: Need to write lots of drivers, '''functionality may

Sandwich testing
down testing. layer remains on which to perform the final set of tests.
 * Sandwich testing is a hybrid between bottom-up and top
 * Test the user interface in isolation, using stubs.
 * Test the very lowest level functions, using drivers.
 * When the complete system is integrated, only the middle

The test-fix-test cycle
When a failure occurs during testing:


 * Each failure report is entered into a failure tracking system.
 * It is then screened and assigned a priority.
 * Low-priority failures might be put on a known bugs list that is included with the software’s release notes.
 * Some failure reports might be merged if they appear to result from the same defects.
 * Somebody is assigned to investigate a failure.
 * That person tracks down the defect and fixes it.
 * Finally a new version of the system is created, ready to be tested again.

The ripple effect
state
 * The person that analysizes the software does NOT fix the problem he/she finds
 * There is a high probability that the efforts to remove the defects may have actually added new defects
 * The maintainer tries to fix problems without fully understanding the ramifications of the changes
 * The maintainer makes ordinary human errors
 * The system regresses into a more and more failure-prone
 * Use regression testing to rerun important tests

Regression Testing
case every time a change is made to software. is actually re-run. much of the system as possible. proportional to the number of bugs already fixed''' - you often inject as many defects to the system when you try to correct defects
 * It tends to be far too expensive to re-run every single test
 * Hence only a subset of the previously-successful test cases
 * This process is called regression testing.
 * The tests that are re-run are called regression tests.
 * Regression test cases are carefully selected to cover as
 * The “law of conservation of bugs”:
 * '''The number of bugs remaining in a large system is

Deciding when to stop testing

 * What is defined as level 1, 2, 3 defects is dependant on your software
 * Does some not get paid because of your system- important
 * Or does a web menu not get updated - not important

successfully executed. cases must have been executed successfully.
 * All of the level 1 (“critical”) test cases must have been
 * Certain pre-defined percentages of level 2 and level 3 test

Test planning

 * testing is as important as the code
 * if ther are no tests, hwo do you know if code works

Software Engineering I – SE361
 * Decide on overall test strategy
 * What type of integration
 * Whether to automate system tests
 * Whether there is an independent test team
 * Decide on the coverage strategy for system tests
 * Compute the number of test cases needed
 * Identify the test cases and implement them
 * The set of test cases constitutes a “test suite”
 * May categorize into critical, important, optional tests (level 1, 2, 3)
 * Identify a subset of the tests as regression tests

Testing performed by users and clients

 * Alpha testing - very limited release - very controlled
 * Beta testing - Performed by the user or client in a normal work environment.
 * Acceptance testing - Performed by users and customers.

Inspect software before testing
testing it. get rid of many defects.
 * It is important to inspect software before extensively
 * The reason for this is that inspecting allows you to quickly

Cross-Tem testing
better software
 * Each team will serve as “independent test team” for another team
 * Package your R2 software for delivery to other team
 * Create a JAR file, README with execution instructions, requirements doc and acceptance test doc
 * Designate someone in your team to serve as support contact
 * Run acceptance test cases on received software, check for pass/fail
 * If failed, note down what inputs were and how it failed
 * Perform any other testing you consider necessary
 * Can add missed test cases to acceptance test doc
 * Can include any other comments at bottom of acceptance test doc
 * Deliver cross-team test report to other team
 * Objective: To test the software thoroughly, help other team produce