Workflow diagram: quality assurance testing lifecycle

QA Testing
Software testing verifies that the software meets its requirements and that it is complete and ready for delivery.
Objectives of QA Testing

1. Assure the quality of client deliverable. 
2. Design, assemble, and execute a full testing life cycle. 
3. Confirm the full functional capabilities of the final product. 
4. Confirm stability and performance (response time, etc.) of the final product. 
5. Confirm that deliverable meet client expectations/requirements. 
6. Report, document and verify code and design defects.

Preparing for QA Testing
Prior to conducting formal software testing, QA develops testing documentation (including test plans, test specifications, and test procedures) and reviews the documentation for completeness and adherence to standards. QA confirms that:

1. The test cases are testing the software requirements in accordance with test plans. 
2. The test cases are verifiable. 
3. The correct or "advertised" version of the software is being tested (by QA monitoring of the Configuration Management activity).

QA then conducts the testing in accordance with procedure, documents and reports defects, and reviews the test reports.
The Key to Productive QA Testing
It is crucial to recognize that all testing will be conducted by comparing the final product to the product’s set requirements; therefore, product requirements must state all functionality of the software and must be updated as changes are made. Any functionality that does not meet the requirements will be recorded as a defect until resolution is delivered.
Twelve Types of QA Testing
1. Unit testing (conducted by Development)
Unit test case design begins after a technical review approves the high level design. The unit test cases shall be designed to test the validity of the program's correctness. White box testing is used to test the modules and procedures that support the modules. The white box testing technique ignores the function of the program under test and focuses only on its code and the structure of that code. To accomplish this, a statement and condition technique shall be used. Test case designers shall generate cases that not only cause each condition to take on all possible values at least once, but that cause each such condition to be executed at least once. In other words:

1. Each decision statement in the program shall take on a true value and a false value at least once during testing. 
2. Each condition shall take on each possible outcome at least once during testing. 

2. Configuration Management

1. The configuration management team prepares the testing environment

3. Build Verification
When a build has met completion criteria and is ready to be tested, the QA team runs an initial battery of basic tests to verify the build.

1. If the build is not testable at all, then the QA team will reject the build 
2. If portions of the website are testable and some portions are not yet available, the project manager, technical lead and QA team will reassign the build schedule and deliverable dates. 
3. If all portions of the build pass for testing, the QA team will proceed with testing 

4. Integration Testing

1. Integration testing proves that all areas of the system interface with each other correctly and that there are no gaps in the data flow. The final integration test proves that the system works as an integrated unit when all the fixes are complete.

5. Functional Testing

1. Functional testing assures that each element of the application meets the functional requirements of the business as outlined in the requirements document/functional brief, system design specification, and other functional documents produced during the course of the project (such as records of change requests, feedback, and resolution of issues).

6. Non-functional Testing (Performance Testing)

1. Non-functional testing proves that the documented performance standards or requirements are met. Examples of testable standards include response time and compatibility with specified browsers and operating systems.
2. If the system hardware specifications state that the system can handle a specific amount of traffic or data volume, then the system will be tested for those levels as well. 

7. Defect Fix Validation

1. If any known defects or issues existed during development, QA tests specifically in those areas to validate the fixes.

8. Ad Hoc Testing

1. This type of testing is conducted to simulate actual user scenarios. QA engineers simulate a user conducting a set of intended actions and behaving as a user would in case of slow response, such as clicking ahead before the page is done loading, etc.

9. Regression Testing

1. Regression testing is performed after the release of each phase to ensure that there is no impact on previously released software. Regression testing cannot be conducted on the initial build because the test cases are taken from defects found in previous builds.
2. Regression testing ensures that there is an continual increase in the functionality and stability of the software. 

10. Error Management

1. During the QA testing workflow, all defects will be reported using the error management workflow.
2. Regular meetings will take place between QA, system development, interface development and project management to discuss defects, priority of defects, and fixes.

11. QA Reporting

1. QA states the results of testing, reports outstanding defects/known issues, and makes a recommendation for release into production.

12. Release into production

1. If the project team decides that the build is acceptable for production, the configuration management team will migrate the build into production.

CONFIGURATION MANAGEMENT

The configuration management (CM) team is responsible for maintaining the validity of code as it moved from one environment to the next. The CM team takes part in the development workflow every time there is a change of code status. For example, when a software build phase is completed and ready to be tested, the code must be moved from the development environment to testing environment. This task belongs to the CM team.
CM follows a controlled workflow to promote and deploy all code. Additionally, CM is responsible for maintaining all source control databases (SCD) and the standards and processes related to SCD.
Objectives and Benefits

1. Improve the complete software development and maintenance cycle. 
2. Make QA testing easier, reliable, and effective. 
3. Remove error-prone steps from product release management. 
4. Support and ease the challenges of change management and defect tracking. 
5. Improve the quality and validity of code through version control. 
6. Provide code roll-back if necessary, or trace related code components. 
7. Maintain a history of code changes (who, what, when and why). 

CM Responsibilities
The CM team provides the following services throughout the course of product development and deployment:

1. Implement and maintain version control in development and testing environments. 
2. Prepare and configure QA servers per the specifications provided by system designers. 
3. Move code into the testing environment (servers and SCD) and establish the initial (or “baseline”) code base. 
4. Give the go-ahead to QA to commence testing after the code is moved. 
5. Simplified CM Workflow
6. CM’s role in software development and deployment may be summarized as follows:
7. In preparation for QA testing, CM moves all labeled files from the development SCD to the QA servers and the QA SCD, where they are labeled again. 
8. During QA testing, CM moves all new builds and defect fixes to the QA SCD and QA servers and labels them. 
9. When the files are approved, CM labels and moves them from the QA SCD into the production staging servers. 
10. Upon approval from project management and QA, CM promotes the final code base to production servers for deployment. 
11. Any defect fixes or enhancements after the initial build will also be deployed following the above steps. 

QA and Configuration Management

1. QA audits all CM functions for adherence to CM plans and procedures and prepares reports of its findings. QA reviews the CM plans for compliance with software CM policies and verifies that the plans include follow-up for defects. 
2. The CM functions monitored and audited by QA include baseline control, configuration identification, and configuration control. QA assures that:
3. Baselines are established and consistently maintained for use in subsequent baseline development and control. 
4. Approved changes to baseline software are made properly and consistently in all products, and no unauthorized changes are made.

Testing Roles and Responsibilities -Senior Test Leads,Test Leads,Software Testing Specialists

Senior Test Leads’ roles include the following:
Senior Test Leads

1. Lead a team of test leads on multiple projects including:
2. Supervise test leads and assist with test lead tasks as appropriate. 
3. Verify all projects are smoke, build, and regression tested on release build. 
4. Verify all projects have undergone at least two complete cycles of testing (all test cases executed) for the alpha and final release testing milestones. 
5. Verify test plans and other test documentation is prepared. 
6. Act as Test Lead on larger projects. 
7. Monitor releases and builds going out to production and track production bugs over time. 
8. Provide test documentation and reports to test and project management as required. 
9. Train and Mentor test leads. Verify all appropriate testing deliverable are met including: 
10. Weekly Roll-up Status Reports 
11. Test Schedules and requirements for test staffing. 
12. Test Environment prepared & maintained. 
13. Risk and Quality Assessments in conjunction with project test lead 
14. Final Release “Certification” Report 
15. Post Mortem Reports in conjunction with project test lead 
16. CD’s and Testing Manual of the project in conjunction with project test lead Provide the following services to Project Management 
17. Non team based sanity check for all software going out to production. 
18. Provide all above documentation 
19. Training, hiring, and mentoring for all contract test staff. 
20. Quality feedback for use in go/no go meetings and roll outs 
21. Make sure testing is on schedule and identify and work with project management to fix test issues that are delaying implementation Test Leads 

Test Leads’ roles include the following:
Lead a team of testers on a project including:

1. Supervise testers and allocate testing tasks to appropriate testers. 
2. Verify all test cases assigned to testers are executed. 
3. Verify all test cases assigned to testers are designed properly. 
4. Verify all bugs are put into bug tracking system. 
5. Verify all bugs marked resolved are regressed by the proper testers. 

Assist with testing.Verify all appropriate testing deliverable are met including:
Weekly Status Reports

1. Test Plans 
2. Test Schedules 
3. Bug tracking & TCM are setup 
4. Requirements for staffing, back end testing, and automation testing are identified. 
5. Test Cases 
6. Test Environment prepared & Maintained 
7. Bug Tracking & TCM Reports 
8. Build Acceptance Reports or Certification Reports 
9. Risk and Quality Assessments in conjunction with Senior test lead 

Release Checklists

1. Post Implementation Reports in conjunction with Senior test lead 
2. CD’s and Testing Manual of the project in conjunction with Senior test lead 
3. Act as member of core project team 
4. Provide the following services to Project Management 
5. Setup and run Bug Triages (Bug Review Meetings, minimum of weekly meeting once in development) 
6. Provide all above documentation 
7. Maintain bug tracking system, TCM, and the Test Environment 
8. Provide quality feedback for go/no go meetings and rollouts 

Software Testing Specialists
Test Specialists roles include the following:

1. Execute all assigned test cases. 
2. Design and prioritize test cases. 
3. Put bugs into bug tracking system. 
4. Retest resolved bugs on each release. 
5. Regresses existing bugs on previous software builds or alternate database/server configurations. 
6. Assist the test lead with test lead duties as needed. 
7. Provide feedback in Bug triages. 
8. Assist with preparation of test plans. 

In addition Test Specialists may be called to perform any of these tasks:
Automate test cases.

1. Test SQL Interfaces and have a working knowledge of SQL. 
2. Run Smoke tests. 
3. Build code from Source Safe or other Version Control utility. 
4. Run Build Acceptance tests on any code.