TAPAS Home

QA Plan

Release Information

Project:	TAPAS\|ems
Internal Release Number:	1.0
Release Audience:	Developer release
Attached Worksheets:
Related Documents:	Software Requirements Specification Design Project plan Software development methodology

Process impact: This document specifies quality goals, selects strategies for assuring that those goals have been met, and details a plan of action to carry out those strategies.

Introduction

"Quality" refers to all the good things that we would like to see in the TAPAS product. In particular, we adopt the standardized quality factor description according to ISO 9126, in which quality is defined in regard to the following factors:

    1.     Functionality - suitability, accuracy, interoperability, security, compliance, completeness
    2.     Reliability - maturity, fault tolerance, recoverability compliance
    3.     Usability - understandability, learnability, operability, attractiveness, compliance
    4.     Efficiency - time behavior, resource utilization, compliance
    5.     Maintainability - analyzability, changeability, stability, testability, compliance
    6.     Portability - adaptability, installability, co-existence, replaceability, compliance

The above terminology is used throughout this document and determines its structure. The TAPAS project plan schedules five mile stone releases. Each of these releases will include different, respectively revised versions of work products that must comply with different quality goals. An overview on the scheduled release plan, the work products to be produced and the quality goals to be achieved is given below. The next sections will define mechanisms used to assure the quality goals in detail.

Project phase	Schedule	Deliverables / Quality objectives
Elaboration	Jan 1- March 31, 2005	"Paperchild" release Specification Requirements Should be functionally complete, suitabile, easy to understand and should cover the security aspect Design Should maximize portability, adaptability, replaceability, and be compliant to standards Should detail mechanisms on how to implement security requirements Should detail repository draft structure for storing e-MS documents Implementation UI Prototypes (multiple screen captures) Should allow clarifying operability of thick client application
Construction I	April 1- June 31, 2005	"Straw man" release Specification Requirements Should be revised for accuracy Should be compliant to ISO/IEEE documentation standards Design Should give an accurate structural and behavioural description of the component-level architecture Implementation Functionality Interoperability with e-MS (import/export) Use cases related to viewing and adding basic summaries are implemented at the thick client Viewer implemented on PDA client Maintainability: Testability, understandability, installability Usability
Construction II	July 1- September 30, 2005	"Woodwoman" release Specification Design Should give an accurate structural and behavioural description of the component-level and class-level architecture Component-level architecture should comply to ISO/IEEE documentation standards Mission-critical parts of the specification should be formal Implementation Functionality All requirements implemented All security requirements implemented Maintainability: adaptability (plugin architecture) Usability Efficiency
Construction III	October 1 - December 31, 2005	"Ironcat" release Specification All parts of the specification should comply to ISO/IEEE documentation standards Compliance with implementation Implementation Reliability: correctness, fault tolerance, recoverable, maturity
Deployment	January 1 - March 31, 2006	"Diamonddog" release Specification User documentation Should be complete and understandable Implementation Usability, deployability

Quality assurance for paperchild release

Quality goal	Assurance technique
Requirements spec. should be functionally complete, suitabile, easy to understand and should cover the security aspect	Multiple rounds of internal requirements review cycles, including our internal domain expert (MD). Security part: external review of specification by security expert
Design spec. Should maximize portability, adaptability, replaceability, and be compliant to standards Should detail mechanisms on how to implement security requirements Should detail repository draft structure for storing e-MS documents	Platform evaluation with respect to different criteria Repository draft structure based on e-MS example document
Implementation UI Prototypes (multiple screen captures) Should allow clarifying operability of thick client application	Test against requirements specification

Quality assurance for strawman release

Quality goal	Assurance techniques
Requirements spec. Should be revised for accuracy Should be compliant to ISO/IEEE documentation standards	Comparison to ISO/IEEE documentation standards and best practices as documented in software engineeting literature
Design spec Should give an accurate structural and behavioural description of the component-level architecture	Comparison against component interface implementation and execution traces Design review by entire team
Implementation Functionality Interoperability with e-MS (import/export) Use cases related to viewing and adding basic summaries are implemented at the thick client Viewer implemented on PDA client Maintainability: Testability, understandability, installability Usability	Test of import and export functions against minimum and maximum e-MS sample document (from VIHA). Result must be the same. Adding and viewing the minimum e-MS sample must be possible. A subsequent export must result in the same document as the e-MS sample (apart from non-semantic differences) The minimum e-MS sample document must be viewable at the PDA Test harnesses have to exist for all components (JUnit) Code must comply to Sun Java coding conventions Code must compile with minimal warnings (with compiler switched to "strict") Design and code will be inspected by peers Thick client and thin client should be installable without special development tools (JavaWebstart, HotSync) Usability of dialogues will be assessed and refined based on external user review

Quality assurance for woodwoman release

Quality goal	Assurance techniques
Design spec. Should give an accurate structural and behavioural description of the component-level and class-level architecture Component-level architecture should comply to ISO/IEEE documentation standards Mission-critical parts of the specification should be formal	Accuracy with class-level implementation tested by reverse engineering of reference model Comparison to ISO/IEEE documentation standards and best practices as documented in software engineeting literature Use of formal reasoning techniques to prove critical properties (such as TLA+)
Implementation Functionality All requirements implemented All security requirements implemented Maintainability: adaptability (plugin architecture) Usability Efficiency	Test whether all use cases can be traced to implementation functions Semi-formal analysis whether security implementation is a valid refinement of security requirements Prototypical development of two plugins for extending TAPAS\|ems Usability study with participating external users (MDs) Performance measurements and STP annotations in UML, protocol refactoring if necessary Formal code and design review meetings We use a load testing tool and/or custom scripts to simulate heavy usage of the system. Server load will be minimal, given number of users in the local system and robustness of hardware. Load will be defined by scalability parameters such as number of concurrent users, number of transactions per second, or number/size of data items stored/processed. We will verify that the system can handle loads within its capacity without crashing, producing incorrect results, mixing up results for distinct users, or corrupting the data. We will verify that when capacity limits are exceeded, the system safely rejects, ignores, or defers requests that it cannot handle. PDA load will be defined by number and size of records stored on the system. Load and complexity of data will be simulated and tested by increasing the numbr of stored records to beyond the expected number of patients.

Quality goal

Assurance techniques

Design spec.

Should give an accurate structural and behavioural description of the component-level and class-level architecture
Component-level architecture should comply to ISO/IEEE documentation standards
Mission-critical parts of the specification should be formal

Accuracy with class-level implementation tested by reverse engineering of reference model
Comparison to ISO/IEEE documentation standards and best practices as documented in software engineeting literature
Use of formal reasoning techniques to prove critical properties (such as TLA+)

Implementation

Functionality

All requirements implemented
All security requirements implemented

Maintainability:

adaptability (plugin architecture)

Usability
Efficiency

Test whether all use cases can be traced to implementation functions
Semi-formal analysis whether security implementation is a valid refinement of security requirements
Prototypical development of two plugins for extending TAPAS|ems
Usability study with participating external users (MDs)
Performance measurements and STP annotations in UML, protocol refactoring if necessary
Formal code and design review meetings
We use a load testing tool and/or custom scripts to simulate heavy usage of the system.

Server load will be minimal, given number of users in the local system and robustness of hardware. Load will be defined by scalability parameters such as number of concurrent users, number of transactions per second, or number/size of data items stored/processed. We will verify that the system can handle loads within its capacity without crashing, producing incorrect results, mixing up results for distinct users, or corrupting the data. We will verify that when capacity limits are exceeded, the system safely rejects, ignores, or defers requests that it cannot handle.

PDA load will be defined by number and size of records stored on the system. Load and complexity of data will be simulated and tested by increasing the numbr of stored records to beyond the expected number of patients.

Quality assurance for ironcat release

Quality goal	Assurance techniques
Specification All parts of the specification should comply to ISO/IEEE documentation standards Compliance with implementation	Accuracy with class-level implementation tested by reverse engineering of reference model Comparison to ISO/IEEE documentation standards and best practices as documented in software engineeting literature Documentation audit by ISO/IEEE expert
Implementation Reliability: correctness, fault tolerance, recoverable, maturity	Development and application of systematic reliability test plan Use of "Design-by-contract" paradigm of specifying pre-conditions, post-conditions and class invariants for all implementation classes, using Java assertions. For implementation of invariants, each class will have a method "invariant_check", in each pre- and post-condition of each method. We will monitor the behavior of servers to automatically detect service outages or performance degradation. We have policies and procedures in place for failure notification, escalation, and correction.

Quality goal

Assurance techniques

Specification

All parts of the specification should comply to ISO/IEEE documentation standards
Compliance with implementation

Accuracy with class-level implementation tested by reverse engineering of reference model
Comparison to ISO/IEEE documentation standards and best practices as documented in software engineeting literature
Documentation audit by ISO/IEEE expert

Implementation

Reliability: correctness, fault tolerance, recoverable, maturity

Development and application of systematic reliability test plan
Use of "Design-by-contract" paradigm of specifying pre-conditions, post-conditions and class invariants for all implementation classes, using Java assertions. For implementation of invariants, each class will have a method "invariant_check", in each pre- and post-condition of each method.
We will monitor the behavior of servers to automatically detect service outages or performance degradation. We have policies and procedures in place for failure notification, escalation, and correction.

Quality assurance for diamondog release

Quality goal	Assurance techniques
Specification User documentation Should be complete and understandable	As well as involving clinicians in our UI design phases. We will involve commited memners of the NSMobile health network to beta test TAPAS. We will provide beta testers directions to focus on specific features of the system as they are incorporated into the modular framework. We will actively follow up with beta testers to encourage them to report issues. (4 current MD members of NSMobile and 2 MOAs.)
Implementation Usability, deployability	We want to understand each post-deployment system failure and actively take steps to correct the defect. The system has built-in capabilities for gathering detailed information from each system failure (e.g., error message, stack traceback, operating system version). This information will be transmitted back to us so that we may analyze it and act on it.

Quality goal

Assurance techniques

Specification

User documentation

Should be complete and understandable

As well as involving clinicians in our UI design phases. We will involve commited memners of the NSMobile health network to beta test TAPAS. We will provide beta testers directions to focus on specific features of the system as they are incorporated into the modular framework. We will actively follow up with beta testers to encourage them to report issues. (4 current MD members of NSMobile and 2 MOAs.)

Implementation

Usability, deployability

We want to understand each post-deployment system failure and actively take steps to correct the defect. The system has built-in capabilities for gathering detailed information from each system failure (e.g., error message, stack traceback, operating system version). This information will be transmitted back to us so that we may analyze it and act on it.

Company Proprietary