Common use of Test Planning Clause in Contracts

Test Planning. Test planning will be accomplished in the manner prescribed by the lead OTA’s directives. The lead OTA invites supporting OTAs to participate in early activities (between acquisition entities, developmental testers and operational testers) which focus on developing strategies to leverage and integrate test efforts and use of data between developmental and operational testing (DT and OT). Examples would include the activities of integrated test teams (ITT), early involvement, test and evaluation Working-level IPT (T&E WIPT), integrated product teams (IPT), and program test integration working groups which produce a test and evaluation strategy (TES) per ▇▇▇▇ 5000.(series). Supporting OTAs will participate early in MOT&E planning and remain proactive throughout the test planning process. Safety will be addressed throughout all phases MOT&E test planning. The lead OTA will produce the OTA test plan with concurrence from the Supporting OTAs. (1) The lead OTA for a MOT&E (Annex G, Glossary) is responsible for initiating the OT&E inputs to the test and evaluation strategy (TES) and test and evaluation master plan (TEMP), ensuring supporting OTA participation in the appropriate multi-Service integrated test team or T&E WIPT, providing lead OTA document guidance, and preparing all OT documents. (2) The lead OTA is responsible for providing input to the documents, participating in meetings, briefs and working groups as required, participating in data generating events, and providing mutually agreed upon support. (3) The Lead OTA will utilize integrated DT and OT whenever possible. (4) Each Service OTA plans resource requirements in accordance with their Service procedures and directives. Some POM (Program Objective Memorandum) for test funding and some rely on the Program Manager (PM)/Joint Program Office (JPO) to fund testing resources. Consequently, the lead OTA will ensure that the TEMP clearly identifies each Service’s specific test resources (assets and funding) and the source of funding (specific PM/JPO, POM, etc). (5) The lead OTA will begin the planning process by forming a core team comprised of the participating OTAs. The OTAs will communicate Service test requirements, critical operational issues (COI), test objectives, and key resource requirements. (6) The lead OTA will consolidate test requirements, test objectives, key resource requirements, and test scenarios and gain agreement by all involved Service OTAs. Service- unique issues will be included as COIs or additional issues when deemed appropriate by that Service. (7) The lead OTA will consolidate and provide MOT&E TEMP input. The lead OTA will accommodate supporting Service peculiar OT&E requirements and inputs in the formal coordination action of the TEMP. Coordination actions will accommodate Service-unique staffing approval requirements. The TEMP is prepared in accordance with the Defense Acquisition Guidebook. (8) Participating OTA representatives will meet with the lead OTA for the purpose of assigning OTA specific responsibilities for accomplishment of test objectives. These assignments will be made in a mutually agreeable manner. Each OTA will be responsible for resource identification and accomplishment of its assigned test objectives under the direction of the lead OTA. (9) The lead OTA, with assistance from supporting OTAs, will develop a matrix to provide a comparison of the user’s requirements, and Service operational criteria. It is not a source document, but it increases visibility of program requirements, increases communications, and identifies areas for resolution. The lead OTA’s format will be used when developing the matrix. (10) Each OTA will prepare and identify service specific data requirements, and provide the requirements to the lead OTA in the lead OTA format. (11) The lead OTA will prepare the OTA test plan(s), consolidating the inputs from all supporting activities. After consolidation, the OTA test plan(s) will be approved by the supporting OTAs. See Annex E for the recommended format for the signature page. (12) The lead OTA is responsible for scheduling OTA test plan brief for programs requiring OSD oversight. The brief may be presented jointly by all OTAs involved. (13) The lead OTA will invite Joint Interoperability Test Command (JITC) to participate in test planning to address interoperability certification and operational interoperability reporting. (14) The lead OTA in coordination with supporting OTAs should ensure the T&E WIPT Charter contains an event-driven deliverables table identifying deliverables needed by the T&E WIPT to plan and execute integrated test activities. The table will also identify the offices responsible for those deliverables (see Table 3)

Test Planning. Test planning will be accomplished in the manner prescribed by the lead OTA’s directives. The lead OTA invites supporting participating OTAs to participate in early activities (between acquisition entities, developmental testers testers, and operational testers) which focus on developing strategies to leverage and integrate test efforts and use of data between developmental and operational testing (DT and OT). Examples would include the activities of integrated test teams (ITT), early involvement, test and evaluation T&E Working-level IPT Integrated Product Team (T&E WIPT), integrated product teams Integrated Product Team (IPT), and program test integration working groups groups, which produce a test and evaluation strategy (TES) Milestone A TEMP per ▇▇▇▇ 5000.(series5000. (Series). Supporting Participating OTAs will participate early in MOT&E planning and remain proactive throughout the test planning process. Safety will be addressed throughout all phases of MOT&E test planning. The lead OTA will produce the OTA test plan with concurrence from the Supporting participating OTAs. (1) The lead OTA for a MOT&E (Annex G, Glossary) is responsible for initiating the OT&E operational test and evaluation (OT&E) inputs to the TEMP. The participating OTA will provide Service-unique test and evaluation strategy (TES) and test and evaluation master plan (requirements for the TEMP), ensuring supporting . The lead OTA will ensure participating OTA participation in the appropriate multi-Service integrated test team ITT or T&E WIPT, providing lead OTA document guidance, and preparing all OT documents. (2) The lead OTA is responsible for providing input to the documents, participating in meetings, briefs briefs, and working groups groups, as required, participating in data generating events, events and providing mutually agreed upon support. (3) The Lead lead OTA will utilize integrated integrate DT and OT whenever possiblecost and feasibility allow. (4) Each Service OTA plans resource requirements in accordance with their Service procedures and directives. Some POM (Services rely on Program Objective MemorandumMemorandums (POMs) for test funding and some rely on the Program Manager (PM)/Joint Program Office (JPO) to fund testing resources. Consequently, the lead OTA will ensure that the TEMP clearly identifies each Service’s specific test resources (assets and funding) and the source of funding (specific PM/JPO, POM, etc.). (5) The lead OTA will begin the planning process by forming a core team comprised of the participating OTAs. The OTAs will communicate Service Service-unique test requirements, critical operational issues (COI), test objectives, concerns, and key resource requirements. (6) The lead OTA will consolidate test requirements, test objectives, key resource requirements, and test scenarios and gain agreement by all involved Service OTAs. Service- unique issues will be included as COIs COI or additional issues when deemed appropriate by that Service. (7) The lead OTA will consolidate and provide MOT&E TEMP input. The lead OTA will accommodate supporting participating Service peculiar OT&E requirements and inputs in the formal coordination action of the TEMP. Coordination actions will accommodate Service-unique staffing approval requirements. The TEMP is prepared in accordance with the Defense Acquisition Guidebook and the DOT&E TEMP Guidebook. (8) Participating OTA representatives will meet with the lead OTA for the purpose of assigning OTA specific responsibilities for accomplishment of test objectives. These assignments will be made in a mutually agreeable manner. Each OTA will be responsible for resource identification and accomplishment of its assigned test objectives under the direction of the lead OTA. (9) The lead OTA, with assistance from supporting OTAs, will develop a matrix to provide a comparison of the user’s requirements, and Service operational criteria. It is not a source document, but it increases visibility of program requirements, increases communications, and identifies areas for resolution. The lead OTA’s format will be used when developing the matrix. (10) Each OTA will prepare and identify service Service specific data requirements, requirements and provide the requirements to the lead OTA in the lead OTA format. (1110) The lead OTA will prepare the OTA test plan(s), consolidating the inputs from all supporting participating activities. After consolidation, the OTA test plan(s) will be approved by the supporting participating OTAs. See Annex E for OTAs will integrate their cybersecurity test requirements into the recommended format for test plan. (11) The lead OTA will ensure that all planning and execution documents not captured in an evaluation plan or operational test plan are reviewed and approved by the signature pageparticipating OTAs. This includes the detailed schedule, data collection plans to include forms (quantitative, qualitative, and verification), instrumentation plans, and Safety Plan. (12) The Based upon the program’s inclusion in one or more of the categories of OSD T&E oversight, the lead OTA is responsible for scheduling OTA test plan brief for programs requiring briefs to the cognizant OSD oversightauthority. The brief may be presented jointly by all OTAs involved. (13) The lead OTA will invite Joint Interoperability Test Command (JITC) to participate in test planning to address interoperability certification and operational interoperability reporting. (14) The lead OTA OTA, in coordination with supporting OTAs participating OTAs, should ensure the T&E WIPT Charter contains an event-driven deliverables table identifying deliverables needed by the T&E WIPT to plan and execute integrated test activities. The table will also identify the offices responsible for those deliverables (see Table 3). (15) The lead OTA, in coordination with participating OTAs, will ensure that Cybersecurity testing is in compliance with DOT&E Procedures for Operational Test and Evaluation of Cybersecurity in Acquisition Programs, 1 August 2014. The lead OTA will also lead coordination efforts for Cybersecurity testing with participating OTAs. This responsibility is applicable to all acquisition systems under test and not specifically oversight programs. The described guidance from DOT&E should be used across all acquisition systems when testing Cybersecurity. (16) The lead OTA for the program will provide the DOT&E with a memorandum that assesses the T&E implications of the initial concept of operations provided by the user as soon as practical after the Materiel Development Decision. (17) For software acquisitions, the lead OTA will conduct an analysis of operational risk to mission accomplishment covering all planned capabilities or features in the system. The analysis will include commercial and non-developmental items. The initial analysis will be documented in the Milestone A TEMP and updated thereafter.

Test Planning. Test planning will be accomplished in the manner prescribed by the lead OTA’s directives. The lead OTA invites supporting OTAs to participate in early activities (between acquisition entities, developmental testers and operational testers) which focus on developing strategies to leverage and integrate test efforts and use of data between developmental and operational testing (DT and OT). Examples would include the activities of integrated test teams (ITT), early involvement, test and evaluation Working-level IPT Integrated Product Team (T&E WIPT), integrated product teams Integrated Product Team (IPT), and program test integration working groups groups, which produce a test Test and evaluation strategy Evaluation Strategy (TES) per ▇▇▇▇ 5000.(series5000. (series). Supporting OTAs will participate early in MOT&E planning and remain proactive throughout the test planning process. Safety will be addressed throughout all phases of MOT&E test planning. The lead OTA will produce the OTA test plan with concurrence from the Supporting OTAs. (1) The lead OTA for a MOT&E (Annex G, Glossary) is responsible for initiating the OT&E inputs to the test and evaluation strategy (TES) and test and evaluation master plan (TEMP), ensuring TES andTEMP. The supporting OTA will clearly provide for TEMP inclusion of Service-unique testing that may be required. The lead OTA will ensure supporting OTA participation in the appropriate multi-Service integrated test team ITT or T&E WIPT, providing lead OTA document guidance, and preparing all OT documents. (2) The lead OTA is responsible for providing input to the documents, participating in meetings, briefs and working groups as required, participating in data generating events, and providing mutually agreed upon support. (3) The Lead OTA will utilize integrated integrate DT and OT whenever possiblecost and feasibility allow. (4) Each Service OTA plans resource requirements in accordance with their Service procedures and directives. Some POM (Program Objective MemorandumMemorandums (POM ) for test funding and some rely on the Program Manager (PM)/Joint Program Office (JPO) to fund testing resources. Consequently, the lead OTA will ensure that the TEMP clearly identifies each Service’s specific test resources (assets and funding) and the source of funding (specific PM/JPO, POM, etc.). (5) The lead OTA will begin the planning process by forming a core team comprised of the participating OTAs. The OTAs will communicate Service Service-unique test requirements, critical operational issues (COI), test objectives, concerns, and key resource requirements. (6) The lead OTA will consolidate test requirements, test objectives, key resource requirements, and test scenarios and gain agreement by all involved Service OTAs. Service- Service-unique issues will be included as COIs or additional issues when deemed appropriate by that Service. (7) The lead OTA will consolidate and provide MOT&E TEMP input. The lead OTA will accommodate supporting Service peculiar OT&E requirements and inputs in the formal coordination action of the TEMP. Coordination actions will accommodate Service-unique staffing approval requirements. The TEMP is prepared in accordance with the Defense Acquisition Guidebook. (8) Participating OTA representatives will meet with the lead OTA for the purpose of assigning OTA specific responsibilities for accomplishment of test objectives. These assignments will be made in a mutually agreeable manner. Each OTA will be responsible for resource identification and accomplishment of its assigned test objectives under the direction of the lead OTA. (9) The lead OTAOTA will prepare the evaluation plan, with assistance consolidating the inputs from all supporting activities. After consolidation, the evaluation plan will be approved by all supporting OTAs, will develop a matrix to provide a comparison of . See Annex E for the user’s requirements, and Service operational criteria. It is not a source document, but it increases visibility of program requirements, increases communications, and identifies areas recommended format for resolution. The lead OTA’s format will be used when developing the matrixsignature page. (10) Each OTA will prepare and identify service specific data requirements, and provide the requirements to the lead OTA in the lead OTA format. (11) The lead OTA will prepare the OTA test plan(s), consolidating the inputs from all supporting activities. After consolidation, the OTA test plan(s) will be approved by the supporting OTAs. See Annex E for the recommended format for the signature page. (12) The lead OTA is responsible for scheduling program OTA test plan brief for programs requiring briefs to the cognizant OSD oversightauthority. The brief may be presented jointly by all OTAs involved. (13) The lead OTA will invite Joint Interoperability Test Command (JITC) to participate in test planning to address interoperability certification and operational interoperability reporting. (14) The lead OTA in coordination with supporting OTAs should ensure the T&E WIPT Charter contains an event-driven deliverables table identifying deliverables needed by the T&E WIPT to plan and execute integrated test activities. The table will also identify the offices responsible for those deliverables (see Table 3)