Functional Responsibilities. Providing knowledge and recognized expertise in the specific subject area of the project • Understanding and providing in-depth knowledge of the business processes of the project • Providing consultative expertise in fulfillment of contract deliverables and Statements of Work • Articulating and implementing best practices related to the subject area of project • Applying expertise to support the project’s vision and strategic direction CohnReznick’s (IT) IV&V Methodology 1 RACI Matrix 7 Communication Strategy 8 As part of CohnReznick’s (IT) IV&V methodology, our team uses both technical and non-technical methods to assess and prioritize risks in an organization’s technology environment and to develop strategies to reduce risk, cost, and complexity, as well as improve IT performance and value. We will use our in-house developed methodology based on our Cybersecurity, Technology Risk, and Privacy Practice’s extensive experience performing similar reviews. Our methodology is rooted in industry recognized standard setting bodies such as: • National Institute of Standards and Technology (NIST) • Information Systems Audit and Control Association (ISACA) • International Standards Organization (ISO) • Project Management Institute’s (PMI) Project Management Book of Knowledge (PMBOK) • Institute of Electrical and Electronics Engineers (IEEE) Standard (Std.) 1490-2003 • Capability Maturity Model Integration (CMMI) Development Standards • International Information System Security Certification Consortium (ISC2) • IEEE Std 1058-1998 Standard for Software Project Management Plans • IEEE Std 828-2005 Standard for Software Configuration Management Plans • IEEE Std 730-2002 Standard for Software Quality Assurance Plans • IEEE Std 1061-1998 Standard for Software Quality Metrics Methodology • Industry standards for IV&V IEEE Std.1012-2004 • IT Infrastructure Library (ITIL) version 3 Our work plan consists of four key phases as illustrated in the diagram below and further detailed in the following pages. Our team invests time up front to clearly define a project plan, which provides a framework for operations. It will map out the project’s goals and clearly indicate efficient paths to attain them. Our approach to providing (IT) IV&V services involves defining a formal project management approach, including a project schedule. The (IT) IV&V project schedule is based on the defined work breakdown structure (“WBS”), which shows the relationship between all deliverables, activities, and resources required to complete the (IT) IV&V project. The approach we use to develop the project plan and schedule consists of the following activities: Define and Document (IT) IV&V Management Plan Develop (IT) IV&V Project Schedule Document Communication Plan Project Assessment Report Baseline assessment of the Project Management Plan and the Projects' WBS to assess the suitability of the current plan and future activities Identify and communicate any forseable impacts/ roadblocks of the Project Plan related to procurements and major deadlines Establish reporting criteria for project progress, assessment results, budget, and scope Identify thresholds or triggers as to when to notify the PM and key stakeholders of significant risk or impact to the overall project schedule, scope, or budget Define the project activities and deliverables to be tracked by the WBS Assign resources and estimate duration for each project activity Align major milestones and phase gate reviews with the master Project schedule Meet with key project teams to discuss and develop the (IT) IV&V Management Plan Establish agreed-upon goals and targets Review historical data to identify previous issues, root causes, or potential roadblocks We recommend approaching the (IT) IV&V project as a formal project that runs alongside the main project. The (IT) IV&V project approach and schedule should be synchronized with the main project schedule and continually monitored and adjusted, as required, to remain synchronized. • Kickoff Presentation: Used to formally kick off the (IT) IV&V project to make sure that all stakeholders are aligned and that the project is set up for success. • Project Charter: Authorizes the (IT) IV&V project and identifies the initial requirements that satisfy stakeholder needs and expectations. • (IT) IV&V Management Plan: Identifies how the (IT) IV&V project will be planned, executed, monitored, controlled, and closed, and guides the decision-making that occurs throughout the (IT) IV&V project. • (IT) IV&V Project Schedule and WBS: Show the relationship between all deliverables, activities, and resources required to complete the (IT) IV&V project. • (IT) IV&V Weekly and Monthly Status Reports/Meetings: Provides comprehensive updates of (IT) IV&V project progress and assessment results, reports on strengths and deficiencies, and identifies recommendations. • Meeting Minutes: Summarizes any formal (IT) IV&V project meetings. • Tracking Tool: Tracks project risks, issues, action items, and decisions. Continuing with our iterative approach, the next phase is focused on conducting verification and validation activities to assess the work products (i.e., deliverables) produced by a project team and determine whether they meet expectations. We believe it is very important to instill structure and discipline in the work product creation, submission, and review process as early in the program as possible to prevent rework. CohnReznick’s approach to reviewing implementation deliverables, documentations and artifacts includes the following activities that show the typical process used to review project deliverables and provide a formal deliverable review report: Define and Document Baseline Expectations Documents (BEDA) Conduct review of the deliverables Conduct Test of the deliverables where applies Gather documents, artifacts and relavant information Analyze deliverables versus BEDA Assemble Test and review results Provide initial drafts of deliverable baseline expectations documents or artifacts Define acceptance criteria for deliverables Define Test Enviroment Achieve concurence amonst key stakeholders and communicate Program governance • Baseline Expectation Document and Acceptance (BEDA) Criteria: For every deliverable, create a BEDA that defines what is required for the execution of the agency project. CohnReznick strongly suggests that BEDA criteria be created and agreed upon by each of the key stakeholders within the program. The small incremental effort associated with creating a thorough BEDA prior to creating a deliverable will yield benefits to the project in terms of greater efficiency and lower risk by limiting deliverable re-work, which negatively impacts both the developer of the deliverable and the review and acceptance teams.
Appears in 1 contract
Sources: Information Technology Independent Verification and Validation Services Contract
Functional Responsibilities. Providing knowledge and recognized expertise in the specific subject area of the project • Understanding and providing in-depth knowledge of the business processes of the project • Providing consultative expertise in fulfillment of contract deliverables and Statements of Work • Articulating and implementing best practices related to the subject area of project • Applying expertise to support the project’s vision and strategic direction CohnReznickEXHIB DocuSign Envelope ID: 2E44A81B-4C03-4469-B714-1C7B4A83F047 IT D Overview of RSM’s (IT) IV&V Methodology 1 RACI Matrix 7 Communication Strategy 8 As RSM’s IV&V assessment methodology is baselined on the IEEE standards for software verification and validation, but enhanced with several frameworks including ISO2700, PMBOK/PMP, COBIT5, ITILv3, COSI, SOX, NIST, PCI, HIPAA, FDA. Our methodology is flexible to align with any ERP, any industry, with any type of project methodology (i.e. agile, waterfall, or hybrid). We work with many customers to customize our IV&V assessments to contain the specific areas of high risk that our customers would like us to focus on. The our IV&V approach covers determine if software, hardware, interfaces, documentation, and user requirements have been designed completely, accurately, consistently and can be proven via testing. We bring a proprietary set of technology, enablers, and risk frameworks, which will intersect, with our deep experience in helping clients reduce risk and maximize the return on their investments in complex IT programs RSM has identified grouped IV&V implementation risk into domains in the graphic to the right, based on success factors of an implementation. The following describes each of the domains and what areas are covered in each domain. Project Governance RSM’s Project Governance methodology contains a 70 point inspection review of all areas of project governance including review of project plan and resource scheduling using analytics, budgetary requirements, including phase/gate and acceptance criteria, initial cutover plan review, and assessment of system integrator contracts against project objectives. RSM’s comprehensive objectives reviewed are below: • Allocate resources • Onboarding team • Project governance methodology and approach • Project organization • Roles and responsibilities • Other key initiatives • SI implementation roles and responsibilities • Engagement status reporting plan • Project communication management plan(s) • Project charter (preparation) • Stakeholder identification • Project charter (sign-off) • Kick-off workshop • Customer requirements • Project plan • Project management plan (change process) • Solution documentation procedure • Solution implementation procedure • Template management procedure • Test management procedure • Change control management procedure • Application incident management procedure • Technical operations • Business process operations • Maintenance management • Project execution • Project engagement monitoring and control • KPI engagement monitoring • Issue, risk, and change management • Cutover plan review • Testing strategy • Technical requirements and design • Interface Inventory • Initial hardware sizing proposal • Project support tools • Project quality gate scorecard • Knowledge management gate • Project phase gate reviews • Project management review • Phase gate completion sign-off • Exception and gap management • Project scope statement • Risk ratings • Escalation procedures • Capital budget • Status meetings • Acceptance criteria refinement • Project engagement approach and methodology • Project schedule • Project budget • Project management plan (initial development) • Project Costing • Data migration approach and strategy • Project progress and status communications • System integrator contract review • Project training strategy and plan • Project team training • Fit/gap analysis • Value determination strategy (ROI) • Project coordination • Solution customizations • Change control access • External contractor usage (knowledge transition) • External contractor usage (roles/responsibilities) • Program quality assurance RSM IV&V PROJECT GOVERNANCE RISK ASSESSMENT METHODOLOGY DOCUMENTATION Business requirements Business requirements includes requirement traceability from identification through design, through testing reviewing the testing strategy, plan and procedures, and training of business, functional and or technical requirements. Business requirements are the foundation and the purpose of an ERP implementation or ERP upgrade. Business requirements are also where a majority of the scope changes (increases or decreases) happen. Scope increases are always monitored, but scope decreases may slip through the cracks. Business requirements are critical to identify those possible scope decreases, by tracing requirements from design, to testing, to training to go-live—what RSM calls business requirement traceability. Without that traceability, how do you know you got what you paid for? RSM’s business requirement methodology and approach consists of multiple types of assessments consisting of business requirement traceability, review of testing strategy covering a phased approach/gates, schedules, defect management plan, resources, and review of actual testing performed (RICEFWs). We provide several examples of our methodology and testing below: EXAMPLE OF A BUSINESS REQUIREMENT TRACEABILITY ASSESMENT. THESE ARE CUSTOM DEVELOPED PER EACH CUSTOMERS TRACEABILITY STRATEGY. RSM IV&V TESTING STRATEGY RISK ASSESSMENT METHODOLOGY RSM re-performs testing of RICEFWs (Reports, Interfaces, Configuration, Enhancements, Forms, Workflows), to determine if we can reach the same conclusion the testers did. Below is an example of an assessment: Data RSM’s Data migration assessment which is an 80 point inspection that includes data governance strategy, data privacy, cleansing, mapping, conversion, and migration activities including both inspection reviews and using analytics. This review includes external interfaces, which constitute a high-risk area and a typical cause of implementation failure due to data accuracy and completeness issues. We often find that roles and responsibilities between the system integrator and the business are not clear, which causes most failure points. RSM recommends data be assessed at multiple project check points. Below is a table of our leading practice recommendations, aligned to the implementation-testing phase, where leading practice is using live data for testing. Often, we see clients still using fabricated data beyond the interface phase, which usually leads to ineffective testing and functional issues—post-go-live—that were not detected during testing. RSM reviews this plan as part of CohnReznick’s our testing strategy review. Testing phase Leading practice data activity Unit Round 1 master data conversion testing System Fabricated data String Fabricated data Data conversion Should be testing only the data that has been converted; not testing functionality Integration Real master data; transactional should be 75% real Interface 100% real data End-to-end 100% real data as a result of a data load UAT (ITUser Acceptance Testing) IV&V methodology100% real data Security Start with 50% real data; end with 100% real data Performance 100% real data with some fictitious data to mirror a day in the life of using the system Cut-over/dry run Data sign offs Regression/smoke testing 100% real data (post-go-live) Regulatory, our team uses Security and Controls RSM both technical designs security and non-technical methods to assess and prioritize risks in an organization’s technology environment and to develop strategies to reduce risk, cost, and complexitycontrols, as well as improve performs security and control IV&V assessments—not just for implementations, but for large complex internal and external audits. We bring both the audit and implementation risk knowledge to our IV&Vs. We identify controls into the following categories: IT performance general controls (ITGC), IT automated application controls (ITACs), sensitive access/segregation of duties (SOD), cyber security, hybrid or key reports, and manual controls. We use our catalog of accelerators, which includes automation and tools to test controls where we can provide recommendations if needed depending on the application, database, or operating system being implemented. So this experience helps when we perform IV&Vs because if there are gaps or issues found, RSM has the experience to specifically provide recommendations on how to close those gaps. Our IV&V methodology for this area of focus covers ERPs, application, databases, operating systems, and networks implementations. More specific qualifications regarding RSM’s methodology for this domain can be provided at the time of an SOW once a specific technology type is known. Organizational Change Management Recommending organizational or process changes brought on by large-scale ERP implementation or transformation are easy compared to the people aspects which are often harder to control. The goal of any change strategy is to re-establish a sense of control that is lost when the status quo is changed for employees, customers, and leaders. RSM’s Organizational Performance and Change Management (OPCM) practice is customizable, scalable, and developed over three decades of successfully planning the transition to a new future state operating model. Our eight-step methodology and related tools focus on taking the emotion out of the change process, which delays decision making and often leads to higher levels of customization during ERP installations. Our approach is based on the fact that human behavior during change can be predicted and measured. In fact, the goal of any change plan should be to accelerate the time to value. Simply put, we begin each project or audit by partnering with our clients or their integrators to identify the financial and non-financial benefit drivers and targets, and measure the behaviors required to deliver those benefits. The sooner employees and leaders adopt and sustain a new way of working, the sooner the value from the investment can be realized. We will use utilize analytics to measure employee commitment, deployment readiness, and solution compliance and work to educate our inclient teams on how to effectively lead and deliver “change.” We share 100% of our tools and workplan to enable our clients to monitor and reinforce the change, or new opportunities, after our engagement has been completed. We believe in helping our clients achieve their desired business goals and manage changes in their organization successfully through our proven methodologies. RSM has a comprehensive 45-house point review of your organizational change management strategy. Coverage includes: change capability and business case, change planning, sponsorship and commitment, change enablement, and solution adoption and behavior change. RSM OCM Methodology Operations Operational alignment is critical to a successful outcome before and after go-live. Do you know all the critical projects being taken on by all areas of your company, where your employees are splitting time between their “day job” and the major implementation? Many times we see other projects not as large as the major implementation are not viewed as a possible impact to phase gate deadlines and the go-live. Examples would be large marketing campaigns generating more products or acquiring another company. We can assist with accumulating this list and reviewing it with leadership to help determine if you have enough employees on the implementation or if you need shift around other jobs. We also see many times where new vendors and prior vendors are not identified before go-live thus SLAs do not get reviewed. Many times we see where vendors are part of the operations of a key SOX control, but they were not informed, and it results in key control failures simply due to lack of communication. We can perform a review to make sure all vendors were identified and that their SLAs were reviewed and updated. Another area critical to operations is effective execution of the cut-over plan. Unfortunately, we have seen too many times where the plans are not detailed enough to be followed, or developed at the last minute, and/or not widely communicated enough to be effective, resulting in a lot of stumbling or forgotten steps before or just at go-live. RSMs methodology reviews the strategy, the plan looking for detail line items, not mile-stones, sign-offs for training completion, data conversion, testing, security, outstanding unresolved issues, etc. RSM CUT-OVER/DEPLOYMENT READINESS ASSESSMENT Technology RSM views the technology domain critical risk factors covering technical architecture reviews, or coding standard reviews, BCP, DR, IR updates revised programs, interfaces development, and working with IT control owners. RSM has defined a structured methodology for developing BCPs and deploying sustainable business continuity planning programs. Our five-phase process is based on a cyclical approach; which includes not only the creation of BCPs, but also the ongoing revision, maintenance and validation activities that help to protect your investment while supporting a state of readiness for disaster response. Our standard business continuity planning methodology is shown on the following page, with subsequent sections of this proposal further describing how we will tailor our Cybersecurityapproach to meet your specific objectives and requirements. We leverage processes and technology to help you mitigate risk, Technology Risk, monitor compliance and Privacy Practice’s extensive experience performing similar reviewsadd value to your organization. Our methodology is rooted grounded in industry recognized standard setting bodies such as: • National Institute of Standards understanding your needs and Technology (NIST) • Information Systems Audit working with you to develop a responsive approach to meet and Control Association (ISACA) • International Standards Organization (ISO) • Project Management Institute’s (PMI) Project Management Book of Knowledge (PMBOK) • Institute of Electrical exceed your expectations. In addition, we integrate quality review and Electronics Engineers (IEEE) Standard (Std.) 1490-2003 • Capability Maturity Model Integration (CMMI) Development Standards • International Information System Security Certification Consortium (ISC2) • IEEE Std 1058-1998 Standard for Software Project Management Plans • IEEE Std 828-2005 Standard for Software Configuration Management Plans • IEEE Std 730-2002 Standard for Software Quality Assurance Plans • IEEE Std 1061-1998 Standard for Software Quality Metrics Methodology • Industry standards project management resources to increase visibility into your business continuity activities, providing results and insight into progress. RSM BCP/DR/IR READINESS ASSESSMENT IV&V METHODOLOGY RSM Proprietary Tool Auditor Assistant- utilized for IV&V IEEE Std.1012assessments RSM believes real-2004 • IT Infrastructure Library (ITIL) version 3 Our work plan consists of four key phases as illustrated in the diagram below time communication is important and further detailed in the following pages. Our team invests time up front to clearly define a project plan, which provides a framework for operations. It will map out the project’s goals and clearly indicate efficient paths to attain themcritical when performing IV&Vs. Our approach is to providing (IT) identify findings, validate them, and communicate findings as soon as possible so that the implementation team can react and change course promptly. RSM uses a priority tool for continues and multiple phase roll-out implementation IV&Vs, called Auditor Assistant. This tool is used to track progress on the engagement real-time. It is used to house our IV&V services involves defining a formal project management approach, including a project schedulerisk assessment templates that have been described throughout this proposal. It is used to manage quality of assessments through workflows approvals. The (IT) IV&V project schedule tool is based on the defined work breakdown structure (“WBS”)used not to just to store issues but monitor identified issues, which shows the relationship between all deliverables, activitiesconfirmation with issue owners, and resources required resolution to complete issues. We will be able to report on findings with risk ratings that you will see progress of risk ratings lowered and progress of risks being closed through the (IT) multiple phases to determine if lessons are being learned and resolved as the phase roll-outs continue. Snapshots of Auditor Assistant is below: REAL-TIME PROGRESS TRACKING RSM ERP IMPLEMENTATION RISK ASSESSMENT FRAMEWORK STORAGE FOR QUALITY CONTROL AND MANAGEMENT TRACKING AND COMMUNICATION OF EACH ASSESSMENT STATUS AND RESULTS ABILITY TO ENTER ISSUES WITH RISK RATINGS TRACKING OF FINDINGS INCLUDING THE ABILITY TO TRACK AGING RSM’s IV&V projectWork Plan RSM’s work plan aligns with IEEE methodology described above and is modified based upon the verification or validation phase. The approach we use to develop the project plan and schedule consists of RSM’s work program methodology/ IV&V plans contains the following activities: Define and Document (IT) IV&V Management Plan Develop (IT) IV&V Project Schedule Document Communication Plan Project Assessment Report Baseline assessment of the Project Management Plan and the Projects' WBS to assess the suitability of the current plan and future activities Identify and communicate any forseable impacts/ roadblocks of the Project Plan related to procurements and major deadlines Establish reporting criteria for project progress, assessment results, budget, and scope Identify thresholds or triggers as to when to notify the PM and key stakeholders of significant risk or impact to the overall project schedule, scope, or budget Define the project activities and deliverables to be tracked by the WBS Assign resources and estimate duration for each project activity Align major milestones and phase gate reviews with the master Project schedule Meet with key project teams to discuss and develop the (IT) IV&V Management Plan Establish agreedmile-upon goals and targets Review historical data to identify previous issues, root causes, or potential roadblocks We recommend approaching the (IT) IV&V project as a formal project that runs alongside the main project. The (IT) IV&V project approach and schedule should be synchronized with the main project schedule and continually monitored and adjusted, as required, to remain synchronized. • Kickoff Presentation: Used to formally kick off the (IT) IV&V project to make sure that all stakeholders are aligned and that the project is set up for success. • Project Charter: Authorizes the (IT) IV&V project and identifies the initial requirements that satisfy stakeholder needs and expectations. • (IT) IV&V Management Plan: Identifies how the (IT) IV&V project will be planned, executed, monitored, controlled, and closed, and guides the decision-making that occurs throughout the (IT) IV&V project. • (IT) IV&V Project Schedule and WBS: Show the relationship between all deliverables, activities, and resources required to complete the (IT) IV&V project. • (IT) IV&V Weekly and Monthly Status Reports/Meetings: Provides comprehensive updates of (IT) IV&V project progress and assessment results, reports on strengths and deficiencies, and identifies recommendations. • Meeting Minutes: Summarizes any formal (IT) IV&V project meetings. • Tracking Tool: Tracks project risks, issues, action items, and decisions. Continuing with our iterative approach, the next phase is focused on conducting verification and validation activities to assess the work products (i.e., deliverables) produced by a project team and determine whether they meet expectations. We believe it is very important to instill structure and discipline in the work product creation, submission, and review process as early in the program as possible to prevent rework. CohnReznick’s approach to reviewing implementation deliverables, documentations and artifacts includes the following activities that show the typical process used to review project deliverables and provide a formal deliverable review report: Define and Document Baseline Expectations Documents (BEDA) Conduct review of the deliverables Conduct Test of the deliverables where applies Gather documents, artifacts and relavant information Analyze deliverables versus BEDA Assemble Test and review results Provide initial drafts of deliverable baseline expectations documents or artifacts Define acceptance criteria for deliverables Define Test Enviroment Achieve concurence amonst key stakeholders and communicate Program governance • Baseline Expectation Document and Acceptance (BEDA) Criteria: For every deliverable, create a BEDA that defines what is required for the execution of the agency project. CohnReznick strongly suggests that BEDA criteria be created and agreed upon by each of the key stakeholders within the program. The small incremental effort associated with creating a thorough BEDA prior to creating a deliverable will yield benefits to the project in terms of greater efficiency and lower risk by limiting deliverable re-work, which negatively impacts both the developer of the deliverable and the review and acceptance teams.stones:
Appears in 1 contract
Sources: Information Technology Independent Verification and Validation Services Contract