EX-10.17 4 s100412_ex10-17.htm EX-10.17 JOINT RESEARCH AND DEVELOPMENT AGREEMENT Between THE JOHNS HOPKINS UNIVERSITY APPLIED PHYSICS LABORATORY AND SECOND SIGHT MEDICAL PRODUCTS, INC.
JOINT RESEARCH AND DEVELOPMENT AGREEMENT
Between
THE ▇▇▇▇▇ ▇▇▇▇▇▇▇ UNIVERSITY APPLIED
PHYSICS LABORATORY AND
SECOND SIGHT MEDICAL PRODUCTS, INC.
This Joint Research and Development Agreement (“Agreement”) is made and entered into by and between The ▇▇▇▇▇ ▇▇▇▇▇▇▇ University Applied Physics Laboratory LLC (“JHU/APL”), and Second Sight Medical Products, Inc. (“Company”) hereinafter collectively referred to as the Parties.
Article 1 STATEMENT OF WORK
1.1 The Statement of Work (SOW) sets forth the nature and scope of the cooperative work to be performed under this Agreement, including any tasks, material, equipment or other support and any associated reporting requirements. The Statement of Work is attached as Appendix A to this Agreement.
1.2 The parties recognize that the Statement of Work describes the collaborative work they will undertake and that its goals are good faith guidelines. Periodic reviews can be held between the Parties for the purpose of assessing progress. It is understood that the nature of the collaborative work is such that completion within the period of performance specified cannot be guaranteed.
Article 2 DISCLOSURE AND PUBLICATION
2.1 "Proprietary Information" means information that is privileged or confidential, provided that such information: is not known or available from other sources without obligations concerning its confidentiality; has not been made available by the owners to others without obligations concerning its confidentiality; is not already available without obligations concerning its confidentiality; and has not been developed independently by persons who have had no access to the information.
2.2 The Parties agree to confer and consult with each other prior to publication or other public disclosure of the results of the collaborative work to ensure that no Proprietary Information, or other confidential information is released. Prior to submitting a manuscript for publication or before any other public disclosure, each Party will offer the other Party at least thirty (30) days to review the proposed publication or disclosure, submit objections, and file applications for patents, as applicable, in a timely manner.
2.3 If necessary, the Parties will exchange information which they consider to be Proprietary Information. The recipient of such information agrees to accept the disclosure of said information which is marked as confidential or proprietary or other similar marking, at the time it is sent to the recipient, and to employ all reasonable efforts to maintain the information secret and confidential, such efforts to be no less than the degree of care employed by the recipient to preserve and safeguard its own Proprietary Information. The information shall not be disclosed or revealed to anyone except employees of the recipient who have a need to know the information and who are under a secrecy obligation with the recipient under which such employees are required to maintain confidential the Proprietary Information of the recipient and such employees shall be advised by the recipient of the confidential nature of the Proprietary Information and that the information shall be treated accordingly.
2.4 Exceptions. The recipient's obligations shall not extend to any part of the information:
(a) that can be demonstrated to have been in the public domain or publicly known and readily available to the trade or the public prior to the date of the disclosure;
(b) that can be demonstrated, from written records to have been in the recipient's possession or readily available to the recipient from another source not under obligation of secrecy to the disclosing Party prior to the disclosure;
(c) that becomes part of the public domain or publicly known by publication or otherwise, not due to any unauthorized act by the recipient;
(d) that is demonstrated from written records to have been developed by or for the recipient without reference to confidential information disclosed by the disclosing Party; or
(e ) that is required to be disclosed by law, government regulation or court order.
Article 3 FUNDING
3.1 JHU/APL’s obligations under this agreement and the SOW (Appendix A) are subject to the availability and disbursement of funds from The ▇▇▇▇ Fund. No legal liability on the part of JHU/APL for any payment may arise until funds are made available to JHU/APL from The ▇▇▇▇ Fund.
3.2 JHU/APL will provide approximately four million (i.e., $ 4.075 million) dollars of the ▇▇▇▇ Fund to Company, within 30 days of receiving funds from the ▇▇▇▇ Fund, for research and development as stated in the SOW. JHU/APL will use funds from the ▇▇▇▇ Fund to conduct research as stated in the SOW, said research being within JHU/APL’s non-profit research, development and educational purposes. If funding is not received by JHU/APL within three (3) months of execution of this Agreement, all rights and obligations of both parties will terminate.
Article 4 INTELLECTUAL PROPERTY
4.1 Background Intellectual Property
"Background Intellectual Property" (also referred to as "Background IP") means property and the legal rights therein of either or both Parties developed before and independent of this Agreement including inventions, patent applications, patents, copyrights, trademarks, mask works, trade secrets and any information embodying proprietary data such as technical data and computer software, or any other legally protectable information. This Agreement shall not be construed as implying that either Party hereto shall have the right to use Background IP of the other in connection with this Agreement except as otherwise provided hereunder. Background IP of either Party may be used non-exclusively and without compensation by either Party in connection with the performance of work under this Agreement, and limited to the term of this Agreement.
4.2 Subject Intellectual Property
| 1. | "Subject Intellectual Property" (also referred to as "Subject IP") means property and the legal rights therein relating to inventions which may be patentable or otherwise protectable, test data and reports, patent applications, patents, copyrights, trademarks, mask works, trade secrets and any other legally protectable information, including technical data and computer software, first made or generated during the performance of work under this Agreement. |
| 2. | Ownership of Subject IP shall vest in the Party whose employee(s), consultants, or agents make or generate Subject IP, and such Party may perfect legal protection therein in its own name and at its own expense. Jointly made or generated Subject IP shall be jointly owned by the Parties. |
| 3. | Each Party agrees to disclose to the other Party in writing and in reasonable detail any jointly made or generated Subject IP. Such disclosure shall be made promptly and no later than thirty (30) days before any patent application directed to such invention is filed, or no later than thirty (30) days after receipt of disclosure by the patent department of the disclosing Party of the Subject IP. Any decision to file a patent application directed to jointly made or generated Subject IP and associated costs shall be negotiated between the Parties. Procedures for seeking and maintaining statutory protection such as patent protection, for jointly made or generated Subject IP shall be agreed in good faith by the Parties, provided that neither Party shall unreasonably withhold its agreement to seeking such statutory protection. |
| 4. | Each Party hereto may use, import, copy, modify, prepare derivative works, distribute, publicly display and perform jointly made or generated Subject IP of the other nonexclusively and without compensation solely in connection with the performance of work as directed by the SOW under this Agreement. |
| 5. | The nonexclusive license granted in sub-paragraph 4 above is non-assignable and cannot be sub-licensed, unless either Party obtains written permission from the other Party. |
| 6. | Upon the execution of this Agreement, Company shall issue to JHU/APL 1,000 shares of common stock (SHARES). Company shall deliver to JHU/APL a stock certificate, duly signed by appropriate officers of Company and issued to The ▇▇▇▇▇ ▇▇▇▇▇▇▇ University, representing all of the SHARES. Issuance of SHARES is subject to Company’ s Board approval, If Company’s Board does not grant approval, which will not be unreasonably withheld, an equivalent dollar amount of the shares will be paid to JHU/APL based on the public market price. If there is no public market within three months of the effective date of this agreement, and the Company has not issued the SHARES to JHU/APL, Company shall pay JHU/APL $14,000.00. |
| 7. | Company agrees to execute a license with JHU/APL upon incorporation of documented JHU/APL Subject IP, documented either internally or consistent with paragraph three above, into Company’s product(s) with the license terms as follows: |
| a) | Patent Costs: COMPANY shall be required to reimburse JHU/APL for all patent costs for IP developed during performance of the DEVELOPMENT AGREEMENT. The parties agree to work together to obtain cost effective representation. COMPANY must approve prosecuting counsel, which approval will not be unreasonably withheld. To the extent COMPANY is obligated to reimburse prosecuting counsel COMPANY may direct that effort. |
| b) | Running Royalty Rate: COMPANY shall pay to JHU/APL a royalty of .25% on Net Sales generated by the individual components that include JHU/APL Subject IP or jointly made Subject IP. The system is currently divided into three components, the implant, the video processing unit and the glasses. |
| c) | The Licensed Field of Use: restricted to neurally integrated visual prosthetics. |
| d) | Terms for Negotiation: Company shall outline for JHU/APL Company's capability and/or plans to introduce Company’s products into public use. Once due diligence milestone and capabilities are established in a manner reasonably satisfactory to both parties, JHU/APL and Company agree to negotiate in good faith to establish additional terms of a license agreement granting Company rights to make, have made, use, sell, offer to sell and import JHU/APL Subject IP . In addition to the terms outlined in 7a-c, the license agreement shall include at least the following additional provisions: a commitment by Company to exert their best efforts to introduce products and services incorporating JHU/APL Subject IP into public use as rapidly as practicable, and the right of JHU/APL to terminate the license should Company not meet specified, and mutually agreed due diligence milestones. At the sole discretion of JHU/APL, such license agreement may include provisions for an option to additional fields of use. |
| 8. | The Parties make NO EXPRESS OR IMPLIED WARRANTY AS TO ANY MATTER WHATSOEVER, including the conditions of the research or any invention or other intellectual property, or product, whether tangible or intangible, provided, made or generated under this Agreement, or the merchantability, or fitness for a particular purpose of the research or JHU/APL Subject IP. The Parties further make no warranty that the use of any intellectual property made, or generated under this Agreement will not infringe any other United States or foreign patent or other intellectual property right. |
ARTICLE 5 EFFECTIVE DATE AND DURATION
The effective date of this Agreement is the date of last signature by the Parties. This agreement will terminate thirty-six (36) months from the effective date, unless terminated earlier per Article 6, or due to lack of funding per Article 3.
ARTICLE 6 TERMINATION
6.1 Termination By Either Party. This Agreement may be terminated by either Party, in the event that the other Party
1. Files or has filed against it a petition under the Bankruptcy Act, makes an assignment for the benefit of creditors, has a receiver appointed for it or a substantial part of its assets, or otherwise takes advantage of any statute or law designed for relief of debtors, or
2. Fails to perform or otherwise breaches any of its obligations hereunder, if, following the giving of notice by the terminating Party of its intent to terminate and stating the grounds therefor, the Party receiving such notice shall not have cured the failure or breach within thirty (30) days. In no event, however, shall such notice or intention to terminate be deemed to waive any rights to damages or any other remedy which the Party giving notice of breach may have as a consequence of such failure or breach. Either Party may terminate this Agreement and the license granted herein, for any reason, upon giving the other Party sixty (60) days written notice.
6.2 Obligations and Duties upon Termination If this Agreement is terminated or expires, both Parties shall be released from all obligations and duties imposed or assumed hereunder to the extent so terminated, except as expressly provided to the contrary in this Agreement. Upon expiration or termination, both Parties shall cease any further use of the Proprietary Information disclosed to the recipient by the disclosing Party.
ARTICLE 7 GENERAL PROVISIONS
7.1 This Agreement constitutes the entire agreement between the Parties concerning the Collaborative Work and supersedes any prior understanding either written or oral. Additionally, nothing in this Agreement is intended to prevent either Party from performing similar or related work under other existing or future agreements and such work shall not be included within the scope of this Agreement.
7.2 The relationship of the Parties is that of independent parties and not as agents of each other, partners, or participants in a joint venture. Each Party shall maintain sole and exclusive control over its personnel and operations.
7.3 Neither Party shall use the name of the other Party on any product or service that is directly or indirectly related to either this Agreement or any patent license or assignment associated with this Agreement without the prior approval of the other Party.
7.4 IN NO EVENT WILL EITHER PARTY BE LIABLE TO THE OTHER PARTY FOR PUNITIVE, EXEMPLARY, OR CONSEQUENTIAL DAMAGES.
7.5 Neither Party will be in breach of this Agreement for any failure of performance caused by any event beyond its reasonable control and not caused by the fault or negligence of that Party. In the event such a force majeure event occurs, the Party unable to perform must promptly notify the other Party and in good faith maintain such part performance as is reasonably possible and resume full performance as soon as is reasonably possible
7.6 This Agreement shall not be construed as an agreement between the Parties to have any future business dealings.
7.7 This Agreement shall be governed by the laws of the State of Maryland. No consideration shall be given to Maryland's conflict of laws rules.
7.8 COMPANY understands and agrees that JHU/APL has a technical direction agent relationship with the United States Government which requires that JHU/APL avoid any work under any contract or agreement that would jeopardize its or its employees' ability to act for the United States Government as an impartial or neutral evaluator. Therefore, JHU/APL shall at all times under this Agreement retain the right to refuse to accept any subcontract or other agreement to perform any work under any such subcontract or other agreement between JHU/APL and COMPANY which in JHU/APL's sole discretion would create an actual or perceived organizational or individual conflict of interest.
7.9 This Agreement may be amended and any of its terms or conditions may be waived only by a written instrument executed by the authorized officials of the parties or, in the case of a waiver, by the party waiving compliance. The failure of either party at any time or times to require performance of any provision hereof shall in no manner affect its right at a later time to enforce the same. No waiver by either party of any condition or term in any one or more instances shall be construed as a further or continuing waiver of such condition or term or of any other condition or term.
Article 8 SURVIVING PROVISIONS
8.1. The provisions covering Liability, Funding, General Provisions, and Surviving Provisions shall survive the completion, termination, or expiration of this Agreement.
8.2 The export regulations of the United States Government may prohibit, except under a special validated license, the exportation from the United States of certain commodities and/or related technical data. In order to facilitate the exchange of technical information under this Agreement, COMPANY hereby gives its assurance to JHU/APL that COMPANY will not knowingly, unless prior authorization is obtained from the appropriate United States Government agency or agencies, export any apparatus or technical data received from JHU/APL under this Agreement or LICENSED PRODUCT(S) to any restricted country specified in such regulations. JHU/APL neither represents that a license is not required nor that, if required, it will be issued by the United States Government.
| Second Sight Medical Products, Inc. | The ▇▇▇▇▇ ▇▇▇▇▇▇▇ University | |
| Applied Physics Laboratory | ||
| Date | Date |
Appendix A
Executive Summary
The ▇▇▇▇▇ ▇▇▇▇▇▇▇ University Applied Physics Laboratory (JHU/APL) is proposing to conduct a multi-year research and development (R&D) effort with the goal of creating a set of core intellectual property (IP) with applications in multiple biomedical and consumer device markets (Figure 1) and deploying that IP to a next-generation retinal prosthesis. This 36 month project is highly innovative and achievable by utilizing the financial resources available in the ▇▇▇▇▇▇ ▇▇▇▇ Fund at ▇▇▇▇▇ ▇▇▇▇▇▇▇ and will focus on development of a retinal prosthesis. This work will be conducted in close collaboration with Second Sight Medical Products, Inc. (SSMP; see attached “Joint R&D Agreement”). This project is synergistic with ongoing, independent R&D efforts at the ▇▇▇▇▇▇ ▇. ▇▇▇▇ Foundation. IP developed during this effort will also be applicable for a next-generation retinal prosthesis system and a semi-autonomous controller for assistive robotic manipulators and remote devices (called “HARMONIE”) currently under development by JHU/APL.
Figure 1: Areas of innovation for proposed R&D effort. The HARMONIE system lies at the intersection of all four areas, whereas the retinal prosthesis system occupies the upper-left quadrant. Other markets are listed at the appropriate intersection of technologies.
System Components
The retinal prosthesis system will rely on a common set of components to be developed in this effort including a wearable eye-tracking device, a wearable three-dimensional (3D) image sensor, and a library of software routines that perform scene segmentation and object recognition on the captured images (Figure 2). The eye tracker will facilitate identification of regions of interest within the patient’s extended field of view, after which the computer vision algorithms can parse the scenes captured by the 3D image sensor to detect salient entities including both navigational cues (doorways, steps, obstacles etc.) and manipulable objects (e.g. coffee cup, pencil, etc.). For the retinal prosthesis system, this information can then be optimally configured for presentation to the patient through the implant.
Work Plan
This 36 month effort will be primarily dedicated to designing, implementing and testing the technologies required by the next-generation retinal prosthesis system, and our work plan is integrated and coordinated with the ongoing design and development effort at SSMP (Figure 3). This work will span a three-phase development cycle resulting in a final deliverable suitable for testing with retinal prosthesis users. During each phase, the JHU/APL team will meet regularly with the SSMP team and conduct regular progress reviews. JHU/APL intends to use an agile development approach, and will work with SSMP to evaluate emerging technologies as candidates for inclusion in the retinal prosthesis system.
Figure 2: HARMONIE system components. Items shown in green are shared with the next-generation retinal prosthesis system
Phase I spans months 1 through 18 in which APL’s role will be to deliver a system that integrates a stereo camera sensor and eye tracking sensor into the latest generation of glasses provided by SSMP. In addition, this effort will also include algorithms required for interfacing with the eye tracking sensor, image rectification, stereo correspondence, and preliminary 2D and 3D computer vision algorithms for wall, door and obstacle detection.
In parallel to this effort, Second Sight will perform substantial hardware and software re-design of the existing Argus II Externals System (Glasses, Video Processor, Fitting System & Data Management) with the goal of improving the ease-of-use, aesthetics, performance and battery life of the existing system.
Phase 2 is the subsequent phase spanning months 3 through 27. APL’s objective during this phase is to develop an out-of-form-factor brassboard-based solution that builds on the Phase 1 by adding a time-of-flight (ToF) or structured light 3D sensor as well as modifying the COTS eye tracking sensor if needed to reduce the footprint. In addition, computer vision algorithms will be developed that utilize the new 3D sensor to perform a preliminary version of common household object detection. When used in conjunction with the stereo camera, this prototype will contain the necessary sensors to generate high resolution point clouds that can work for both indoor and outdoor lighting conditions.
Second Sight will use this phase to add capabilities related to improving the patient experience, streamline data management, developing an interface to COTS tablets/ smartphones as well as integrating remote diagnostics, fitting and troubleshooting capabilities.
Phase 3 is the final phase which spans months 14 through 36. APL will incorporate all of the lessons learned from the previous two phases resulting in an integrated advanced prototype solution that can be used for retinal prosthesis testing at SSMP. APL will develop a smaller form factor version of the eye tracking and computer vision sensors in Prototype 1 with integrated, onboard power as well as a more robust and complete suite of household object detection algorithms.
Once complete, the final prototype will be able to leverage the eye-tracking sensor to facilitate identification of the user’s field of view and the 3D sensors with corresponding algorithms to segment obstacles, walls, doors, and household objects such as a mug, pitcher, silverware, remote control, etc. in real time to provide visual cues to the retinal prosthesis user.
Second Sight will focus the development efforts during this phase on building a fully integrated, head-mounted, all-in-one video processor / glasses device that integrates the various components provided by JHU/APL including the wearable 3D image sensor(s), eye tracking device and the 2D/3D vision algorithms.
Figure 3: JHU/APL and Second Sight development timeline.
Budget
The total budget of this proposal is $8.08M of which $4.04M is allocated to APL’s role in developing components for the next-generation retinal prosthesis system and $4.04M is allocated to Second Sight. This total includes support for project management, design and development, systems engineering processes, and quarterly travel to Second Sight (Figure 4).
Figure 4: Expected costs illustrated as percentages of the overall ▇▇▇▇ Fund value (approximately $8M).
Value to JHU/APL
Over the past decade, JHU/APL has developed the world’s most advanced upper limb prosthesis, the Modular Prosthetic Limb (MPL). However, bringing an MPL or similar device to market requires control paradigms and Human Computer Interfaces (HCI) beyond those available today. The HARMONIE platform would enable a broad base of users to efficiently and effectively control dexterous manipulators like the MPL, and would also allow JHU/APL to develop innovative solutions in contemporary general-purpose HCI technologies such as head-mounted displays, augmented reality, eye-tracking, and brain-computer interfaces. Moreover, the synergies between the HARMONIE system and SSMP’s next-generation retinal prosthesis provide an opportunity for JHU/APL to expand its footprint in neuroprosthetic R&D and enhance our reputation in this field.
JHU/APL Statement of Work
JHU/APL tasks:
Phase 1:
1. Systems Engineering
1.1. Establish design controls in compliance with JHU/APL’s Quality Management System and the FDA’s Quality System regulations
1.2. Conduct trade studies to evaluate currently-available commercial devices versus custom solutions for critical system components
1.3. Establish requirements database
1.4. Establish interface control documentation
1.5. Establish risk management plan
1.6. Establish test plan
1.7. Conduct periodic design reviews (with Second Sight)
2. Integrate OEM eye tracking sensor suitable for use with existing retinal prosthesis
2.1. Hardware engineering tasks
2.1.1. Acquire eye tracking sensor
2.2. Software engineering tasks
2.2.1. Create firmware to interact with eye tracking sensor
2.2.2. Develop software for identifying location of eye gaze and eye gaze trajectory
3. Integrate stereo camera based depth map sensor for use with existing retinal prosthesis
3.1. Hardware engineering tasks
3.1.1. Acquire depth map sensor
3.2. Software engineering tasks
3.2.1. Create firmware to interact with depth map sensor
3.2.2. Develop software for generating point cloud data
3.2.3. Develop software for segmenting visual scene
4. Create computer vision algorithms for identifying environmental cues for orientation and navigation
4.1. Develop a framework for evaluating computer vision algorithms
4.2. Develop software to co-register information from depth map sensor and eye tracking sensor
4.3. Develop algorithms for identifying physical objects and structures within the depth map sensor’s field of view
Phase 2:
1. Systems Engineering
1.1. Update design controls in compliance with JHU/APL’s Quality Management System and the FDA’s Quality System regulations
1.2. Conduct trade studies to evaluate currently-available commercial devices versus custom solutions for critical system components
1.3. Maintain requirements database
1.4. Maintain interface control documentation
1.5. Maintain risk management plan
1.6. Maintain test plan
1.7. Conduct periodic design reviews (with Second Sight)
1.8. Provide support to Second Sight for clinical testing preparation
2. Integrate OEM eye tracking sensor suitable for use with out-of-form-factor brassboard-based retinal prosthesis system
2.1. Hardware engineering tasks
2.1.1. Modify existing COTS eye tracking sensor or develop custom eye tracking sensor to reduce footprint
2.1.2. Integrate sensor with Second Sight’s development board
2.2. Software engineering tasks
2.2.1. Create firmware to interact with eye tracking sensor
2.2.2. Develop software for identifying location of eye gaze and eye gaze trajectory
3. Integrate additional time-of-flight or structured light 3D camera for use with out-of-form-factor brassboard-based retinal prosthesis system
3.1. Hardware engineering tasks
3.1.1. Build depth map sensor
3.1.2. Integrate sensor with Second Sight’s development board
3.2. Software engineering tasks
3.2.1. Create firmware to interact with depth map sensor
3.2.2. Develop software for generating point cloud data
3.2.3. Develop software for segmenting visual scene
4. Enhance computer vision algorithms for identifying environmental cues for orientation and navigation
4.1. Develop software to co-register information from depth map sensor and eye tracking sensor
4.3. Enhance algorithms for identifying physical objects and structures within the depth map sensor’s field of view
4.4. Integrate algorithms with development board
Phase 3:
1. Systems Engineering
1.1. Update design controls in compliance with JHU/APL’s Quality Management System and the FDA’s Quality System regulations
1.2. Conduct trade studies to evaluate currently-available commercial devices versus custom solutions for critical system components
1.3. Maintain requirements database
1.4. Maintain interface control documentation
1.5. Maintain risk management plan
1.6. Maintain test plan
1.7. Conduct periodic design reviews (with Second Sight)
1.8 Provide support to Second Sight for clinical testing
2. Integrate eye tracking sensor for use with final form factor retinal prosthesis system
2.1. Hardware engineering tasks
2.1.1. Integrate sensor with Second Sight’s final form factor retinal prosthesis system
2.2. Software engineering tasks
2.2.1. Enhance firmware to interact with eye tracking sensor
2.2.2. Enhance software for identifying location of eye gaze and eye gaze trajectory
3. Integrate additional time-of-flight or structured light 3D camera for use with final form factor retinal prosthesis system
3.1. Hardware engineering tasks
3.1.2. Integrate sensor with Second Sight’s final form factor retinal prosthesis system
3.2. Software engineering tasks
3.2.1. Enhance firmware to interact with depth map sensor
3.2.2. Enhance software for generating point cloud data
3.2.3. Enhance software for segmenting visual scene
4. Enhance computer vision algorithms for identifying environmental cues for orientation and navigation
4.1. Develop software to co-register information from depth map sensor and eye tracking sensor
4.3. Enhance algorithms for identifying physical objects and structures within the depth map sensor’s field of view
4.4. Integrate algorithms with final form factor retinal prosthesis system
JHU/APL deliverables:
Phase 1:
1. Design documents – Month 2
1.1. Eye tracking sensor
1.2. Depth map sensor
1.3. Computer vision algorithms
2. Interface control documents – Month 4
2.1. Eye tracking sensor
2.2. Depth map sensor
2.3. Computer vision algorithms
3. Test description documents – Month 6
3.1. Eye tracking sensor
3.2. Depth map sensor
3.3. Computer vision algorithms
4. Component prototypes
4.1. OEM eye tracking sensor – Month 10
4.2. Stereo vision based depth sensor – Month 11
4.3. Computer vision algorithms – Month 13
Phase 2:
1. Component prototypes integrated in out-of-form-factor brassboard-based retinal prosthesis system
1.1. Custom eye tracking sensor – Month 18
1.2. Time-of-flight based depth sensor – Month 20
1.3. Computer vision algorithms – Month 20
Phase 3:
1. Component prototypes integrated in final form factor retinal prosthesis system
1.1. Custom eye tracking sensor – Month 25
1.2. Time-of-flight + stereo camera based depth sensor – Month 27
1.3. Computer vision algorithms – Month 27
Second Sight Statement of Work
Phase 1:
| 1. | Improved industrial design (Glasses & Video processor) for better aesthetics and greater patient satisfaction |
| 2. | Increased processing power of the Externals System (dual-core OMAP processor + FPGA devices) |
| 3. | Improved industrial design (User Interface of Fitting System) and fitting algorithms to shorten patient fitting / performance evaluation time, reduce training and support burden on clinical staff and improve usability/user experience |
| 4. | Untethered (wireless) and browser-based Fitting System for ease of rehabilitation |
| 5. | Simplified and integrated data reporting and analysis features (e.g. threshold history, patient performance charts) for more efficient patient performance monitoring |
| 6. | Chromatic based image processing algorithms and custom digital camera design for better performance by patients in orientation and mobility tasks in inclement lighting conditions (low-light, glare situations etc.) |
| 7. | Patient adjustable controls (brightness/sensitivity and zoom) on the Video processor |
| 8. | Improved Glasses electronics/coil design to increase system battery life, increase telemetry range, reduce temperature of patient-touching surfaces and support better ergonomics all for greater patient satisfaction |
| 9. | Custom battery design for improved ergonomics and better charge capacity |
| 10. | Framework for downloadable image processing filters |
| 11. | Support for Next Generation Implant/ASIC advanced features (e.g. current steering) |
| 12. | Streamlined data management (“Cloud” connectivity) for more efficient access control management, Implant matching, Cloning and data transfer |
| 13. | Perform patient testing in conjunction with JHU (IRB approvals, patient enrollment, generation and execution of clinical validation protocol) |
Phase 2:
| 1. | Internationalization of the Fitting System User Interface |
| 2. | Support for assistive technologies (e.g. voice feedback, vibratory) for better patient experience |
| 3. | Visual Psychophysics Research Platform (Experiment Builder) |
| 4. | Fitting System support for tablets and smartphones (‘Bring Your Own Device’) |
| 5. | Support for Remote Fitting and Troubleshooting to reduce cost and support burden on clinical staff |
| 6. | Customizable Glasses frames |
Phase 3:
| 1. | Wearable 3D image sensor (developed as part of the APL R&D effort) |
| 2. | Wearable eye tracking device (developed as part of the APL R&D effort) |
| 3. | Complete suite of 3D scene segmentation and household object detection image processing algorithms (developed as part of the APL R&D effort) |
| 4. | Support for Next Generation Implant/ASIC advanced features (e.g. neural response imaging etc.) |
| 5. | Inertial sensors (accelerometers/gyroscopes) for tracking patient head coordinates |
| 6. | Perform patient testing in conjunction with JHU (IRB approvals, patient enrollment, generation and execution of clinical validation protocol) |
Second Sight Deliverables
Phase 1:
| 1. | Re-designed version of the Argus II Externals System (Glasses, Video Processor, Fitting System & Data Management) with improved ease-of-use, aesthetics, performance and battery life – Month 14 |
| 2. | Clinical Validation report detailing clinical efficacy of the device – Month 18 |
| 3. | Documentation needed by APL for integration – Month 18 |
Phase 2:
| 1. | Additional capabilities to the retinal prosthesis system including internationalization of Fitting System UI, voice and vibratory feedback, Visual Psychophysics Research Platform and remote fitting and troubleshooting – Month 27 |
| 2. | Documentation needed by APL for integration – Month 27 |
Phase 3:
| 1. | A final form factor retinal prosthesis device that consists of a wearable 3D sensor, eye tracking sensor and3D vision algorithms – Month 32 |
| 2. | Clinical Validation report detailing clinical efficacy of the device – Month 36 |