Duke Human Vaccine Institute Science Culture and Accountability Plan
The Mission of the Duke Human Vaccine Institute (DHVI) is to perform the basic and translational research necessary to discover the enabling technology for development of vaccines to prevent global emerging and re-emerging infections. Maintaining quality and integrity of basic, translational and clinical research in the DHVI is a central goal and integral to the mission of DHVI. The DHVI science culture and accountability plan is underpinned by ten principles. All trainees, staff and faculty are educated on the 10 principles for work ethics and carrying out research in the DHVI.
- Creation of a research environment in which research results are open, transparent, and DHVI scientists, trainees and technical staff feel comfortable in discussion of the data generated. Communication and trust in each other is the key to an open and transparent environment. In addition, the leadership continually reminds DHVI staff that due to the resources that are entrusted to us, that the expectations are similarly great, and there can be no road in any of our activities but the high road. This notion is reinforced in all DHVI meetings, lab meetings, and by modeling of behavior in all aspects of science by DHVI leadership and faculty. Specifically, we repeatedly discuss the pitfalls of publishing incorrect data, the factors that contribute to scientific fraud, and discuss ways for DHVI members to handle work stress
- Create a research environment in which each research team member feels comfortable and supported to say, “I don’t know the answer to the question” and “I made a mistake”. This is an expectation by leadership and DHVI leaders guarantee that no employee, trainee or senior staff is criticized for admitting that they made a mistake. At DHVI meetings and all lab meetings, DHVI members are told that the only way to becoming an A+ scientist is to be able to admit a mistake and to rapidly correct it. The expectation is put forth that there must be no emotional impact to admitting that a mistake has been made. This mantra has been accepted throughout DHVI and daily the leadership learns of mistakes and their corrections.
- Written and verbal communication of data from and to science team members (faculty, trainees, technical staff) that enable open discussion and sharing of data generated in a timely manner each week. This practice has been in place since 1990 at DHVI’s inception. Every scientist, trainee and technician is expected to report, through written communication, the primary data they have generated that week and what the plans are for the next week. This reporting can take the form of weekly email updates to the senior investigators of each team and/or reporting to the entire work group. Senior investigators document scientific directions for each team member in the form of written and verbal communication during group meetings with the overall goal of transparency and letting all team members see what everyone else is doing. For the Duke Center for HIV/AIDS Vaccine Immunogen Discovery (CHAVI-ID) team for example, Dr. Bart Haynes receives and sends material to ~30 people. This practice serves three purposes. It establishes trust in the junior scientists that the senior people know what they are doing and provide weekly feedback, and trust that no one else is working on exactly the same thing as they are. Second it creates an open forum of information allowing the community of investigators working on similar project to learn from each other. Finally, it allows each senior person to personally review all primary data and creates an environment of openness, sharing and the opportunity for many people to critique what others are doing. Other examples of documenting the work at DHVI include electronically tracking and assigning projects that by design are open access to all in the work group as well as weekly work group meetings where all data from the previous week is presented and discussed.
- All DHVI laboratory books and electronic files are open to DHVI leadership.This is a stated requirement for each employee during interviews and at the time of employment. This creates the expectation that all at DHVI make primary data available to anyone who wishes to review it and as well, establishes the practice of many eyes reviewing primary data. The hard copy DHVI laboratory notebooks are numbered and labelled with each scientists’ name. Electronic notebooks have a log in account and are automatically time and date stamped specific for each scientist. Both hard copy and electronic notebooks are open for viewing by any member of the work group and DHVI leadership. The expectation for all scientists is that they document their work each day. All DHVI laboratory books and electronic files are property of Duke University, and remain at DHVI when team members leave Duke. This is also a stated requirement of DHVI and similarly creates the expectation of open books and repetitive oversight of all work and primary data.
- All research findings are discussed weekly within DHVI in lab meetings and ad hoc team meetings, weekly with outside collaborating team members, and reviewed yearly by external Scientific Advisory Board members. The DHVI is supported primarily by the National Institutes of Health (NIH) and Bill & Melinda Gates Foundation (BMGF) grants, and these grants require outside Scientific Advisory Boards (SABs). Thus for most of the DHVI work, each year, the science performed is rigorously reviewed by independent outside SABs made up of the most prominent and preeminent scientists in the fields of immunology, host defense, genetics and cellular, molecular and structural biology. Internally, all research findings are discussed within DHVI in lab meetings, and with CHAVI-ID and BMGF grants collaborators on weekly and monthly calls respectively. For work that leads to human clinical trials, before any clinical trials can be contemplated, the concepts must be approved by the CHAVI-ID/Gates SABs, by the NIH Division of AIDS Program Staff, and by the Vaccine Development Working Group at the National Institute of Allergy and Infectious Diseases (NIAID). In addition, all clinical protocols at DHVI are approved both by the Duke University Health System Institutional Review Board (IRB) and the NIH IRB after exhaustive reviews. All of our clinical trials are performed by the HIV Vaccine Trials Network, and their review committees similarly review all proposed clinical trials and science performed on them.
- When key discoveries are made, they are independently replicated by others. Key to open science is replication of experimental data and statistical analyses by others. At DHVI there is an expectation that all work that is published is replicated in the laboratory that generated the data, and critical discoveries are independently confirmed. This independent confirmation of critical discoveries can take the form of another scientist replicating the experiment in a blinded fashion and/or confirmation by a different laboratory group. In accordance with the above principle that all data are open and transparent and documented in a timely fashion, DHVI works with statistical teams to review, analyze and apply the most rigorous standards to publication of data. DHVI has a consortium of statistical teams comprised of statistical teams at Duke University, Boston University, Statistical Center for HIV/AIDS Research and Prevention (SCHARP), University of North Carolina, and Los Alamos National Laboratory. Members of the biostatistics teams that are affiliated with DHVI are involved in all projects from their inception, and are key members of all the project teams. Data sets that are relevant to clinical trials and patient care decisions are independently replicated by an independent statistical group.
- Before papers are published, they are reviewed by outside reviewers, and comments incorporated into the papers. After publication, data are made public on websites and/or on upon request.Before publication, each manuscript is reviewed and approved by each member of the scientific and statistical teams involved in the work. For consortium papers (e.g. CHAVI-ID, BMGF, HVTN), there is an additional layer of review by the publication committee before manuscript submission. For example, CHAVI-ID publications committee is made up of scientists from University of Pennsylvania, Harvard University, University of Oxford and Duke University. For all published data generated at DHVI, the data are shared freely with the scientific community, either by posting raw and summary data on BMGF websites or on sites such as GenBank or others. When requests for data, experimental methods, and/or statistical methods/analytical tools are made, they are freely provided in the spirit of open access.
- Concerns regarding scientific or research data integrity issues raised by students, post-doctoral fellows, laboratory staff, and faculty are all taken seriously and vetted by DHVI leadership. By creation of the open environment by policies 1-7 above, we work to create an environment in which all DHVI members and trainees feel completely comfortable to question data and ongoing studies. This is accomplished by robust open conversations about all data by DHVI leaders and PIs and good modeling of constructive critical discussions of all data. Each technician, trainee and PI is treated equally with regard to ability to raise issues and question either data or the interpretation of data. If disagreements arise, they are resolved prior to public presentations or publications.
- A mentoring program is in place for all faculty and trainees. Each faculty member receives a mentoring letter in the context of a dedicated mentoring session and includes expectations of accountability of science and compliance with all university and federal regulations. In addition, Dr. Georgia Tomaras Director of Training and Dr. Bart Haynes, DHVI Director, have an open door policy and encourage all trainees to come meet to review data together and hear concerns either individually or in dedicated forums for communication with trainees in the absence of other senior staff. Additionally, each Monday, DHVI holds a 1-hour lunch meeting for trainees to discuss and review the scientific literature and these meetings have included presentations and discussions on data provenance, data reproducibility and best practices for analyzing, reporting and communicating data.
- Data provenance is a priority. Data provenance is an institute priority. Regular meetings with faculty and key staff members are conducted to ensure SOPs are developed and maintained across the institute. We are approaching this issue from two different angles. The first is to require each PI to develop, implement, and maintain data management SOPs in each individual lab to ensure the integrity and reproducibility of all scientific data. This process, facilitated at the Institute level, has been completed for several labs and continues to be developed by the rest of the faculty. Data Provenance principles continue to be refined for the labs, and are living documents that will be updated at least once per year. The second aspect of the data provenance project is the infusion of DP principles and Best Practices into research and administrative IT systems. Currently under development are improved electronic record keeping systems for wet lab data to conform to Data Provenance principles and meet Best Practices. An example of this is the implementation of electronic notebook systems (a 21 CFR Part 11 compliant, fully auditable electronic notebook system that maintains raw data integrity and secure data sharing) coupled with a secure data management system (secure repository). These systems provide automated file indexing and version control, secure and centralized permanent archive, and a fully traceable audit trail for all files generated from data acquisition from an instrument to reporting data for manuscripts and clinical trials.