Harnessing the power of knockout validation for research reproducibility

Video Transcript

00:00 - 00:10: Hello and welcome to everyone joining us for today’s Select Science webinar entitled 00:10 - 00:16: Harnessing Knockout Validation for Reproducible Research. My name is Carrie Haslam and I will 00:16 - 00:22: be moderating today’s presentation. Today I’m delighted to be joined by Dr. Hanna Dreja, 00:22 - 00:28: Principal Scientist at Abcam. After the presentation we’ll move on to our question and answer session 00:28 - 00:33: so please feel free to ask any questions for the Q&A at any time during the webinar 00:33 - 00:37: and you can submit your questions to the left of the screen. We will also be releasing some 00:37 - 00:41: poll questions so please do answer these as the webinar goes along, that would be greatly 00:41 - 00:47: appreciated. So without further delay, I’d like to hand over to Hanna for today’s presentation 00:47 - 00:49: and I would like to thank her again for speaking with us today. 00:51 - 00:57: So good afternoon and thank you Carrie for that introduction. My name is Hanna Dreja 00:57 - 01:04: and I will talk about Harnessing Knockout Validation for Reproducible Research. My career 01:04 - 01:10: has covered areas of immune responses to viruses and vaccines, monoclonal antibody characterization 01:10 - 01:16: as well as gene therapy and here at Abcam I lead this scientific quality control team which is 01:16 - 01:22: involved in new product development, making sure that our new products are of the highest specificity 01:22 - 01:29: and quality. We also re-evaluate and audit our antibody catalog continuously by using a portfolio 01:29 - 01:36: of knockout cell lines and lysates using mainly western blotting. The objective of this book is 01:36 - 01:43: to highlight the importance of experimental validation where edited cell lines can help 01:43 - 01:48: and I will also provide you with some tips on how to get your samples to work from me 01:49 - 01:52: and I will also take questions at the end. 01:59 - 02:05: So as you can see I have a lot of first-hand experience using knockout samples 02:07 - 02:13: and so have researchers across the scientific discipline. Cell line editing has found a firm 02:13 - 02:18: place in academic research as well as in the biotechnology and pharmaceutical industries. 02:20 - 02:27: There are principally three areas where such edited cell samples prove extremely valuable. 02:28 - 02:36: It’s for experimental validation, confirming the specificity of the immunoassay, for target 02:36 - 02:42: selection, so early stage identification and validation to make sure the target is relevant 02:42 - 02:49: in your research. As well as phenotypic assay, helping answer those biological 02:49 - 02:55: functional questions, so what are the effects of removing or adding a protein on the state 02:55 - 03:01: of differentiation as is depicted in this example on the right. 03:02 - 03:07: But to go back to that first area validation, why am I so passionate about it? 03:08 - 03:17: Well you may be aware that there is an issue with reproducibility of results in biomedical 03:17 - 03:28: research. In 2012 a study led by Dr. Begley at Amgen was only able to reproduce six out of 53 03:29 - 03:40: cancer studies. So this study raised concerns regarding irreproducible research. It highlighted 03:40 - 03:45: issues such as selective presentation of favorable results and poor reporting. 03:45 - 03:52: And one study found that only about half of publications included enough information about 03:52 - 03:57: reagents such as the suppliers and clones to allow other researchers to purchase the same 03:58 - 04:03: product and reproduce experiments. It also highlighted a lack of blinded studies and 04:03 - 04:14: controls and a lack of reagent validation. And this lack of a validated reagent has been 04:14 - 04:21: highlighted in multiple reviews and publications recently. And proper reporting and experimental 04:21 - 04:26: controls as well as reagent validation are vital for successful experimentation. 04:28 - 04:41: And this experimental validation is essential to succeed in detecting the target of interest. 04:42 - 04:48: Here are some examples which help the target identification. Resources are available to use 04:48 - 04:55: transgenic mice to overexpress or not express the gene of interest. Genome editing via CRISPR 04:55 - 05:03: technology and others which can modify the expression of a gene. And RNA interference 05:03 - 05:09: can shut off expression. Overexpression will help as a positive control in your immunoassay. 05:09 - 05:15: And proteins can also be removed from the cells in question using new technologies such as 05:15 - 05:20: RNA. But to go back to the lack of reproducibility. 05:26 - 05:33: So reproducibility is a responsibility shared by many stakeholders. For antibodies, 05:33 - 05:39: the suppliers should of course demonstrate specificity, adhere to controlled manufacturing 05:39 - 05:47: standards. Also scientific journals must ensure that complete methods and reagents are reported 05:47 - 05:53: in publications. Researchers themselves must also take steps to address this by sharing 05:53 - 05:59: validation data and using appropriate controls in their experiments. And funding models should 05:59 - 06:06: expect this. A lot of useful guidelines have been written on how to validate the binders 06:06 - 06:11: which should influence both suppliers, journals, funding bodies, and researchers. 06:18 - 06:26: A proposal for validation of antibodies was published by Mattias Uhlen in Nature 06:26 - 06:33: Methods in 2016. This followed an international working group for antibody validation summit 06:34 - 06:41: which formulated the best approaches for validating antibodies. This group recommends 06:41 - 06:48: five conceptual pillars for antibody validation to be used in the application-specific manner. 06:49 - 06:56: The genetic strategy, such as knockout strategy, is considered the most robust of these processes. 07:04 - 07:10: And that the knockout strategy is considered the most robust way to validate immunoassays 07:10 - 07:18: is also recognized by grant bodies. The NIH grant application guidelines demand that the knockout 07:18 - 07:26: samples are incorporated in the project plan. So the data that you will see throughout this 07:27 - 07:33: presentation relies on samples for CRISPR-Cas9 has been employed to cause a mutation in the gene 07:33 - 07:38: of interest which results in a frameshift and ultimately lack of protein expression. 07:39 - 07:48: And one example here is the EZR knockout cell lines that you can see on your right-hand side. 07:48 - 07:55: So, as I mentioned, reproducibility is the responsibility of funding bodies, journals, 07:55 - 08:02: and researchers. But suppliers have obviously an important role to play. However, we, the suppliers, 08:03 - 08:09: can sometimes get things wrong. In this case, I compared two different antibodies, 08:10 - 08:25: GAIN-CD73. Antibody 81720 had been cited already 17 times in the literature. And for that reason, 08:25 - 08:32: I was expecting this to work. But it did not detect the protein of interest. And this was 08:32 - 08:38: confirmed when I compared the results with one of our recombinant rabbit monoclonal antibodies 08:39 - 08:50: on your right-hand side. This antibody had much fewer citations. And as a consequence, 08:53 - 09:03: we have removed this 181720 from sales to ensure that customers only choose the correct 09:03 - 09:11: antibody and avoid the mishap of using one that does not detect the protein of interest. 09:12 - 09:18: So, what I want to show from this set of data is that sometimes the supplier can get it wrong. 09:19 - 09:25: And I warmly recommend that you, the scientist, at the same time, have full validation 09:25 - 09:33: so you can draw the correct conclusion. And it is also important to flag, as is shown, 09:33 - 09:39: that just because an antibody has garnered a lot of citations, 09:39 - 09:43: does not necessarily mean that it is the best one out there. 09:44 - 09:51: And as a scientist, of course, you will check your chosen immunoassay for specificity. 09:52 - 09:58: So, if you’re interested in a secretive protein, you may choose an ELISA rather than a Western 09:58 - 10:08: blot. The data on the left shows CXCL10 from interferon and TNF-treated A549 cells. 10:09 - 10:17: And as expected, this protein is not detected in the supernatant of knockout cells where the gene 10:17 - 10:26: IP10 is dysfunctional. Similarly, imaging applications allow us to determine where in 10:26 - 10:32: the cell the protein is. And with our knockout cells, we can demonstrate that the signal is 10:32 - 10:40: indeed correct. You can see on the two lower panels that the green signal is lost in this 10:40 - 10:49: particular knockout cell line. So, please, try to use good negative controls in your types of assays. 11:02 - 11:09: Having these types of control samples in academic research is important. In 11:09 - 11:16: assay development, in diagnostic or therapeutic areas, such robustness is essential. As you can 11:16 - 11:23: imagine, this is valuable in novel therapeutic generation, in control of pharmacodynamic 11:23 - 11:28: assay development, drug substance analysis, as well as recapitulation of disease. 11:29 - 11:37: And one laboratory head that we spoke with claims that the validation and in-depth study of the 11:38 - 11:45: underlying biology of targets now provides a first-rate foundation for the development 11:45 - 11:46: of novel therapeutics. 11:47 - 11:58: So, I hope I have made my case clear. Validating experiments is crucial. But why do scientists 12:00 - 12:07: don’t, why don’t scientists do that all the time? Well, apparently, over 70% of researchers 12:07 - 12:14: do not validate antibodies for their applications. So, why is that? Well, 12:14 - 12:27: because it takes too much time. And CRISPR technology is complex. You are expected, 12:27 - 12:34: and you will have to design your guide RNA, perform the transfection, go through the 12:34 - 12:43: enrichment cycle, perform single-cell isolation. You need to confirm the edits, validate it, 12:43 - 12:50: and then get on with the functional assays. The average time for a CRISPR gene edit and 12:50 - 12:58: clone selection is around 19 weeks. Most researchers also need to repeat their experiments 12:58 - 13:06: three to four times before obtaining the edit they are looking for. So, understandably, 13:06 - 13:12: not all scientists have the time, knowledge, or even the facilities to perform these desired 13:12 - 13:21: tests. So, where can one access such samples if you’re not a CRISPR expert? Well, you can 13:21 - 13:27: hope to get the cell lines for free from the collaborators, of course. And yes, hopefully, 13:27 - 13:34: you would trust your buddy Dave in the lab next door. You can buy it off a catalog, 13:34 - 13:40: and I would warmly recommend to look for a reputable company to supply these types of samples. 13:41 - 13:47: Or you can have them custom made. If you have specific requirements, this may be an option as 13:47 - 14:04: well. So, this is a very simplified decision tree. Whether you would want to produce 14:04 - 14:12: or purchase your knockout sample. So, if you are or want to become a CRISPR expert and have the 14:12 - 14:18: time and facility, you may decide to carefully follow the protocols and perform the engineering 14:18 - 14:26: in your laboratory. If this is not feasible, Abcam offers a catalog of lysates as well as cell lines. 14:27 - 14:37: These can be grown as normal, regular cells, so no amendments are needed to make this cell culture 14:37 - 14:44: successful. So, in case we do not have the knockout or the mutation of interest, we also 14:44 - 14:50: offer a custom service option where the cells are delivered with the edits of your choice. 14:56 - 15:06: So, as I mentioned, my team has spent about five years working with knockout cell lines 15:07 - 15:10: for antibody validation, and we have gathered a lot of experience. 15:11 - 15:18: The most important one, though, is know your target. So, ensure that the cell line background 15:18 - 15:24: is suitable for your biological question. If you are interested in exploring signaling 15:24 - 15:31: in axonal regeneration, for example, use knockout material that is of neuronal origin. 15:31 - 15:39: You may not understand what the effects are in a ROA knockout if the gene is taken out of 15:39 - 15:46: a HEK293T cell, for example. If you want to use knockout samples to demonstrate that your 15:46 - 15:52: immunoassays are working, you need to ensure that the wild-type cells express a sufficient amount 15:52 - 16:00: for the protein to be detected in your assay. We have demonstrated knockout and antibody pairs 16:00 - 16:07: at work in Western blot, but be aware that this relies on both antibodies and the cell 16:07 - 16:13: background to be true and specific. If you have not yet validated your cell line, 16:13 - 16:16: try to include additional negative controls in your experiments. 16:17 - 16:25: So, another question that we have gotten is, do your cells need stimulation to produce the 16:25 - 16:32: protein of interest? As you remember from a few slides earlier, I gave an example of the CXCL10, 16:33 - 16:39: which required stimulation for expression. So, do learn about your target. 16:39 - 16:49: Also, know your antibodies. So, in this case here, on the image on the left, 16:51 - 17:01: we have antibody 201959, which detects pre-CASPase 7, as you can see running, which is the 17:02 - 17:12: top green band, and also P20, which is a cleavage format, and this gets cleaved following treatment. 17:13 - 17:22: The next panel in the middle shows you 92842. It also sees a pro-CASPase 7, 17:22 - 17:29: but recognizes cleaved P12. And with the help of the knockout samples, we could determine that 17:29 - 17:38: this is not CASPase 7 protein, as it remains in the knockout sample, in the sample names, 17:38 - 17:50: as you can see. So, and on the right, we have an antibody that binds both to GSK3-alpha and beta. 17:51 - 17:55: And we could elegantly confirm this by using two different knockout cell lines, 17:56 - 18:04: which shows that the top band of the two that is detected is indeed the alpha, and the lower one 18:04 - 18:12: is the beta. There’s also the challenge to generate cell lines where one wants to remove an essential 18:12 - 18:20: gene. This is generally not viable for the cell line. So, alternatives, such as temporarily 18:20 - 18:26: removing the protein using siRNA, or trim away, or an inducible ON-OFF switch can be solutions. 18:28 - 18:31: Permanent CRISPR knockout silencing is not. 18:41 - 18:46: So, in summary, I have been telling you about the importance of validating antibodies, 18:46 - 18:54: proteins, and research tools, how essential that is, and it should be performed in an application- 18:55 - 19:01: and experimental setup-specific manner. Knockout validation is the most trusted method of 19:01 - 19:06: validation. It’s increasingly being required for research publication and funding. 19:06 - 19:12: The generation of knockout models is a complex and resource-intensive process. 19:13 - 19:16: Pre-made knockout cell lines and cell engineering services 19:16 - 19:20: are available at Abcam to support your validation and research. 19:29 - 19:33: And with that, I would like to thank you all.