The strength of the ELISPOT technique lies in its outstanding sensitivity to detect antigen-specific T and B cells in even very low frequencies, on a single cell level. In most scenarios, the assay can be performed without any in vitro expansion of cells or addition of exogenous cytokines, offering the possibility to attain a precise estimate of reactive cells in a donor. Further, these measurements can be achieved in relatively short time with a straightforward protocol that can be standardized and exposed to qualification and validation procedures following available guidance. The assay can be adapted to high-throughput sample screening which is supported by the demonstration that cryopreserved cells can perform comparable to fresh cells in ELISPOT assays. Further, a wide range of qualified reagents, materials, and equipment exists, and various controls and quality assurance parameters have been described and made available to scientists performing the assay. While the advantage of ELISPOT testing is its superb screening ability for cells secreting a specific cytokine (most commonly, IFN-G), it has to be noted that it can be adapted to the simultaneous detection of two cytokines, as well as a variety of secreted molecules, including granzyme B and perforin.
As in every assay, the outcome is dependent on the protocol choices made and the established laboratory environment the assay is conducted in. It is well-known and reviewed elsewhere that choices, like ELISPOT plate, antibody coating concentration, spot development system, and other protocol variables, can influence the final spot numbers. Further adding to possible sources of result variation is the final analysis approach of ELISPOT plates. Another complicating issue is the nonlinearity of responses in dependence of the cell number plated in a well. While linearity is preserved within a specific cell range (typically,<150,000 effector cells per well) if sufficient costimulation as well as antigen presentation by separate cells are provided, there is only a limited linearity range existent when peripheral blood mononuclear cells (PBMCs) are used as effectors and antigen presenters at the same time. This observation is most likely influenced by the fact that less than 200,000 PBMCs per well do not guarantee optimal antigen presentation conditions while more than 200,000 cells start to pile up on each other, thus providing good cell-to-cell contact, but limiting the percentage of cells with direct contact to the coating antibody bound to the well membrane, which is essential for spot formation.
These findings are not new, and the field has responded with the establishment of Standardized Operating Procedures (SOPs), especially in clinical immune-monitoring labs. During this process, labs typically test variations of different protocol choices and select those with the most desired outcome as the standard to adhere to. A logical conclusion would be that all standardized laboratories have similar protocols since it can be assumed that each one opted for the most desired results (highest specific spot numbers, lowest background reactivity levels, and lowest variability within replicates), which should be achievable with the most optimized reagents, materials, and general protocol procedures. Nonetheless, countless different SOPs exist, even for closely related experimental requirements. Certainly, some of this divergence can be explained by factors already mentioned earlier, like local availability or preference of reagents and materials and their vendors, previous experience of operators, or recommendations obtained from collaborators.
However, parts of this development might be accounted for by the predicament of the lack of a true gold reference standard for ELISPOT. Some groups attempt to solve this challenge by using T-cell lines or clones, others PBMC reference samples. While the first option is of limited wider applicability, the latter one does not truly represent a reference standard since the actual number of antigen-specific T cells able to secrete a given cytokine in these preparations is not precisely known. PBMC reference samples can be an excellent tool for standardization and validation approaches, as well as external controls for ELISPOT experiments; they are not, however, a reference standard for the amount of analyte or, in the case of ELISPOT, the number of antigen-specific cytokine secreting cells. Hence, the question always remains: Is the measurement perceived as optimal with a given protocol indeed the correct measurement? Or with other words: Does the protocol permit optimal sensitivity and specificity (all cells detected without false-positive signals)? The key question that arises from these challenges is: How comparable are ELISPOT measurements across laboratories?
Proficiency panel programs are typically conducted to provide participants a feedback about their test performance relative to a predefined reference value (see Note 1). This feedback can be of additional importance, as regular and successful (e.g., results within a given range) participation in proficiency panel programs might be requested by regulatory frameworks, depending on the specific setting.
In addition to quality assessment, proficiency panel programs can serve as a tool to define the extent and specifics of assay harmonization necessary. In order to allow the identification of critical assay steps that influence the assay outcome and to generate harmonization guidelines, proficiency panels need to be properly designed and conducted in such a way that a large enough number of representative data sets are obtained. Successful assay harmonization can first and foremost increase the comparability of results generated across institutions. This goal clearly is of high interest to the scientific community, but might not be the main interest of participating labs. Here, the question how individual labs can benefit from participating in harmonization activities is addressed.
It has been clearly stated that each method implemented for patient testing needs to undergo an external quality assessment via proficiency panel testing. For such testing, the same samples are sent to participating laboratories, where they need to be tested with the established assay. Results are centrally collected and analyzed. A feedback about each lab’s performance is given in comparison to the entire panel. If a lab’s performance is not in acceptable consensus with the overall panel results, necessary steps to correct and improve the assay outcome within that lab need to be taken.
It has been suggested that the expected accuracy for proficiency panel testing should be >90%. However, as accuracy describes the closeness of results to the true value, determining the accuracy level for ELISPOT testing is a challenge due to the difficulty to ascertain the actual number of antigen-specific cells in PBMC samples. A solution to this impediment could be offered by the proficiency panel itself. Given a well-designed panel with a sufficient number of participating laboratories with their own established protocol (providing an acceptable cross-section of applied protocols in the field), it can be assumed that the measurement median of the entire panel for a given sample provides a representative estimate of antigen-reactive cells in that sample. In fact, an accumulation of participants’ measurements around the panel median has been demonstrated for previously conducted ELISPOT and other proficiency panels. With this in mind, it appears reasonable to propose that the median measurement values of large, open panels could provide a range for an alternative reference standard for certain biological assays, like ELISPOT (see Note 2).
The use of ELISPOT assays as an immune-monitoring tool in clinical trial context has consequently led to the initiation of various large international proficiency panel programs. A main goal of these panels is to offer an external quality assessment for laboratories using ELISPOT for patient testing in the cancer and HIV vaccine and related fields. A central aspect of these programs is their thought-out design that allows comparability of results while including laboratories with different protocols in place. Not surprisingly, the interlaboratory variability observed was high, and labs were identified that were not able to detect all responses even on a yes/no basis. Recent harmonization efforts evolving out of these activities have dramatically improved these initial observations. Furthermore, panels with strict overall standardization as required in specific vaccine networks were able to demonstrate encouraging concordance of results (see Note 3).
Several smaller ELISPOT proficiency panels with a limited number of participating centers were conducted by groups in the field of cancer, autoimmunity, and infectious diseases. Larger, more systematic approaches to identify critical assay variables were initiated in 2005 and mainly driven by the HIV and cancer vaccine field.
The design of large international ELISPOT proficiency panels with the inclusion of labs employing different SOPs has opened the door to a process that allows the investigation of crucial protocol variables which influence the assay outcome in either direction. Once such variables have been identified, measures can be taken to harmonize the field toward a uniform approach of dealing with them. During the past few years, two collaborating programs have made significant contributions to the harmonization of ELISPOT testing: the proficiency panel program of the Cancer Immunotherapy Consortium of the Cancer Research Institute (CIC/CRI) and the Cancer Immunoguiding Program (CIP) of the Association for Cancer Immunotherapy (CIMT). Both programs were able to systematically investigate specific protocol variables for their influence on ELISPOT testing by analyzing data and protocol specifics obtained from their recurring large-scale proficiency panels. Their findings are summarized in initial ELISPOT harmonization guidelines which were made available to the community. Interestingly, these initial guidelines address rather general assay steps, which do not require major protocol changes and, importantly, do not impose strict overall standardization measures to the field. Most importantly, they are continuously being adapted by panelists, and their implementation has assisted remarkably in improving the overall panel outcome. Notably, these harmonization efforts do not end with the publication of initial guidelines, but continue with constant refinements (see Note 4). For instance, both programs have initiated a thorough investigation of the influence of serum and the use of serum-free media for the ELISPOT assay. It could be shown that serum is not required for ELISPOT performance and that commercially available serum-free media can perform at least equally well in human IFN-G ELISPOT assays as extensively pretested serum-supplemented media. A logical next study is underway testing the influence of different freezing media on ELISPOT outcome.
In addition, proficiency panel projects have demonstrated that even after an experiment was done and spots were counted, considerable variation can occur due to the interpretation of raw data from ELISPOT assays. As various methods for response determination lead to variable outcomes, harmonization of ELISPOT assays has to include the harmonization of response determination as addressed in Chapter 15 in this book. The assay harmonization efforts conducted over the past 5 years led to the identification of several critical experimental process steps based on the analysis of large, representative data sets. Obviously, any published report of ELISPOT experiments should include sufficient information on critical test variables and process steps (see Note 5). To this end, the MIATA project was launched which addresses the minimal information that needs to be published when reporting results from T-cell assays (see Note 6).
Figure 1. Improvement of ELISPOT performance during the harmonization process.
The typical evolution of assay development can be divided into six subsequent steps:
Assay development begins even prior to the first experiment by defining the actual assay (what will it measure, how it will be measured) and the first selection of reagents, materials, and protocol variables. The initial assay test runs are typically followed by systematic benchmarking studies for all (or the most critical) assay variables/ steps. Results from internal benchmarking studies can be used to further optimize the protocol. Obviously, investigators in these early stages of assay evolution can benefit considerably from integrating recommendations and guidelines deduced from harmonization efforts.
Protocol optimization is followed by standardization which is typically achieved by generating and implementing SOPs. Once a working SOP is in place, assay qualification and validation can be tackled which is supported by first describing the purpose and design of planned validation studies and how each of the critical parameters is addressed in detail (validation plan). The prevalidation stage establishes the parameters for qualifying the assay by performing a series of exploratory experiments addressing each of the defined validation parameters. The validation stage involves conducting a series of experiments to determine whether the specifications established during the prevalidation stage can be consistently met.
The organizers of proficiency panels acknowledge that the most advanced labs generally contribute best to harmonization efforts as they can generate robust data sets. Nevertheless, labs with newly developed and nonvalidated assay protocols can also achieve outstanding test sensitivity and performance and thus contribute valuable data sets.
Obviously, an investigator who has already validated an ELISPOT assay might prefer not to change any assay component as this would ask for time-consuming revalidation of the new protocol. However, in more than one instance, panel participation was regarded as an eye opener and has led to modifications in even long-established protocols (see Note 7).
The increased comparability of results generated across institutions that can be reached by harmonization efforts represents a clear advancement for the scientific community. Without doubt, even investigators who use validated assays within clinical studies can value the possibility to better compare own results with data sets that were generated by peers who use similar antigens and drug formats and treat similar patient groups. In addition, it seems reasonable to argue that participation in proficiency panels can benefit a lab’s assay development independent of its stage. During assay evolution, one constantly needs to compare assay performance to a reference standard. Proficiency panels can offer an alternative reference standard, as described earlier, thus providing a solution to the lack of a true gold reference standard in ELISPOT. Further, participation in proficiency testing projects allows the performance comparison with the field at each step of assay evolution. This contributes to enhanced confidence in optimization and standardization procedures and the actual performance of appointed staff members.
In summary, the output from harmonization activities can help to develop and optimize an assay at early stages of assay evolution. By repetitively comparing the performance of many different protocols, large data sets are generated which can be used to define typical and extreme performance characteristics for the ELISPOT assay. This knowledge can be used to set specifications for assay validation. Finally, even experienced and validated labs can profit from participating in harmonization activities due to the feedback of performance they obtain, which would expose the quality of their assay performance.
Reference
