Comments for:

---

Response to ‘Letter to the Editor’ by E.P. Diamandis, V. Kulasingam, G. Sardana

Submitter: Paul Tempst | p-tempst@mskcc.org

Memorial Sloan-Kettering Cancer Center

Published February 21, 2006

As already pointed out in our JCI article [1], we were acutely aware of several problems, past and present, in clinical proteomics-based biomarker discovery. The common denominator of these problems is bias, which can be easily and inadvertently introduced at different stages of the overall process: (i) study design, (ii) demographics of the patient and control groups, (iii) clinical chemistry, (iv) analytical chemistry, and (v) signal processing / data extraction. We had previously investigated various sources of bias that can affect stages (iii) through (v) and have taken great precautions to avoid it, as described in two earlier publications [2, 3] and in the Supplementary Methods of the JCI paper. We will, therefore, only comment on the remarks of Diamandis, Kulasingam and Sardana as they relate to study design, age and gender of the patient / control population, and on the concept of serum peptidomics as ‘functional’ proteomics.
The paper was not intended as a definitive clinical validation of serum peptidomics methodology, and certainly not as part of early detection or localized prostate cancer programs, or to distinguish prostate cancer from BPH or prostatitis. Our panel of authors, including clinicians who manage cancer patients on a daily basis, are committed to the long-term goal of developing clinically useful serum biomarkers. In parallel with the study already published in the JCI, we are also well along in other IRB-approved studies, using the same peptidomics platform, designed to address clinically relevant questions related to patients with early and late stage cancer, and these will ideally be reported in clinical journals if appropriately validated.
While extensive validation of candidate “biomarkers” is one important objective in peptidomics that we wholeheartedly agree with, it is also important to learn more about the biology underlying generation of the specific peptide markers, and that is what we aimed to do in the present study. A number of groups, including our own, asking a variety of cancer- related questions, have repeatedly found useful serum markers of cancer in the low molecular weight (<4 kDa) fraction of the serum proteome. Before moving forward towards larger clinical validation studies, we wanted to understand how these peptides came to be produced, believing that such knowledge might help in optimizing the methodology and in defining appropriate clinical questions. We believe that these aims were clearly stated in our paper.
Whereas there remained a small possibility that the results of our analysis may have been affected by differences in median age between patients and control individuals, we did address the gender issue, more specifically in Supplemental Figure 1, where it was shown that spectral peaks selected as part of marker patterns didn’t have any appreciable differences in ion-intensity between samples obtained from healthy men or women. Furthermore, while the JCI manuscript was under review and subsequently awaiting publication, we collected and analyzed plasma and serum samples from 200 healthy volunteers (46% men; 54% women) in the following age brackets: 20-35 years (35%), 35-50 years (33.5%), and >50 years old (31.5%). Peptide profiles of all serum samples were analyzed by average-linkage hierarchical clustering using standard correlation or by principal component analysis (PCA). No correlations whatsoever were observed. Supervised analysis of the same 200 profiles in an attempt to identify peptide-ions that showed statistical differences in ion intensity between any of the age groups, or between man and women, did not yield a single feature when we applied an adjusted (Benjamini and Hochberg algorithm) p-value cut-off of 1x10-5. By comparison, that same cut-off was used in the JCI study to select the dozens of cancer type-specific features listed in Figures 5 and 6, and in Supplementary Table S2. When 60 profiles from sera of one group of cancer patients (30 men; 30 women) were added to this set of 200 controls, unequivocal partitioning between cancer and no-cancer was obtained in both unsupervised hierarchical clustering and in PCA. For these reasons and others already set forth in the JCI paper, we are convinced that our results have not been tainted by age or gender bias. The detailed results of this ‘age and gender’ study will be published elsewhere.
In some of the other remarks, Diamandis and colleagues are criticizing a paper we didn’t write. Nowhere do we claim that the observed peptides are biomarkers, let alone “magic” type biomarkers that appear ex vivo after blood samples have been collected. Instead, we have clearly made the point that serum peptidome analysis by direct MALDI TOF MS is a form of functional proteomics, as it provides a ‘peptide metabolomic’ read-out of protease activity. Granted, we didn’t start out with this concept in mind, but the analytical results didn’t leave much doubt about the origin and mechanisms underlying peptide diversity and cancer-type specificity. The peptide ‘byproducts’ of the coagulation and complement cascades are actually irrelevant, except that a subset unexpectedly and fortuitously served as suitable substrates for a metabolomic ‘assay’. Some, like FPA and C3f, are ideal substrates; but others, like for instance FPB, are not. Testing of a large number of synthetic peptide substrates in our laboratory indicated that very few peptide sequences are suitable for this purpose (Villanueva and Tempst, unpublished). However, synthetic analogues of endogenous ‘founder’ peptides can be readily converted in plasma to identical peptide ladders as initially observed in serum, suggestive of equivalent exoprotease activities in plasma and serum, independent of coagulation (Figure 10 in the article).
We have no information whether the patients that were part of our JCI study had any inflammations or infections, or whether there may have been coagulation disturbances. If so, how much did it contribute to the patterns? We don’t know, but we believe it’s very unlikely to correlate with cancer type-specific differences in the patterns. Indeed, infections and inflammations, if present, will likely occur across the board, not in one specific group (type) of cancer patients. Furthermore, even if any of these conditions did affect clotting, the founder peptides are typically generated in such huge abundance that it is unlikely to present a limiting factor for the read-out. Alternatively, the read-out can be readily performed using external substrates, permitting carefully controlled, analytical laboratory-style assay conditions.
We appreciate the suggestion to focus on the tumor-specific proteases as candidate diagnostic molecules. However, presence / absence or concentrations in serum alone may not provide the full picture, and simple ELISA-based assays may miss out on important, functional information related to these enzymes in cancer patients. We are therefore pursuing vigorous research programs in two parallel areas: one on developing quantitative assays, using isotope-coded substrate analogues, and another on purification and identification of plasma/serum derived proteases.
Finally, Diamandis and colleagues advocate the use of plasma, instead of serum, for peptidomic analysis. This advice is not supported by the facts. First, Koomen et al. [4] identified several hundred peptides, purified from 5 mL of plasma (Note: a 100-fold larger volume as compared to the 0.05-mL serum used in our study), and all derived from more or less the same common blood proteins as the ones we observed. The very nature of these peptides (e.g., fibrinopeptide A) indicated that limited clotting must have occurred in the plasma. Secondly, the Marshall et al. study [5] showed a virtual absence of peptides in fresh plasma. Subsequent time- dependent accumulation of selected species was also the result of ‘leaky’ clotting in conjunction with the same exoprotease activities that we have observed in plasma using externally added substrates. This isn’t surprising, as whatever small (<4kDa) peptides that may be generated in vivo in the bloodstream are rapidly cleared by the kidneys. Peptides resulting from ex vivo proteolysis therefore represent the overwhelming majority of the total pool, completely obscuring whatever little else may be present, if anything. Due to low abundance, the real cancer biomarkers are usually invisible to the mass spectrometers, both as proteins or as fragments thereof, but functionally measurable in the case of (exo)proteases when appropriate measures are taken to avoid bias, as already explained. These proteases are present in blood, plasma and serum; endogenous peptide read-out products are absent in vivo in blood, but present ex vivo at very low levels in plasma and at high levels in serum.
In conclusion, finding bona fide biomarkers by serum peptidomic analysis appears to be just as big a challenge and frustration as all recent MS sequence identification-based proteomics efforts. But at least, it now offers the option of a targeted, functional proteomic read-out that may be either a supplement or a practical alternative for the classical biomarker discovery techniques.
References
1.Villanueva, J., Shaffer, D.R., Philip, J., Chaparro, C.A., Erdjument- Bromage, H., Olshen, A.B., Fleisher, M., Lilja, H., Brogi, E., Boyd, J., et al. 2006. Differential exoprotease activities confer tumor-specific serum peptidome patterns. J. Clin. Invest. 116:271-284.
2.Villanueva, J., Philip, J., Entenberg, D., Chaparro, C.A., Tanwar, M.K., Holland, E.C., and Tempst, P. 2004. Serum peptide profiling by magnetic particle-assisted, automated sample processing and MALDI-TOF mass spectrometry. Anal. Chem. 76:1560-1570.
3.Villanueva, J., Philip, J., Chaparro, C.A., Li. Y., Toledo-Crow, R., DeNoyer, L., Fleisher, M., Robbins, R.J., and Tempst, P. 2005. Correcting common errors in identifying cancer-specific serum peptide signatures. J. Proteome Res. 4:1060-1072.
4.Koomen, J.M., Li, D., Xiao, L-C., Liu, T.C., Coombes, K.R., Abbruzzese, J., Kobayashi, R. 2005. Direct tandem mass spectrometry reveals limitations in protein profiling experiments for plasma biomarker discovery. J. Proteome Res. 4: 972-981.
5.Marshall, J., Kupchak, P., Zhu, W., Yantha, J., Vrees, T., Furesz, S., Jacks, K., Smith, C., Kireeva, I., Zhang, R., et al. 2003. Processing of serum proteins underlies the mass spectral fingerprinting of myocardial infarction. J. Proteome Res. 2: 361-372.

---

Letter to the Editor about Differential exoprotease activities confer tumor-specific serum peptidome

Submitter: Eleftherios P. Diamandis | ediamandis@mtsinai.on.ca

Department of Pathology and Laboratory Medicine, Mount Sinai Hospital Toronto, Canada

Published February 21, 2006

Villanueva et al. recently published in this journal, a provocative methodology which appears to be able to discriminate between normal subjects and patients with three types of solid tumors; prostate, breast and bladder cancer (1). An accompanying Commentary by Liotta and Petricoin (2) suggested that this is an important advancement in the field of cancer diagnostics. The original diagnostic methods based on mass spectrometric profiling of serum, proposed approximately four years ago by Liotta, Petricoin, and others (3), have been contested by this and other authors (4-11). Important methodological and bioinformatic artifacts and biases are responsible for the inability of others to reproduce the data, and the published method (3) has not as yet been validated. In short, after 4 years of debate, the authors should present convincing validation data.
The paper by Villanueva et al. has many technical similarities to the previously published paper (3) but it is based on a somewhat different hypothesis, i.e. that the informative diagnostic peptides, identified by mass spectrometry, originate after ex vivo proteolysis by tumor-specific proteases of high abundance protein fragments primarily generated by the coagulation and complement enzymatic cascades. These authors speculate that the putative tumor-associated exoproteases could generate proteomic patterns that can distinguish, not only normals from cancer patients, but also between various types of cancers. The working hypothesis is purely speculative and does not fit into what we currently know about cancer biomarker generation and dynamics in serum. This new approach, introduces for the first time an element of “magic” in the field of cancer diagnostics, in the sense that the informative biomarkers are not there when the blood is drawn, but are derived after a certain incubation (“cooking”) period. One way to confirm the validity of this method is through validation studies by independent laboratories. However, we question the validity of the findings now, based on what has been presented in the published paper.
It is surprising to us that some obvious and important study design problems have not been identified by the reviewers, the Editors or cancer biomarker specialists who co-authored the paper. Here are some points for consideration:
1. It is well-known that the problem of current cancer biomarkers is not their inability to diagnose late stage cancer, but their inability to diagnose efficiently early stage cancer. To this effect, Villanueva et al. chose to include only late-stage cancers in their study population. For example, the median prostate-specific antigen (PSA) levels for both the prostate cancer training set and the validation set were 66 and 133 µg/L, respectively. The real clinical problem, which was not addressed, is diagnosis of localized prostate cancer, in which case PSA levels are usually below 10 µg/L.
2. Another major difficulty with prostate cancer diagnosis is the discrimination between patients with prostate cancer and patients with benign prostatic hyperplasia (BPH) or prostatitis, who also have frequent elevations of PSA. No such group was included in this investigation, raising the question of whether their methodology could differentiate these groups of patients based on their serum peptide profiling. It is crucial to include patients with BPH or prostatitis to evaluate the specificity of their approach.
3. The control group used consisted of 19 females and 14 males whose median age was 31 years. The median age of all cancer patients used in this study was much higher. For example, for the prostate cancer group, it was 66 years for the training set and 67 years for the validation set. Furthermore, the prostate cancer sets were compared to a control group that consisted of more than 50% females. One would reasonably wonder, as we have indicated in the past (4, 5), if this is a test for age and gender discrimination, rather than a test for patients with or without cancer. A possible explanation for the above deficiencies is that this paper represents a “proof-of concept”, rather than definitive results. However, as outlined by Ransohoff (7), introduction of a serious bias in the design could invalidate all the findings of the paper, including the “proof-of- concept”. Consequently, only after removal of the bias, the concept can be further examined.
4. In standard textbooks on cancer biomarkers (12), the authors will find the attributes of an “ideal” tumor marker, including correlation to tumor burden (stage) and other clinicopathological variables, disappearance or decrease of the marker after tumor removal, relative insensitivity to various common variables such as diet, exercise, common drug ingestion and possible dependence on ethnicity, age, etc. None of these parameters have been addressed in this paper.
5. Since the method is heavily dependent on the ex vivo activity of the coagulation and complement cascades, one would predict that coagulation disturbances or infections will have an effect on the results, rendering the method non-specific.
6. Their hypothesis, no matter how provocative and original, is extremely unlikely to hold true because it does not fit with what we have learned about tumor marker generation, release and dynamics in serum over the last 100 years. Since the putative biomarkers are not there when the blood is drawn, but they appear ex-vivo after an incubation period, we predict that it will likely be a nightmare to control such ex vivo incubation in a real-life laboratory environment. We believe that the authors should have gone after the putative tumor-specific exoproteases as diagnostic molecules.
Earlier, Koomen et al. (13) performed an excellent study of peptides in serum and concluded that these are derived from a surprisingly low number of high abundance serum proteins and that plasma, not serum, would be the preferred sample for peptidomic analysis. Marshall et al. presented similar findings (14).
We conclude that this paper has serious methodological biases, as outlined above, which call all of the findings into serious question.
References
1. Villanueva, J., Shaffer, D.R., Philip, J., Chaparro, C.A., Erdjument- Bromage, H., Olshen, A.B., Fleisher, M., Lilja, H., Brogi, E., Boyd, J., et al. 2006. Differential exoprotease activities confer tumor-specific serum peptidome patterns. J. Clin. Invest. 116:271-284.
2. Liotta, L.A., and Petricoin, E.F. 2006. Serum peptidome for cancer detection: spinning biologic trash into diagnostic gold. J Clin. Invest. 116:25-30.
3. Petricoin III, E.F., Ardekani, A.M., Hitt, B.A., Levine, P.J., Fusaro, V.A., Steinberg, S.M., Mills, G.B., Simone, C., Fishman, D.A., Kohn, E.C., et al. 2002. Use of proteomic patterns in serum to identify ovarian cancer. Lancet. 359:572-577.
4. Diamandis, E.P. 2003. Proteomic patterns in biological fluids: Do they represent the future of cancer diagnostics? Clin. Chem. 49:1272- 1278.
5. Diamandis, E.P. 2004. Analysis of serum proteomic patterns for early cancer diagnosis: Drawing attention to potential problems. J. Natl. Cancer Inst. 96:353-356.
6. Diamandis, E.P., van der Merwe, D-E. 2005. Plasma protein profiling by mass spectrometry for cancer diagnosis: opportunities and limitations. Clin. Cancer Res. 11:963-965.
7. Ransohoff, D.F. 2005. Bias as a threat to the validity of cancer molecular-marker research. Nat. Rev. Cancer 5:152-149.
8. Baggerly, K.A., Morris, J.S., Edmonson, S.R., Coombes, K.R. 2005. Signal in noise: evaluating reported reproducibility of serum proteomic tests for ovarian cancer. J. Natl. Cancer Inst. 97:307-309.
9. Ransohoff, D.F. 2005. Lessons from controversy: ovarian cancer screening and serum proteomics. J. Natl. Cancer Inst. 97:315-319.
10. Banks, R.E., Stanley, A.J., Cairns, D.A., Barrett, J.H., Clarke, P., Thompson, D., Selby, P.J. 2005. Influences of blood sample processing on low-molecular-weight proteome identified by surface-enhanced laser desorption/ionization mass spectrometry. Clin. Chem. 51:1637-1649.
11. Karsan, A., Eigl, B.J., Flibotte, S., Gelmon, K., Switzer, P., Hassell, P., Harrison, D., Law, J., Hayes, M., Stillwell, M., et al. 2005. Analytical and preanalytical biases in serum proteomic pattern analysis for breast cancer diagnosis. Clin. Chem. 51: 1525-1528.
12. Diamandis, E.P. 2002. Tumour Markers: Past, Present, and Future. In: Tumor Markers: Physiology, Pathobiology, Technology and Clinical Applications. E.P. Diamandis, H.A. Fritsche, H. Lilja, D.W. Chan, M.K. Schwartz (Eds), pp 3-8, AACC Press.
13. Koomen, J.M., Li, D., Xiao, L-C., Liu, T.C., Coombes, K.R., Abbruzzese, J., Kobayashi, R. 2005. Direct tandem mass spectrometry reveals limitations in protein profiling experiments for plasma biomarker discovery. J. Proteome Res. 4: 972-981.
14. Marshall, J., Kupchak, P., Zhu, W., Yantha, J., Vrees, T., Furesz, S., Jacks, K., Smith, C., Kireeva, I., Zhang, R., et al. 2003. Processing of serum proteins underlies the mass spectral fingerprinting of myocardial infarction. J. Proteome Res. 2: 361-372.