Important BC article....NY Times

voraciousreader · July 2011

Add Patience to a Leap of Faith to Discover Cancer Signatures

Are we outsmarting cancer? Or just ourselves?

Over the past several years, scientists have begun looking not just for individual genes linked to cancer, but for collections of genes and molecules, like proteins, that form telltale patterns, or signatures, that can be used to identify a cancer cell and reveal what drugs might kill it.

Signatures can be used to diagnose the disease, scientists hope, and to give a prognosis to patients who have cancer. But there have been few successes in this brave new world of cancer research, and some notable failures.

Genetic tests devised at Duke University by researchers using the new methodology have turned out to be worthless, though they were once hailed as breakthroughs. Two new blood tests for ovarian cancer have also been abandoned.

Despite the setbacks, researchers say they cannot give up on their quest for cancer signatures. Dr. Lajos Pusztai, a breast cancer researcher at the University of Texas MD Anderson Cancer Center, is one of them.

How many new drugs, he asks, were approved for breast cancer treatment in the past decade? His answer: seven. None was much different from drugs already on the market.

Yet in the same decade, he said, there were 8,000 publications in medical and scientific journals on breast cancer and more than 3,000 clinical trials at a cost of over $1 billion. "What came out of this is seven ‘me too' drugs," Dr. Pusztai said.

Yes, there have been studies showing single genes can go awry and fuel certain cancers, he and other scientists say. And yes, those studies have led to new drugs, so-called targeted therapies, that block the genes, extending the lives of some patients with some types of cancer.

But for a major advance in the way cancer is diagnosed and treated, Dr. Pusztai and other researchers believe that work must continue on genomic signatures. It's a very different sort of science, an elaborate enterprise that involves complicated analyses of patterns of genes or other components of cells.

The hunt for cancer signatures also is a type of work that requires a leap of faith. It is impossible for scientists to use their intuition to know whether a signature has any biological meaning - it is just a pattern, and the meaning comes from its statistical association with a result.

By looking for these signatures, scientists are looking for a sort of next generation of biomarkers, and biomarkers have a troubled history in cancer research.

About 15 years ago, when the world was simpler, the American Society for Clinical Oncology called together a group of experts and asked them to develop some guidelines. Which cancer biomarkers are useful for making clinical decisions?

Dr. Daniel Hayes, a breast cancer researcher at the University of Michigan, was part of the group.

"We all kind of sat around and looked at each other and said, ‘We have no idea how to do this,' " Dr. Hayes recalled. The field of tumor biomarkers, he said, "had been so chaotic." All too often, researchers claimed to have discovered reliable ways to identify a particular cancer, but studies confirming they were valid had never been done.

The group ended up writing a paper with what they called a "level of evidence scale" outlining the results they needed before they would believe a claim. Only a handful of tumor biomarkers met their criteria.

Yet, Dr. Hayes said, "during that time there had been hundreds of putative markers reported for breast cancer alone, let alone other diseases."

With genomic signatures, the situation is more involved than it was 15 years ago, but the many of the same problems remain, Dr. Hayes and others say.

Signatures can be used to identify cancer cells in the way a tartan can identify a Scottish clan, Dr. Hayes said. "Each tartan is composed of threads of several different colors, but when woven into a single cloth, presents a distinctive pattern or signature that distinguishes one clan from another," he said.

Yet the tools used to find signatures are so complex they can be misleading if the results are not tested properly. Investigators look for patterns in huge arrays of genes or proteins or RNA molecules, and they constantly find spurious associations with cancer that look for all the world like true ones.

"The question is, what is noise and what is real?" said Dr. Steven Goodman, a biostatistician at Johns Hopkins University. In these studies, he added, "you are guaranteed to find things," and almost always what is found is nonsense.

Gene or protein patterns, said Donald Berry, a statistician at MD Anderson Cancer Center, "are very difficult to get right." Finding them, he said, "is like looking for a needle in a haystack when you can't tell the needle from the hay."

Adding to the confusion is that the research requires a group of experts, each of whom has a different, highly technical skill. Each person on a team may be so specialized that no one is qualified to know exactly what his or her colleagues are doing.

Dr. Pusztai's team, for instance, includes pathologists, molecular biologists and biostatisticians. "No one person on the team sees all the pieces together," he said.

For example, he said, just analyzing cancer tissue for a genomics study involves 200 to 3,000 steps and takes several days to complete. "Any one of these steps can go wrong, and a good researcher should know what can influence the success of each step and control for the quality," he said.

What comes out of this analysis is "reams of numbers," Dr. Pusztai said. "If one were to print these out it would amount to thousands of pages. The interpretation of these numbers is purely statistical and mathematical."

Even when researchers find a real association, it may not be clinically useful. A genomic signature test that correctly identifies most tumors that will respond to a drug but incorrectly identifies others may not be of net benefit to patients.

What is needed, and what rarely has been done, is research to see if a test based on a new biomarker does more good than harm.

It's expensive and time-consuming, but it is the sort of evidence-gathering that is almost always done to see if a new drug is effective, Dr. Hayes noted. Yet there is little incentive to evaluate biomarkers, because the Food and Drug Administration does not require it and companies are not reimbursed as much for tests based on biomarkers as they are for new cancer drugs.

"There is a whole series of things in this cycle that need to be broken," Dr. Hayes said.

One step that could make a big difference, scientists said, would be for researchers who think they have found a new genomic signature for cancer to publish enough of their data and analysis for others to verify their work. Surprisingly few have done so.

The only reason the Duke research was discovered to be flawed, in fact, was that it relied on publicly available data sets and algorithms. Even so, unraveling the details of the work was so complicated that it took Keith Baggerly and Kevin Coombes, two statisticians at MD Anderson, 2,000 hours to find all the errors.

Perhaps illustrating the perils of the needle-in-a-haystack approach, the only two genomic signatures for cancer that have been validated, used in the tests Oncotype DX and MammaPrint, were discovered in a very different ways.

In both cases, researchers started with hundreds of genes that they had some reason to believe were important. They winnowed the collection down to those that seemed to be clinically useful. The Oncotype DX assay relies on 21 genes, and MammaPrint on 70. Both companies then rigorously tested their signatures to be certain that they were accurate in women with breast cancer. Oncotype DX showed its signature could predict prognosis and whether women would benefit from chemotherapy.

Yet the two signatures used to make these tests have just one gene in common.

"What it means, as I suppose everybody is beginning to know, is that cancer is a very complicated thing," Dr. Berry said.

voraciousreader · July 2011

Last week, I quoted from Ian Brown's book, The Boy in the Moon, about his son with a catastrophic genetic illness. In the book he quotes researchers who study genetics. He mentions that the easy part of studying genetics was mapping out the genes. The hard part is understanding the mechanics of how the genes actually cause illness and ultimately lead to treatments.....

Gina Kolata's article echos that sentiment.

AnacortesGirl · July 2011

An excellent article describing such a complicated process. The Scottish kilt is a great analogy! It also feels good to know that there are researching out there doing this work and getting funding because "They want to know!" and not just throwing out a drug because it might work and therefore bring in a bunch of money for a pharma. (Note: I have no particular rant against the pharmaceuticals. It just that there is nothing altruistic about their work. Investment (i.e. research) is done for ROI and not just because they want the answer(s).

voraciousreader · July 2011

AnacortesGirl....Check out the book, The Boy in the Moon and you will see how humbling it is to work in the field of genetics.......

gpawelski · July 2011

Predictive accuracy is the only data existing to validate the Oncotype DX test, which wasn't a prospective study and certainly wasn't a "real world" study. The Oncotype DX test has been independently validated by the original laboratory group which published the results.

The other molecular-targeted breast prognostic test Mammostrat is validated with the usual, retrospective, non-randomized study using archival tissues and uniform batch processing and slide interpretation. It utilizes five immunohistochemical (IHC) biomarkers to classify patients into high, moderate, or low-risk categories for disease recurrence.

No one is seriously proposing that any of the molecular tests now available (Oncotype DX, EGFR amplification/mutation) should have to be proven efficacious, as opposed to merely accurate, before they are used in clinical decisions regarding treatment selection.

These new gene expression profiling tests enable the oncologist and breast cancer surgeon to more accurately determine who should be treated and who should not be treated with chemotherapy, but they cannot predict chemo response (clinical responders).

The OncotypeDX test is somewhat problematic. I think that work like this is tremendously important, but it is an example of herd mentality thinking, which almost always causes problems. There's been a stampede to endorse it, and many laudatory comments were made about it at the Ovarian Cancer State of the Science meeting in Bethesda in 2005.

I did get to read all the supplemental materials with the New England Journal of Medicine (NEJM) article from a medical oncologist friend of mine. All of the assays were completed in a two week period on specimens archived from the 1980s. So all were done by the same team within a short time. The same pathologist did the micro-dissection of the paraffin slides. It was all retrospective and about as non-real world as you can get.

In the real world, specimens are accessioned in real time over days, weeks, months, and years. The people working on the different days are different. The assay is very complicated (the CEO even made a big point of how complicated it was, to justify the pricing). It involves a whole lot of steps and a whole lot of micropipetting. As far as for quality control, they stated that only two specimens had been tested twice (again, within a very narrow time frame) to affirm reproducibility.

Then, when they applied the same test at a later time to a slightly different patient population (at MD Anderson, rather than at the National Surgical Adjuvant Breast and Bowel Project), the correlations were not significant. And the only thing the test was useful for was identifying a small group of patients who ostensibly don't need tamoxifen and/or anastrozole therapy.

The challenge is to identify which patients the targeted treatment will be most effective. Tumors can become resistant to a targeted treatment, or the drug no longer works, even if it has previously been effective in shrinking a tumor. Drugs are combined with existing ones to target the tumor more effectively. Most cancers cannot be effectively treated with targeted drugs alone.

What is needed is to measure the net effect of all processes within the cancer, acting with and against each other in real time, and test living cells actually exposed to drugs and drug combinations of interest. The key to understanding the genome is understanding how cells work. How is the cell being killed regardless of the mechanism?

The core understanding is the cell, composed of hundreds of complex molecules that regulate the pathways necessary for vital cellular functions. If a targeted drug could perturb any of these pathways, it is important to examine the effects of drug combinations within the context of the cell. Both genomics and proeomics can identify potential therapeutic targets, but these targets require the determination of cellular endpoints.

It's not just Potti, and it's not just microarrays. The whole concept of using molecular "signatures" of any kind to do anything beyond the most straightforward of cases (i.e. single gene mutations, etc.) is so flawed that everyone should have seen the problems at the beginning.

The reason that no one seemingly sees it now can be explained by the facts that the technology itself is so elegant and beautiful. But a beautiful biological technology is no different than a beautiful computer technology - it's not worth much without some very good applications ("apps"), and personalized molecular medicine is still waiting for its first killer app.

Until such time as cancer patients are selected for therapies predicated upon their own unique biology (and not population studies), we will confront one targeted drug after another.

The solution to this problem has been to investigate the targeting agents in each individual patient's tissue culture, alone and in combination with other drugs, to gauge the likelihood that the targeting will favorably influence each patient's outcome.

apple · July 2011

interesting article.. thanks.

It is not up to us to judge who is and who is not altruistic.. some people live their lives with that intent. research projects are often the same way. (not too elegantly stated but i have brain compromise).

AnacortesGirl · July 2011

Mary,

You're right that I should not judge who is altruistic and who is not. Whether it's a large corp or small company, it is the people who work there that comprise the entity and it's the sum of their intentions that make up the larger entity. Unfortunately, too many CEOs and other top management have very inflated egos that make them believe it's their leadership that makes good things happen when it's the people that are actually performing the core work. They are the facilitators, not the workers.

You were right to point this out that my brush was much too broad. The particular corp that I'm working has totally turned it's back on it's people and many of it's tenets for the sake of return on investment (ROI) and so I'm applying my current experience too broadly.

But I do not want to take this thread off topic! If anyone feels the need to discuss my above comments further then let's take it to a new thread. Believe me, I'm just scratching the surface with how I feel after 25 years......

voraciousreader,

Thank you for the book recommendation! I know have 3 that I need to get: The Emperor of all Maladies, The Boy in the Moon and Born to Run.

The last one isn't cancer related. A friend of mine just finished the BadWater race. It was his first year to enter and he came in 15th. It starts in Death Valley and cover 135 miles, non-stop to Mount Whitney. Maybe this book will give me some insight into this guy!

gpawelski,

Your post was very interesting. I'm not a chemist, molecular biologist or statistician but I definitely appreciate the important of correlating and reproducible data. I never delved too far into Oncotype or MammoPrint since my need for chemo was a moot point at Stage III. But I did wonder (just never bothered to ask my onc), if these tests could prove a benefit to chemo then doesn't that imply the ability to indicate the responsiveness to a particular chemo? Your post clarifies why the answer would have been "no".

As a cancer survivor, I choose to view my BRCA 2 mutation as beneficial. It provides somewhat of a grounding point for researchers. And as a result, more studies. It's also clearly demonstrated the ability to become resistant to carbo. Since I"ve already had some carbo I've wondered if, should any cancer be found in the future, it will have already made that adaptation. But research continues on the parp inhibitors - more hope for the future. If they can develop targeted therapies for single gene mutations then maybe the jigsaw puzzle will start coming together.

I found it interesting that you used a computer application analogy. Now we're talking my field. I feel fortunate that I chose a field where destructive testing is often an option since it can provide reproducibility.

If we go back 35 years (yes, I am actually familiar with the first micros, minis and mainframes from back then) the technology was considered successful if you could develop a single process app. For instance an app that allowed you to schedule an airplane flight. Or perform the accounting process for entering transaction, close interim periods, close the fiscal year and produce the pertinent reports to visually review the data. But even those apps had multiple functions, the processes they fulfilled were very limited. Chances are you needed a separate application to perform capital asset management, project management, tax preparation. And that booking was only on a specific airline and it required a booking agent -- it was not self-service.

Then we moved into the era of one system to do everything. Integrated applications versus "best of breed". SAP is the most notable. Do your accounting, track your inventory, track your maintenance work and whether you have the spare parts in your inventory, maintain your employees information including which department they work for and whether they can look at the accounting information based on their reporting department, etc. Do everything but at the price of doing no one thing well. Best of breed software was comprehensive for a single large process such as maintenance work or project work.

But with android and iphone apps we've moved into a new generation. Hundreds of apps, but now the app (the process) may only be defined by a handful of functional steps. Not unlike 30 years ago when WordStar or WordPerfect did nothing more than create a document. But different because those documents could have complexity, be retained, printed, or modified. Now you have an app on a phone that does nothing other than send a text message to someone else. Or an app that gives you a different keyboard. Or tells you your GPS coordinates.

The most profound result that I have seen from this evolution is how it has changed us as humans. Not just the socialization, which is what the news media focuses on. But I believe it has profoundly changed people in how they think and analyze. Thirty years ago there were those people who could adapt their thinking to work with computers and those that couldn't. Ten to twenty years ago the workers in their prime embraced the computer apps and actually expected more information and feedback than the apps were capable of providing. Today, it seems the younger generation thinking process is getting disjointed (I don't mean that in a negative way) in the sense that apps have been pulled apart into handful of functions rather than one fully integrated app that does it all. Yet the individual apps flow together on the device an can result in a very smooth and effective tool. Not at all the same effect as those first computers.

So how is all the rambling that I've done relate to cancer? It's the thought process. It's the need to continually step back and attack. Not only using different methods but sometimes radically different approaches. I'm not trying to predict the future but I do wonder if the same evolution in people's response to digital applications needs to happen in the medical world. Maybe some of the fundamental requirements for reproducibility, correlations, statistical significance, whatever, needs to be looked at completely differently.

voraciousreader · July 2011

Off topic, but still on topic are two films to watch regarding medical genetics. Lorenzo's Oil and the Harrison Ford film, Extraordinary Measures. Both films serve as a primer to understand how complicated the world of genetics really is.

On another note, I have a rare breast cancer. While the Oncotypedx test hasn't been validated as well for mucinous bc as it has for more traditional breast cancers, I have questioned the test and wondered whether or not there were other genes that the Oncotypedx test doesn't test for that offers protection against cancer. Similar to the HDL cholesterol number....

I really think we have a very long way to go in understanding the genetics of breast cancer. Likewise , I think our understanding of cholesterol numbers as they relate to heart disease is way too simplistic as well. Just MHO.

Important BC article....NY Times

Comments

Categories