Skip Navigation


NDT Advance Access originally published online on October 31, 2007
Nephrology Dialysis Transplantation 2008 23(1):52-55; doi:10.1093/ndt/gfm712
This Article
Right arrow Extract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
23/1/52    most recent
gfm712v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (1)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Garg, A. X.
Right arrow Articles by Levin, N. W.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Garg, A. X.
Right arrow Articles by Levin, N. W.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author [2007]. Published by Oxford University Press on behalf of ERA-EDTA. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.org


A well-conducted randomized trial that establishes no benefit of therapy is an important medical advance

Amit X. Garg1, Tom Greene and Nathan W. Levin2

1Associate Professor Medicine and Epidemiology, University of Western Ontario, London, Ontario, Canada and 2Medical and Research Director, Renal Research Institute, 207 East 94th Street, Suite 303, New York, NY 10128

Correspondence and offprint requests to: Nathan W. Levin, Room ELL-101 Kidney Clinical Research Unit London Health Sciences Centre 800 Commissioners Road East London, ON N6A 4G5 Tel: 519.685.8502; Fax: 519.685.8269; E-mail: nlevin{at}rriny.com

Keywords: randomized controlled trials; kidney diseases; dialysis; epidemiologic methods



   Background
Top
 Background
Assumptions for the argument
 Negative RCTs do influence...
 Negative RCTs do influence...
 Conclusion
 References
 
As physicians, we continually strive to understand disease processes and predict how they will affect our patients. However, our greatest skill lies in our ability to intervene and change the natural history of a disease, i.e. change a poor outcome which otherwise would have occurred. Many types of interventions are used in the care of renal patients, from pills, to procedures and dialysis, to alternative ways to deliver health care. All would agree that interventions need to be evaluated to determine if they are beneficial, without harm, and cost-effective in a system of finite resources.

A randomized, controlled clinical trial (RCT) is an experimental method used to evaluate the effectiveness of an intervention. RCTs are conducted when an intervention shows the potential for health care improvement, but there is collective uncertainty as to the true benefits of the intervention [1]. This uncertainty is in fact essential to the clinical trial paradigm, as otherwise clinical equipoise would be violated, and in most cases it would not be ethical to randomly assign patients between the intervention to be tested and a control intervention. This is true both when the control treatment is a placebo and when the control entails administration of an active treatment reflecting standard therapy [2]. A logical consequence of the uncertainty required to maintain equipoise is that a substantial proportion, in fact the majority, of well-conducted RCTs must be negative.

When an intervention is found to be efficacious in a well-conducted RCT (referred to here as a positive trial), the therapy is often rapidly adopted into standard medical care [3–5]. While positive trials still need subsequent real-world evaluations for unanticipated deleterious effects [6, 7], in general a positive trial is viewed as an important medical advance for the betterment of patient health. Examples of recent positive trials in nephrology include mycophenolate for active lupus nephritis [8], angiotensin-converting enzyme inhibitors in advanced chronic kidney disease [9], combination ACE inhibitor and angiotensin-receptor blocker therapy in non-diabetic renal disease [10], education for patients with chronic kidney disease to choose self-care dialysis [11], polysporin triple antibiotic ointment for haemodialysis central venous catheter insertion sites [12] and hydration with sodium bicarbonate plus N-acetylcysteine in patients undergoing emergency percutaneous coronary intervention [13].

A RCT can also establish the absence of a clinically important benefit of an intervention on a primary outcome, a result that in this context is referred to as a ‘negative’ trial. Examples of recent negative trials in nephrology include atorvastatin for diabetic patients receiving haemodialysis [14], intravenous dopamine in critically ill patients [15], high dialysis dose and membrane flux in maintenance haemodialysis [16], angioplasty for renal artery stenosis [17], and plasma exchange when myeloma presents as acute renal failure [18]. Aside from the issue of equipoise, the occurrence of such negative findings is essential to the conduct of evidence-based medicine; the absence of negative trials would indicate a failure of the RCT paradigm to refute the clinical hypotheses evaluated in RCTs when these hypotheses are in fact false. This capacity of experimental evidence to refute untrue hypotheses is fundamental to the advancement of science [19]. Given the reality of finite resources for medical treatment, the identification of non-efficacious therapies is also essential to allow the available resources to be distributed to those therapies that are demonstrated to be efficacious. Further, it is our position that negative RCTs, when they occur, often represent important medical advances in their own right for the care of our patients. We highlight examples where negative clinical trials have influenced theory and practice.



   Assumptions for the argument
Top
 Background
 Assumptions for the argument
Negative RCTs do influence...
 Negative RCTs do influence...
 Conclusion
 References
 
We take our position under the following assumptions, which apply irrespective of whether the trial turns out to be negative or positive:

(1) It is ethical and practical to study the intervention in a RCT. For example, technologies in some areas of medicine are advancing so quickly that the intervention studied in a RCT is no longer relevant by the time the RCT is reported.

(2) The RCT is well conducted using methods to minimize bias (e.g. concealment of allocation, adequate generation of the allocation sequence, double blinding and a minimal number of patients lost to follow-up [20–22]). The trial also has adequate statistical power to rule out a minimal clinically important benefit of the intervention, if in truth this benefit exists [23].

(3) The outcome studied in the trial is clinically important, avoiding a separate debate on the potential for misleading results with surrogate measures [24].

(4) The intervention is applied appropriately in the RCT, and patients are followed for a necessary period to allow a ‘biological’ effect of the intervention to be exerted [25].

(5) RCTs represent a rigorous method to convince physicians, the public and government regulatory agencies as to the benefits (or lack thereof) of an intervention. For this reason they are accepted as one of the highest levels of evidence to evaluate treatment effects [26,27]. Despite the intensive resources needed to conduct RCTs, there is an agreement that some need to be conducted.

(6) The debate excludes RCTs designed to prove non-inferiority, which have a different framework for the definitions of ‘positive’ and ‘negative’ in the current discussion.



   Negative RCTs do influence theory and future research
Top
 Background
 Assumptions for the argument
 Negative RCTs do influence...
Negative RCTs do influence...
 Conclusion
 References
 
Well-conducted negative RCTs can result in a paradigm shift and a revolution in scientific thinking. To illustrate the point, take the scenario that major observational studies suggested that the high mortality in ESRD patients on haemodialysis might be improved by increasing the delivered dose of dialysis [28]. This theory was tested in a large RCT [16]. Patients on conventional, three times weekly haemodialysis were randomized to receive usual dose (equilibrated Kt/V of 1.05 per session) versus high dose (equilibrated Kt/V of 1.45 per session). These increased doses were delivered primarily by increasing dialysis session time. The RCT was negative; there were no significant differences in either the primary analysis or in four pre-specified secondary analyses between the two groups with respect to overall mortality, hospitalizations or serum albumin-based end-points. This RCT prompted a re-evaluation of existing theories for the kinetics of small and middle molecule solute removal on haemodialysis [29]. Urea and other small toxic solutes follow first-order kinetics, with the rate of solute removal proportional to the concentration of solute [30]. Consequently, most solute removal occurs at the start of haemodialysis, with decreasing removal rates as the haemodialysis session proceeds [31]. During the last hour of a 4.5-h haemodialysis session, very little solute is removed in comparison to the first 3 h. Thus, increasing dose by increasing dialysis session time on conventional haemodialysis results in very minimal increments in total urea and small toxic solute removal. Similarly, the relatively short increases in time on conventional haemodialysis do not result in substantial increases in removal of toxic middle molecules [32], such as β2-microglobulin, implicated in dialysis amyloidosis [33], nor in phosphate, implicated in cardiovascular risk and death [34]. Finally, increasing haemodialysis session time within the setting of conventional, three times per week dialysis does not ameliorate the problem of chronic extracellular fluid accumulation, a major contributing factor in the development of hypertension and cardiovascular risk [35].

The negative results of the HEMO trial redirected attention from modifications to therapy within the constraints of conventional, three times per week dialysis to alternative treatment schedules that may be better suited to address these physiological considerations. For example, dialysis performed daily for 2 h, six times per week, would be predicted to be an improvement over conventional haemodialysis being performed for 4 h, three times per week, despite total weekly dialysis time remaining constant. Given that one is dialyzing against the steepest portion of the urea concentration curve each day, daily haemodialysis would be predicted to have higher efficiency than conventional haemodialysis, resulting in greater weekly small solute, middle molecule and phosphate removal [32,36]. Preliminary studies have highlighted the potential merits of daily haemodialysis [37], and the intervention is now being tested in a larger RCT [38].

In addition to redirecting the focus of research from hypotheses that are falsified by their primary analyses, secondary analyses of data collected in negative trials have also often had a major scientific impact. For example, while the primary analyses of the Modification of Diet in Renal Disease Study failed to demonstrate a benefit of protein restriction for CKD patients, secondary results from this study led to the development of widely used methods for quantification of renal function [39] and have contributed to the characterization of the course of progression of kidney disease and it consequences [40]. Secondary analyses from the dialysis study noted above have similarly led to advances in the quantification of dialysis dose [41] and the understanding of potential biases in observational studies [42]. Results from this study in subgroups such as female gender [43] have contributed to the ongoing debate on the question of whether optimum dialysis dose is characterized by utilization of a constant Kt/V in all patients.



   Negative RCTs do influence practice
Top
 Background
 Assumptions for the argument
 Negative RCTs do influence...
 Negative RCTs do influence...
Conclusion
 References
 
Many interventions are used in patient care even prior to being studied in a RCT. There is ample evidence that the results of completed RCTs influence the subsequent use of the therapy, with negative studies followed by negative shifts in use. Take for example the use of prolonged plasma exchange for multiple sclerosis, a technical procedure not unlike dialysis, with a comparable risk profile [44,45]. A small prospective study indicated a superior outcome in patients who received immunosuppression and plasma exchange compared to those who received immunosuppression alone [46]. Use of plasma exchange for multiple sclerosis increased the year after this report was published. Subsequently a larger RCT was conducted, which included a placebo group and used end-points based on blinded observation [47]. The trial showed no benefit for plasma exchange, which resulted in a decrease in activity that persisted over the subsequent decade. Although to our knowledge no empirical evaluation has been reported, based on our clinical experience similar negative shifts in use also occurred after important negative RCTs in renal medicine: angioplasty for renal artery stenosis [17], protein restriction in chronic kidney disease [48], plasma exchange for myeloma kidney [18], normalization of haemoglobin in patients receiving haemodialysis [49] and deliberate attempts to target a higher dialysis dose (equilibrated Kt/V of 1.45 per session) amongst haemodialysis patients [16]. The negative RCTs also influence recommendations in subsequent clinical practice guidelines. Negative shift in therapy use may be even more dramatic, if the trial demonstrates evidence of harm in addition to lack of benefit [50].



   Conclusion
Top
 Background
 Assumptions for the argument
 Negative RCTs do influence...
 Negative RCTs do influence...
 Conclusion
References
 
The need for careful scientific evaluation of new interventions is undeniable. As with any type of research, RCTs have their advantages and disadvantages, and can be subject to abuse. However, when a RCT is conducted with due diligence, the result, whether positive or negative, provides an important opportunity to advance knowledge and improve patient care.

Conflicts of interest statement. None declared.



   References
Top
 Background
 Assumptions for the argument
 Negative RCTs do influence...
 Negative RCTs do influence...
 Conclusion
 References
 

  1. Weijer C, Shapiro SH, Cranley Glass K. For and against: clinical equipoise and not the uncertainty principle is the moral underpinning of the randomised controlled trial. BMJ (2000) 321:756–758.[Free Full Text]
  2. Chow S, Liu J. Section 3.4: Selection of Controls. In: Design and Analysis of Clinical Trials: Concepts and Methodologies. (2004) New York: Wiley. 100–109.
  3. Mamdani MM, Tu JV. Did the major clinical trials of statins affect prescribing behaviour? CMAJ (2001) 164:1695–1696.[Free Full Text]
  4. Tu JV, Hannan EL, Anderson GM, et al. The fall and rise of carotid endarterectomy in the United States and Canada. N Engl J Med (1998) 339:1441–1447.[Abstract/Free Full Text]
  5. Tu K, Mamdani MM, Jacka RM, et al. The striking effect of the Heart Outcomes Prevention Evaluation (HOPE) on ramipril prescribing in Ontario. CMAJ (2003) 168:553–557.[Abstract/Free Full Text]
  6. Juurlink DN, Mamdani MM, Lee DS, et al. Rates of hyperkalemia after publication of the Randomized Aldactone Evaluation Study. N Engl J Med (2004) 351:543–551.[Abstract/Free Full Text]
  7. Pitt B, Zannad F, Remme WJ, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. Randomized Aldactone Evaluation Study Investigators. N Engl J Med (1999) 341:709–717.[Abstract/Free Full Text]
  8. Ginzler EM, Dooley MA, Aranow C, et al. Mycophenolate mofetil or intravenous cyclophosphamide for lupus nephritis. N Engl J Med (2005) 353:2219–2228.[Abstract/Free Full Text]
  9. Hou FF, Zhang X, Zhang GH, et al. Efficacy and safety of benazepril for advanced chronic renal insufficiency. N Engl J Med (2006) 354:131–140.[Abstract/Free Full Text]
  10. Nakao N, Yoshimura A, Morita H, et al. Combination treatment of angiotensin-II receptor blocker and angiotensin-converting-enzyme inhibitor in non-diabetic renal disease (COOPERATE): a randomised controlled trial. Lancet (2003) 361:117–124.[CrossRef][Web of Science][Medline]
  11. Manns BJ, Taub K, Vanderstraeten C, et al. The impact of education on chronic kidney disease patients’ plans to initiate dialysis with self-care dialysis: a randomized trial. Kidney Int (2005) 68:1777–1783.[CrossRef][Web of Science][Medline]
  12. Lok CE, Stanley KE, Hux JE, et al. Hemodialysis infection prevention with polysporin ointment. J Am Soc Nephrol (2003) 14:169–179.[Abstract/Free Full Text]
  13. Recio-Mayoral A, Chaparro M, Prado B, et al. The reno-protective effect of hydration with sodium bicarbonate plus N-acetylcysteine in patients undergoing emergency percutaneous coronary intervention: the RENO Study. J Am Coll Cardiol (2007) 49:1283–1288.[Abstract/Free Full Text]
  14. Wanner C, Krane V, Marz W, et al. Atorvastatin in patients with type 2 diabetes mellitus undergoing hemodialysis. N Engl J Med (2005) 353:238–248.[Abstract/Free Full Text]
  15. Bellomo R, Chapman M, Finfer S, et al. Low-dose dopamine in patients with early renal dysfunction: a placebo-controlled randomised trial Australian and New Zealand Intensive Care Society (ANZICS) Clinical Trials Group. Lancet (2000) 356:2139–2143.[CrossRef][Web of Science][Medline]
  16. Eknoyan G, Beck GJ, Cheung AK, et al. Effect of dialysis dose and membrane flux in maintenance hemodialysis. N Engl J Med (2002) 347:2010–2019.[Abstract/Free Full Text]
  17. van Jaarsveld BC, Krijnen P, Pieterman H, et al. Man in ‘t The effect of balloon angioplasty on hypertension in atherosclerotic renal-artery stenosis. Dutch Renal Artery Stenosis Intervention Cooperative Study Group. N Engl J Med (2000) 342:1007–1014.[Abstract/Free Full Text]
  18. Clark WF, Stewart AK, Rock GA, et al. Plasma exchange when myeloma presents as acute renal failure: a randomized, controlled trial. Ann Intern Med (2005) 143:777–784.[Abstract/Free Full Text]
  19. Popper K. Logic of Scientific Discovery. (1959) New York: Routledge.
  20. Juni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ (2001) 323:42–46.[Free Full Text]
  21. Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet (1998) 352:609–613.[CrossRef][Web of Science][Medline]
  22. Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA (1995) 273:408–412.[Abstract/Free Full Text]
  23. Scales DC, Rubenfeld GD. Estimating sample size in critical care clinical trials. J Crit Care (2005) 20:6–11.[CrossRef][Web of Science][Medline]
  24. Manns B, Owen WF Jr, Winkelmayer WC, et al. Surrogate markers in clinical studies: problems solved or created? Am J Kidney Dis (2006) 48:159–166.[CrossRef][Web of Science][Medline]
  25. Levey AS, Greene T, Sarnak MJ, et al. Effect of dietary protein restriction on the progression of kidney disease: long-term follow-up of the Modification of Diet in Renal Disease (MDRD) Study. Am J Kidney Dis (2006) 48:879–888.[CrossRef][Web of Science][Medline]
  26. Guyatt G, Schunemann HJ, Cook D, et al. Applying the grades of recommendation for antithrombotic and thrombolytic therapy: the Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy. In: Chest (2004) 126:179S–187S.[CrossRef][Web of Science][Medline]
  27. Harbour R, Miller J. A new system for grading recommendations in evidence based guidelines. BMJ (2001) 323:334–336.[Free Full Text]
  28. Saran R, Canaud BJ, Depner TA, et al. Dose of dialysis: key lessons from major observational studies and clinical trials. Am J Kidney Dis (2004) 44:47–53.[CrossRef][Medline]
  29. Depner TA, Bhat A. Quantifying daily hemodialysis. Semin Dial (2004) 17:79–84.[CrossRef][Web of Science][Medline]
  30. Depner TA. Benefits of more frequent dialysis: lower TAC at the same Kt/V. Nephrol Dial Transplant (1998) 13(Suppl 6):20–24.[Free Full Text]
  31. Depner TA. Assessing adequacy of hemodialysis: urea modeling. Kidney Int (1994) 45:1522–1535.[Web of Science][Medline]
  32. Pierratos A. Effect of therapy time and frequency on effective solute removal. Semin Dial (2001) 14:284–288.[CrossRef][Web of Science][Medline]
  33. Floege J, Ketteler M. beta2-microglobulin-derived amyloidosis: an update. Kidney Int Suppl (2001) 78:S164–S171.[Medline]
  34. Block GA, Klassen PS, Lazarus JM, et al. Mineral metabolism, mortality, and morbidity in maintenance hemodialysis. J Am Soc Nephrol (2004) 15:2208–2218.[Abstract/Free Full Text]
  35. Jaeger JQ, Mehta RL. Assessment of dry weight in hemodialysis: an overview. J Am Soc Nephrol (1999) 10:392–403.[Abstract/Free Full Text]
  36. Depner TA. Daily hemodialysis efficiency: an analysis of solute kinetics. Adv Ren Replace Ther (2001) 8:227–235.[CrossRef][Web of Science][Medline]
  37. Suri RS, Nesrallah GE, Mainra R, et al. Daily Hemodialysis: a systematic review. Clin J Am Soc Nephrol (2006) 1:33–42.[Abstract/Free Full Text]
  38. Suri RS, Garg AX, Chertow GM, et al. Frequent hemodialysis network (FHN) randomized trials: study design. Kidney Int (2007) 71:349–359.[CrossRef][Web of Science][Medline]
  39. Levey AS, Bosch JP, Lewis JB, et al. A more accurate method to estimate glomerular filtration rate from serum creatinine: a new prediction equation. Modification of Diet in Renal Disease Study Group. Ann Intern Med (1999) 130:461–470.[Abstract/Free Full Text]
  40. Hunsicker LG, Adler S, Caggiula A, et al. Predictors of the progression of renal disease in the Modification of Diet in Renal Disease Study. Kidney Int (1997) 51:1908–1919.[Web of Science][Medline]
  41. Daugirdas JT, Greene T, Depner TA, et al. Relationship between apparent (single-pool) and true (double-pool) urea distribution volume. Kidney Int (1999) 56:1928–1933.[CrossRef][Web of Science][Medline]
  42. Greene T, Daugirdas J, Depner T, et al. Association of achieved dialysis dose with mortality in the hemodialysis study: an example of "dose-targeting bias". J Am Soc Nephrol (2005) 16:3371–3380.[Abstract/Free Full Text]
  43. Depner T, Daugirdas J, Greene T, et al. Dialysis dose and the effect of gender and body size on outcome in the HEMO Study. Kidney Int (2004) 65:1386–1394.[CrossRef][Web of Science][Medline]
  44. Clark WF, Garg AX, Blake PG, et al. Effect of awareness of a randomized controlled trial on use of experimental therapy. JAMA (2003) 290:1351–1355.[Abstract/Free Full Text]
  45. Clark WF, Rock GA, Buskard N, et al. Therapeutic plasma exchange: an update from the Canadian Apheresis Group. Ann Intern Med (1999) 131:453–462.[Abstract/Free Full Text]
  46. Khatri BO, McQuillen MP, Harrington GJ, et al. Chronic progressive multiple sclerosis: double-blind controlled study of plasmapheresis in patients taking immunosuppressive drugs. Neurology (1985) 35:312–319.[Abstract/Free Full Text]
  47. The Canadian cooperative trial of cyclophosphamide and plasma exchange in progressive multiple sclerosis. The Canadian Cooperative Multiple Sclerosis Study Group. Lancet (1991) 337:441–446.[Web of Science][Medline]
  48. Klahr S, Levey AS, Beck GJ, et al. The effects of dietary protein restriction and blood-pressure control on the progression of chronic renal disease Modification of Diet in Renal Disease Study Group. N Engl J Med (1994) 330:877–884.[Abstract/Free Full Text]
  49. Phrommintikul A, Haas SJ, Elsik M, et al. Mortality and target haemoglobin concentrations in anaemic patients with chronic kidney disease treated with erythropoietin: a meta-analysis. Lancet (2007) 369:381–388.[CrossRef][Web of Science][Medline]
  50. Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA (1998) 280:605–613.[Abstract/Free Full Text]
Received for publication: 17. 7.07
Accepted in revised form: 12. 9.07


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 pdiHome page
E. Vonesh
ON SMALL SOLUTE CLEARANCE AND PATIENT OUTCOMES: EVIDENTIAL PRACTICE OR OBSERVATIONAL TREPIDATION?
Perit. Dial. Int., November 1, 2009; 29(6): 623 - 629.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Extract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
23/1/52    most recent
gfm712v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (1)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Garg, A. X.
Right arrow Articles by Levin, N. W.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Garg, A. X.
Right arrow Articles by Levin, N. W.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?