A new international consortium of medical professionals, academic researchers, and dialysis patients and their care partners has determined that treatment outcomes most sought by patients are not supported by the bulk of medical research done in the nephrology field.
The SONG-HD (Standardized Outcomes in Nephrology—Hemodialysis) survey project is the first time patient preference in research outcomes has been considered on a large, international scale and given equal consideration with those of medical professionals. The study involved 1,181 participants from 73 countries over three rounds of a Delphi study where patients, caregivers, and health professionals ranked desired outcomes on a 9-point Likert scale. Twenty desired health outcomes were charted in the final results.1
Table 1 and the accompanying chart (Fig. 3) from the SONG-HD report shows that with few exceptions, patients and health professional are far apart in understanding what outcomes matter most.2
In hemodialysis, biochemical markers such as serum phosphorus, calcium, and parathyroid hormone levels are commonly reported but are not strongly and consistently associated with mortality, cardiovascular disease or quality of life. Hemodialysis patients and their caregivers prioritize outcomes relevant to their well-being and quality of lifestyle, factors usually absent as outcomes reported in clinical trials. These priorities often include empowerment, independence, social acceptance, concerns for family and care partners, normality, free time from dialysis, and freedom to travel. 3-7
At the start of the SONG-HD Initiative, the consortium reviewed research outcomes over the last 15 years. This period had an alarming growth in the number of new dialysis patients, which in turn led to a plethora of studies. The Cochrane Renal Group Specialized Register shows that from 2004 to 2014, there were 1,500 randomized, controlled trials in hemodialysis.8 Yet there has been very limited improvement in clinical, quality-of-life, and mortality outcomes for dialysis patients.9-10
In 2014, the Lancet reported that 85% of the $240 billion expended on health research in 2010 was wasted because of problems in the design, conduct, analysis and reporting of research, much of which ignored patient’s non-medical needs.11
Chronic kidney disease is all about loss and the life-changing deprivations kidney dialysis brings. Today, CKD is a leading contributor of mortality and morbidity the world over 12-14 and its standard form of treatment, 3-times-per-week in-center dialysis, has proven to be inadequate to rehabilitate people with kidney failure. Patients on dialysis have an annual mortality rate 15-100 times higher than that of the general population.15-18 And studies consistently show that dialysis patients report a significant loss in quality of life, even greater than people with metastatic cancer and other chronic diseases.19-20
SONG is designed to help establish consensus on a prescribed list of dialysis outcomes most desired by health care professionals and patients, to which future randomized trials and other research on dialysis should adhere. It is based on the precept that patient-centered care involves shared decision making, which takes into account patients’ priorities and values as well as the biomedical goals of the clinician.
The initiative takes its cue from the success of the Outcome Measures in Rheumatology (OMERACT) initiative which began in 1992. OMERACT outcomes have improved the reporting and relevance of outcomes in rheumatology trials and its methodology has been applied successfully in cancer, middle ear infection, eczema, and chronic pain. Both the World Health Organization and the U.S. Food and Drug Administration have endorsed OMERACT outcomes. In 2010, the international Core Outcome Measures in Effectiveness Trials (COMET) organization was launched to facilitate the development and application of core outcome sets.21-29
Core outcome sets represent the minimum that should be measured and reported in all clinical trials for a specific condition. While they are not meant to be definitive, the intention is that the core outcomes be collected and reported to allow the results of trials and other studies to be compared, contrasted, and combined as appropriate; researchers are at liberty to collect and explore other outcomes. A core outcome set does not exist for CKD.
Phases of the initiative
SONG-HD is a five-phase, multi-method project that includes systematic reviews, focus groups, semi-structured interviews, Delphi surveys with best-worst/choice experiments, and a consensus workshop.
In Phase 1, SONG-HD reviewed 265 research trials published in Cochrane, involving 19,393 patients on hemodialysis. In those trials, 1,507 different outcomes were reported with a mean of 12 outcomes per trial. Biochemical outcomes were most frequently reported (e.g. phosphorus, calcium, hemoglobin, parathyroid hormone measurements).30
In Phase 2, 58 hemodialysis patients and 24 care partners from 11 dialysis units in Sydney and Melbourne, Australia, and Calgary, Canada identified, defined, and ranked outcomes they considered important. The top 10 were:
3) Ability to travel
4) Dialysis-free time
5) Impact on family
6) Ability to work
9) Drop in blood pressure
10) Lack of appetite
In interviews with participants in Phase 3, patients spoke openly about not fully understanding the biomedical markers so valued by nephrologists, like potassium and phosphate levels or clearance formulas. Instead, they wanted to maintain as normal a life as possible while on dialysis. Mortality was not high on their list of priorities as they live with death daily; they were more concerned about the quality of the life left to them than their longevity.31
Phase 4: The Delphi survey
The Delphi method, developed at the beginning of the Cold War to forecast the impact of technology on warfare, is based on the principle that decisions from a structured group of individuals are more accurate than those from unstructured groups.32
In a typical Delphi survey, participants are asked to give their opinions over several rounds, until a consensus emerges. The mean (average) or median scores of the final rounds determine the results. All participants remain anonymous, even after completion of the final report, to prevent the personality or reputation of some participants from dominating others. Arguably, anonymity also frees participants from their personal biases, allows free expression of opinions, encourages open critique, and facilitates admission of errors when revising earlier judgments.33-35
Approximately 200+ patients and caregivers, along with more than 1,100 health professionals (nephrologists, surgeons, nurses, allied health care professionals, researchers, policy makers and people from industry), took part in the three rounds of the SONG-HD Delphi survey. There was a similar proportion of males and females, and a wide range of ages (18 years to over 81 years). Round 1 participants were from 73 countries and began by ranking 34 desired HD outcomes on a scale of 1 (least important) to 9 (critical). In Round 2, five outcomes with medians <7 were excluded, leaving 29 to rank. In Round 3, nine outcomes with a median <7 were excluded, leaving 20 to rank. Outcomes that were removed are still important—but not of critical importance to both patients and health professionals.36
Consensus was defined a priority based on the OMERACT definition using proportion scores. However, because most participants rated all outcomes in round 3 as critically important with scores of 7 to 9, these criteria resulted in a list of 16 outcomes, which exceeded the recommended 3 to 5 outcomes for a core outcome set. Therefore, the definition and threshold for “consensus in” were revised to determine a maximum of 5 core outcomes to be considered for the core outcome set.
The highest priority outcomes shared among patients/caregivers and health professionals were 1) vascular access problems, 2) dialysis adequacy, 3) fatigue, 4) cardiovascular disease, and 5) mortality. Frequently reported biochemical outcomes in HD trials, such as phosphate, calcium, and parathyroid hormone levels, were consistently rated to be of lower importance by both stakeholder groups.37
Consensus workshops were held with Delphi participants to further define “fatigue”—the condition that occurs after in-center dialysis treatment session.
Future SONG projects
SONG-HD is first focusing on hemodialysis, since it is the main dialysis modality worldwide. The SONG Initiative is now working to develop core outcomes across the CKD treatment spectrum, including kidney transplantation (SONG-Tx), childhood CKD (SONG-Kids), polycystic kidney disease (SONG-PKD), and peritoneal dialysis (PD).
- Evangelidis N, Tong A, Manns B, Hemmelgarn B, Wheeler DC, Tugwell P, Crowe S, Harris T, Van Biesen W, Winkelmayer WC, Sautenet B, O’Donoghue D, Tam-Tham H, Youssouf S, Mandayam S, Ju A, Hawley C, Pollock C, Harris DC, Johnson, DW, Rifkin DE, Tentori F, Agar J, Polkinghorne KR, Gallagher M, Kerr PG, McDonald SP, Howard K, Howell M, Craig JC, Developing a Set of Core Outcomes for Trials in Hemodialysis: An International Delphi Survey, Am J Kidney Dis. DOI: http://dx.doi.org/10.1053/j.ajkd.2016.11.029, 3.
- Halimi JM, Sautenet B, Gatault P, Roland M, Giraudeau B.Renal endpoints in renal and cardiovascular randomized clinicaltrials: time for a consensus? Fundam Clin Pharmacol. 2012;26(6):771-782.
- EVOLVE Trial Investigators. Effect of cinacalcet on cardiovasculardisease in patients undergoing dialysis. N Engl J Med.2012;367(26):2482-2494.
- Palmer SC, Hayen A, Macaskill P, et al. Serum levels of phosphorus, parathyroid hormone, and calcium and risks of death and cardiovascular disease in individuals with chronic kidney disease: a systematic review and meta-analysis. JAMA. 2011;305(11): 1119-1127.
- Palmer SC, Teixeira-Pinto A, Saglimbene V, et al. Association of drug effects on serum parathyroid hormone, phosphorus, and calcium levels with mortality in CKD: a meta-analysis. Am J Kidney Dis. 2015;66(6):962-971.
- Perkovic V, Neal B. Trials in kidney disease–time to EVOLVE. N Engl J Med. 2012;367(26):2541-2542.
- Jones A, Edwards L, Higgins GY, Webster AC, Craig JC. Cochrane Renal Group. About the Cochrane Collaboration (Cochrane Review Groups (CRGS)). onlinelibrary.wiley.com/o/cochrane/clabout/articles/RENAL/sect0-meta.html. Accessed 31 March 2015.
- Inrig JK, Califf RM, Tasneem A, Vegunta RK, Molina C, Stanifer JW, et al. The landscape of clinical trials in nephrology: a systematic review of Clinicaltrials.gov.. Am J Kidney Dis. 2014;63:771–80
- Nissenson AR. Improving outcomes for ESRD patients: shifting the quality paradigm. Clin J Am Soc Nephrol. 2014;9:430–34.
- Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383:156–65.
- Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY. Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. N Eng J Med. 2004;351:1296–305
- Levey AS, Atkins R, Coresh J, Cohen EP, Collins AJ, Eckardt KU, et al. Chronic kidney disease as a global public health problem: approaches and initiatives – a position statement from Kidney Disease Improving Global Outcomes. Am J Kidney Dis. 2007;72:247–59
- Coresh J, Turin TC, Matsushita K, Sang Y, Ballew SH, Appel LJ, et al. Decline in estimated glomerular filtration rate and subsequent risk of end-stage renal disease and mortality. JAMA. 2014;311:2518–31.
- Neovius M, Jacobson SH, Eriksson JK, Elinder CG, Hylander B. Mortality in chronic kidney disease and renal replacement therapy: a population-based cohort study. BMJ Open. 2014;4:e004251 Robinson BM, Zhang J, Morgenstern H, Bradbury BD, Ng LJ, McCullough KP, et al. Worldwide mortality risk is high soon after initiation of hemodialysis. Kidney Int. 2014;85:158–65
- Robinson BM, Zhang J, Morgenstern H, Bradbury BD, Ng LJ, McCullough KP, et al. Worldwide mortality risk is high soon after initiation of hemodialysis. Kidney Int. 2014;85:158–65
- de Jager DJ, Grootendorst DC, Jager KJ, van Dijk PC, Tomas LM, Ansell D, et al. Cardiovascular and noncardiovascular mortality among patients starting dialysis. JAMA. 2009;302:1782–89
ANZDATA. The 36th Annual Report. Adelaide, Australia: Australian and New Zealand Dialysis and Transplantation Registry; 2013.
- Wong G, Howard K, Chapman JR, Pollock C, Chadban SC, Salkeld G, et al. How do people with chronic kidney disease value cancer-related quality of life? Nephrology. 2012;17:32–41
- Wyld M, Morton RL, Hayen A, Howard K, Webster AC. A systematic review and meta-analysis of utility-based quality of life in chronic kidney disease treatments. PLoS Med. 2012;9:e1001307.
- Boers M, Kirwan JR, Wells G, Beaton D, Gossec L, d’Agostino MA, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol. 2014;67:745–53
- Kirkham JJ, Boers M, Tugwell P, Clarke M, Williamson PR. Outcome measures in rheumatoid arthritis randomised trials over the last 50 years. Trials. 2013;14:324
- Kirkham JJ, Gargon E, Clarke M, Williamson PR. Can a core outcome set improve the quality of systematic reviews?–a survey of the Co-ordinating Editors of Cochrane Review Groups. Trials. 2013;14:21
- Waters AM, Tudur Smith C, Young B, Jones TM. The CONSENSUS study: protocol for a mixed methods study to establish which outcomes should be included in a core outcome set for oropharyngeal cancer. Trials. 2014;15:168
- Harman NL, Bruce IA, Callery P, Tierney S, Sharif MO, O’Brien K, et al. MOMENT–Management of Otitis Media with Effusion in Cleft Palate: protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials. 2013;14:70
- Schmitt J, Spuls PI, Thomas KS, Simpson E, Furue M, Deckert S, et al. The Harmonising Outcome Measures for Eczema (HOME) statement to assess clinical signs of atopic eczema in trials. J Allergy Clin Immunol.2014;134:800–7;
- Chiarotto A, Terwee CB, Deyo RA, Boers M, Lin CW, Buchbinder R, et al. A core outcome set for clinical trials on non-specific low back pain: study protocol for the development of a core domain set. Trials. 2014;15:511
- Prinsen CA, Vohra S, Rose MR, King-Jones S, Ishaque S, Bhaloo Z. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a ‘core outcome set. Trials. 2014;15:247
- Clark M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8:39
- Tong, A. et. al. Standardised Outcomes in Nephrology – Haemodialysis (SONG-HD). Summary Report, Oct. 2015, 6, 7, 8, 16
- JVTE v15n2: The Modified Delphi Technique – A Rotational Modification, Journal of Vocational and Technical Education, Volume 15 Number 2, Spring 1999, web: VT-edu-JVTE-v15n2: of Delphi Technique developed by Olaf Helmer and Norman Dalkey.
- Rowe and Wright (1999): The Delphi technique as a forecasting tool: issues and analysis. International Journal of Forecasting, Volume 15, Issue 4, October 1999
- Rowe and Wright (2001): Expert Opinions in Forecasting. Role of the Delphi Technique. In: Armstrong (Ed.): Principles of Forecasting: A Handbook of Researchers and Practitioners, Boston: Kluwer Academic Publishers
- Green, Armstrong, and Graefe (2007): Methods to Elicit Forecasts from Groups: Delphi and Prediction Markets Compared. Forthcoming in Foresight: The International Journal of Applied Forecasting (Fall 2007).
- Tong, A. et.al. Standardised Outcomes in Nephrology – Haemodialysis (SONG-HD). Program & Summary Report, Consensus Workshop/San Diego. Nov. 7, 2015, 14.
- Evangelidis N, Tong A, Manns B, Hemmelgarn B, Wheeler DC, et al., Developing a Set of Core Outcomes for Trials in Hemodialysis: An International Delphi Survey, Am J Kidney Dis. DOI: http://dx.doi.org/10.1053/j.ajkd.2016.11.029, 3-10.