- Journal List
- Am J Public Health
- v.92(1); Jan 2002
- PMC1447380
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsem*nt of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer | PMC Copyright Notice
Am J Public Health. 2002 January; 92(1): 30–33.
PMCID: PMC1447380
PMID: 11772753
Joseph S. Coyne, DrPH and Peter Hilsenrath, PhD
Author information Article notes Copyright and License information PMC Disclaimer
COMPARATIVE STUDIES have been part of health services research literature for decades. The benefits of these analyses include documenting how the more successful practices can be adapted in another country. Such has been the case in France, where many US health care delivery practices have been adopted in market reforms.1
The World Health Organization (WHO) studied the health systems of 191 countries for its World Health Report 2000.2 The study is provocative and has stimulated significant analysis of the structure and performance of health systems.3 We examine the variables and methodology used by the WHO to measure efficiency and performance of health systems.
METHODOLOGY OF THE WORLD HEALTH REPORT
The methodology employed in the WHO report relies on the following major components: (1) goal attainment (effectiveness), (2) health expenditures per capita, and (3) efficiency and the overall level of health performance.
Goal Attainment (Effectiveness)
The first component, goal attainment (effectiveness), has 5 subcomponents (respective weights in parentheses): level of health (25%), distribution of health (25%), level of responsiveness (12.5%), distribution of responsiveness (12.5 %), and fairness of financial contribution (25%).
The first of these subcomponents is reported in terms of disability-adjusted life expectancy (DALE), for which life tables are used to calculate the average number of healthy years of life for a population. Japan ranked 1st on this measure, Australia 2nd, and the United States 24th. The second subcomponent measures the equality of child survival for a population. Chile ranked 1st on this measure, the United Kingdom 2nd, and the United States 32nd. The third subcomponent measures the level of system responsiveness; it is based on surveys of approximately 2000 key informants from selected countries about the performance of their health system in terms of such concerns as access to social services and choice of provider. The United States ranked 1st on this measure, and Switzerland ranked 2nd.
The fourth subcomponent is the distribution-of-responsiveness variable, used to measure the proportion of the population judged by the 2000 key informants to be part of a disadvantaged group (e.g., racially disadvantaged, indigenous, elderly, or poor). On this measure, for which a country that has greater equality would score higher than one with more inequality, the United Arab Emirates ranked 1st, Bulgaria 2nd, and the United States fell into a group of countries that were tied for 3rd to 38th place.
The fifth subcomponent measures the equality of household contributions to the financing of the health system, based on the proportion of permanent income above subsistence level spent as out-of-pocket outlays. On this measure, Colombia ranked 1st, Luxembourg 2nd, and the United States was tied with Fiji for 54th and 55th place.
The 5 subcomponents were weighted as specified above to produce one overall measure constructed on a scale of 0 to 100. On this overall goal attainment measure, Japan ranked 1st, Switzerland 2nd, and the United States 15th.
Performance and Efficiency
The second component, health expenditure per capita, is a variable considered in both efficiency and performance measures. The United States ranked 1st in health expenditure per capita, with expenditures well beyond those of Switzerland (2nd) and Germany (3rd).
The third component measures performance of health systems, including efficiency. Efficiency has been defined as follows:HSE = (DALEO – DALEWO) / (DALEM – DALEWO),where HSE is the efficiency performance of the health system; DALEO is the observed DALE; DALEWO is the DALE without a “functioning modern health system” given the nonhealth attributes that affect health, represented by education; and DALEM is the maximum DALE given the level of expenditure per capita. A frontier production model was used to estimate maximum DALE levels. A similar model was used to produce an overall indicator of performance, but in this model a measure of composite health system attainment was used in place of life expectancy.
The results reported by the WHO have received wide publicity and drawn attention to the shortcomings of many health systems, including that of the United States. Oman ranked 1st, Malta 2nd, and the United States 72nd in terms of HSE. France and Italy ranked 1st and 2nd, respectively, in overall health system performance; the United States ranked 37th.
THE CASE OF SOUTH AFRICA
The emphasis on DALE can be misleading and undermines rankings for countries with low life expectancy but otherwise good health systems. DALE is driven by many factors other than health systems. The WHO also emphasizes equity in the distribution of health, the distribution of responsiveness, and the fairness of financial contribution. Equity is not universally considered desirable and is difficult to achieve in heterogeneous societies.
Consider the case of South Africa, which is home to perhaps the most modern health care system in Africa. It was the first nation to perform a human heart transplant in the 1960s. It has modern hospitals and clinics and well-trained providers, with most health spending occurring in the private sector.4 Yet South Africa was ranked 175th in overall performance and 182nd in efficiency among 191 countries.
How is it possible that such a well-developed infrastructure supports one of the worst health systems in the world, according to the WHO? Some empirical studies show that public health measures matter, not medical care.5 The answer lies largely with the tremendous impact of AIDS in driving down DALE in South Africa. A significant additional factor has been the continuing inequity that prevails in the post-apartheid era.
The United Nations estimates that 20% of the adult population in South Africa is HIV-positive, but there is considerable variation within the country.6 Life expectancy in South Africa is expected to fall to 35 years by 2010.7 It is not clear how much of this epidemic is due to a flawed health system. Other factors, more appropriately classified as cultural, anthropological, or social, are driving AIDS in Southern Africa. A similar argument can be made about inequality. The conditions driving inequality are often complex, with deep historical roots. It can be misleading to attribute severe inequality, such as is found in South Africa, to the health system. The rankings, it can be argued, reflect much more than shortcomings of the health system.
THE CASE OF THE UNITED STATES
There is another dimension of efficiency that should also be considered. A low-cost, highly effective health system that sustains a healthy population is efficient in a static sense, but over the long run advances in medical and other technologies play a significant role.8 The above data and the WHO report do not account for this important aspect of efficiency.
The United States spent an estimated $22 billion on research in the health sector in 1999, exclusive of substantial privatesector research and development occurring in pharmaceutical, medical electronics, and other organizations.9 These large allocations have generated major advances in health technologies that are not adequately captured by the WHO methodology. The responsiveness of health systems, at least in a static sense, is captured in the WHO methodology. On the measure of responsiveness, the United States ranked 1st and South Africa tied for 73rd and 74th, and this ranking was achieved while South Africa was implementing conservative public spending programs.10
A CONTINUING CONTROVERSY
Shortcomings of health systems are identified in the WHO report, but much of what is measured has to do with broad socioeconomic conditions. Consider the case of the country ranked first in overall health system performance. France's health policies are driven by the national culture and preferred philosophies of its people: freedom of choice and regulatory protection.11 The importance of such traditions and other nonsystem factors becomes even more apparent when France is compared with other countries.12
It is misleading for an efficiency or system performance indicator to rely heavily on life expectancy when many determinants of life expectancy are beyond the realm of the health sector. Another criticism of the report is that the WHO has not made an adequate effort to distinguish between efficiency and equity. This results in a bias against countries with greater inequality, such as the United States and South Africa. While this bias may be defensible politically, it is more difficult to defend in a performance analysis of health systems.
The controversy over the WHO report, has in our view, been constructive and contributed to an important dialogue among students of health systems.13,14 Such dialogue has encompassed a full spectrum of cross-national comparisons.15–18 We hope this continues.
Notes
Peer Reviewed
References
1. Coyne JS. Financing international partnerships: a case study approach. Paper presented at: First International Conference on Healthcare Systems for the New Millennium; May 17–20, 2000; Queretaro, Mexico.
2. World Health Report 2000. Available at: http://www.who.int/whr/2001/archives/2000/en/index.htm. Accessed November 4, 2001.
3. Anderson G, Hussey P. Comparing health system performance in OECD countries. Health Aff (Millwood). 2001; 20(3):219–232. [PubMed] [Google Scholar]
4. McIntyre D, Bloom G, Doherty J, Brijlal P. Health Expenditure and Finance in South Africa. Durban, South Africa: Health Systems Trust and World Bank; 1995.
5. Hadley J. Medicare spending and mortality rates of the elderly. Inquiry. 1988;25:485–493. [PubMed] [Google Scholar]
6. Report on the global HIV/AIDS epidemic—June 2000. Available at: http://www.unaids.org/epidemic_update/report/#table. Accessed November 5, 2001.
7. A turning point for AIDS? Economist. July 15, 2000:77–79. [PubMed]
8. Manning W, Newhouse J, Duan N, et al. Health insurance and the demand for medical care: evidence from a randomized experiment. American Economic Review. 1987;77:251–277. [PubMed] [Google Scholar]
9. Heffler S, Levit K, Smith S, et al. Health spending growth up in 1999; faster growth expected in the future. Health Aff (Millwood). 2001;20(2): 193–203. [PubMed] [Google Scholar]
10. Hilsenrath P. Fiscal policy dilemmas and health spending in South Africa. J Health Hum Serv Adm. 1999; 21:310–324. [PubMed] [Google Scholar]
11. Poullier J, Sandler S. Reconsidering the role of competition in health care markets: France. J Health Polit Policy Law. 2000;5:899–905. [Google Scholar]
12. Coyne J, Hilsenrath P. Health system performance in South Africa vs. France: World Health Report 2000. Safundi: Journal of South African and American Comparative Studies. 2001; 3(1):1–9. [Google Scholar]
13. Murray C, Frenk J. World Health Report 2000: a step towards evidence-based health policy. Lancet. 2001;357: 1698–1700. [PubMed] [Google Scholar]
14. Navarro V. World Health Report 2000: response to Murray and Frenk. Lancet. 2001:357;1701–1702. [PubMed] [Google Scholar]
15. Murray C, Kawabata K, Valentine N. People's experience versus people's expectations. Health Aff (Millwood). 2001;20(3):21–24. [PubMed] [Google Scholar]
16. Blendon R, Kim M, Benson J. The public versus the World Health Organization on health system performance. Health Aff (Millwood). 2001;20(3): 10–20. [PubMed] [Google Scholar]
17. Blendon R, Schoen C, Donelan K, et al. Physicians' views on quality of care: a five-country comparison. Health Aff (Millwood). 2001;20(3):233–243. [PubMed] [Google Scholar]
18. Coulter A, Cleary PD. Patients' experiences with hospital care in five countries. Health Aff (Millwood). 2001;20(3):244–252. [PubMed] [Google Scholar]
Articles from American Journal of Public Health are provided here courtesy of American Public Health Association
