Academic Articles Hospitals

Accounting for quality in the measurement of hospital performance in Costa Rica

Many papers have attempted to calculate hospital efficiency before and after a policy change. Most research, however, does not take into account quality levels when analyzing these changes in efficiency. For instance, a doctor may increase the number of patients per hour that they visit by 50%, but if this is accomplished simply by lowering the quality of each visit, there may not necessarily be a quality improvement.

Pablo Aroncena and Ariadna García-Prado (2007) look at efficiency measures in Costa Rica after the implementation of management contracts with the local public hospital administrators. Unlike most papers, however, they attempt to take into account how quality plays a role in efficiency metrics.

Costa Rican Health Care System

In Costa Rica, health care is primarily provided by the government. In 2001, 83% of providers worked in the public sector. Approximately 77% of health care expenditures are public compared to only 23% private. The Ministry of Health provides regulatory oversight of the health care system and the Caja Costarricense de Seguro Social (CCSS)–Costa Rican Social Security Institute is in charge of public health care service delivery and financing. Most all physicians in public health care system are salaried and receive civil service status. Physicians and hospital managers receive little or no merit-based pay and thus have little incentive to improve performance.

The 1990s brought much restructuring of the Costa Rican health care system. For example:

  • 1994: a population-based model of primary care was created. Basic health care teams–Equipo Básico de Atención Integrada a la Salud (EBAIS) were established to decentralize health care services and offer primary care and preventative services to the entire population.
  • 1998: Decentralization Law. This law gave managerial autonomy for public hospitals.
  • 2000: Management agreements fully implemented. “These contracts specified targets that managers pledged to achieve in a given time frame and were mainly deemed to get quality improvements and a better utilization of hospital resources.” Spain introduced similar management contracts in the 1990s and while hospital efficiency did increase, the measures were ineffective in primary care centers.

Data and Methods

The authors use Costa Rican public hospital data between 1997 and 2001. The authors define a productive process as follows:

  • P(x)={(y,b) : x can produce (x,b)}

In the paper, x are inputs, y are good outputs, and b are bad outputs. The vector x consists of measures of MD and RN labor hours, the number of beds as a proxy for capital, and real monetary expenditures on goods and services. The good outcomes y are the number of discharges and number of outpatient hospital services used, while the bad outcome, b, is given by hospital readmissions.

The authors claim that hospital readmissions is a good proxy for quality. This may not be true. A sicker patient base may have more hospital readmissions. The authors do try to control for health status using DRGs and also their study examines changes in readmission rates so the baseline case mix of a hospital is not as important. Similarly, if a hospital has poorer patients, they may be less likely to comply with the doctors directives (either due to financial issues or misunderstanding the physician). If the case mix remains unchanged, however, this will not bias the results. A final issue noted by Arocena and García-Prado is that managers may have misreported re-admission rates in order to receive favorable reviews on their performance contracts. This problem is mitigated somewhat by the fact that the CCSS conducted audits of each hospital facility.

The distance function employs a nonparametric data envelopment analysis approach. The function used by the authors is:

  • Dt(xt,yt,bt,α)=min {λ: (λ1-αb, λy) ∈ Pt(x)}

Results

The authors find that hospital efficiency increased significantly after the management contracts only in small hospitals. In large hospitals, there was neither an increase or decrease in efficiency. The paper believes that increased physician compliance with clinical protocols and better record keeping drove the improved performance for small hospitals. Peer pressure mechanisms are less effective in large hospitals, and the finding that absenteeism rates increased in large hospitals after the management agreements may show one reason why efficiency improvements did not increase in large hospitals.

Further, the authors actually find an efficiency decrease when quality is not taken into account. While the management contracts may not have strictly increased throughput, they do seem to have increased quality in small hospitals and the quality gains outweigh the quantity losses, at least in the case of small hospitals.

  • Aroncena P, García-Prado A (2007). “Accounting for quality in the measurement of hospital performance: Evidence from Costa Rica,” Health Economics. Volume 16, Issue 7, Pages 667 – 685.