Transcript of Introduction to Economic Evaluation.
Stephen Jan: My name is Stephen Jan. I'm Professor of Health Economics at the George Institute for Global Health. Today's lecture will be an introduction to economic evaluation. The focus of the lecture will be on the use of economic evaluation in relation to population health programs. What you should be aware of is that there's no assumed knowledge of economics or economic evaluation, so I'm pretty much going to be starting from basic principles.
So you can see from the first slide I have some background statistics in relation to expenditure in Australia and as you can see there's a trend towards increasing spending from generally all sources, in particular from the Australian government. So you can see over the last several years there's been an increase in spending in relation to government expenditure, private expenditure, expenditure from states and territories. And all this all this some highlights is that there is increasing demands on health care resources, and so we we really need, do need to think about how we spend our resources and achieving value for money. And so that's the context in which I'd like to present today's lecture, is the idea that there are increasing demands on healthcare resources and that's a universal phenomenon, not just Australia. And we need to work out ways in which we can get value for money, from healthcare.
So the outline of the lecture - the first part of the lecture, I'll just give you a very brief definition of what we mean by economic evaluation and then I'll go into four different types of economic evaluation: cost minimization analysis, cost-effectiveness analysis, cost utility analysis and cost-benefit analysis. This isn't an exhaustive list, but it covers the main types of economic evaluation that have been used in the healthcare setting. And in the last bit of the lecture I'll discuss how we interpret the findings of an economic evaluation study.
So here's a definition that's of of economic evaluation as it applies to health sector. It's a definition that comes from a textbook by Michael Drummond, which is probably the most commonly used resource in this area. And the definition is: the comparative analysis of alternative courses of action in terms of both their resource costs and their health consequences. And as you can see that, there are a number of elements in this definition that highlight important features of an economic evaluation. Firstly, it's a comparative analysis - so involves comparing two or more options. The other aspect is that it involves looking at both what goes into a program, so resource costs and what comes out in terms of health and other consequences. And that's generally the framework that we're operating when we're doing an economic evaluation. You can see that with the next slide.
So this is a diagram that basically shows you the structure of an economic evaluation. We have two programs, program A and program B. And what we're interested in is both the costs, so what goes into both of them, and consequences, what comes out the other end. And importantly what we do within an economic evaluation is that we value the differences in the costs and consequences between A and B. So in a sense we subtract one from the other, subtract the costs from A from subtract the cost of A from B and subtract the consequences of A from B. So A might be the new program or treatment, so it might be a new drug or might be a new health promotion program and B, while B is usually framed is in to a standard practice or the status quo. So what economic evaluation generally does then is assess what is the additional costs associated with moving from the status quo to the new program and what are the additional benefits or consequences we achieve when we do so? So the the fundamental principle underlying this is that economic evaluation is sense in incremental analysis. It looks at the incremental costs and the incremental consequences of an investment in a new program.
So if we look into a bit more detail what we're trying to measure, on the cost side there are a number of components that we are interested in. So we interested in in terms of the intervention and also the comparator, which is often the usual kit. And the first component of costs is the cost of the actual intervention itself. So when we're doing the economic evaluation, whether it's a new drug or a new health promotion program, the most obvious thing that we need to cost is the cost of that program itself. So the cost of implementing that program on top of usual care. So if we have a usual care or a standard practice if we're implementing the new program, we want to find out how much does it cost to the health sector or the community to implement this new program on top of what's already been happening. And that comprises a number of components and might include staffing, materials and drugs.
The other element of costs are the costs that are incurred by individuals that are involved in the intervention or the comparator. So these are costs of healthcare that are consumed by patients, or by individuals within both groups of patients or individuals. So if we're talking about the cost of healthcare we're talking about the costs associated with hospitalisations, primary care visits and medicines and if we're doing a study or for instance a randomized trial of a new intervention then we, what we're interested in is following up the individuals within that trial in terms of the healthcare use, measured in terms of hospitalization, primary care visits and medicines. And we do that for both patients and individuals in the intervention arm, as well as the comparator arm, people who are receiving usual care. So they're the sorts of costs that we're interested in measuring if we're taking a health sector perspective. So essentially these are the health care costs the costs that are incurred by the health sector associated with the intervention.
Sometimes however that that may be too narrower at perspective and we might be interested in a broader social perspective. From that point of view who might be interested in measuring broader social costs. These might include for instance lost productivity associated with an illness or a condition that's been assessed. So if we're interested in social costs then you might have for instance an intervention that keeps people out of hospital if it's effective. If that program is shown to be effective and keep people out of hospital it, in turn, may enable those people to return to work earlier and be more productive. And if we're taking a societal perspective we might want to then measure those costs or cost savings to the health system associated.
So in terms of how we measure those costs, there are a number of sources of data. The the most obvious of financial records associated with running a study, so we generally have budget statements pertaining to how much staff come in, materials, how much training. Those sorts of things our records which enable us to determine how much intervention cost. So this might come from a particular trial that we're running, but it may also come from questionnaires that we administered to patients or participants within a study. Sometimes we might be possible to use diaries to record healthcare utilisation, so and that's quite sometimes quite a useful tool for measuring hospitalisations. Diaries are ways in which we can measure the use of health services from patients on an ongoing basis and not have to rely on them to to remember how many hospitalisations or how many medical consultations they had over the past year, if we're just relying on following them up over a long period of time.
Other ways in which we can record resource use is using hospital records the advantage of that of course is that it doesn't rely on patients or participants to remember their utilisation. And perhaps the gold stand for for measuring healthcare resource is using the link that Ministry of data. In New South Wales we have the benefit of a resource core the Centre for Health Economic Record Linkage, which is often referred to as Cherel, which enables us to link across multiple data sets and enables us to then follow up patients not only within the hospital system but across the entire health sector. So that's, there's some of the practical aspects associated with the measurement of costs for an economic evaluation.
I just now want to go into some of the different types of economic evaluation that are available for us to use. The most common or the most basic form of economic evaluation is referred to as a cost minimisation analysis and I can the best way to describe that is if we go back to the original diagram that I showed you and and in the next slide I've reproduced that diagram but showed you how a cost minimisation analysis fits in. So you can see here we've got program A and program B, and going in to prepare both are costs and coming out are consequences. The feature of cost minimization analysis is that it assumes the consequences of A are equal to the consequences of B, so it means that the outcomes of A and B are equal and therefore all we have to do is compare the two in terms of costs. This is something that you know we can occasionally assume. For instance, if we're doing an evaluation of two drugs one might be a generic and the other one might be a brand-name drug, the same active ingredients, we may be able to then make the assumption that the outcomes of the two interventions are going to be the same and therefore the comparison is really just going to be on the basis of costs. In relation to population health that might be that sort of comparison may be a bit more difficult to make because it's often difficult to assume that two programs are going to be equivalent without doing the studies to determine whether that is in fact the case. So in that instance it's always probably safer not to do a cost minimisation analysis and move on to the next type of evaluation, which is referred to as cost-effectiveness analysis
So cost-effectiveness analysis is probably the most commonly used type of economic evaluation in the health sector and if we go back to the diagram it's the same schema - program A and program B. But the feature of cost-effectiveness analysis is that it assumes the consequences of A and B can be measured in the same units of health gain. So that might be lives saved or life years gained or cases prevented. It doesn't assume like in cost minimization analysis they were equal. It just means it all it does is it assumes that they can be measured in the same unit. Again so program, so an example of how the findings of a cost-effectiveness analysis might be presented, might be in terms of an incremental cost per health gain of A versus B. So an incremental cost per life year saved of A versus B. So that's essentially the type of comparison that we're doing with a cost-effectiveness analysis, where we're comparing the costs per unit of health gain achieved. In this case, it's its cost per life year saved.
A variation on cost-effectiveness analysis is cost utility analysis. It's essentially the same as cost-effectiveness analysis, except the unit of measure that we have is quality adjusted life year or QUALYs and is essentially a composite measure of life years gained, adjusted by quality of life. And I'll show you just briefly how we measure QUALys. Essentially quality of life is measured on a scale or between 0 and 1 - so 0 represents death and 1 represents full health. And a quality of life weight for instance of 0.5 would represent some degree of impairment, which is then multiplied by survival. So if somebody has a quality of life of 0.5 and survival of 10 years then their QUALY is 0.5 times 10 which equals 5 QUALY. So it's a simple matter of just measuring the quality of life by the length of survival and so when we're measuring QUALY gained we're interested in the quality of program A subtracted from program B. So that's essentially the gain that we achieved from a particular intervention, the difference between A and B in terms of QUALY.
So the final type of economic evaluation is cost-benefit analysis and as you can see it's slightly different in the way it's presented. It really doesn't involve a comparison between two different alternatives, it's essentially a standalone evaluation of a single program. So it looks at program A in terms of costs and its consequences. The way, what defines a cost-benefit analysis is that the consequences of A are measured in terms of social benefits and cost in terms of social cost, but they're both measured in monetary terms. So the point here is it we generally take a societal perspective. So you remember the original diagram that I showed you? We have a health sector perspective and a societal perspective. Well in cost-benefit analysis we generally take a societal perspective and we measure costs as you always as you generally do in monetary terms and dollars, but what's interesting in cost-benefit analysis is we measure the benefit also in dollar terms. And that can be that can propose a bit of a challenge when we measure when we are evaluating health care programs. So the question arises how do you measure that the the benefit of a health care program in monetary terms if the aim of the program is to save lives? Well there are ways in which we can do that and I'll show you in the next slide, but essentially what we're trying to do with a cost-benefit analysis is if we can measure the cost and benefits in dollar terms, then it gives us a very simple decision rule when we're assimilating the results. So if the benefits exceed the costs, then what it says is that overall in terms of the community we should go ahead with the program, the value of the program is greater than the cost then we should accept the program and invest in it. If the benefit of the program is less than than the cost then obviously we reject it. So it, that, it enables us to boil down the decision in relation to funding or not funding a program on the basis of a simple comparison of the benefit versus the cost, and if the benefits lower than the cost we we reject and if benefits higher than the cost we accept. Often it's presented as ratio so you may have heard people refer to benefit cost ratios and if the obviously if the benefit cost ratio is greater than one, it means we should accept and fund that program.
So I'll just go back to that question I raised earlier - how do we value benefits in dollar terms? Well there are two ways in which we can do it. One is to value the benefits of health care programs using what's known as a human capital approach. That means valuing human life or survival in terms of the productivity of the person whose life you're valuing. So if you're for instance save 10 years of life as a result of the intervention, that might translate into 10 years of productive capability for that person. So 10 years in the workforce, we then value that that benefit in terms of the wage that that person would have earned in that 10 years. So that's a fairly crude way of measuring the benefit of a healthcare program but it is an approach that that has been used in the literature.
Another way of valuing benefits is to use what's known as a willingness to pay, and that's essentially involves asking consumers or the users of the program, how much they're willing to pay for it. So for instance if we have a new program that's proved potentially going to prevent road accidents and people, and reduce the risk of death for people who may benefit from it. Well one way of valuing that program may be to ask users of that program how much they're willing to pay for instance in extra taxes per year. And and that's essentially a way in which we can ask people to value a health care or a health sector product, much in the same way as we value things in the marketplace. So the the rationale behind this is that the value of goods and services that we transact on a day-to-day basis in the marketplace is essentially based on how much people are willing to pay, how much consumers. So we're now sort of transposing that principle to health care setting and asking healthcare users how much they're willing to pay in order to gain evaluation of that particular product.
So they're the ways in which we conduct an economic evaluation. So I've gone through the four main approaches: cost minimization analysis, cost-effectiveness analysis, cost utility analysis and cost-effectiveness analysis. And as I mentioned cost-effectiveness analysis is probably the most commonly used in the healthcare setting.
I'll now just go through a few slides just to explain how the findings of a cost-effectiveness analysis are presented and how they might be interpreted. So we have once again an example of treatment A versus treatment B. Treatment A costs $800 and increases life expectancy by one year, treatment B costs $500 and increases life expectancy by half a year. So in this instance we could probably assume fairly safely that treatment B is probably the status quo option, the standard care. The usual practice, option, treatment A is probably the new intervention so when we're doing cost-effectiveness analysis we generally do it in relation to programs that may cost more but have a claim to greater effectiveness and that's the case in this example so this new program costs a bit more $300 more than the old program but has an advantage of increasing life expectancy by an extra half a year. So we can plot that we can plot that in this diagram here treatment B improves life expectancies by half a year, treatment A improves life expectancy by one year but A cost $300 more. Another way of plotting that is on what's known as an incremental cost-effectiveness plane, so you can see here that the intervention that we're looking at is more effective but also more costly, so it belongs what on is it belongs on what's known as the northeast quadrant, so that's where that red dot is. And that represents a situation where the new treatment is $300 more costly than the existing treatment but is half a year or half a year of life, in terms of effectiveness, more effective. So we're, it's costing more but we're gaining more from the program.
There are programs that sometimes are, new treatments sometimes more effective and less costly, in those instances as a decision maker we have a fairly sort of simple choice. So if the new program is is saving money and more effective than what we have is a situation where the intervention sits on the southeast quadrant and the new treatment dominates the old likewise or conversely if we have a program a new program that is is more costly and less effective then it's being dominated by the existing treatment in that instance, the new program belongs on the northwest quadrant. Finally we have the situation the southwest quadrant where the new treatment is less costly but less effective and it's this is probably a situation that doesn't occur very often in a high-income country, sitting where we're considering a new treatment that's actually less effective than the old treatment but we would potentially fund because it simply saves money. It might be a live policy option in much more resource-constrained settings but generally speaking you know high-income country settings it'd be difficult politically for a government or a healthcare funded to invest in a new treatment that's less effective purely in order to save money.
So the way we can put the example that I gave you of this new program in terms of costing $800 versus $500 and saving one year of life as opposed to half year in life can be now presented as an incremental cost-effectiveness ratio, which is essentially the the punchline in an economic evaluation. That the summary statistic, that the final sort of number that we generate when we're doing an economic evaluation. And essentially in this example what we come up with is an incremental cost-effectiveness ratio of six hundred dollars per life year gain. So we have an icer of six hundred dollars per life year gained. The question that that poses is is whether it's cost-effective, so from a government's point of view is it value for money and it's something that we should be investing in. Thankfully there are thresholds that are available, that governments can use as a benchmark for determining whether various ices are cost representing cost-effectiveness. So the the UK has published a cost-effectiveness threshold through the National Institute for Clinical Excellence and that's at 30,000 pounds of quality adjusted life year gained.
So what that means is that if we come up with an icer so that's less than 30,000 pounds it would be considered cost-effective anything above 30,000 pounds would be considered cost-effective. In Australia we we we don't have an explicit threshold, we have something that's been implicitly derived from decisions that have been previously made by government, and generally speaking the threshold falls in the vicinity of 50 to 70 thousand dollars per quality-adjusted life year gained. So anything below that it's cost-effective, anything above that it's generally not cost-effective. WHO has threshold as well. It's published and it's generally been used in context of low and middle income countries and it's said that three times per capita gross national income. The point of all these thresholds is that gives decision makers and people who read these cost-effectiveness analyses some sort of benchmark to determin whether an icer represents value for money or not. The problem however is that a lot of these thresholds are being arbitrarily set, but they've been used and basically take on a life of their own. So the initial setting of these thresholds has been pretty arbitrary, they take on a life of their own and everybody starts using them and they continually then get cited.
So just a very quick example of how a cost-effectiveness finding is presented in the literature. So this is a study that I was involved in a couple of years ago that evaluated the cost-effectiveness of a text message messaging program for the prevention of recurrent cardiovascular events. So basically it involved sending out text messages to people who have been discharged from hospital following hospitalisation for a heart condition and the text messages were generally providing information and reassurance about lifestyle modifications, so advice about smoking cessation, exercise and diet. And the trial that was done showed that the intervention was very effective in terms of reducing risk factors for cardiovascular disease, particularly blood pressure and smoking, and cholesterol levels. So we used that data, we conducted an economic evaluation and we modelled the impact if the intervention was being rolled out to a target population of 50,000 patients. And we predicted that in that 50,000 patients we would see 563 fewer myocardial infarctions, 361 fewer strokes, 100 1143 additional quality adjusted life years, and an overall saving to the health system of 10.5 million dollars. So in effect what we found was that the intervention not only was health improving, in terms of all those outcomes, but also saved the healthcare system money. So what it meant was that if we refer back to the slide earlier showing the incremental cost-effectiveness plan, we had a new intervention that was sitting in the southeast quadrant - it was both less costly, because it saved money, and more effective than the the status quo. So that was the findings of a economic evaluation which showed result which was we're pretty emphatic, but it indicates a way in which we can sort of present the findings of our economic evaluation in the context of that cost-effectiveness plane that I showed you earlier.
So in conclusion, just the main points that arise from the presentation. Economic evaluation is a tool essentially for guiding the way in which we allocate resources within the health sector. There are four types of economic evaluation: cost minimization analysis, cost-effectiveness analysis, cost utility analysis and cost-benefit analysis. When we're interpreting and presenting cost-effectiveness analyses, cost-effectiveness thresholds can be useful for determining whether the findings of our study, interpreting the findings of our study in determining whether the intervention is cost-effective or not. Now the problem with these thresholds is that they generally arbitrarily sit, but nonetheless they're probably that they're widely used and in a sense and take on a life of their own. So yes, so that's essentially the economic evaluation. Thank you.
