Site downtime

The site will be down for about half an hour after 10pm on the 26th of March for those nice people at Mythic Beasts to do some essential work on the server. Sorry!

Making exception: 3 - Is analysis possible?

Scotland and England have both now published some data on exception reporting. It is not co-incidence that these two countries published the data as both use the QMAS software. For the second year of running this collected data on exception reporting from practices. Indeed on the QMAS website practices and PCTs could see the breakdown of exceptions by reason. This could be compared with the national average.

Now this was a fairly positive development. Where people can see what others are doing they tend to fall into line. I have always been of the opinion that exceptions levels should be unexceptional. However in the published data England does not get down to practice level and Scotland does not break down the reasons for exceptions. There are a couple of reasons for this, one being that each patient can only be included in one exemption e.g. a new dissenting patient would only classify as one of these. This could be dealt with at the analysis stage an at least produce comparable results if it were not for a more significant problem. The English document states

The testing of patient exceptions on national QOF systems (such as QMAS) is primarily focused on ensuring that data values used for achievement calculations are accurate for payment purposes. Therefore any testing of the order of sequencing (ie the order whereby Different GP clinical information systems may follow different sequencing without this impacting on payment accuracy.

To translate into English this is simply to state that the method of deciding which exception applies was not actually tested on systems deployed to GP surgeries. Different computer systems may work this out differently. There is no way of checking as there is no set of business rules published for exception finding.

This hits plans for looking at individual practices quite hard. It become impossible to see whether the exceptions for an individual practice are entirely down to rapid practice turnover or mass patient dissent.

Analysis is still possible though. The Scottish data goes down to practice level and gives figures for exemptions and exceptions. Exemptions are simply those on the register to whom a particular criteria does not apply. An example would be a non smoker who would be exempt from smoking cessation advice. In theory the denominator of an indicator plus the exceptions plus the exemptions should add up to the register size. In practice it doesn't exactly due the the difference in the dates they are measured but it does get there roughly!

I expect we will see more advance analysis of the Scottish data in the coming weeks and months but it is certainly possible to identify practices at either end of the exception spectrum. Being out of ordinary does not automatically mean bad though.

In publishing the English data there is not a practice breakdown, but rather look at the individual indicators. Unsurprisingly there is more exception reporting achievement indicators than with monitoring one. There are, of course, more possible exceptions in these areas. Top of the list for exceptions is the use of beta blockers in CHD. Simply they are contraindicated in asthma, COPD and peripheral vascular disease - all more common in patient with CHD. Next at 18.8% was flu jabs in asthma, probably reflecting guidance from the chief medical officer that it was not indicated in large numbers of asthmatics. Epilepsy 4 at 16.8% reflects the fact that it is not always possible to completely control epilepsy no matter how many drugs you can persuade the patient to take. The rest of the top ten is more about flu jabs and getting to target.

In short there does not seem to be any evidence of systematic manipulation of exception reporting. More than that is difficult to say, other than the whole of the exception data is much less exciting than many people hoped, or possible feared!

Making exception: 2 - How?

Last week I looked at the reasons for exception reporting. In this entry I will go into some detail about how exception reporting actually works in practice. In particular how the business rules work out the exceptions and how practices decide what codes to enter.

When it comes to the business rules the exceptions fall into three main groups. Firstly there patients who are recently registered with the practice or recently diagnosed are automatically excepted. There is no need for practice intervention in this - the number of potential exceptions are simply dependant on the practice turnover and the number of new diagnoses. The length of the exemption is three months for most 'process' areas (e.g. blood pressure measurement) and nine months for 'outcome' measures (e.g. blood pressure below 150/90)

Secondly are the exceptions which apply to a whole domain. These are generally speaking due to reasons of patient dissent or unsuitability (e.g. hypertension in the terminally ill). Patient dissent is taken as being either actively expressed or a failure to respond to three invitations to review.

Thirdly exceptions may apply to a specific indicator. Patients may decline to have a flu jab or be allergic to a particular drug. Alternatively they may be on the maximum possible dose of treatment drugs and there simply is no further treatment.

To add to the complication each of the exceptions only count if the target is missed. If the patient subsequently makes the target in an area then the exception is ignored in that area. Thus a new patient will only be exempted from a target about having blood pressure measured until they actually have it measured or three months, whichever comes sooner.

In actually applying the codes there are further complications around whether the codes need to be repeatedly entered each year or not, but the above explanation should be enough to understand the basic process.

The latter two types of exception are controlled by the insertion of Read codes by the practice. Now it would be nice to think that the practice sat down in April and worked out who would be inappropriate to test or treat and entered the codes appropriately. They might invite all their patients to the surgery for review and code those who declined.

In reality, of course, it doesn't work like that. Most GPs don't particularly enjoy exception coding - it somehow feels like failure. Well it certainly doesn't in my surgery or any that I know of. Explicit dissent is recorded throughout the year until about January time then the figures are looked at closely. It is then that unsuitable patients are coded and the letters sent out. If the maximum threshold is crossed then we can all relax and stop exception coding.

So much for anecdote, but is there any sign that this is happening over a wide area? The answer is yes, at least in Brighton. A study there showed everyone getting much the same level of achievement in the areas that they looked at. There was a difference in that deprived areas had a much higher level of exception reporting. This could be interpreted as an increased level of exception reporting in reaction to targets being more difficult to reach. The alternative, and less politically correct interpretation, would be that patient in deprived areas are more resistant to treatment.

In the model presented here the two drivers to exception reporting are thus the practice list turnover and the practice's desire to seek out codes - the latter may be driven by likely achievement levels. There is also likely to be a direct population consent effect similar to that we see with immunisation uptake around the country.

Next time I will look a the currently published data and, using what we have explored so far, look at how they can be analysed. I will also look at what level of detail we can look at.

Making exception: 1 - Why?

Exceptions have to be one of the most contentious issues in the QOF. Considered essential to many practices and built into the very fabric of the QOF. However it seems that few people other than GPs actually like them.

PCTs hate them as they have the vague and ultimately unprovable feeling that they may be being cheated. Statisticians hate them because it makes it very difficult to say what the real results are for the practice population. Certainly this latter argument misses the point somewhat. The point, of course being that they make the QOF rather saner than it would be otherwise.

This is not to say that there are not things that can be learnt through the exception reporting and it is those issues that I will explore over a series of articles. The actual nitty gritty of dealing with them I will leave to another day and for the moment concentrate on the question of "What are they for?

1. Patients who have been recorded as refusing to attend review who have been invited on at least three occasions during the preceding twelve months.
2. Patients for whom it is not appropriate to review the chronic disease parameters due to particular circumstances e.g. terminal illness, extreme frailty.
3. Patients newly diagnosed within the practice or who have recently registered with the practice, who should have measurements made within three months and delivery of clinical standards within nine months e.g. blood pressure or cholesterol measurements within target levels.
4. Patients who are on maximum tolerated doses of medication whose levels remain sub-optimal.
5. Patients for whom prescribing a medication is not clinically appropriate e.g. those who have an allergy, another contraindication or have experienced an adverse reaction.
6. Where a patient has not tolerated medication.
7. Where a patient does not agree to investigation or treatment (informed dissent), and this has been recorded in their medical records.
8. Where the patient has a supervening condition which makes treatment of their condition inappropriate eg cholesterol reduction where the patient has liver disease.
9. Where an investigative service or secondary care service is unavailable.

The current criteria are listed in the box. Their actual number seem to vary from source to source but this is more about layout than content. What is quite apparent is that they are designed to keep the QOF relevant. Some are about not penalising practices for patients informed decisions, something that had controversially not been included in the childhood vaccination targets. In a similar vein other exceptions are there to make sure that GPs are not encouraged to give treatments that are inappropriate or even harmfull. Finally some of the exceptions allow some time for the number to be produced after diagnosis or registration.

All of these codes provide a valuable services to prevent inappropriate care being incentives. There have been some calls for the abolition of exception codes though. There are some who would argue for the abolition of the codes, sometimes arguing that the fact that the points only score up to an achievement of 90% or less does the same job. The reality is that this latter mechanism is a blunt instrument, unresponsive to local circumstances. If anything it is these top thresholds that should be abolished with a continuation of the scoring up to 100%. It is a bizarre system that encourages clinicians to get to 90% and then stop.

It is however very clear that GPs are not stopping at the upper thresholds. Most of the achievement on this site is well over these thresholds. Exception reporting is essential in removing undue pressure on patients to conform to the medical model. It is probably the easiest target of cheap shots against QOF but the alternatives are likely contain more perverse incentives.

Having made the case for their existence, next time I will look at how they are implemented in practice by both practices and the business rules.

Exception reporting data for Scotland

ISD Scotland has published some of the exception reporting data for Scotland. Once again they are ahead of the rest of the UK. It is, as they point out as frequently as they can, difficult data to use in any sensible way.

Over the next week or so I am planning a series of short articles about exception reporting and how it may be interpreted.