Making exception: 3 - Is analysis possible?

Scotland and England have both now published some data on exception reporting. It is not co-incidence that these two countries published the data as both use the QMAS software. For the second year of running this collected data on exception reporting from practices. Indeed on the QMAS website practices and PCTs could see the breakdown of exceptions by reason. This could be compared with the national average.

Now this was a fairly positive development. Where people can see what others are doing they tend to fall into line. I have always been of the opinion that exceptions levels should be unexceptional. However in the published data England does not get down to practice level and Scotland does not break down the reasons for exceptions. There are a couple of reasons for this, one being that each patient can only be included in one exemption e.g. a new dissenting patient would only classify as one of these. This could be dealt with at the analysis stage an at least produce comparable results if it were not for a more significant problem. The English document states

The testing of patient exceptions on national QOF systems (such as QMAS) is primarily focused on ensuring that data values used for achievement calculations are accurate for payment purposes. Therefore any testing of the order of sequencing (ie the order whereby Different GP clinical information systems may follow different sequencing without this impacting on payment accuracy.

To translate into English this is simply to state that the method of deciding which exception applies was not actually tested on systems deployed to GP surgeries. Different computer systems may work this out differently. There is no way of checking as there is no set of business rules published for exception finding.

This hits plans for looking at individual practices quite hard. It become impossible to see whether the exceptions for an individual practice are entirely down to rapid practice turnover or mass patient dissent.

Analysis is still possible though. The Scottish data goes down to practice level and gives figures for exemptions and exceptions. Exemptions are simply those on the register to whom a particular criteria does not apply. An example would be a non smoker who would be exempt from smoking cessation advice. In theory the denominator of an indicator plus the exceptions plus the exemptions should add up to the register size. In practice it doesn't exactly due the the difference in the dates they are measured but it does get there roughly!

I expect we will see more advance analysis of the Scottish data in the coming weeks and months but it is certainly possible to identify practices at either end of the exception spectrum. Being out of ordinary does not automatically mean bad though.

In publishing the English data there is not a practice breakdown, but rather look at the individual indicators. Unsurprisingly there is more exception reporting achievement indicators than with monitoring one. There are, of course, more possible exceptions in these areas. Top of the list for exceptions is the use of beta blockers in CHD. Simply they are contraindicated in asthma, COPD and peripheral vascular disease - all more common in patient with CHD. Next at 18.8% was flu jabs in asthma, probably reflecting guidance from the chief medical officer that it was not indicated in large numbers of asthmatics. Epilepsy 4 at 16.8% reflects the fact that it is not always possible to completely control epilepsy no matter how many drugs you can persuade the patient to take. The rest of the top ten is more about flu jabs and getting to target.

In short there does not seem to be any evidence of systematic manipulation of exception reporting. More than that is difficult to say, other than the whole of the exception data is much less exciting than many people hoped, or possible feared!