2006/7 Data Publication Dates

I do get asked quite a bit when the new data is due to come onto the site. Well all the data comes from the various departments of health in the four countries. The current plans from England and Scotland are to release the data sometime in September. Something of a relief for me, at least, as I have to get all of the new data into the database.

As an aside the English GP patient survey 2007 has been released. As this is down to practice level and is in a reasonably friendly format I will try to put this onto the site in addition. It also includes interesting figures such as rurality (really horrible word!) and deprivation factors. For largely presentation reasons however this is unlikely to precede the full QOF data (it will be linked from the 2006/7 QOF data).

Gaming, and report writing

A few weeks ago the Centre for Health Economics at York University produced a report looking at some of the statistics in QOF. It looks in some detail at both disease prevalence and to some degree at exception reporting. They are particularly interested in the difference in behaviour between high scoring practice and lower scoring ones, although they also look at social and societal differences between practices.

They only looked at Scottish practices due to the rather better data that was available for them, which has got to be a pat on the back for ISD Scotland.

I won't go into detail about the mechanics of the analysis - you can read it yourself although I would warn you that some knowledge of statistics is needed. It is not a light read. health economics papers rarely are. Most of the really interesting findings are related to the differences between 2005 and 2006 in practices that did, and did not, get maximum points in a given area.

The results are interesting. In general terms those practices who hit the top indicator thresholds in the first year increased their prevalences in the second year relative to those practices which did not. Conversely those practices who did not reach the top thresholds tended to increase the amount of exception reporting they did.

Now there is probably nothing too surprising in that. It would be a rather worrying situation for an incentive scheme not to lead to changes in behaviour in the direction of the incentive. That is exactly what is happening here. Practices are tending to most work in the areas that lead to the greatest incentive. There are certainly issues with the underdiagnosis of chronic diseases and there are probably many people who could be exception reported and are not.

The report talks a lot about "gaming". It does not define this however and I struggle to find a good definition on the Internet. Perhaps the most benign definition would be, in this context "undertaking actions to increase revenue that would not improve patient care". Actually this would encompass all exception reporting. This is not a bad definition as they define altruism as precisely the converse (personally I think that is professionalism but lets not get bogged down in semantics)

The authors of the report do not look so kindly on gaming. They define it thus:

However, exception reporting also gives GPs the opportunity to exclude patients who should in fact be treated in order to achieve higher financial rewards. This is inappropriate use of exception reporting or "gaming".

You can see where we are going here, can't you? By page 15 they are just calling it cheating.

That is not to say that I disagree with their mathematical analysis. I actually think it is rather brilliant and represents an attempt to model QOF mathematically in a way that has not been seen before - in public at least.

However they fall over in the conclusions. They cannot see any reason for these variations except cheating and dishonesty. Now that is one possible explanation for their findings but it is not the only one by any means. They seem to have very little idea of how exceptions are actually used. They don't see practices a living organisations with priorities. If you incentivise them to look for more patients they will find them - there certainly seem to be plenty undiagnosed with diabetes and hypertension. If they are going to get extra cash for a more efficient exception reporting system then they are likely to do that. It could simply be an indication of priorities.

None of this needs dishonest exception reporting or fraudulent diagnosis, simply an understanding of where the statics come from. So are GPs cheating lying scoundrels? We some might be but there is no solid evidence of this on a large scale. It is reassuring (as a GP) to read their first conclusion.

The fact that practices could have treated substantially fewer patients (12.5%) without falling below the upper thresholds for indicators and thereby reducing practice revenue is compatible with altruistic motivation.

Not so bad after all!

Encouraging news from Leaminton Spa

Readers outside the Leamington Spa area may not have seen this article in the local paper giving the rather good news that more practices than last year had gained maximum points (1000 this year, 1050 last year.

This is a little surprising as there are certainly more targets to reach this year. If this were repeated across the country it would certainly be a major achievement for practices.

Full details will be released by the NHS in September. Last years details are on this site. In the meantime congratulations to practices in Warwickshire.

QOF reduces admissions - or does it?

I like to be positive here. It is nice to find positive things about the QOF. I was very interested to see reports that higher QOF scores in asthma were associated to a reduction in emergency asthma admissions. Good news - or was it?

The original report (1.7M pdf) was produced by Asthma UK. The report, to be fair, is a glossy affair putting a political message rather than a scientific paper. There are virtually no figures, although some, partially processed, have been put in a couple of appendices. There are some graphs but even these do not seem to support some of the conclusions given.

There is undoubtedly a great variation in the number of emergency admissions with asthma. The greatest factor appears to be latitude with the number going up as you go north and pages six and seven make this clear. So far, so good. There is then a brief pause for a full page photograph of a nurse clinging to a bag and mask and a name and shame list for PCTs. The high admitters tend to be city PCTs and the lower admitters leafy southern PCTs, a fact not commented on. The next page is titled "Why the Divide?". It starts with the sentence The difference in hospital admissions across England is unlikely to reflect differences in the number of people with asthma.. Asthma UK appears to be saying, without offering any real justification that the number of people admitted with asthma is unrelated to the number of people with asthma. Intuitively it seems incredible and unfortunately no evidence if given to back up this bold statement. In fact it is printed above a graph showing pretty much the opposite.

Lastly we get to the correlation with QOF points. There certainly seems to be a weak correlation between QOF score and asthma admissions in 2004/6 - the first year of QOF. This may be something of an underestimate as they use QOF score rather than total QOF achievement. Why should that make a difference? Well QOF scores are capped at 70%. Any extra achievement above this is not counted. In 2004/5 over a third of practices got every single point in the asthma section of QOF. The extra achievement of these practices has been thrown away in the analysis.

In any event all that we can say is there is a correlation. Cause and effect is impossible to suggest without at least some data from previous years.

I would love to see some data that QOF is making a difference. I was disappointed that this report shows little other than a large variation in asthma admission around the country. It does not answer the questions of why half as well as proper peer reviewed study (no mention of QOF though!).

Prevalence data for England and Northern Ireland

Prevalence data is starting to get out! In the table below you can see the data from England and Northern Ireland. The English data was taken from QMAS at the start of April and the NI data from their official prevalence bulletin. I would recommend the NI bulletin for further reading as there are a lot of nice charts showing the spread of the prevalence. When comparing the data with previous years it is important to remember that there have been big rule changes in mental health and smaller one in LVD. Also of note is that the palliative care prevalence is for information only and does not change the cash value of points as the others do.

There are couple of figures in the NI bulletin I don't understand - mainly the depression 2 and LVD 3 listings. I can't quite see the relevance but I will ask!

Prevalence Area	England	Northern Ireland
CHD	3.551	4.196
LVD	0.790	0.818
Stroke	1.615	1.619
Hypertension	12.466	11.651
Diabetes	3.629	3.138
COPD	1.425	15.33
Epilepsy	0.590	0.745
Thyroid	2.490	2.872
Cancer	0.897	0.778
Palliative Care	0.087	0.090
Mental Health	0.716	0.753
Asthma	5.771	5.78
Dementia	0.400	0.526
Depression*	7.004	6.5
Kidney Disease	2.242	2.307
Atrial Fibrillation	1.295	1.252
Obesity	7.223	7.989
Learning disabilities	0.256	0.316
Smoking - for recording**	19.557	18.55

* I am not sure exactly what this depression figure means. I think it is the number of people eligible for depression screening.

** This is the number of people eligible to be asked regularly about their smoking. It is the combined prevalence of diabetes, hypertension, heart disease, COPD and stroke

Measuring Prevalence

One of the questions often asked by GPs about QOF data is what their prevalence data should be. There are three ways to measure the prevalence of a condition. First you can ask doctors, second ask patients and thirdly you can get out there and thoroughly examine a random group of people.

The data from QOF on this site is definitely in the first camp. I have come across some interesting disease prevalence models which try to compare QOF data against data from the Health Survey for England which is largely, as the name suggests a survey of the asking patients variety. It does however features some objective measurement by nurses as well.

There is only analysis for heart disease and hypertension. In heart disease there is a small reduction in prevalence in QOF compared to the HSE estimation. This is probably down to a lack of coding. The differences in hypertension prevalence are much larger with the HSE prevalence over double the QOF prevalence.

Now I have to admit that I boggled at this for a while. Could it really be that a quarter of all my patients had hypertension? Well the answer by strict interpretation seems to be "Yes". In fact the data, including the difference between the diagnosed and the actively treated has been observed for some time.

Now I don't propose to go through the rights and wrongs of this but the fact remains that prevalence varies widely possibly predictably depending who you ask. We don't have the official figures for this year's prevalence but kidney disease seems certain to come in well under expectations.

So before making comparisons make sure that data is all coming from the same sources. We await the official figures (traditionally Wales has been early with them but not this year).

Waiting for the Data

So far this month the data has been submitted to QMAS - which stumbled and recovered a couple of days later. PCTs should have received the final data and I have had a short holiday.

I came back from my holiday to an email asking about the status of the data currently showing on QMAS. This is not available to the public as yet as it has not been finalised. We can expect the final data, for England at least, around September. I can see from emails and server logs however that many users of this site work in the NHS itself and so may have access to the QMAS data. Can it be relied upon?

It is useful to know the stages the data passes through. Data from practices should have been signed off on QMAS by now. The signing of process signifies the agreement with the submitted data by the practice and the guidance describes it as a legal declaration that the submission is accurate. In the vast majority of cases this will lead directly to payment without further changes, often within a few weeks.

In some cases there is disagreement from the PCT, most often querying a claim. This normally relates to some disagreement over the interpretation of an indicator although occasionally it may be used to customise QOF to some local and special circumstance - particularly in the clinical areas where the data is automatically extracted. It is this process which must be completed by June, although it is rare that it takes this long.

In any case PCTs should be aware by now of any disputed figures. If you are working in the PCT it is hopefully a simple matter to find out who is dealing with these disputes, make them a coffee and ask if they are aware of any possible changes to the data. If there is no dispute the data can probably be considered as final.

The rest of us will just have to wait.