Bringing QRISK to Facebook

This has nothing to do with QOF. Earlier this year the the good people at QResearch put the QRisk2 formula onto the internet with an open licence. This is freely available to download and use. This is quite something. There have always been problems with trying to apply JBS formulae as, for most people, the data to input is not there. Qrisk2 is based in large part on information that is easily available and the QResearch group provide formula that can estimate some of the other data.

I have put the formula onto Facebook with a fairly simple form. There is a limit to how simple it can be but hopefully it is at least usable.

The results are presented again as simply as seems reasonable. One of the things that can be done with the formula is some "what if" calculations. We can see the effects of stopping smoking or losing weight. There is even data drawn from NHS Choices which gives the closest NHS Smoking Cessation Clinics if you are in England, although this is not currently available due to a bug at NHS Choices.

The app is at http://apps.facebook.com/cvdrisk . Feel free to have a play around. You can change your data as often as you wish and see the effects. All your data is deleted when (or hopefully if) you remove the app from your Facebook account. Facebook will post that you are using the app but none of your data is posted back to Facebook unless you specifically type it in. You can post reviews or questions on the Facebook profile page.

Ooops

Many thanks to the reader who pointed out that, in England, I had transposed the two depression prevalence figures. I have now corrected this on the site. If you have downloaded the data there is now an updated version on the download page.

If you don't fancy downloading it all again you can switch in Access with some queries. I have not worked how to export these. If you are using MySQL the code is

update `achievement` set area='' WHERE `practiceid`regexp '^[ABCDEFGHIJKMLMNOPQRSTUVXYZ][0-9][0-9][0-9][0-9][0-9]$' and area ='DEP Prev 1';
update `achievement` set area='Dep Prev 1' WHERE `practiceid`regexp '^[ABCDEFGHIJKMLMNOPQRSTUVXYZ][0-9][0-9][0-9][0-9][0-9]$' and area ='DEP Prev 2';
update `achievement` set area='Dep Prev 2' WHERE area ='';
update `pcoach` set area='' WHERE `pco`regexp '5[A-Z][A-Z]' and area ='DEP Prev 1';
update `pcoach` set area='DEP PREV 1' WHERE `pco`regexp '5[A-Z][A-Z]' and area ='DEP Prev 2';
update `pcoach` set area='DEP PREV 2' WHERE area ='';

I had also missed the reconfiguration of the PCT in Hertfordshire which is now corrected in the downloads also (there is not a simple patch for that).

Sorry!

CPD+ on QOF Database

I tend to have a few ideas for projects floating around at a time, most of which come to nothing. One of these was an improved method for saving all of the stuff that I read for appraisal. At the time all that was out there was the official Appraisal Toolkit that was too horrible to use. Thankfully this is, if not quite dead, certainly having the curtains pulled around it.

I never actually managed more than about 20 lines of code as a proof of concept on this one but am very excited to find that those good folks at Healthcarerepublic have actually put finger to keyboard and written something very much along the same lines with their CPD+ system. It is also free, which is nice.

It is pretty basic at the moment but crucially it does allow one click to enter data into the log. There are instructions on the CPD + site but to make things even easier you can click on the CPD+ logo after every blog post at at the bottom of every table on the site. There is also a button at the bottom of each blog posting which is a stupidly complex thing to actually get into a Blogger template. Feel free to use the code yourselves.

I hope you find this useful.

2010 QOF data now online

I am delighted to be able to say that the data for 2010 is now available on the site. This is a little later than usual this year. This is a little due to a later publication of the data by the Information Centre and rather more due to my hard drive crashing. I still had the data but getting the system set up around it took a bit of time. I talk a little about that process on E-Health-Insider.

There is still a bit of work to do and I will get the database download going in the next few days.

More business rules strangeness - PP1

My thanks to Dr David Fitzsimons who has pointed out that it is not only depression that has suffered a ruleset that differs from the guidance. PP1 - the assessment of CVD risk in patients newly diagnosed with high blood pressure - features an almost identical change. Whilst the guidance suggests only the patients diagnosed in the past year should be counted the business rules carry forward all of the unassessed patients from the previous year. There is not a lot that a practice can do to avoid this, unless of course their hypertension has resolve (read code 212K ) in the meantime.

Depression is not what it seems

This week have been looking at the QOF progress in my own practice and I noticed that the depression figures seemed a little odd. The number of patients needing a second depression assessment (Dep 3) was rather larger than the number of patients needing the first assessement (Dep 2). According to the guidance the former should be a subset of the latter.

Just a reminder of what the indicator says

DEP 3: In those patients with a new diagnosis of depression and assessment of severity recorded between the preceding 1 April to 31 March, the percentage of patients who have had a further assessment of severity 5 – 12 weeks (inclusive) after the initial recording of the assessment of severity. Both assessments should be completed using an assessment tool validated for use in primary care.

Indeed the guidance goes on to say:-

New diagnoses are those which have been made between the preceding 1 April to 31 March.

Clear? There is a slightly odd consequence to this. FIt is difficult to count the need for assessment towards the end of the year. With a first assessment in January the second could be due after all of the data is sent to QMAS. This would tend to work in a practice's favour.

This was not what I actually saw on my practice figures. I looked at the latest version of the business rules (warning - highly geeky) and it seemed all was not well. These are the rules that govern the data that is sent to QMAS for payment and the data that ultimately appears on the QOF Database web site. This year's rule was looking way back past 1st April - back to the start of December 2009 looking for diagnoses.

I got in touch with the NHS Information Centre who are the latest organisation to be in charge of the business rules. It turns out that there is no mistake. The rules were set, in consultation with NHS Employers, to try to fix the issue described earlier. In the process the rules now contradict the wording of the indicator and the guidance.

So what do the rules now say? The "yes" and "no" groups have split slightly so it is worth specifying the numerator and denominator groups separately.

Numerator: Patients who have had a second assessment from the previous 1st April to 31st March. The second assessment must be between 5 and 12 weeks after the first and the first, in turn must be within four weeks after the diagnostic code.

Now the denominator :-

Denominator: Patients in the numerator (i.e. Yes)
and also patients for whom the date of 12 weeks following their first assessment is within the previous 1st April -31 March and who have not had that second assessment (i.e. No)

Those who you failed to get at the end of one year are carried over to the next. The successes are counted in the same year.

What does irk me is not the fact that this was changed - indicators are changed and improved over time - but that this alteration from the original meaning has never been announced anywhere other than the business rules. The description of the indicator, although clearly now misleading, has not been altered. Similarly the guidance remains inaccurate. The business rules are hardly the first port of call for a busy GP.

What's On the Menu

NICE has published its menu of potential indicators. These indicators will then go to the negotiators for the decision of what goes into the QOF for 2011-12.

There is not a lot of "wow" here. There majority of changes reflect a tidying up of the current indicators with some made more explicit. There are no big ideas or even a unifying style to the indicators. Let's see what is ahead.

NM07: The percentage of patients with a history of myocardial infarction from 1 April 2011 currently treated with an ACE inhibitor (or ARB if ACE intolerant), aspirin or an alternative anti-platelet therapy, beta-blocker and statin (unless a contraindication or side effects are recorded)

A big wrap up here. CHD 9, 10 and 11 are combined with a requirement for a statin, regardless of cholesterol level. In fact CHD 7, the requirement to test cholesterol is also scheduled to go - the outcome measure remains. Actually this is just an assumption as the actual notes only suggest the replacement of CHD 11, but they must mean 9 and 10 as well, surely?

Personally I don't like these big indicators - they end up with horribly complex business rules, particularly when exception reporting comes in to play. Other than being awkward there is not much new here.

NM09: The percentage of patients with a new diagnosis of dementia from 1 April 2011 to have FBC, calcium, glucose, renal and liver function, thyroid function tests, serum vitamin B12 and folate levels recorded 6 months before or after entering on to the register

My least favourite form of indicator I'm afraid. Miss out in the first six months of diagnosis and you are stuck with that for ages. Depending where they put the threshold a slow start could affect results for years to come. Similarly success will result in several years payments. Much more sensible would be a new diagnosis within the year studied.

NM12: The percentage of patients with diabetes with a record of testing of foot sensation using a 10 g monofilament or vibration (using biothesiometer or calibrated tuning fork), within the preceding 15 months

A tidy up of the wording of DM 10. Nothing new.

NM13: The percentage of patients with diabetes with a record of a foot examination and risk classification: 1) low risk (normal sensation, palpable pulses), 2) increased risk (neuropathy or absent pulses), 3) high risk (neuropathy or absent pulses plus deformity or skin changes or previous ulcer) or 4) ulcerated foot within the preceding 15 months

DM 9 tidied with the risk classification added. Minor changes only.

NM15: The percentage of patients with schizophrenia, bipolar affective disorder and other psychoses who have a record of alcohol consumption in the preceding 15 months

I suspect that the wording here is misleading and they are not going to insist on the mentally ill having a couple of beers. They seem actually mean a record of the amount of alcohol consumed. This is the start of an extensive mental health section.

NM16: The percentage of patients with schizophrenia, bipolar affective disorder and other psychoses who have a record of BMI in the preceding 15 months

There is little doubt that modern antipsychotics make you put on weight; indeed AstraZenica have just paid out on this. This is probably a large part of their potential to increase the risk of diabetes. I'm sure the guidance will explain what to do with the overweight in this group.

NM17: The percentage of patients with schizophrenia, bipolar affective disorder and other psychoses who have a record of blood pressure in the preceding 15 months

Currently MH9 is a general annual MOT for those on these registers. This includes a physical review. Most GPs do a blood pressure as part of this. This group of indicators replaceme MH9. Attentive readers may be wondering why NM07 combined various indicators and MH4 is being unbundled and whether there is any guiding strategy here. I know I am.

NM18: The percentage of patients aged 40 and over with schizophrenia, bipolar affective disorder and other psychoses who have a record of total cholesterol: hdl ratio in the preceding 15 months

I expect some changes to this one in negotiations most likely to the use of total cholesterol rather than the ratio. Certainly our local lab advise HDL measurement sparingly and only once per person. Currently we have only recently agreed for the lab to do HDL measurement when the total cholesterol is low for QOF purposes in new hypertensives. Expect objections from chemical pathologists. This would be a closer monitoring of cholesterol, and particularly HDL than for patients with heart disease, strokes or diabetes. The associated documents don't comment on this.

NM19: The percentage of patients aged 40 and over with schizophrenia, bipolar affective disorder and other psychoses who have a record of blood glucose level or HbA1c in the preceding 15 months

More checks on the mental health register. Worth doing for patients on antipsychotics but for patients with bipolar the evidence base is less clear.

NM20: The percentage of women aged 25-64 (in Scotland from 21 to 60) with schizophrenia, bipolar affective disorder and other psychoses who have a record of cervical screening within the last 5 years

NICE says they don't like double counting but these patients will also count for the main cervical screening indicators. There could be some quite small numbers here. The average number of patients on the mental health register is just over 48. If we assume roughly half are female and some of those are outside the age range then this indicator will apply to less than 20 patients per typical practice. Likely a lot fewer in some practices.

Percentage of patients on the Learning Disability register with Down's Syndrome aged 18 and over who have a record of blood TSH in the previous 15 months (excluding those who are on the thyroid disease register)

I certainly don't want to put down the importance of thyroid function to patients with Down's but I am left wondering how big the problem is at the population level. The economic analysis (which is fairly dire) suggests one in 1000 live births. Down's can be associated with heart problems and other issues which increase childhood mortality and even after that life expectancy, whilst rising, is still reduced. So lets say two thirds are over 18. That would be fewer than four patients in a typical practice and maybe one or even none at all for a small practice. Small numbers make bad indicators and is this really the best use of an indicator?

The percentage of patients with a new diagnosis of dementia from 1 April 2011 to have FBC, calcium, glucose, renal and liver function, thyroid function tests, serum vitamin B12 and folate levels recorded 6 months before or after entering on to the register.

Simple cheap tests. It is worth mentioning small numbers again. It would be worthwhile practices checking in January whether they have diagnosed someone with dementia in the previous year to get this indicator, if implented. If they have not then they would need to hunt someone out to get the points (tongue slightly in cheek, but only slightly).

The percentage of women with epilepsy under the age of 50 who are taking antiepileptic drugs who have a record of information and counselling about contraception, conception and pregnancy in the previous 15 months

Probably small number here again. Quite how otherwise intelligent women will feel about being told the same thing annually will be seen. There is no lower age limit stated here but the lower limit for the epilsepy register is 18, which is probably too late to start. In reality this will likely become part of the annual review.

And now some more brief points

• The lowest level of HbA1c target for diabetes has been increased from 7% to 7.5% although these are no expressed in mmol/mol.
• The depression assessment indicator changes wording from "the outset of treatment" to "the time of diagnosis". In fact it always was the latter in the business rule. No real change.
• The second assessment moves from 5-12 weeks later to 4-12 weeks later. Probably a little easier
• For patients on lithium the creatinine and TSH must now be measured after the first of July rather than in the previous 15 months.
• For these patients the lithium must now be in the correct range after the First of December rather than July. All blood tests are still only required once a year though, just in more limited months. Unlikely to have the desired effects.
• Risk assessment for new hypertensives has the wording brought up to date with the current practice of only looking at those 30-74. This has been in the business rules for a while.
• CHD 2 is to have the date updated to next April and the emphisis changed from excercise tolerance testing to clinical assessment.

The 5% rule

The square root formula for adjusting prevalences finished a year ago but we are still left with the 5% cut off for the year just gone. This was less well known than the square root formula but its effects could be rather larger. It seems to have become more of an issue for many practices recently - perhaps we are all looking at our budgets just that little bit more closely.

The basic rule is this. You find the practice with the highest prevalence for any given condition and then calculate 5% of that prevalence. Any practices below that 5% value have their prevalence moved up to that level. Simple? No, not really.

The problem is that there are a small number of practices out there that are quite exceptional in their prevalence. You can see the spread of prevalences in the boxplot below (2008-9 data). For those unfamiliar with a boxplot the middle 50% of practices are within the box. The whiskers spread out from this upto 1.5 times the size of the box. The really outlying practices are plotted separately.

As you can see there are high outliers in every area, some more than others. The 5% rule really starts to kick in when the highest outlier is more than 20 times the mean. When this happens more than half of the practices will be bunched together at the 5% level. Prevalence adjustsment can simply stop. Last year, for instance, only three practices in England had a dementia prevalence more than 5% of the maximum. This meaned every other practice received the same prevalence factor.

To illustrate this effect the results for one of these extreme practices below. I am not giving their name as who they are is not really the point. They provide services to a group of patients with significant needs and there is no reason at all to doubt their figures.

Under the current rules this one practice can significantly change the QOF payments to thousands of others. But how many are affected? Well we can use the database to see.

As you can see it is not only in dementia that the 5% rule affects the vast majority of the 8229 practices in England. Learning disabilities, stroke and mental health area all hugely affected and over half of practices are affected in the area of CKD. As a side note you may notice that learning disabilities, heart failure and epilepsy all have the same maximum. This is all down to a single, and highly unusual practice, although this time down to very small numbers of patients. The same practice is also responsible for the highest stroke prevalence. Another "special" practice has the highest rate of mental health problems, although fewer than 100 patient overall.

There is no blame to attach to these practices. They are providing services to often very difficult populations and there is no doubt that they are recording accurately. The problem is with the operation of the rule, now thankfully in its final year. Expect big changes in these areas next year.

Removing Indicators

There have been a couple of new pieces of research in the last week or so relating to the QOF. I am trying to track down a copy of the this month's BJGP but fortunately there is free access to research in the BMJ and the paper The impact of removing financial incentives from clinical quality indicators. Lester et al.

The paper looks at the removal of incentive payments in California and, at the risk of spoiling the end for you, finds that there is a decrease in achievement when the incentives are withdrawn. In fact this decline is continuous over the years so things get worse. Comparisons are drawn with the UK and QOF although there are differences. In the US the payments rarely affect the clinicians directly but rather their employer. There were other programmes associated with the incentive payments that could have made a difference. Things not mentioned in the paper are that the incentives tend to be higher in the UK as a proportion of funding. Additionally most of the targets incentivised were fairly uncontroversial (cervical screening, diabetes control) whilst there is much more scepticism amongst clinicians about some of the QOF targets.

In general though the paper is a pretty easy read everyone except possibly for the NICE QOF advisory committee. Of course we won't really know what happens in the UK until it actually happens and has a chance to work through the system. As the earliest indicators will be removed is the 2011/12 year we won't really know until after the Olympics. Until then this is our best clue.

NICE New Indicators Consultation

NICE has a current consultation about possible new QOF indicators. It is open until the 11th of April (a week after Easter) and is certainly worth a look.

I won't go through the whole thing here, if you are interested you can read my submission. I will admit I am a little disappointed in the general scope of the indicators. Some of the current indicators are joined together and some are split apart. There is little feeling of an overall strategy or direction and even less of how indicators fit into the whole QOF. There is not really enough information to give a proper response. They are knee to know about potential unintended consequences but these tend to appear at the business rule level and there are not even draft versions of these. The document reads more as a list of intentions than indicators ready to be used. They don't even included suggestions of point scores which should be part of the economic evaluations.

I suppose this may get sorted at the negotiation level but we probably won't know that for another nine months. We will also find out how much the DH and GPC think of the QOF Advisory committee's work.

Prevalence Predictions

I am grateful for an email pointing me to some work done to try to predict the prevalence of QOF related diseases. The prediction is based on the age, sex, ethnicity, deprivation and smoking status of a practice's population. Readers with long memories may remember something similar being done by the North East Health Observatory a few years ago. This new work, however, covers more disease areas and is rather easier to use as the predictions are now featured on NHS Comparators (that link only works on NHS computers, sorry)

The methodology is interesting as the "expected" prevalence is based on various household surveys rather than GP data. The national prevalences are all rather lower than the "expected" prevalences except for cancer. The high relative prevalence of cancer in the QOF data is likely due to the predictions being based on one year and QOF registers being based on several years.

It is possible to argue with the methods of determining prevalence. It is not entirely clear that it is desirable that medically recorded prevalence should be the same as some of these surveys. Data about pratice populations is generally limited to age and sex and so various assumptions and approximations have been used in the model with data from other sources. Whilst there is some validity in these objections they do not justify writing off this work. For the first time at an accessible national level there is an attempt to produce corrected prevalence figures for practices. Comparison of one practice with another is still not simple but it is a little simpler. It is only thing that has made me log on to NHS Comparators recently.

QOF upper thresholds

The Marmot report on health inequalities, published a couple of weeks ago, had a few things to say on the subject of the QOF. Somewhat unsurprisingly it suggested that the QOF is used to reduce inequalities. One of the ideas mentioned, although not an official suggestion, was removing the upper threshold on QOF targets. I have to admit some sympathy to this idea in principle although it would certainly add to my work as a GP and the benefits are not entirely clear cut.

There are, however, a number of practical problems which could make implementation hard.

The biggest problem is one of funding. Currently practices receive a certain amount for each patient that meets the criteria between the lower threshold (currently mostly 40%) and the upper threshold (anywhere from 50-90%). If we want to expand that top threshold up to 100% there are a couple of ways of doing this. First we can keep the number of points the same. This means that the incentive per patient falls but the total potential cost to the government remains the same. Where thresholds are at 90% the difference will be 15% - enough to make some unhappiness but probably not earth shattering.

For DM 23 (diabetics with HbA1c less than 7) each patient will be worth a sixth of what they were before.

The other option is to keep the payment per patient the same. This is a lot more costly for the government. DM23 would need 102 points allocated to it (it has 17 at the moment) to keep up the incentive.

None of this takes account of exception reporting. As thresholds have not risen in this way before. It seems very likely that exception reporting would rise, but by how much? Would it incentivise exception reporting more than achievement? Might it simply lead to more organised exception reporting systems to send out the three letters? Most practices already get quite well over thresholds without the incentive of extra points.

And would it really reduce inequality? Could it make it worse? In the first years of the new contract practices in deprived areas did rather worse than those in less deprived areas, however they later caught up. Two years ago when moving the thresholds was mooted before extended hours came in I did some of the maths. It seems effects on practices are likely to be marginal only, but we won't know until it is implemented.

So not an absolutely awful idea but a lot of work would be needed before implementation.

There is more reaction reported at GP magazine.

Site unavailability

I apologise for the unavailability of the main site. This is due to upgrade work by the hosting provider which is not going all that well! This blog moved to a different server a couple of days ago. Should be up again soon and we are promised better, faster stronger etc.

Targets and Indicators

Setting indicators is difficult. It is especially difficult setting indicators attached to incentives as they are in the QOF. It is especially difficult to produce indicators that actually produce the result that you are after without producing perverse incentives.

Required reading for anyone setting a target at any level, from practice to nationwide, should be the very readable report from the King's Fund report Getting the Measure of Quality. It clearly documents the successes and potential failures of information gathering and some of the possible remedies. It would certainly be interesting to see some of these standards applied to QOF indicators although there are signs that the new NICE committee is thinking along broadly similar lines.