There are well-established regimes for rating schools and healthcare providers. Those that work best are stable, detailed and easy to compare
We all know now that the government wants to assess the quality of teaching in higher education through a new Teaching Excellence Framework (Tef). What’s less clear is exactly how this will happen.
There are genuine concerns, but finding imperfections in the model of the Tef put out for consultation is the easy bit. The hard bit is finding improvements that will benefit students and institutions. That is why we are publishing a new paper today, which considers the lessons for the Tef from other sectors – particularly other parts of the education system and from the health and social care sectors.
The problems
This will be different to past initiatives for raising the standard of university teaching in two key ways. First, ministers plan to use statistics as proxies for what goes on in the lecture hall. Their green paper suggests these will be: student satisfaction; the number of drop-outs; and the proportion of graduates finding employment. The problem is that these statistics are flawed. You could go to a prestigious university, receive unimpressive teaching and not engage much with academic work. But so long as you have fun, complete your course and secure a job, your experience would count as successful teaching.
This is not necessarily a killer argument against what the government is doing, however, because alternative metrics are thin on the ground. At the Higher Education Policy Institute (Hepi), we have published a paper which argues that teaching and learning processes should be measured directly, along with one that says we should test students on what they have learned (the so-called value-added measure). Both are sound ideas, but they are years away. If we wait, nothing much will change now – if ever.
The second big difference between the Tef and earlier schemes to improve university teaching is that ministers plan to link the amount that students pay to their institution’s score. So someone who attends a university thought to teach less well will pay fees of £9,000 but someone who attends an institution deemed to be better could pay more.
It is not clear why institutions with below-par teaching should be punished rather than helped. But a bigger problem is that, in truth, all universities deliver some excellent teaching and some not-so-excellent teaching. Yet, to begin with, the Tef will measure performance across an entire institution rather than at a departmental or course level. So students on poorly-taught courses at universities that pass the Tef will pay more than students on well-taught courses at institutions that fail the Tef. That is the opposite of what ministers want.
There is also a risk that a link to fees could encourage perverse behaviour. It is already rumoured that students are being encouraged to mark their institutions down in the National Student Survey because lower rates of satisfaction could block fee increases.
Lessons to learn
Comparing the Tef to the quality assurance regimes of education and health may seem like a no-brainer but, surprisingly, it has not been done before. The exercise produces a number of clear lessons.
These sectors have gradually developed measures of quality which, while of limited value when used on their own, are useful when digested as part of a balanced diet. Ask any parent anxiously waiting to find out if they have the school place they want for their child whether they used any information from Ofsted inspections. I know I did.
Stability in the organisation delivering the quality ratings is crucial. Before 2009, there was considerable instability in the organisations delivering healthcare ratings: for example, the Commission for Health Improvement was replaced by the Healthcare Commission in 2004. The Nuffield Trust claims that such upheavalreduced the time for regulators to develop the system of ratings and to evaluate their impact. Careful thought needs to be given to this, because the Tef is set to be implemented alongside the biggest shake-up of higher education regulation for a generation.
Our report also confirms that it must be possible to disaggregate data on quality for it to be truly meaningful. That is why Ofsted provides separate ratings for early years provision that is delivered within schools and the Care Quality Commission has 48 ratings for a one-site acute hospital NHS trust. This is more effective at engaging staff than broader measures would be. So it will be important to move to subject-level ratings in the Tef as quickly as possible.
Perhaps most uncomfortably, we find that ratings need to be included in league tables if they are to have the impact that policymakers want. In Wales, the performance of secondary schools worsened dramatically in comparison to those in England after the abolition of school league tables at the turn of the century.
Currently, people applying to higher education have access to university league tables that, in the words of the higher education green paper, “are not always an accurate reflection of the quality of education provided in each individual course”. The inclusion of the TEF could alter this.
When Jo Johnson first announced the Tef last year, he promised it would be “proportionate and light-touch, not big, bossy and bureaucratic”. Looking at what happens in other sectors will help to ensure that the Tef keeps to those goals, and that it hits the right targets.
