There is a witty spoof on the ‘contact hours = quality education’ debate (sic) in the THE. AKA why are ministers so stupid.

Ministers are concerned that despite large differences in the quality of novels, they all seem to cost the same. From now on, under the National Assessment of Fiction Framework (NAFF), the quality of novels will be scientifically measured according to the number of pages they contain, and prices will be set accordingly. Ministers are said to be delighted to have finally proved that Riders, Jilly Cooper’s 900-page epic, is three times better than Jane Austen’s insubstantial Pride and Prejudice.

It reminded me (yet again) to track down something I had read many times on Brian Randell’s web page here. Computing was lucky because the inventors got to do so much before the academy tried to close it down.

Masterpiece Engineering, T. H. Simpson, IBM Corporation, (Via Brian Randell’s web page here.)

“Here on this spot in the year 1500 an International Conference was held”.

It seems that a group of people had gotten together to discuss the problems posed by the numbers of art masterpieces being fabricated throughout the world; at that time it was a very flourishing industry. They thought it would be appropriate to find out if this process could be “scientificized” so they held the “International Working Conference on Masterpiece Engineering” to discuss the problem. As I continued walking round the garden, now looking a little closer at the ground, I came across the bones of a group, still in session, attempting to write down the criteria for the design of the “Mona Lisa”. The sight reminded me strangely of our group working on the criteria for the design of an operating system.

Apparently the Conference decided that it should establish an Institute to work in more detail on production problems in the masterpiece field. So they went out into the streets of Rome and solicited a few chariot drivers, gladiators and others and put them through a five week (half-day) masterpiece creation course; then they were all put into a large room and asked to begin creating. They soon realized that they weren’t getting much efficiency out of the Institute, so they set about equipping the masterpiece workers with some more efficient tools to help them create masterpieces. They invented power-driven chisels, automatic paint tube squeezers and so on but all this merely produced a loud outcry from the educators: “All these techniques will give the painters sloppy characteristics”, they said.

Production was still not reaching satisfactory levels so they extended the range of masterpiece support techniques with some further steps. One idea was to take a single canvas and pass it rapidly from painter to painter. While one was applying the brush the others had time to think. The next natural step to take was, of course, to double the number of painters but before taking it they adopted a most interesting device. They decided to carry out some proper measurement of productivity. Two weeks at the Institute were spent in counting the number of brush strokes per day produced by one group of painters, and this criterion was then promptly applied in assessing the value to the enterprise of the rest. If a painter failed to turn in his twenty brush strokes per day he was clearly under-productive.

Regrettably none of these advances in knowledge seemed to have any real impact on masterpiece production and so, at length, the group decided that the basic difficulty was clearly a management problem. One of the brighter students (by the name of L. da Vinci) was instantly promoted to manager of the project, putting him in charge of procuring paints, canvases and brushes for the rest of the organisation.

Some people try to optimise the shit out of everything. Or, as we might say in South Wales, ‘tearing the arse out of it’.

This film from 38 degrees is incredibly moving. And I recognise the places and the hospitals. Worth a critical review, but the abuse of trust is manifest (which is why they are called ‘trusts”). My post  title — ‘Forty year ago we would have  started a revolution’ —  is my favourite line.

The quick retort is, of course, that they are actually treated worse — at least worse than some factory workers. The heading is from an article in BMJ careers. Jane Dacre is quoted as saying we need more juniors, and ‘more boots on the ground’. By contrast, Catherine Calderwood, Scotland’s CMO didn’t agree that an any increase in junior doctors was necessary:

“I’m not sure that more is definitely better. We’ve had a 38% increase in consultants in Scotland in the past eight years, and almost a doubling of our emergency medicine consultants, and I’m not sure if I walked into an emergency department they would tell me it’s half as much work as it was eight years ago.”

She went on to spread the kool-aid a little more:

“I think sometimes doctors have not embraced others doing some tasks, and I would like us to be much more like a conductor for the orchestra. So (only) the really difficult stuff, the really responsible stuff, and the really clever stuff is what comes to the doctor as the senior leader.”

I think some of what they both say is correct, but I also fear some of what they both say is politics. The UK — including Scotland — is desperately short of doctors. Appointment times are too brief, waiting times out of control, clinical expertise increasingly patchy, and doubts about the adequacy of training, widespread. Demand is rising, and wants and needs increasingly confused. From where I stand, clinical service in some areas is getting worse. Scotland’s NHS is, in some parts, second world standard, as a colleague from mainland Europe once reminded me.

The issue about paramedical staff is important, but we have been here before. I may be misquoting the figures a little — and they are for the US— but around 1900 one in three health care workers was a doctor. Now the figure is around 1 in 14. This trend in the ratio will and should continue. The discussions about physician assistants and the like have been going on for over half a century, with little evidence of action. The key issue is that if you want to encourage people to move into new roles, you have to create a certification system that rewards and encourages people to do these jobs. That is why we have radiographers, pharmacists and the like. But successive UK government officials hate the idea of certification, and prefer that ‘nurses’ moves into roles that they have little formal training for, and instead, end up existing free of any meaningful regulation. Governments cannot face up to the fact that these ‘health care workers’ (apologies: an awful bloody phrase) will need more than subsistence wages, and setting the system up will require upfront funding. I used to laugh at the image of a doctor’s office with all the certificates on the wall behind the desk, but now I advise patients always to ask whether the person providing diagnostic or therapeutic activities has a recognised medical qualification. How many melanomas do you diagnose a year?; how many times have you performed this procedure?; what exams did you have to pass etc.

If you ask whether you can train graduates to become physician assistants in dermatology, dermatological surgery and the like, the answer is a clear yes. But to make this a sensible career choice, we need certification — theory and practical exams and so on — and job titles that are transferable wherever somebody works. At present we simply to not have this. The dentists have done this and — key to any debate — practitioners (hygienists etc), whether they are dentists or not are registered with the General Dental Council. Now, the GDC is not a very popular organisation, but the idea of formal certification —something that means that practitioners can move jobs easily — is a key component of making this system work. Institutions matter, as do incentives.

Undergraduates frequently complain that they don’t have enough contact hours. But a major study in the UK suggests that students develop skills better out of – rather than in – the classroom.

I do not find this claim too surprising, but if you read the article and go to the HEA ‘engagement report’, I find it hard to know how they claim to have established this as a fact. Confounding and hidden variables all around. But as has been said before, classroom learning has the air of an oxymoron. Contact hours are both relevant and  irrelevant. Context matters most.

In all places, I came across the following in Dylan Wiliam’s most recent book (if you want to understand what teaching feedback really is, read it). After pointing out how some machine learning techniques can outperform some medics in some contexts, he writes

However, it is important to realize that the key factor in making jobs suitable for automating is not that they are manual or low skill. It is that they are routine. A task can require many years of training for humans to become good at it, but it can still be relatively routine, thus making it relatively straightforward to automate. This is just one example of a much more general principle, which is that many of the things that we thought would be easy to automate turn out to be rather complex, while many of the things that we thought would be hard to automate turn out to be reasonably simple. …. This observation—that high-level reasoning seems to require very little in the way of machine power, while many low-level sensorimotor skills appear to require huge computational resources—is known as Moravec’s paradox, named after Hans Moravec, who pointed out that “it is comparatively easy to make computers exhibit adult-level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility” (Moravec, 1988, p. 15).

Now, of course I work in a domain that is heavily perceptual and, as yet, AI systems have made little inroad. This may well be because the task is difficult (and that the human visual system is powerful) but also (critically) that the available training sets are orders of magnitude too small. This will only change if the clinical workflow is fully digital. We have published some work in this area, and if you visit the Dermofit app on the iOS store you will see an app that uses some machine learning. But a long, long way to go, and the humans can still pay the bills. For the moment.

The question for medical education is that as we (reasonably) concentrate our procedures around high level processing, the sort of environments we need to develop perceptual skills are neglected. You can do both.

There is a nice piece by Nassim Nicholas Taleb in Medium. It is from a forward to a book (I think) on physical / strength training. If you have read Taleb you will know this is not too surprising.

You will never get an idea of the strength of a bridge by driving several hundred cars on it, making sure they are all of different colors and makes, which would correspond to representative traffic. No, an engineer would subject it instead to a few multi-ton vehicles. You may not thus map all the risks, as heavy trucks will not show material fatigue, but you can get a solid picture of the overall safety.

Likewise, to train pilots, we do not make them spend time on the tarmac flirting with flight attendants, then switch the autopilot on and start daydreaming about vacations, thinking about mortgages or meditating about corporate airline intrigues — which represent about the bulk of the life of a pilot. We make pilots learn from storms, difficult landings, and intricate situations — again, from the tails.

In one sense he is saying something that is easy to agree with. But if you delve a little deeper, it is not what we always do in medical education.

The structures we create to enable learning in a clinical discipline are not mirrors of what goes on in the real world. Pace the airline example. We shouldn’t expect teaching time to mirror disease prevalence; we don’t spend most of our time in dermatology teaching students about viral warts, or dandruff, or toxic erythema. When you try to recognise objects, you do not just study those particularly objects. Rather, you have to study all the other objects. If you want to be able to ‘call out’ whenever you see a dog, you have to study cats. And chimps, and wolves and so on. This is one of the reasons why just learning about the top ten conditions makes little sense, if acts of recognition are involved. Most things are defined by what they are not. To think in the box, you have to know what is outside the box. This is what makes medical education a hard problem.

There are implications for clinical practice for the expert, too. Everyday practice appears to minimise the role of the statistical tails. Your learning about common condition may be ‘everyday stuff’ requiring little formal study. But for rare conditions, or odd presentations of common conditions, everyday practice, may not be sufficient — simply put, you do not see rare events frequently enough to consolidate and strengthen your memories. Everyday practice rarely provides enough critical mass, you might say. A practical example.

When I was a trainee in Newcastle if we saw an ‘interesting patient’ or a patient in which the diagnosis was unclear, we pressed a buzzer. The buzzer and flashing light went off in all the clinic rooms, the laboratories, the professor’s office and the seminar room. What happened then, resembled the Stepford wives. All descended on the particularly clinic room, as though under some malign influence. There were times when this was quite funny, although some patients might have told this differently.

This simple tool was just an implementation of another one of Rees’s rules: routine clinical practice is not sufficient to consolidate or acquire the skills you need to provide routine clinical practice. This seems like a paradox, but it isn’t. “A sailor gets to know the sea only after he has waded ashore.”  Rather, I always view it as a solution to the forgetting curve that Ebbinghaus described (although I think there may be other justifications)

There is a simple learning point here. The acquisition or maintenance of clinical competence requires much more than seeing patients (and by this, I do not just mean reading research papers). Software, and virtual worlds that we control, might help. But the Rees maxim remains: routine clinical practice is not sufficient to consolidate or acquire the skills you need to provide routine clinical practice

Interesting story in Nature highlighting instances where instead of doing post-docs, young biologists have raised funding to set up their own companies. Of course, most start-ups fail, but then most really interesting research projects should fail. Y Combinator is getting into this area, which surprised me. As the age of getting your first grant gets higher, and with the increasingly dysfunctional nature of much academic (medical) science, the attractions are obvious. I was sceptical (and still am)  that the ‘software’ model would work in this area.

“It is a variant of the (at present unfashionable) historical method. It consists, simply, in trying to find out what other people have thought and said about the problem in hand: why they had to face it: how they formulated it: how they tried to solve it. This seems to me important because it is part of the general method of rational discussion. If we ignore what other people are thinking, or have thought in the past, then rational discussion must come to an end, though each of us may go on happily talking to himself.” from “The Logic of Scientific Discovery (Routledge Classics)” by Karl Popper

There was a story in the FT a few weeks back (paywall). It concerned the painting ‘Portrait of a Man’, by the Dutch artist Frans Hals. Apparently, the Louvre had wanted to buy the painting some time back, but were unable to raise the funds. However, a few weeks ago, the painting was declared a “modern forgery” by Sotheby’s — trace elements of synthetic 20th-century materials have been discovered in it. The story has a wider resonance however. The FT writes:

But if anything the fake Hals merely highlights an existing problem in how we determine attribution. In their quest to confirm attributions, dealers and auction houses seek the imprimatur of independent, usually academic, experts. Often that person’s “expertise” is deduced by whether they have published anything on a particular artist. But the skills required to publish a book are different to those needed to recognise whether a painting is genuine. Many academics are also fine connoisseurs. One of the few to doubt the attribution to Parmigianino of the St Jerome allegedly connected to Ruffini was the English scholar, David Ekserdjian. But too often the market values being a published writer over having a good “eye”.

Here is a non trivial problem: how can we designate expertise, and to what extent can you formalise it. In some domains — research for example — it is easier than in others. But as anybody who reads Nature or the broadsheets knows, research publication is increasingly dysfunctional, partly because of the scale of modern science; partly because the ‘personal knowledge’ and community has been exiled; and partly because it has become subjugated to academic accountancy because the people running universities cannot admit that they do not possess the necessary judgment to predict the future. To use George Steiner’s tidy phrase, there is also the ‘stench of money’.

But the real danger is when the ‘research model’ is used in areas where it not only does not work, but does active harm. I wrote some time back in a paper in PLoS Medicine:

Herbert Simon, the polymath and Nobel laureate in economics, observed many years ago that medical schools resembled schools of molecular biology rather than of medicine . He drew parallels with what had happened to business schools. The art and science of design, be it of companies or health care, or even the type of design that we call engineering, lost out to the kudos of pure science. Producing an economics paper densely laden with mathematical symbols, with its patently mistaken assumptions about rational man, was a more secure way to gain tenure than studying the mess of how real people make decisions.

Many of the important problems that face us cannot be solved using the paradigm that has come to dominate institutional science (or I fear, the structures of many universities). For many areas (think: teaching or clinical expertise), we need to think in ‘design’ mode. We are concerned more with engineering and practice, than is normal in the world of science. I do not know to what extent this expertise can be formalised — it certainly isn’t going to be as easy as whether you published in ‘glossy’ or ’non-glossy’ cover journals, but reputations existed long before the digital age and the digital age offers new opportunities. Publishing science is one skill, diagnosing is another, but there is a lot of dark matter linking the two activities. What seems certain to me, is that we have got it wrong, and we are accelerating in the wrong direction.

“Applications from the EU to Oxford, Cambridge, medicine, dentistry and veterinary courses have fallen by 9%, with a 16% fall in those who applied to medicine. This is in spite of the government’s recent confirmation of continued funding for EU students applying for 2017, though the announcement may have come too late for many applicants. It’s still early in the cycle though, and numbers could yet recover, but the numbers are not helping the jitters.”

(BTW, the previous article is titled ‘Are we academically adrift?’ after the US book and survey (AKA: ‘students don’t learn nuttin at college’). Troubled waters. All puns intended.)

WONKHE