*
In 2009, Arsenal’s manager Arsene Wenger likened being a football manager to living on a volcano: any day could be your last. Three years on and his view has surely been reinforced this season, a campaign in which he endured sustained discontent from some fans and some sections of the media for the first time in his 16-year reign.
The issue of when to sack a manager remains one of the most contentious in modern football. Should bosses always be given time to make their mark in a new job, or work through a rough patch in one they’ve held for years? Should chairman be more trigger-happy to find success, not less? And how can clubs assess, objectively, how well their manager is doing?
TOM MARKHAM, a qualified accountant and former foreign exchange trader with an MBA in Football Industries from the University of Liverpool, has collaborated on projects with national associations, clubs, agencies and sports consultancies. Currently researching for a PhD in Football Finance at Henley Business School’s ICMA Centre, here he outlines exclusive new research, done with Adrian Bell and Chris Brooks, that has led to a model that shows precisely when it’s time to show a failing manager the door – or reconsider a decision to make a change.
.
.
By Tom Markham
23 April 2012
The volatility of managing at the highest level has been reiterated in the Premier League in recent weeks. The impressive David Moyes celebrated a decade at Everton’s helm. Conversely, Mick McCarthy and André Villas-Boas were both dismissed from their positions at Wolves and Chelsea respectively for underperforming based on the clubs’ expectations.
Managers who leave a club before the expiration of their contract typically receive substantial compensation in line with their remuneration package. It has been reported that André Villas-Boas received an £11m severance payment from Chelsea on leaving the club. Combine this with the £13.3m (€15m) the club paid FC Porto in compensation for his services. Now add pay-off costs for his backroom staff, and compensation for Chelsea’s previous manager Carlo Ancelotti. What have Chelsea got? A bill in excess of £40m – close to the entire turnover of some smaller Premier League clubs – and the club is still without a permanent manager.
This is the most extreme example of why firing and hiring with little apparent discernment is costly. But it is evident that the selection and appointment of the right manager is vital to a club’s short to medium term success, both on the pitch and financially.
Players have a multitude of analysts from the disciplines of coaching, sports science, psychology and nutrition tracking their every move. So why not develop a form of analysis to gauge managerial performance? The research outlined in this article aims to achieve exactly that.
A comprehensive literature review was undertaken to examine material relating to managerial dismissals and change within a sporting and business context.
Recent research showed that the average football manager in England lasts a little over two years in a job position. This is mirrored in professional US sport with head coaches averaging less than 2.5 years in their job.
Managers are usually dismissed for either poor on-field performance, a breakdown in relations with club directors/owners, and / or increasing media pressure to deliver.
In the modern game, spending power has become increasingly important for a club to be successful. However, there are contrasting views on whether sacking a football manager leads to improved team performance on average.
Of nine studies scrutinised from six countries, five of those nine – from England and Spain – stated that replacing a manager does improve a club’s on-field performance while four studies – from Belgium, the Netherlands, Italy and Germany – found that hiring a new manager did not improve team performance on average.
The results from England suggested that a club’s short-term performance increases when a new manager is appointed which may explain the rationale behind relegation-threatened sides changing their manager towards the end of the season. Change at Wolves recently, with the departure of Mick McCarthy (right), did not work as Wolves expected.
Dr Chris Hope, of the Judge Business School, Cambridge University, published a model in 2003 determining when a manager should be sacked. His methodology is based on a ‘trapdoor’ which relates to an average number of league points a manager achieves per game. If the manager falls below this level, he should be sacked according to the model. It also allows a weighting to be assigned to a manager’s most recent games in charge.
According to his results based on a sample period between 1996-97 and 2001-02 in the Premier League, a manager must gain at least 56.81 points in a season and a manager amassing less than 0.74 points a game should be sacked. This research represented a significant step in the area of academic managerial analysis for football but it was missing something vital – expectation.
If Terry Connor had managed to end this season on 57 points the Wolves board would have been elated with his miraculous powers. But if Roberto Mancini’s Manchester City finished with 57 points at the end of this season it would have been a miserable failure of a season. My objective, therefore, was to build on this model to develop a practical model comprising of real world factors.
Fund managers are often compared to football managers in that both are in control of ‘assets’ worth a massive amount of money. Econometric models (models applying mathematics and statistical methods to economic data) have been used for more than 40 years to monitor the performance of mutual fund managers. More recently Dr Robert Kosowski and his research team in this area managed to separate managerial skill from luck with regard to fund managers. Their research found that there were very few skilled fund managers who could pick winning stocks on a consistent basis.
Can this approach work with regard to football mangers? I think it can, and have produced an analytical model to demonstrate it, using five seasons of Premier League data (and sackings) as the basis of that model.
The model effectively needs to separate the impact a manager has on a team’s performance from all the other factors that are beyond his control. This is obviously an extremely difficult task and requires a number of assumptions to be made. The factors deemed beyond the control of a manager for the purpose of the model are: total player wage bill, net transfer expenditure (we assume that the club directors are control of a club’s budget), player injuries, suspensions and unavailability.
The data to fuel the model was compiled from a variety of reliable sources which are detailed in a full academic paper that outlines this work and can be downloaded at this link.
All the aforementioned factors were built into a bespoke econometric model looking at the five seasons 2004-05 to 2008-09 inclusive.
The initial results showed that there was a positive relationship between league points accumulation and wages spent. This is a well-known relationship but the model also showed that increasing a club’s wage bill by £100m would lead to an average expected increase of 0.8 points a game.
There is a negative relationship between points accumulation and player unavailability as one might expect. Five players missing with injuries equates to a loss of 0.2 points a game on average. Five players being suspended simultaneously results in an average drop of 0.5 points a game.
The unavailability of five players – at the Africa Cup of Nations, for example – is equal to a drop of 0.6 points per game on average.
Jol: Sacked too early by Spurs
A statistical approach called bootstrapping, proven in the mutual fund industry research outlined earlier, is then applied to the model. In simple terms this compares the impact of all 60 managers who worked in the Premier League between 2004-05 and 2008-09 against 10,000 randomly generated managers using the model.
The randomised managers are based on the permutations of real managers. We generate random managers to give a large scale against which we can measure the performance of the real managers. At any point in time, after any single match, we can produce a figure for any given manager that tells us how he is performing, having taken into account the advantages and disadvantages beyond his control as outlined above.
The output of the model is a percentage. This percentage outlines the manager’s performance in relation his peers. A result of 20 per cent, for example, means that 20 per cent of managers performed better than the manager in question. A result of 50 per cent means 50 per cent of managers (in our large randomised sample) perform better. A result of zero percent means no manager performed better, given the same circumstances the manager under scrutiny is facing.
The higher the percentage the rating, the worse the manager. The lower, the better. The model recommends that a manager receiving a score worse than 95 per cent should be a candidate for the sack whilst a manager receiving a score better (lower) than 5 per cent is among the best performing managers.
The results the model generated with regard to the top performing managers were unsurprising in as much as it told me that good managers were good. Sir Alex Ferguson, Guus Hiddink, Arsène Wenger, José Mourinho and Rafa Benítez all consistently registered a score of 0.00% during the sample period. This means that no manager performed better than them given the resources at their disposal. Sam Allardyce, David Moyes, Steve McClaren and Martin O’Neill all performed very well with limited resources throughout the five year period.
The major advantage of this model is that it is run on a weekly basis. For this reason it can recommend an exact date of potential dismissal for a manager.
Let’s examine Steve Wigley’s tenure at Southampton in 2004-05. He crossed the 95 per cent barrier with a score of 99.12 per cent on 4 November 2004 following 2-2 home draw against West Bromwich Albion. The model recommendation is dismissal on this date. Wigley remained in charge for a further four league matches before being sacked on 8 December 2004. An extra two points would have kept Southampton in the Premier League which may have been attainable had the club parted company with the manager earlier.
Looking at Sammy Lee’s stint as Bolton manager in 2007-08, the model recommended that he was fired on29 September 2007, but the club took action only one game later and his successor Gary Megson managed to keep the club in the Premier League that season. In all, the model found that seven managers stayed in their jobs for longer than it recommended.
Alternatively, the model found that there were five managers with improving performance that were sacked unnecessarily during the sample period.
Managers sacked unnecessarily:
Glenn Roeder at Newcastle in 2006-07 who had a score of 30.41 per cent at the start of his tenure which had come down to 8.14 per cent before he was sacked; Chris Coleman of Fulham in 2006-07 who was at 38.43 per cent at the start of the sample period and 15.21 per cent when he was fired; Martin Jol at Tottenham Hotspur in 2007-08 who had 17.68 per cent at start of his tenure, finished at a highly impressive 0.01% (just missing out on a Champions League place when he was dismissed); Avram Grant at Chelsea in 2007-08 who came down from 20.21 per cent to an excellent 2.25 per cent before being sacked (having lost the Champions League final on penalties) and Sven-Göran Eriksson of Manchester City in 2007-08 who brought his score down from 21.55 per cent to 5.30 per cent before getting fired by the then owner, Thaksin Shinawatra.
This provides a brief overview of how the managerial performance analysis model works and the results it is capable of generating.
The objective of the model is not to dictate hard and fast dates to sack managers but to provide guidance to club directors or owners, objectively, on how their managers are performing.
It goes without saying that no manager should be sacked because a computer model says so; but analysis of this type could either support a decision that a change is needed, or actually highlight that a manager is doing a much better job, objectively assessed, than might appear the case after a few poor results and in the heat of fan and media pressure.
As well as promoting optimum on-field success, the model’s financial element is designed to encourage prudence – especially with Uefa’s Financial Fair Play now in operation.
The model can be run in real time to analyse a manager’s performance and understandably, in this analysis era, there has been a large interest from clubs in adopting it. It would be naive to rely on the model alone to make such an important decision as firing a manager and obviously other mitigating factors should also be considered in deciding a manager’s future, but nevertheless the model does provide users with quick, reliable feedback on managerial performance.
..
Clubs, researchers or other interested parties wanting to know more about the model can contact Tom Markham here, and can see his full academic paper here. Tom can be followed on Twitter @TMFootyFinance.
.
.
This article appears in the May issue (left) of FC Business Magazine, the business magazine for the Football Industry in Britain.
The FC Business website is linked here while the May issue and back issues are available as eversion downloads via the FCB website.
FC Business’s editor, Ryan McKnight, is on Twitter here.
..
..
.
