Everybody wants their own Pep Guardiola. That is the dream. You take a club legend just beginning his coaching career, stick him in charge of the reserves for a season, give him the top job, then watch as he revolutionises football with a squad based around academy products and wins three league titles and two Champions Leagues. It’s not just winning, but winning your way.

That’s why so many major clubs have turned to former players with limited or no first-hand managerial experience: Juventus with Andrea Pirlo, Chelsea with Frank Lampard, Arsenal with Mikel Arteta. It fosters the exceptionalist dream that underlies so much of what it is to support a club, the feeling you are different, better, more worthy than the others, and it’s good for the branding, taking a popular, recognisable figure to sell that feeling.

Andrea Pirlo with Juventus players including Cristiano Ronaldo

Andrea Pirlo (second left) has been appointed coach at his former club Juventus with no experience of management. Photograph: Alessandro Di Marco/EPA

The problem for all these rookies is similar: while they may have good ideas and have spent most of their lives around football, they are essentially learning on the job and doing so in the full glare of publicity.

They have been in dressing rooms and on training pitches, worked with a variety of coaches and managers, have some idea of what works and what doesn’t, what will motivate players and what will demoralise them, but having been a pupil isn’t alone enough to become a good teacher.

If things go wrong, what then? They have no store of memories to draw on. They can’t look back to that time at, say, Huddersfield when they had that rocky patch but came through it, at Extremadura when they dropped their best striker and found the rest of the team became more focused or at St Mirren when they did what the director suggested and regretted it ever after.

It’s easy then to lose faith, to start second-guessing yourself and making changes almost for changes’ sake. When Arteta suddenly played Pierre-Emerick Aubameyang through the middle and started jabbering self-protectively about the percentage chance of winning games, his self-confidence and authority were almost visibly leaching from him.

The problem then becomes where that experience should be accrued. To point out that Herbert Chapman started at Northampton or Arrigo Sacchi at Fusignano or Alex Ferguson at East Stirlingshire feels almost irrelevant given how different the game was then.

The gulf between top and bottom was nowhere near as stark as it is now. Liverpool, Leeds and Nottingham Forest were all Second Division sides when Bill Shankly, Don Revie and Brian Clough took over. Matt Busby went straight in to the Manchester United job after coaching in the army during the second world war, but when he took charge they had won two league titles and none for more than 30 years. It was possible then for a visionary manager to transform a mid-ranking side into a giant.

Take over a Championship side now, though, and the benefits seem limited. For a former top player, the style of football will not be hugely familiar. You’ll be working with players not capable of doing the things you and your teammates could. Maybe there are issues the wider world doesn’t know or doesn’t care about. You don’t get promoted; you’re seen as a failure.

Or maybe you do get promoted and spend the next couple of years scrapping for survival on a restricted budget. Eventually, the inevitable happens and you go down, to be forgotten or at least doubted, like Eddie Howe. Or you survive and are pigeonholed like Sean Dyche or Sam Allardyce, particularly if you’ve decided a direct approach is the best way to play for your limited squad.

Being able to operate on a tight budget, to keep players going through a run of poor form, to grind out a gutsy point against far superior opponents is no real preparation for handling the egos of celebrity players, devising a coordinated attacking strategy to overwhelm massed defences weaker sides regularly put out against you and satisfying fans and directors who expect to be entertained or coping with the complexity of a fixture list that involves regular long-distance travel.

As a Manchester City director put it a long time ago, considering whether the club should have sacked Joe Royle as soon as he got them promoted to the Premier League in 2000: “You don’t put the bloke who runs the corner shop in charge of a multinational.”

Steven Gerrard at Rangers

Steven Gerrard is enjoying success in Scotland at Rangers but Gary Neville’s venture into management in Spain did not work out so well. Photograph: Ian MacNicol/Getty Images

Or you could go to a different league, which seems at the moment to be working for Steven Gerrard at Rangers but did not for Gary Neville at Valencia. But that brings the complication of operating in a different culture, the lessons of which may not be directly applicable on a return to the Premier League.

The alternative is to gain your practical experience as a coach working under a manager at a super-club – the route taken by Arteta at Manchester City. There are countless assistants who have struggled to make the step up, the advantages of immediate experience of elite-level football offset by never being the one making the final decision, never being the one in the firing line.

That means, whatever route is taken, there is always going to be an element of learning on the job. The nature of modern football, though, is to sack the manager at the first sign of trouble. It is an impatient world that struggles with the idea somebody might be able to put right what has gone wrong. Sacrifice is preferred to incremental development.

Sign up to The Recap, our weekly email of editors’ picks.

Which means what, exactly? That all managers deserve a level of patience? Probably, although equally it can quickly become apparent some simply don’t fit and there is clearly a point after a year or two when it’s not unreasonable to expect some discernible progress to have been made.

But perhaps the most obvious lesson is that the financial structures of modern football make the idea of a steady progression unworkable and very few people taking over a super-club for the first time are likely to have had adequate preparation. Guardiola is an understandable ideal, but he is also a one-off.