Lewis' Olivier book is nowhere as good as his Sellers book--he falls into the trap of discussing Olivier's personality by cataloging his roles, a tactic that leaves him little room to actually discuss the mechanics of his acting.
And it's probably true that "The Greatest Actor of the 20th Century" title probably rests more on his stage work than his cinema--Olivier himself considered Brando to be the greatest screen actor.
On the other hand, I think you're being terribly dismissive in saying that "He didn't leave much behind. Not in terms of quality."
A subjective list of what he left behind in terms of quality would include his performances in Wuthering Heights, Rebecca (I don't think Cary Grant would have been better), Pride and Prejudice, Henry V, Hamlet, Carrie (perhaps Olivier's most underrated and moving performance), The Beggar's Opera, Richard III, The Devil's Disciple, The Entertainer, The Power and the Glory, Uncle Vanya, Bunny Lake Is Missing, The Dance of Death, Sleuth, Long Day's Journey Into Night, The Merchant of Venice, A Voyage Round My Father, and King Lear.
Give or take a few films, and that's still an impressive roster of vivid, intensely memorable performances. I doubt that a list of Gielgud or Richardson's film/TV performances would be any larger (Guinness and Redgrave's certainly wouldn't) or have as much variety.
I'd also take exception to the generalization of his Shakespearean performances as being records of "big fruity" stage performances. In the case of Othello and The Merchant of Venice that would certainly be true--but it's by comparing those to his major Shakespearian performances that we can see how he modulated his acting for the screen.
And he did lots of wonderful TV--The Power and the Glory, Uncle Vanya, The Dance of Death, Long Day's Journey Into Night, A Voyage Round My Father, and King Lear are all evidence. Some are simply records of stage performances, but they display more restraint and modulation for the cameras than his Othello.
I would rather watch Olivier than any of the actor knights. Even at his worst he's interestingly bad because of his vitality, whereas his colleagues at their worst were merely tasteful.
I don't believe there's a single "great film" in that list. Henry V maybe. It doesn't feel like a proper career to me, more a collection of bits and pieces.
I don't think he was rubbish- I just don't think he left behind enough evidence to sustain his reputation. I wish I'd seen him on stage. I could have done- he and I overlapped- but I never did. It was a great missed opportunity.
I think there's enough evidence in the films and shows I listed earlier to showcase a career of great intensity in a bravura range of parts. Part of the problem is that some of his best performances (Carrie, The Beggar's Opera etc) are either in excellent films that flopped or in TV shows lost in the archives.
A great performance can't by itself make a film great, so I don't believe great actors are those necessarily those who've appeared in a list of masterpieces. The great movie actors are people we often are just as happy to see enlivening otherwise mediocre films--that's part of what makes them great. And in Olivier's case I'd personally rank all of his Shakespeare films as great (plus, how many great films did the other actor knights have under their belts?) or at least cinematically important and influential. Beyond that, three or four of his other films are Hollywood classics, a couple are underrated gems, and much of his TV work features great performances in dully-directed material. Had Olivier not given the theater first priority, he might have directed more films or appeared in more films by other auteurs, thus raising his great-film count. As it is, I don't think cinephiles would ever call him the greatest movie actor or star, but his legacy is substantial enough.
The other theatrical knights had patchy film careers too- with the exception of Guinness, who was always a proper film actor. His CV includes-
Kind Hearts and Coronets
The Lavender Hill Mob
The Lady Killers
The Bridge on the River Kwai
A pretty good collection of first rate and/or important movies.
And then there is his performance as George Smiley in
the BBC's Le Carre dramatisations.
It's a good CV, though it seems no better to me than Olivier's--three classic Ealing comedies, a Hollywood classic, a classic adaptation, a couple of good TV series, and Star Wars, a film that's probably more influential than great (and which has Guinness in a strictly supporting role). Then again, in many ways Guinness was the opposite of Olivier--a restrained and unobtrusive craftsman. Actually, I might add a few more films to Guinness's list, including Our Man in Havana, The Man in the White Suit, Last Holiday, Tunes of Glory, and The Horse's Mouth. None great perhaps, but definitely worthy of notice. (Criterion has put out Tunes and Mouth I think.) The 50s were probably Guinness's best period filmwise.
Yes, I limited myself to the four and five star movies. He did a lot of good work in not so good movies too. For example, his Charles I is the one compelling reason for seeing the turgid Brit-epic Cromwell