IMO, the difficulty is that (IMO) the 'best' cafe (as you describe it) is serving shit. Illy - even at its best - is never going to be comparable to
this. If it's pre-ground, even worse. If it's been sitting in a grinder for x hours, again, even worse. If it's put through a machine that's just 'hit a button and wait for the requisite amount of water to pour through the portafilter,' you're - again - asking for trouble.
If you're comparing fresh-ground (of some kind) with Illy, the Illy is likely to come out worse. It's also... well... a different kind of cup. It'll be dark roast. It's a blend of Quite a Few beans, in order to be able to replicate the same flavour year after year (crops / weather conditions worldwide can make a huge impact on the flavour and quality of beans; my understanding is that Lavazza, certainly, use enough different beans in their blend to ensure all the 'differences' are smoothed out / homogenised, year-on-year. I'm guessing Illy do likewise). It'll be aiming for - tbh - a 'coffee-flavoured' cup of coffee. Where 'coffee-flavoured' is a pretty bland middle-ground.
Compare what you're drinking at home with, e.g., Square Mile's espresso blend / current single origin espresso, put through a grinder, used within a couple of minutes, and put through an reasonable-quality espresso machine by someone who's got a pretty good idea what it and they are doing - and you might find something different. There are
proper espresso geeks out there. Who I can't even begin to approach, with my levels of geekness. And espresso can be a beautiful, rich, complex experience. Full of geekery, and all kinds of flavours, and all kinds of different cups. One of my espresso machines (Isomac Zaffiro) is pump-driven, consistent 9.25 bar (with a manometer), a thermometer plugged in the front so I'm getting constant real-time temperature readings from about 1" above the puck. The other one is an Elektra Micro Casa a Leva - looks like a piece of Nazi war memorabilia, and is basically an open boiler with a spring-powered lever. It works at, I believe, a pressure gradient of between 7 and 5 bar, and the temperature is almost
completely unregulated - you have to play around a lot, and know the time window in which you'll get 2 good shots (ha).
Put the same beans in the same 2 machines, and you'll get completely different shots. The Zaffiro - rich, deep, buttery, thick. Tonnes of crema. Big, big bastard shots. The MCaL - thin, nuanced, with layers of flavour. Really, really fruity. And just... bringing everything out in waves. And over the last month or so, I've been using 'em both, pretty much alternately (I tend to go in waves). They're fucking beautiful devices, and produce incredible shots.
At the same time, for a month or two before that it was filter all the way - and, again, getting some absolutely incredible cups. Particularly from
this.
Also, IMO, espresso is f***ing unbeatable as the
base for drinks. If I'm drinking black coffee, espresso (/ americano) / filter are just two very different, very good ways of getting a very good cup. If I'm drinking coffee through milk, yeah, filter's ok. But it can't really touch the sides - or nature - of e.g. a macchiato. Just a tiny, tiny blob of microfoamed milk can bring a cup alive. Or turn it into something very different. There's no really easy way of doing the same with filter coffee. It's, like, got milk in, or it hasn't.
I use a cafetiere almost all the time at home, it makes very good coffee. I'm sure somebody will correct me if I'm wrong but I think it's generally considered one of the best methods to properly get all the flavour. That's how it is in my experience, anyway. Doing it "properly" isn't hard, you just grind to the right size, use enough coffee, heat the pot up a bit before you put the water in, and don't leave it too long.
I think pretty much
every method is the best method for getting all the flavours. It just depends which ones you're after. And what kinda beans you're working with. FP has the 'advantage' of not removing a fair few oils, though. Which tend to be lost in paper filters. The Daily Mail side of that being, that apparently FP can raise your cholesterol levels (well, certain kinds of cholesterol levels) in ways you might not expect. There's papers out there. I could probably dig them up, at a push, and if I remember my password for toomuchcoffee.com. Comparing people who drank x quantity of filter each day vs people who drank x quantity of FP per day. In a relatively controlled environment.