why do all recommendations suck?
even in the age of AI, almost all recommendation engines are insanely bad.
I was browsing LinkedIn the other day and it suggested new career opportunities for me:
If you tried to create three more incoherent job recommendations for someone, you’d be hard-pressed to come up with this gem. How is it possible that LinkedIn … owned by Microsoft and has free access to all the OpenAI models, with its superstar engineers and massive data troves, came up with this?
The answer: no one is even trying.
That’s right. NO ONE is trying to make good recommendations. Even for the highest-value things in your life, like your career, the recommendations you get on the internet truly and utterly suck.
Look at those three job recommendations again. They’re completely incoherent. Sure, they’re all near DC (geographically desirable for me), but beyond that? The salary ranges are wildly divergent. The roles have nothing to do with each other. The skill sets are entirely different.
It seems really unlikely that anyone would be a fit for all three jobs.
And these were the TOP THREE recommendations LinkedIn surfaced.
Here’s what a close friend, an accomplished lawyer, got:
In the same recommendation feed: a suit attendant ($17/hr) and head of legal ($663k)!!
This isn’t a bug. This is the system working as designed. Or rather, not designed at all.
Here’s the thing that really gets me: LinkedIn has ACCESS to everything. They have your entire work history. Every skill you’ve listed. Every connection you’ve made. Every post you’ve written. Every article you’ve engaged with. Every job you’ve clicked on in the past. And Microsoft just handed them OpenAI models for free.
Yet somehow, with all this data and all this AI, they can’t figure out that someone shouldn’t be recommended three diametrically opposed jobs in the same breath.
Yes Jacob … so would i … but they are out to lunch …
amazon recommends like its 2002
Amazon knows everything you’ve ever bought. They could build an incredibly sophisticated profile of who you are and what you want and even what you need.
And their recommendations STILL haven’t improved since Bush (W.). yes, the last time Amazon really improved its recommendations was BEFORE the US invaded Iraq.
Instead, they just recommend more of what you just bought.
The same pattern plays out for everything: buy razors → here are 200 more razor options.
The fundamental assumption seems to be that people want to endlessly comparison shop the thing they just bought. Which is INSANE.
What we really want is delightful recommendations. We want to see products that we do not know about that would be good fits for us. This is very doable in the age of AI. but even mighty Amazon does not even try. They are completely out to lunch.
recommendations are just ads
At the end of the day, recommendations are ads. The more valuable the recommendation, the more valuable the ad space.
Think about it: if LinkedIn actually recommended you a job that was perfect for you with the right salary, right role, right company, and right location .. you’d click IMMEDIATELY. You’d apply. You’d even thank them. That recommendation slot just became incredibly valuable.
Good ads should feel like features. Good ads should feel like true recommendations. When an ad is genuinely useful, you don’t even realize it’s an ad. You just think “oh, that’s exactly what I needed.”
we already solved this problem
What makes this infuriating is that recommendation systems used to be a hot research area.
Back in 2006, Netflix offered $1 million to anyone who could improve their movie recommendation algorithm by just 10%. Over 20,000 teams from 150+ countries competed. The brightest minds in machine learning spent three years obsessing over how to predict whether you’d give a movie 3 stars or 4 stars.
In 2009, a team called BellKor’s Pragmatic Chaos won by hitting that 10% improvement target. They used ensemble methods, matrix factorization, sophisticated models that actually learned user preferences.
The crazy part is that Netflix never deployed the winning solution because it was too complex to implement at scale. Even so, the research sparked a revolution in recommendation systems. Companies realized they could apply these techniques everywhere - E-commerce, music, streaming, you name it.
We know how to build good recommendation systems. The Netflix Prize proved it was possible. Amazon wrote the playbook in 2003 with their paper on collaborative filtering which became one of the most cited works in computer science. Instead of comparing millions of users to each other (which doesn’t scale), they compared items to items.
YouTube’s algorithm (which actually works) is built on these same principles.
there’s a monopoly factor
LinkedIn doesn’t lose users because their job recommendations suck. You’re not switching to a different professional network because their algorithm is better. There isn’t one.
Amazon doesn’t lose sales because they recommend the wrong products. You still buy what you came for.
Netflix doesn’t lose subscribers over bad recommendations. You eventually scroll past the junk and find something to watch.
So there’s no competitive pressure to fix it.
Plus, there’s another incentive at play: companies want to push their own stuff.
Netflix wants you to watch Stranger Things instead of Seinfeld. One of those is their IP that they own outright. The other costs them licensing fees. Guess which one shows up more prominently in your recommendations?
Amazon wants you to buy Amazon Basics products. LinkedIn wants you to engage with LinkedIn Learning. The recommendation engine doesn’t purely exist to serve you. It’s trying to serve the company’s other business objectives.
why youtube actually works
YouTube’s recommendations are actually good. You watch a video about mechanical keyboards, and suddenly your homepage is filled with related content and at least half of those recommendations are genuinely interesting.
Why does YouTube work when everything else fails? They’re actually trying and have the right incentives.
YouTube’s recommendation engine considers things like what you watch and for how long or what similar viewers enjoy. More importantly, they measure success correctly. They’re optimizing for watch time, yes, but not just clicks. A click that leads to 10 seconds of watching is a failure. A click that leads to watching the full video and then three more is success.
The key is YouTube’s business model, which is advertising. The longer you watch, the more ads they can show you. So they’re heavily incentivized to keep you engaged with good content. Bad recommendations that make you leave the platform directly hurt their bottom line.
Plus, YouTube doesn’t have its “own” content to push. There’s YouTube Originals, but that’s a tiny fraction of the platform. They’re relatively neutral about what you watch. They just want you watching something.
Meta’s ads on Facebook and Instagram work on similar principles. They’ve invested in state-of-the-art models because their entire business depends on showing you ads you’ll actually click on. Bad ad targeting means lost revenue immediately.
Both Google and Meta use AI to make recommendations. To the best of my knowledge, there is still no AI in Amazon or LinkedIn or Netflix: they are still operating like the AI revolution never happend.
what would good recommendations actually look like?
Imagine if LinkedIn’s job recommendations actually worked. Instead of showing three random jobs in a single city, they could look at your profile and see your background, notice the kinds of companies you have worked for and engage with, see your educational background, understand your network, check out what topics you write about … and then actually give you GREAT recommendations.
Ok … GREAT recommendations is hard. But GOOD recommendations is easy. 3 decent engineers should be able to do it in a week. Come on LinkedIn, you can do better: you can actually try..
The data is all there and the technology exists. Someone just needs to care enough to build it.
Now it’s your turn: Go to https://www.linkedin.com/jobs/ and look at your top three recommendations. Post them in the comments. Let’s crowdsource just how bad this really is.
note: Flex Capital invests in 50+ seed-stage start-ups per year (1+ per week). typical first check is $500k. please reach out if you know amazing founders that want to change the world.
if you like this article, please do three things:
share it with someone you admire
follow me (@auren) on X
subscribe to the “Summation” podcast (formerly known as World of DaaS)
(if you hate this, please share with all your enemies)
Pleasantly surprised to learn I'd apparently be an excellent candidate for roofing project management
My top recommendation was Hungarian Voice Acting...
I don't speak Hungarian.