Rise, Marginalization & Return of the Niche Internet

Let’s imagine you’re sitting on a couch in front of your laptop. You have a question to ask. Where do you go?

Probably, it depends. 

If you need to ask, “How long is the flight from New York to LA?”, then Google would do.

If you need to ask, “How to repair a broken dishwasher?” then YouTube would be great.

If you need to ask, “What are the top tools for product management?” then ChatGPT would handle that.

But what if your question falls outside of the mainstream? What if you’re looking for an underground rap song, a niche JS library, or an honest review of crypto wallets?

Then you would probably chat with your friends. Or ask on some specialized forum. Or open Google and type in “{keyword} reddit”.

You would be looking for answers from humans. You would be looking for answers from the Niche Internet.

What’s Niche Internet? Let’s start with the Internet itself. 

We can look at it as two networks connected with each other:
(1) content network that lets us read and react to content such as social media posts, blogs, and videos,
(2) community network that lets us connect with people through chat, follows, and likes.

On the Mainstream Internet, the content is focused on subjects you might discuss at a family dinner, such as the economy, presidential elections, or the Super Bowl. The community there consists of celebrities, sports figures, Big Tech CEOs, and normal people.

Niche Internet, on the other hand, is focused on deep interest in one particular subject. These smaller-scale communities can talk all day about rationality, mathematics, or geo investigations. Although the Niche Internet creators might have some local fame, if you mention their (nick)name to your normie friends, they’d have no idea who these people are.

I believe that - in the last few years - Niche Internet has faced significant problems. 

Our digital space has been dominated by Big Tech, where Google, Facebook, and Twitter algorithms decide what content to surface and what content to hide. 

Big Tech leans into the mainstream content at the expense of the niche since this is where they can make the most money. It means that niches have less distribution, and it’s hard for these small communities to reach new people.

But there’s hope. 

I believe that - thanks to recent changes in tech & business models - the Niche internet can return stronger than ever.

And just like Netscape’s bundling and unbundling and Packy’s centralization and decentralization trends don’t repeat but rhyme, so does Mainstream & Niche dominance of the Internet

So, I’d like to invite you to examine the past, present, and future of the Internet through this lens and see what the future might hold.

Let’s start from the beginning so we can understand how we got here.

In the early days, the Internet was a very niche place. It was full of academics, Department of Defense employees, and hardcore geeks.

If 80s Internet users wanted to find some content, they typically asked their friends or strangers for advice (Do you remember sending emails with interesting links and jokes, anon?). And if they wanted to find some people to chat with and maintain their digital friendships, they typically used Usenet & email.

Later on, after we got the world’s first website, people could also “browse the web” by reading websites and clicking links that took them to other, sometimes very obscure and niche websites.

The Niche Internet was growing, and people  - via e-mails, Usenet, or personal websites - were natural brokers of the content.

***

The years passed, but the Internet was still in its early stages. It also became more popular, as did interactive services such as IRC, forums, and web chats.

Those tools made it easy to find people with similar interests, and instead of writing electronic letters, we could have a more back-and-forth conversation.

IRC, forums, and web chats were typically subject-oriented (cat lovers, PC geeks, football fans) or location-oriented (NYC, US, Europe). That meant that, by joining a forum, people naturally segmented themselves into these niches, so it was easy to have a focused conversation.

These were also places where we exchanged interesting content to consume or even programs to test. The Internet was still pretty niche, and we still learned about new content, primarily from people who were geeking out on a particular subject. 

***

But the Internet grew, and we needed to find our way around it. 

So, two Stanford students started curating a list of the most interesting websites:

“There were no commercial search engines back in 1993. But a Stanford student named David Filo compiled a list of his 200 favorite websites.

His buddy Jerry Yang helped turn this into an online list. They called it “Jerry’s Guide to the Worldwide Web.” Filo and Yang added new websites every day to their list—and classified them according to categories.

This turned into Yahoo.”

It has been a great place to discover interesting websites. And although we learned about new websites from a service, not from individual users like in the Usenet/chat/e-mail era, it still had a strong human touch and “browsing the web” experience.

Filo and Yang even included a “Search only in Forums” option.

But as Yahoo grew and became the Internet’s wunderkind, it needed more users.

So they did what all businesses do when they need to grow—they turned to bigger markets and showed more mainstream content and websites to generate more traffic and sell more ads. 

It wasn’t Jerry sharing the coolest websites anymore. If you look at their website from 2003, it also looks very different from the 1996 Niche Internet-Yahoo.

The Mainstream Internet was on the rise.

***

And then, as the Internet grew even bigger, a new way to find content gained popularity. 

Found by two other Stanford (PhD) students, Google allowed us to find websites faster than ever. Instead of asking a friend or even Yahoo for a recommendation, we started asking Google. If we did otherwise, it was even sometimes ridiculed (“Just Google it!”), and people’s role in content discovery became even more limited than in the Yahoo era.

Since the World Wide Web was much smaller and less SEO-optimized then, we could use Google to find both mainstream and niche content. Niche, interest-based communities were easier to discover and, as a consequence, got many new members. 

There were even Yahoo-like Directories and Groups back then.

Google website in 2003

Early 2000s Google was how I found most places I hung out online back then. Thanks to its search I found the Jedi Knight II game forum to look for new Star Wars comic books and spent time on Wolfenstein-related IRC channels to learn about the new metal songs.

So, although the websites were not picked by Yahoo employees and users but by Google’s algorithm - it was pretty easy to find niches. The Mainstream Internet was growing, but the Niche Internet was doing fine.

2003 Google even had Yahoo-like categories

Then, social media entered the chat. 

MySpace was something between the old websites and the new ones. 

You could customize the look and feel of your pages like on old, personal websites. And there were many conversations on forums and blogs, creating a new, tight social network. 

But if you look closely, there’s a subtle difference here. MySpace didn’t point to external forums & blogs. They wanted to have all the content on their website. And these tactics worked - their engagement was skyrocketing.

At some point, MySpace got so dominant that Google paid $900M to provide search results and ads, and some journalists started asking if MySpace monopoly could ever be broken.

Although MySpace was a much more centralized solution than a myriad of old-school forums, it was still a pretty good way to discover interesting content from real people. Primarily because - for the first time ever - we had so many people in one place.

Soon after MySpace’s peak, Facebook took over. Back then, their feed was also a good source of content. Yes, sometimes my feed was focused on what my friends had for lunch, but I could also discover tons of interesting stuff.

An obscure YouTube love song that my friend posted so that her crush “just get it.” A niche blog post about the 2008 financial crisis that generated heated debates. Or a DeviantArt gallery by my amateur photographer friend. 

Since my friends were there, it was pretty easy to find niche content - I could just check the wall of a friend who was into a particular subject or DM them. 

Although MySpace and Facebook started to gate content and centralize Internet engagement, the Niche Internet was doing fine. 

This was primarily because these networks created a space for millions of people to discuss things they found interesting and share links to articles, websites, and videos related to these subjects. Once again, people were brokers of content, creating a counterbalance to algorithm-driven Google.

Until something changed.

***

Soon, social media executives started receiving memos from the investors. The message was simple: “You need to finally start making real money.”

Since their platforms earned money by distributing ads to their users, almost all social media decided to limit the reach of external links. The goal was simple - make people stay longer in the app so they watch more ads, so social media would make more money.

So, if you posted your thoughts on Facebook instead of linking to your blog post, you reached more people. This is what some users started doing, and the era of “social media influencers” went into full swing. 

This move had a subtle but important consequence.

When you think about all the previous eras - e-mail, IRC, Yahoo, Google - it was all about helping you find some external content. Even early MySpace and Facebook were full of links.

Now, these social networks have closed the gates and stopped being a good tool for discovering external content. In the meantime, Google search results got more populated by ads and SEO-optimized content. That meant that many niche websites, blogs, and forums couldn't be found as easily as before. 

And in this new era, people didn't explore the Internet the way the 80s & 90s users did. 2010s users relied on Google and social media, which was like taking a guided tour through a huge, wild Internet forest. They were only shown what the Big Tech algorithm guides wanted them to see, missing the vastness and diversity off the beaten path.

In other words, Niche Internet became marginalized. But there was one way to reclaim its spotlight.

One lesson we learned from Yahoo, MySpace, and Facebook stories is that ad-based business models have one thing in common. When companies relying on ads need to grow, they look to bigger and more profitable markets. In other words, they lean more into the mainstream.

And that makes economic sense. 

Consumer brands with the biggest budgets, such as P&G, Mercedes, or Emirates Airlines, must reach millions of people to achieve their marketing goals. So, if a tech platform wants a share of this budget, it needs to provide access to millions of people.

And although niche ads can generate a lot of money - that’s how podcasters make a living - it’s not enough for a $100B+ business (it's one of the reasons why Reddit’s revenue is 150X smaller than Meta’s).

Relying on displaying ads to the mainstream population has a pretty predictable outcome. As we said before, when ads become the platform’s primary source of revenue, it needs users to stay on its platform to show them more ads. That means it’ll eventually have to penalize external links, just like most major social platforms did before.

We said that penalizing links can make it harder for niche websites to get discovered, but it has much more far-fetched consequences.

Niche communities - by definition - are looking for depth. Whether the niche is focused on fishing, IT security, or New York restaurants, people are looking for specialized knowledge that can’t be found on the first Google page.

And niches often move fast. There is always a new fishing rod, new malware, or a new small restaurant opening in NYC. 

In other words, niche communities rely on people who have a deep understanding of the subject and stay up-to-date with these ever-changing domains. These people are rare and are typically scattered across different social media, blogs, forums, and group chats. 

The chances of getting them all on one platform are probably not much higher than Google or Microsoft employing all the brightest engineers in the world. As Bill Joy said: “No matter who you are, most of the smartest people work for someone else.”

If we wanted to adapt Joy’s law to social networks, we could say: "No matter what platform you are, most of the smartest niche people don’t publish there.” 

This means that to reach a satisfying depth of conversation, the niche community can’t rely only on content published on one platform, such as Facebook or Twitter. The community needs links. Links to the best tools, videos, blogs, or websites covering a certain subject that can enrich the conversation and provide more depth.

So, if you run social media and you want to serve niches, you can’t penalize links because the niches will eventually run away, looking for deeper conversation in a place where links are welcome.

And there’s one social platform that got it.

When other social platforms got a memo about making more money through ads, there was one rebellious enough not to sacrifice its links. It was geeky and sometimes even weird. And it was as close to Internet culture as it got. This platform was Reddit. 

What made Reddit different from Facebook or Instagram was its long-tail strategy, with thousands of niche subreddits.

Source: https://www.researchgate.net/figure/Distribution-of-the-number-of-subscribers-of-subreddits-The-largest-subreddits-in-the_fig2_333085087

Reddit understood niches inside out and knew that penalizing links would be the beginning of the end for them. So, they kept them intact, and many online communities migrated there.

They also remained the only major social network on which you don’t need to log in to read the content. It soon turned into a symbiotic relationship with Google, where we started adding “Reddit” to our keywords to find niche answers.  And - although Reddit sells ads - they try to fund themselves with alternative sources such as Reddit Premium, Gold, or Collectibles.

But Reddit isn’t everyone’s cup of tea, and Niche Internet growth wasn't confined to one platform. So, in that era, it felt as if Niche Internet plants started growing in all parts of the Internet forest.

Niche content stormed podcasts, YouTube videos, group chats, and newsletters. Some STEM-oriented communities were growing on StackExchange, which welcomed world-class scientists, including Nobel prize & Fields medal winners. Some culture-oriented communities were growing on Wikia (a.k.a. Fandom), where they learned about books, movies, and video game lore.

New platforms also became more open to exploring business models that are not ad-reliant. Partially thanks to Netflix, Substack, and Patreon, which taught consumers that not all content on the Internet is free.

In addition, Substack revived the Niche Internet idea of people, not algorithms, recommending content. And it worked—some newsletters like Lenny’s get 70% of their growth through recommendations from other writers.

Patreon, on the other hand, gave birth to countless Slack, Telegram, and Facebook groups for niche-obsessed users who pay $10/month just to be a part of the community.

Also, our casual group chats on Messenger, WhatsApp, and Telegram became the next incarnations of the mailing list. And since the chat and messaging apps are not feed-driven, they don’t penalize link sharing.

In the same period, we got the podcast boom. 

Few people would read three-hour-long materials about gardening, martial arts, or parenting. But listening to a three-hour-long niche podcast? No problem, it just makes the workout or a traffic jam more pleasant. What's made the podcasts even better is that they rely on RSS feeds, which means the distribution is less centralized.

YouTube was also helping the cause. Although it is centralized like Facebook or Twitter, its wildly efficient discovery algorithms and generous ad revenue split with creators helped niche content creators reach the right audience.

All these things together made the tech community dive deep (again) into the ideas of the creator economy and 1,000 true fans. I think it’s fair to say that the Niche Internet has returned to the public consciousness.

There's still one barrier in front of us, though.

YouTube, Reddit, and Substack are great for niches, but they are still centralized platforms. This means that if they decide to bet more on the mainstream content, it can come at the expense of the niches. To make Niche Internet more independent, we need better discovery.

How do we discover niche content today? Typically, we ask a friend or an Internet stranger who’d tell us, “This community/podcast/newsletter is really cool; you should check it out.”

Although this idea of people being manual brokers of content has a nice retro vibe, it doesn’t feel optimal that our content discovery in 2024 works as if it were 1994.

So what could we do?

1) The most battle-tested idea is link aggregators.

HackerNews has been around for over 15 years, and it’s still the prime source of tech-related news. Although there is some mainstream news there, you can also find obscure blogs, niche GitHub repos, and eccentric websites.

I believe that more niches should adopt this way of discovering interesting content. This idea is close to my heart because Kiwi - the project I’ve been working on with Tim for almost a year - is about aggregating links. We curate top content and discussions in the same way HackerNews does, so we can keep the discussion quality high, like on old-school forums.

The difference is that our product is decentralized, which means that if you don't like our app, you can build an alternative app with your own moderation design and algorithm. And it can access all links, upvotes, and comments that we display in our app, so you don't need to build the content & community from scratch.

At this point, the content is focused on crypto, but we plan to scale horizontally and, in the future, cover other subjects in a subreddit-like manner.

If you’d like to learn more about it, here's a good link to check.

2) The second idea I find interesting is boutique search engines. 

These engines are focused on a particular niche. For example, we use DoorDash to find a good restaurant, Airbnb to find a good place to stay, and Behance to find a good graphic designer. What makes these engines different from Google is that they combine search with curation, typically through likes and 5-star reviews. 

So, the idea is to use the same concept but for different niches like programming, parenting, or gardening. 

Of course, the examples provided above are centralized in the sense that you can’t use Airbnb to find hotels on Booking. And these marketplaces make money by selling products and services, which might not be the perfect model for an open search engine. But this is definitely an interesting idea to explore, especially given the recent advancements in AI and Chatbots in general.

3) The last idea is SaaS-based social media.

I think it has the potential because of my old Facebook experience, which I recalled before—my feed was a source of different links from the niches my friends were interested in.

One fast-growing project in this area is Farcaster, which uses Reddit-like channels to segment people into niches. Since it charges users and not advertisers, it doesn’t penalize links, which means that some of these channels are good sources of high-quality information.

There are also other projects, such as Mastodon, Nostr, and BlueSky, that are even more popular than Farcaster. From what I know, they don't use SaaS-based pricing and rely on hobbyists and donations, but I have to admit I haven't checked them in a while.

***

All these three methods have their superpowers and limitations. Some of them can also be combined (e.g., link aggregator & boutique search engine), and I think we have just scratched the surface of what's possible.

So, generally speaking, the future of the Niche Internet is bright (if we do the work to make it bright).

This has been a longish post. We started by examining Internet history through the lens of Mainstream and Niche. 

For decades, these worlds grew harmoniously. Mainstream products, such as Yahoo, Google, and early social media, supported the discoverability of niche content and communities. In the last few years, the Mainstream Internet has started growing at the expense of its Niche counterpart.

Since most users learn new content and people via search or social media, Google, Meta, and Twitter have gained huge power over niche creators. As these platforms relied on ads, they eventually had to limit the external links' reach, which decreased the potential of niche communities.

Happily, we saw the renaissance of the Niche Internet, with Reddit, Substack, Patreon, and many others helping niche creators and communities blossom. One big challenge is discoverability, which I believe can be solved via link aggregators, boutique search engines, and SaaS-driven social media. There are already some promising projects in these areas, and I think we are on the right track to reclaim the attention that was taken by Big Tech's algorithms.

And I believe it's an important problem to solve.

Niches are what make the world go round. Hardcore geeks and enthusiasts are the ones who explore new ideas and move their domains forward. Eventually - once their ideas reach mainstream - we all can reap the rewards of the new technologies, frameworks, and methods these people came up with.

So, let's keep pushing to give niche communities their own spaces that can be easily found by anyone who's interested. If we do it right, everyone gets to find their tribe and share what they love, making the internet - and the world - a cooler place for all of us.

Thanks to Tim Daub & freeatnet for feedback on the early drafts of this text

PS: If you'd like to chat about this post, I'd also happily discuss this with you in the Kiwi comments.

PS2: I'd also highly appreciate all likes and retweets of the Twitter thread version of this post.

PS3: If you'd like to dig into the Niche Internet lore, here are the articles and talks I recommend (in no particular order):

1) "Boutique search engines" by Saria Zout

2) "What happened to my search engine" by Honest Broker

3) "Why do news aggregator apps keep failing" by Baekdal

4) "The human medium is the human message" by Simon de la Rouviere

5) "The Balkanization & Babelification of the Internet" by Ruby Justice Thelot and Rue Yi

6) "The tyranny of the marginal user" by Ivan Vendrov

​​​​7) "Content Liquidity and Protocol Lock" by rafa

8)"The Internet's distribution problem" by Kyle Chayka

9) "Death of the follower" by Jack Conte