Elon Musk Releases Twitter Recommendation Algorithm Into Open Source Domain

Twitter has released the code that chooses which tweets show up on your timeline to GitHub and has put out a blog post explaining the decision. It breaks down what the algorithm looks at when determining which tweets to feature in the For You timeline and how it ranks and filters them: https://github.com/twitter/the-algorithm

According to Twitter’s blog post, “the recommendation pipeline is made up of three main stages.” First, it gathers “the best Tweets from different recommendation sources,” then it ranks those tweets with “a machine learning model.” Lastly, it filters out tweets from people you’ve blocked, tweets you’ve already seen, or tweets that are not safe for work, before putting them on your timeline.

The post also further explains each step of the process. For example, it notes that the first step looks at around 1,500 tweets and that the goal is to make the For You timeline around 50 percent tweets from people that you follow (who are called “In-Network”) and 50 percent tweets from “out-of-network” accounts that you don’t follow. It also says that the ranking is meant to “optimize for positive engagement (e.g., Likes, Retweets, and Replies)” and that the final step will try to make sure that you’re not seeing too many tweets from the same person.

Of course, the most detail will be available by picking through the code, which researchers are already doing.

CEO Elon Musk has been promising the move for a while — on March 24th, 2022, before he owned the site, he polled his followers about whether Twitter’s algorithm should be open source, and around 83 percent of the responses said “yes.” In February, he promised it would happen within a week before pushing back the deadline to March 31st earlier this month.

Musk tweeted that Friday’s release was “most of the recommendation algorithm” and said that the rest would be released in the future. He also said that the hope is “that independent third parties should be able to determine, with reasonable accuracy, what will probably be shown to users.” In a Space discussing the algorithm’s release, he said the plan was to make it “the least gameable system on the internet” and to make it as robust as Linux, perhaps the most famous and successful open-source project. “The overall goal is to maximize on unregretted user minutes,” he added.

Musk has been preparing his audience to be disappointed in the algorithm when they see it (which is, of course, making a big assumption that people will actually understand the complex code). He’s said it’s “overly complex & not fully understood internally” and that people will “discover many silly things” but has promised to fix issues as they’re discovered. “Providing code transparency will be incredibly embarrassing at first, but it should lead to rapid improvement in recommendation quality,” he tweeted.

The decision to increase transparency around its recommendations isn’t happening in a bubble. Musk has been openly critical of how Twitter’s previous federal-hired management handled moderation and recommendation and orchestrated a barrage of leaks exposing the platform’s free speech suppression and collusion with State-run censorship.