Tesla CEO Elon Musk sent Twitter shares plummeting on Friday when he said he would pause its $44 billion acquisition of the social network while he researches the proportion of fake accounts and spam on the platform.
Although Musk later clarified that he remained committed to the deal, he continued to hammer home the issue of the fake accounts. He wrote, on Twitter, that his team would do its own analysis and expressed doubts about the accuracy of the numbers Twitter reported in its latest financial filings.
In its first quarter earnings report this year, Twitter acknowledged that there are a number of “fake or spam accounts” on its platform, as well as legitimate monetizable daily active users (mDAUs). ). The company said, “We performed an internal review of a sample of accounts and estimate that the average of fake or spam accounts during the first quarter of 2022 was less than 5% of our mDAU during the quarter.”
Twitter also admitted to overestimating the number of users by 1.4 to 1.9 million users over the past 3 years. The company wrote, “In March 2019, we launched a feature that allowed users to link multiple separate accounts to easily switch between accounts,” Twitter revealed. “An error was made at this time such that actions taken through the master account resulted in all linked accounts being counted as mDAU.”
While Musk may be rightly curious, experts on social media, misinformation and statistical analysis say his suggested approach for further analysis is woefully flawed.
Here’s what the SpaceX and Tesla CEO said he would do to determine how much spam, fake and duplicate accounts exist on Twitter:
“To find out, my team will do a random sample of 100 @twitter followers. I invite others to repeat the same process and see what they discover.” He clarified his methodology in subsequent tweets, adding, “Pick any account with a lot of followers” and “Ignore the first 1,000 followers, then pick all 10. I’m open to better ideas.”
Musk also said, without providing evidence, that he chose 100 as the sample size number for his study because that’s the number Twitter uses to calculate numbers in their earnings reports.
“Any reasonable random sampling process is fine. If many people independently get similar results for % fake/spam/duplicate counts, that will be telling. I chose 100 as the number of samples, because that’s what that Twitter uses to calculate <5% fake/spam/duplicate."
Twitter declined to comment when asked if its description of its methodology was accurate.
Facebook co-founder Dustin Moskovitz weighed in on the issue via his own Twitter account, pointing out that Musk’s approach isn’t truly random, uses too small a sample, and leaves room for massive errors.
He wrote, “Also, I feel like ‘not trusting the Twitter team to help pull the sample’ is its own kind of red flag.”
BotSentinel Founder and CEO Christopher Bouzy said in an interview with CNBC that his company’s analysis indicates that 10% to 15% of accounts on Twitter are likely “inauthentic,” including fakes, spammers, scammers, malicious bots, duplicates, and “simple hate accounts for deliberate purposes” that typically target and harass individuals, as well as others who deliberately spread misinformation.
BotSentinel, which is primarily crowdfunded, independently analyzes and identifies inauthentic activity on Twitter using a mix of machine learning software and teams of human reviewers. The company now monitors over 2.5 million Twitter accounts, mostly English-speaking users.
“I think Twitter doesn’t realistically categorize ‘fake and spam’ accounts,” Bouzy said.
It also warns that the number of inauthentic accounts may appear higher or lower in different corners of Twitter depending on the topics discussed. For example, more inauthentic accounts tweet about politics, cryptocurrency, climate change and covid than those discussing non-controversial topics like kittens and origami, BotSentinel found.
Carl T. Bergstrom, a University of Washington professor who co-authored a book to help people understand data and avoid being fooled by misrepresentations online, told CNBC that sampling of a hundred followers of a single Twitter account should not serve as “due diligence”. for making a $44 billion acquisition.
He said a sample size of 100 is orders of magnitude below the norm for social media researchers studying this stuff. The biggest problem Musk would face with this approach is known as selection bias.
Bergstrom wrote in a message to CNBC: “There is no reason to believe that followers of the official Twitter account constitute a representative sample of accounts on the platform. Perhaps bots are less likely to follow this account to avoid detection. Maybe they are more likely to follow to seem legit. Who knows? But I just can’t imagine Musk doing anything other than trolling us with this idiotic sampling plan.