This post will be somewhat of a Twitter inside baseball topic – not what I usually write about – so if you are not interested in Twitter arcana, you might want to skip this one.
Those of us on Twitter could not have missed the uproar that happened a couple of weeks ago when Twitter management decided to make a “small settings update” to its service by eliminating an option they said was “undesirable and confusing.” This change removed the option to see the one side of the conversations people you follow are having with people you don’t. Now you can only see these “half conversations” (called @replies) if you follow both people. Sounds like a small thing, but those of us who had chosen to see all @replies are now missing out on interesting conversations, resources and the opportunity to discover new people.
Fairly quickly, word spread across Twitter about the change and a revolt took place as people started tagging their protests with “#fixreplies,” which became the top trending topic on Twitter for a few days. After first seeming just clueless about how people use their service, Twitter offered a non-solution posing as a fix and then flat-out said they will not be bringing the option back for technical reasons. The reason from their post:
Even though only 3% of all Twitter accounts ever changed this setting away from the default, it was causing a strain and impacting other parts of the system.
Okay, given the millions of new users that have come on board in the past month or two in the wake of Oprah and Ashton Kutcher’s Twitter publicity stunts, it makes sense that the system is strained. But, having been on Twitter since around the end of 2007, I found it hard to believe that only 3% of the other users had touched their @reply settings. And given the extent of the outcry, either this was a very vocal 3% or a lot of people were jumping on the protest bandwagon even though the change did not affect them at all.
More likely, this was a disingenuous statistic chosen by Twitter to make their point, but that does not give the whole story. I suspect that they are counting 3% of anyone who has ever created an account on Twitter – including those who try it out for a day and never come back. A recent Nielsen study found that 60% of those who sign up do not return the following month (though this statistic does not take into account the many who sign up at Twitter.com but actively use TweetDeck or another client application). What if they looked at the percentage of active Twitter users (the people who should actually matter) — particularly those who have been on the service for a while? Would the percentage change?
This question nagged at me for a while until I decided to do a quick survey to see if my suspicions were right. I created a four-question survey, which asked the following questions:
- How long have you been actively using Twitter?
- How many people do you follow on Twitter?
- Before Twitter took away the option, how was your @Replies option set?
- How has the loss of the @Replies option affected your Twitter experience?
I sent out a tweet asking people to complete the survey and to retweet it (repost in Twitter parlance) to their followers as well. My objective was to send it far and wide on Twitter so that it was not just my own Twitter followers responding, but a wide swath of users across the service. The result was that people who were following my account retweeted the post 28 times, with a subsequent total of 118 retweets dispersed around different social circles. I ended up with 402 total responses to the survey.
I will be the first to admit that this sample may not necessarily be statistically representative of all active Twitter users (though if it were, the sample size gives us a 5% margin of error and 95% confidence level). Respondents were not chosen randomly, and the people who decided to participate may be more likely to have a strong opinion on the topic. Nonetheless, I think it may be helpful to take a look at the results because this segment of Twitter user has been strongly impacted by the change. (The results for each question can be seen here. I’m happy to share my full statistical analyses as well if you’d like to see them.)
Most respondents had been actively using Twitter for 3-12 months (40%), with 36% on for more than a year and 24% for less than three months. I figured that the longer someone had been using Twitter, the more likely they are to have played around with the options to see what they prefer rather than leaving the default of only seeing @replies when they follow both people.
A vast majority (63%) follow between 50-500 people on Twitter. Next is 501-5000 follows at 24%, fewer than 50 with 12%, and only 2% follow more than 5000. I hypothesized that those who were following more people would probably not notice much of a change in their cluttered feed.
Now, the kicker here is that before the option was taken away, 63% of the respondents had chosen to show all @replies for the people they followed — much higher than the 3% cited by Twitter. Those who had the default selected – to show only @replies between people they follow – were 19%, plus another 17% who said that they didn’t know what option was selected (and presumably hadn’t changed the default), for a total of 36%. And only 1% had chosen the option not to see any @replies unless directed at them.
Finally, 57% said that the loss of the @replies option had affected their Twitter experience for the worse. These were presumably those who had the option taken away from them, but could also be people who did not want to see all @replies for people who started making their replies visible to everyone, such as by putting a character before the “@” symbol or embedding the @reply name within or at the end of the tweet. Only 5% said their experience was better and 39% reported no change (close to the 36% who were already set at the default option).
I also ran some chi-square stats to see how these variables affected each other and created some nifty charts at Chartle.net. Here’s what I found (only reporting the statistically significant correlations at p< .05 href="http://www.social-marketing.com/blog/uploaded_images/Time-Following-707772.png">
The number of people that respondents were following on Twitter correlated with how long they had been on the service, at the highest and lowest following numbers. But most people – no matter how long they had been on – were comfortably in the 50-500 range.
Users who had been on Twitter for a longer time were more likely to choose the “show all @replies” option, with 72.2% of old-timers who had been on for at least a year and 65.2% of those on 3-12 months. Still, almost half (46.3%) of the newbies on for less than three months also selected that option.
Not surprisingly, given that time on Twitter and number following are correlated, the more people a respondent followed, the more likely they were to select the “show all @replies” option (
The quality of respondents’ experience on Twitter after the policy change, as you would expect, depended on which @reply option they had selected before all defaulted to showing only mutual @replies. For those with the “show all” option, 78.2% said their experience is worse, the direct opposite of the other two options (show only mutual=72.9%, show none=75.0%). The correlation between number following and quality of current experience on Twitter also mirrors the distribution of @replies option selected.
***
So what does this all mean? Even if this sample is not representative of all Twitter users, it does represent a substantial segment of users who are not as happy with their experience on the site since the option was taken away. Twitter would be smart to pay attention to this group, which is not only comprised of crotchety old-timers and “power users.” To avoid losing these disgruntled users, Twitter needs to come up with a way to bring back seeing all @replies in a way that they can live with. At the very least, Twitter needs to be honest about the percentage of its actual active users (not including abandoned accounts) who were using the “show all @replies” option. Whether it’s closer to 3% or 63%, by dismissing those who were upset by the #fixreplies kerfuffle as a tiny group of whiners, Twitter increased user dissatisfaction and the likelihood of defection should a service come along that works harder to meet its users’ needs.
UPDATE (6/1/09):
New research that has just come out from Harvard from a random sample of 300,000 Twitter users in May 2009 shows that the top 10% of Twitter users account for over 90% of tweets. And the median number of lifetime tweets per Twitter users is one. So there is a huge difference between the typical Twitter account and an active Twitter account.
This backs up my survey findings that many more active Twitter users were affected by the recent @replies option change than Twitter was willing to admit. To say that only 3% of users had selected the “see all @replies” option was extremely deceptive when it turns out that 90% or so of the total Twitter accounts are not even being actively used. Those who do use their accounts tend to opt to see all @replies. Twitter should not be able to so easily dismiss this loud, vocal majority.
Image Credit: monettenriquez
Nedra helps nonprofits and public agencies create positive change on health and social issues through social marketing and transmedia storytelling strategies at Weinreich Communications since founding the company in 1995. She helps organizations make a difference for the populations they serve by strategically designing programs that draw on state-of-the-art behavior change techniques, digital media approaches and the power of stories. 
