The Egyptian Revolution on Twitter

This is a preliminary result of the network of retweets with the hashtag #jan25 at February 11 2011, at the time of the announcement of Mubarak’s resignation. If you retweeted someone, or has been retweeted, it is possible that your username is one of these tinny points (or maybe a bigger one?).

To collect the network data, I used the Gephi Graph Streaming plugin, connected it to a Python web server I made myself. This web server works like a bridge, it connects to the Twitter Streaming API using the statuses/filter service and converts the users and retweets to nodes and edges in a network format that can be read by the Gephi Graph Streaming plugin. Nodes are twitter users, and links appear between the nodes A and B when B retweeted a message of A containing the hashtag #jan25.

The static network visualization is just the final result of about one hour of data collection. It is a dynamic network, and it’s possible to get much more information from the collected data. For example, before the announcement, there were few nodes and edges, sparse in time. But when the announcement arrives, a boom of retweets appears on the network. A video with the flow of retweets is available on YouTube. It shows the dynamic network construction during the hour of data collection, compacted in less than four minutes. During the collection, I run Gephi with the Force Atlas layout just adjusting some parameters from default: repulsion strength to 2000, attraction strength to 0.3 and speed to 10.



I was very lucky to get this data. On February 11 afternoon I was testing the Python server that works as bridge and connected to Twitter. I tried some interesting hashtags to see it working, and at the moment #jan25 seemed to be an active hashtag. I let the application run for some time, adjusted some parameters for visualization, and at some point there was a burst in the activity. I didn’t understood what was happening until I checked again my Twitter account and realized that the Egypt’s vice-president had just made the resignation announcement. After it, I proceeded collecting data, and the final result was this network. It was very interesting to see, in real time, the exact moment when Tahrir Square, from a mass protest demonstration, has been transformed in a giant party, and the burst in the Twitter’s activity. It was like covering in real time a virtual event, a big event that was happening in the Twitter virtual world.

After playing with the data, I found that the data I got through the Twitter Streaming API is only approximately 10% of the total. I’m now working to recover all data and hopeful soon I can make available the full graph of retweets.

Dataset available in a GEXF file here. Download it and play with it with Gephi!

André Panisson / www.

—–
This work is part of a research project involving the Computer Science Department of the University of Turin (www.di.unito.it), the Complex Networks and Systems Group of the ISI Foundation (www.isi.it), and the Informatics department of Indiana University (http://cnets.indiana.edu/).
—-

/seadragon-samples/twitter_jan25/seadragon.html

23 Comments

  1. Hi,

    Really interesting how the image resembles a stadium. It shows in a visual way that the community of retweets observe the first tweets. I am interested though on what tools you used to post the results here. I mean, I would love to know what did you use to record the construction of the graph? and what flash thing is that to visualize the network with zoom in and out.

    Congrats for your luck. 😀

    Reply

  2. Hi Arthur,
    The image layout is a direct effect of using the Force Atlas layout. Other layout implementations result in different type of visualization. I just choose the Force Atlas layout because you can run it indefinitely while you are connected to the stream. The main tools I used to collect the data: Gephi with Graph Streaming plugin, connected to a Python web server and Tweepy library to connect to Twitter Streaming API. To produce the network visualization you can zoom in and out, I used the Gephi Seadragon plugin.
    Thank you for your comment!

    Reply

  3. Hi,

    this looks great! Any possibility of sharing the conversion code (tweets-to-graph)?

    There is scary side to this also: what is the Egyptian government had applied this and filtered the active nodes? The ethical side of SNA+ mining needs consideration.

    Reply

  4. I agree with Olli.
    On the first sight it looks fascinating, but my second thought was that this is an also ugly powerfull tool for an oppressing regime to track down activists. Egypts have won so far, but if you look at Bahrain, Libya, Iran .etc. right now I’m bit scared of the opportunities of such a tool…..

    Reply

  5. Hi!
    I was stonished with your work
    I spend 7 years working on a programming language. 2 years ago I dream (yes, I work too on dreams) how to build it with fractals

    The whole reality is made with fractals, even I suspect that Totallity (someone call it God) is a fractal

    Do you realize that this video shows how a cell organism is created in a fractal way?

    Now, I friend of mine ask me to write an post explaining this belive but in order to write it I need to edit your video. I want to erase the black scenes where you explain the time line

    I want to convert the source of this infography in a surprise for the readers

    In order to give them the reallity of the source I will link with this post, thats the surprise: when the readers realize that this fantastic infography is made with twitts they will be as astonished as I was

    So, I need your confirmation that you are agree with this idea and ask you for your permission to do that (remember that the reader will need to visit your post for the solution)

    Are you agree with my idea? May I have your authorization to do that?

    Please, email me with your answer

    Thanks for that beautiful and useful work

    PD: I will contact you when the post will be online. My post will be in spanish: ask me if you have problems to translate it!

    Reply

  6. For those of us who aren’t familiar with the tools: Could you explain informally the meaning of what the graphs show? The animation begins with a a small circular cluster of paired nodes, apparently users who retweeted each other. Does an edge between A & B mean that one retweeted the other just once? Exceeding a frequency threshold? Do idle tweepers fade out; or is it once in, always in? As the action unfolds, clusters of nodes get “pulled in” from the exterior. Some clusters move fast, some more slowly. What does the speed represent? Proximity to center? What is the total elapsed time? Start & stop times? A clock in the corner would have been nice.

    The graph grows to hundreds of nodes. Yet the initial cluster remains stable and motionless, and unconnected from the hordes gathering around the perimeter. As a commenter above noted, the final picture resembles a stadium: big annular crowd watching the initial circle on tyeh central field of play. (For me, it called to mind the process if chromosomal replication & separation during cellular mitosis.) Are those initial pairs continually retweeting each other? What does proximity to the center represent? It would be helpful to have a play-by-play, for a spectator like me who doesn’t much understand this strange sport.

    In the lower picture: What does color mean?

    Finally, it might be interesting to point out some star players. Who and where are the top-10 or -20 “popular” nodes, ranked by degree? How about the famous players? I’m hardly in a position to list a complete roster of Tahrir celebrities, but among them, I wager, are three English-tweeters who I followed during the Tahrir demonstrations: @Ghonim (the Egyptian Google exec who put up the Facebook page which called for the Jan25 protests), @SandMonkey (long-time blogger; was beaten and arrested). and @MonaEltahawy (freelance Egyptian journalist who appeared dozens of times on CNN, MSNBC, CBS, and other US TV media). I looked, briefly, but couldn’t find them in the haystack — perhaps they never (re-)tweeted during the time interval you captured.

    Reply

  7. Hi, again!
    Because I don’t have any kind of communication from you I assume that you are agree with my proposal and I will publish the post today or tomorrow

    I will post here the link to the post when it will be ready

    Thanks again!

    PS: I hope you like it!

    Reply

  8. Ägyptische Revolution visualisiert…

    Die Einfachheit von Twitter und Co. sowie der Hohn, das Internet abzuschalten hat ja viel dazu beigetragen, dass die Revolution on Ägypten stattfinden konnte. ( Der wirkliche Grund/Gründe sind ja andere, aber das war das Futter für die Massen ) Wie sic…

    Reply

  9. Hi!
    As I promise here: http://vimeo.com/20201993 is the link to the video (see I only drop out the black screens that explains the temporary landmark (to avoid watchers clues what the infography represents. I’m trying to evocate them to see what suggest the image for them), add music and some final credits where you will see the reference to this post

    I spect to see my post published this week. I’ll came back here to point you the link

    Thanks again!!!

    Reply

  10. Hi,

    How I can produce the same animation in Gephi with the given twitter data set above as you shown in the video ? Its awesome. I couldnt produce the same animation as you published above. Please help..

    Reply

  11. HI there. I am authoring a book about teaching with emerging technologies and I’d like your permission to include a still image from your visualization of the Egyptian revolution on Twitter. Could you kindly email me? Thank you!

    Reply

  12. Thanks for sharing! I just wonder is there any time stamp attribute in the data since I don’t see temporal variable in the data laboratory. thanks.

    Reply

  13. please suggest me something..from where i can scrap data from twitter….i need a connected set….it should contain username,tweet,retweet,comment..etc…dataset of any other social networking site like fb will be ok…

    Reply

Leave a reply to Opinio Cancel reply