The HTTP Graph plugin

The HTTP Graph plugin provides real-time collection and visualization of HTTP traffic. Using the embeddable Membrane Router, details are extracted from the transaction headers and fed to Gephi for graphing and further analysis. This approach makes the relationships between clients, servers, and resources easily visible.


See the video in HD on Vimeo.

Nodes

There are 4 types of nodes: client, uri, host, domain.

Client: By default, the largest sized nodes with the source IP addresses of clients for labels. If you are the only one pointing to the plugin’s proxy, there will probably be only one of these nodes that says 127.0.0.1. Clients are linked to a domain node of ‘local’ to keep them together on the graph. Another function of the client node is to keep the graph anchored. You may find it interesting to use a filter in Gephi to hide the client type nodes to see a more “free-form” graph of the internet. If you do this, you may see large pieces float away because they didn’t link to the rest of the graph anymore!

URI: By default, the smallest sized nodes with no visible labels. These represent resources like HTML pages, images, javascript, or whatever other resources might be requested through the proxy. The size in bytes and the MIME (Content-Type) reported by the host when returning the resource is available so you can see what it is.

Host: For a given domain (.gephi.org, .google.com, etc.) there can be multiple hosts which serve the different resources. In some cases, you may see the same resource being served from multiple servers in a DNS-based load balancing system. Other interesting details about the underlying architecture of the sites you are viewing can be seen.

Domain: These nodes exist primarily to keep the related hosts close together on the graph. You may want to use a filter in Gephi for this type of node and hide them to see a different perspective.

Edges

HTTP and the web are defined by links, which are essentially directed graph edges, and these occur at the resource level. An HTML page resource will link to CSS, image, and other file resources, both on the same domain, and on remote domains. These inter-domain links are the glue that forms the structure of the world wide web.

Have fun!

~by phreakocious

Get the HTTP Graph plugin on the Plugins Center, or in Gephi go to Tools > Plugins > Available plugins.

4 Comments

  1. Cool!

    I checked your video and looked into the advanced settings of my Mozilla browser: there, the settings (http proxy + port) that you need to copy paste in the plug-in were empty. Would you have any clue on how I should proceed?

    Best,

    Clement

    Reply

  2. Clement,

    Hopefully you will notice the reply. =) The Firefox settings are how you tell the browser to direct the requests through the plugin. In general, the IP address you want is 127.0.0.1. This is a reserved address that means localhost or “this computer”. The port number comes from the dialog from the plugin, which is 8088 by default.

    When you’re done using the plugin, you will need to remove those settings from Firefox to restore your normal browsing.

    Good luck..

    Reply

  3. Hi Gephi folks,

    I’m an archaeologist. I used this plugin as well as the Navicrawler plugin for firefox to do some structural analysis of the networks I participate in as a blogger and archaeologist, as well as some generic ‘archaeological’ search terms on Google. Thought you might all be interested; paper is at:
    http://bit.ly/fVYPYr “Signal vs Noise: Why Academic Blogging Matters, a structural argument”

    Reply

  4. I’ve tried httpgraph on Ubuntu and MacOS, but struggle to get the plugin listener running, any ideas? I ‘ved checked using nmap and NC, unfortunately it doesn’t seem to start.

    Reply

Leave a reply to phreakocious Cancel reply