Exploring the dystopian future of a Javascript Gephi

Despite Graph Commons, Graphistry, Linkurious or Keylines, there is no equivalent of Gephi in web technologies – notably free and open source. But what if?

We gave a talk at the FOSDEM 2019 on that matter. “We” is Eduardo Ramos Ibáñez, our lead developer, and Mathieu Jacomy, co-founder and network science researcher, teaming up with experts of Javascript network visualization Alexis Jacomy (Sigma.js) and Guillaume Plique (Graphology). The FOSDEM (Free and Open Source Developers’ European Meeting), is a two decades old conference hosting about 4000 visitors every year. The 2019 edition featured 711 speakers, 746 events, and 62 tracks. It is a major moment of the European open source community. You can look at our ~40 min talk in video below, served with its slides.

Our slides:
https://docs.google.com/presentation/d/1SAvbDRgDVLOt5VO_hu0QPDc1OL55yVUlyFGBmj7UrSQ

A quick summary of our talk

We have to face it: the multiplatform is moving from Java to web technologies. Oracle wants a Java that powers backends, not a user interface framework. Gephi’s OpenGL engine has maintenance issues (JOGL is not maintained anymore). At the same time, modern Javascript is powerful and online graph visualization is a thing. So, should we move to web technologies?

Eduardo has developed from scratch a new rendering engine that fixes our current OpenGL issues and improves the performance of Gephi. It has a lower CPU overhead, which provides a better scalability and better leverages the GPU power. It can be used as a library, and it crashes less thanks to its ability to fall back on supported features on older graphic cards. A key to these benefits is the shader-based architecture. Though the engine is still lacking some features (labels…) a demo is available on GitHub, (requires you to build).

On the Javascript side of the situation, graph visualization can be surprisingly efficient, but it comes with a specific kind of challenges. On the bright side, developing interfaces for the web is easy, that is even what it is made for! Web apps are portable, can work on mobiles and tablets, and even be packaged as applications (Electron). But on the flip side, memory boundaries are unpredictable (we cannot tell when an app will crash because of RAM usage), there is no proper multi-threading, and WebGL is only a subset of OpenGL. Gephi is in a specific spot because it is not a simple app (graph visualization has its own requirements) but at the same time we want to benefit from the traditional web app development to improve the user experience. Because of that the classic web development strategies are not sufficient, but we do not want to embrace the “web as a JVM” perspective of compiling C++ or Rust to WebAssembly. “Gephi JS” would need a hybrid approach. It would also require to rethink current Gephi, but this is something we are going to do anyway.

We have made a small indicative benchmark comparing current Gephi engine to the new OpenGL engine and to a WebGL engine (Sigma.js v2 alpha). It turns out that the current Gephi engine is sensibly outperformed even by the WebGL engine, as you can see below!

As you can see below, all engines experience a performance drop around 10-100 thousands nodes or edges. The intensity of this drop varies, but it is pretty clear that after 10 million items, a normal computer cannot display a network smoothly enough to allow interactions (it lags too much). That being said, scaling up to hundred thousands nodes/edges is quite a lot already!

Ultimately, we believe that web technologies are the new multiplatform for graph visualization. It comes with very real challenges, but it is also a perfectly valid option. It does not mean that we will drop the Java Gephi, but that we are starting to think Gephi as a project hosting multiple tools and not only as a single piece of software, and that the web technologies will be part of its future.

A screenshot of the new OpenGL rendering engine

9 Comments

  1. I would also like to add http://infranodus.com to the list of network visualization and analysis tools. It’s open-source, free, it makes it possible to add data much faster than GraphCommons, it supports import / export and has graph analytics. It’s based on Sigma.Js, Graphology and Cytoscape. My only question is why it always gets omitted from all the overviews and what are the reasons for it not being taken seriously?

    Reply

    1. Hi Deemeetree!

      From looking at the website and trying the demo, it seems to me that Infranodus is a text analyzer using graph visualization. Which is great! It produces its own network by analyzing text, but it seems to me that it is not designed to load your own network data. In that sense, it is not a graph visualization tool like the others I mention, but rather a text analysis tool using graph visualization. Which is a perfectly respectable design – a good design is always specific, and it clearly allowed you to offer a clear and lean user experience.

      You seem however to say that there are such features in Infranodus. Could I come up with my own network, load it and make use of your analytics?

      I ask this question because I did not perceive Infranodus as a network analysis tool, but as a text-based visualizer/analyzer, which is why I did not think of it when I wrote this piece. It seems to me that the Infranodus website also chooses to frame it as a text tool and not a network analyzer.

      Reply

      1. Hi Mathieu, thank you for your response!

        Yes, in fact one of the advantages of InfraNodus is that it has one of the fastest interfaces to build your own graph from a scratch, as fast as typing.

        You can input nodes very easily like #NodeA #NodeB #NodeC relationship_description to make a simple triplet, for instance. You can also decide to have a central node (@MainNode) which would link to all the other nodes (e.g. @MainNode #NodeA #NodeB #NodeC). I haven’t seen any other tool that would allow you to create a graph so quickly (in GraphCommons you’d have to manually add every node and connection). It’s like a very efficient language to “write” graphs. Text network visualization is just an extension of that main feature.

        You can also import a simple CSV file and visualize it as a graph. If it’s just a list of statements they will be visualized as a text graph, but you could also make a file that has two columns — source and target — and that would make your graph more precise. I’m also working on providing the Gexf import, which should be available soon.

        In any case, your feedback makes me think that these features of InfraNodus should be made more visible, so I’m going to write a Medium post about it and hopefully then InfraNodus will come up more often in people’s mind when they think about web-based graph visualization.

        Also, I really owe the inspiration to create the tool to the Gephi community and also your brother who created the amazing Sigma.Js library, which is at the core of InfraNodus, so I’ll be happy to give you the invitation code if you don’t want to bother installing the hosted version and want to play around with the online one. Just send me a DM or an e-mail!

    1. Not really for the moment. The closest you can get at the moment is to take a try at MiniVan:
      https://medialab.github.io/minivan/

      It’s a prototype project specifically targeted at releasing already made networks. The project is currently under development but it can already be used. It allows testing the technology. You must note, however, that it does not allow making your own network maps. It has been designed more as a complement to Gephi than a replacement. It takes a GEXF as an input. The technology is the same as a possible Gephi JS, however. So if the libraries used here are satisfying, it will pave the way.

      How to articulate the Java software with the JS software is still an unresolved question. A question of design more than technology. We know for sure that none of these options can replace the other one, and that makes everything complicated. It’s clear that a JS Gephi will not be able to scale as much as Java, and very big networks are a real thing.

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s