Gephi is a graph visualization and analysis platform – the entire tool revolves around the graph the user is manipulating. All modules (e.g. filter, ranking, layout etc.) touch the graph in some way or another and everything happens in real-time, reflected in the visualization. It’s therefore extremely important to rely on a robust and fast underlying graph structure. As explained in this article we decided in 2013 to **rewrite** the graph structure and started the *GraphStore* project. Today, this project is mostly complete and it’s time to look at some of the benefits GraphStore is bringing into Gephi (which its 0.9 release is approaching).

Performance is critical when analyzing graphs. A lot can be done to optimize how graphs are represented and accessed in the code but it remains a hard problem. The first versions of Gephi didn’t always shine in that area as the graphs were using a lot of memory and some operations such as filter were slow on large networks. A lot was learnt though and when the time came to start from scratch we knew what would move the needle. Compared to the previous implementation, GraphStore uses simpler data structures (e.g. more arrays, less maps) and cache-friendly collections to make common graph operations faster. Along the way, we relied on many micro-benchmarks to understand what was expensive and what was not. As often with Java, this can lead to surprises but it’s a necessary process to build a world-class graph library.

## Benchmark

We wanted to compare Gephi 0.8.2 and Gephi 0.9 (development version) so we’ve created a benchmark to test the most common graph operations. Here is what we found. The table below represents the relative improvement between the two versions. For instance, “2X” means that the operation is twice faster to complete. A benchmarking utility was used to guarantee the measurements precision and each scenario was performed at least 20 times, and up to 600 times in some cases. We used two different *classic* graphs, one small (1.5K nodes, 19K edges) and one medium (83K nodes, 68K edges) . Larger graphs may be evaluated in a future blog article.

Benchmark / Graph | SMALL (n=1490, e=19025) | MEDIUM (n=82670, e=67851) |
---|---|---|

Node Iteration | 23.0x | 34.6x |

Edge Iteration | 40.1x | 109.4x |

Node Lookup | 1.6x | 2.1x |

Edge Lookup | 1.2x | 2.3x |

Get Edge | 1.1x | 1.2x |

Get Degree | 2.5x | 2.3x |

Get Neighbors | 3.4x | 1.2x |

Set Attributes | 2.3x | 0.1x |

Get Attributes | 3.3x | 4.0x |

Add Nodes | 6.2x | 5.7x |

Add & Remove Nodes | 1.4x | 2.9x |

Add Edges | 7.7x | 3.8x |

Add & Remove Edges | 3.3x | 1.8x |

Create View | 2851.0x | 4762.3x |

Iterate Nodes In View | 2.7x | 1.5x |

Iterate Edges In View | 11.6x | 7.3x |

Save Project | 2.4x | 1.7x |

Load Project | 0.6x | 0.6x |

Project File Size | 1.9x | 1.5x |

These benchmarks show pretty remarkable improvements in common operations, especially *read* ones such as node or edge iteration. For instance, in average it takes **40 to 100 times** less CPU to read all the edges in the graph. Although this benchmark focus on low-level graph operations it will bring material improvements to user-level features such as filter or layout. The way GraphStore creates views is different from what we were doing before, and doesn’t require a deep graph copy anymore – explaining the large difference. Finally, only the set attribute is significantly slower but that can be explained by the introduction of inverted indices, which are updated when attributes are set.

And what about **memory** usage? Saving memory has been one of our obsession and there’s good news to report on that front as well. Below is a quick comparaison between Gephi 0.8.2 and Gephi 0.9 for the same medium graph above.

Benchmark | Gephi 0.8.2 | Gephi 0.9 | Improvement |
---|---|---|---|

Simple graph | 115MB | 52MB | 2.2X |

Graph with 5 attribute columns | 186MB | 55MB | 3.4X |

This benchmark shows a clear reduction of memory usage in Gephi’s next version. How much? It’s hard to say as it really depends on the graph but the denser (i.e. more edges) and the more attributes, the more memory saved as significant improvements have been made in these areas. Dynamic graphs (i.e. graphs that have their topology or attributes change over time) will also see a big boost as we’ve redesigned this part from scratch.

## What’s next?

All of the GraphStore project benefits are included in the upcoming 0.9 release and that’s the most important. However, the work doesn’t end and there’s many more features and performance optimization that can be added.

Then, we count on the community’s help to start collaborating with us on the GraphStore library – calling all database and performance experts. GraphStore will continue to live as an **all-purpose Java graph library**, released under the Apache 2.0 license and independent from Gephi (i.e. Gephi uses GraphStore but not the opposite). We hope to see it used in other projects in the near future.

Reblogged this on 木秀于林.

[…] Gephi boosts its performance with new “GraphStore” core […]

[…] o “core” do Gephi foi redesenhado e agora ele é leve, mas muito mais leve. Segundo o Blog do Gephi a nova versão reduziu de 40 a 100 vezes o tempo de processamento da CPU em algumas operações. Em […]

[…] Gephi boosts its performance with new “GraphStore” core […]

[…] Gephi boosts its performance with new “GraphStore” core by Mathieu Bastian. […]