A close look at the Gephi user community

Last December we asked Gephi users to participate in a survey. The survey’s main objective was to better understand who users are and what kind of projects they work on. One important dimension we wanted to explore was the diversity of the user community. Through the projects we’ve seen in research and on the web we knew that Gephi users were diverse, but we wanted to quantify it. Ultimately, we aim to make the tool better so it supports users’ needs, but this is a process that requires first a good understanding of who the audience is and what are their objectives. Below we summarized our findings about the profile of users, the types of networks they work with and finally useful usage statistics the community can reflect on.

Profile

The largest share of Gephi users work in academia. The project started in the academic sphere from where it has spread into business, artistic and non-profits domains as well. Working at a profit organization is the second most common occupation, which confirms that network analysis is no longer reserved to scientists.

surveyq12

Q12. What is your occupation? n=285; multiple choice

Given that the largest group of users works in academia, it is not surprising that the most common title among Gephi users is a researcher.

surveyq14

Q14. What is your title? n=285; multiple choice

The user community is also widely spread around the world. Users from 46 different countries participated in this study. This confirms the importance of localization for as many languages as possible (Gephi currently supports eight). While many countries were represented by only a handful number of participants in the study, large concentration of users is, as expected, in the US (23%) and in France (15%). Significant presence in France is predetermined by Gephi’s presence in universities and businesses within which Gephi was originally founded.

Networks

Social networks are by far the most commonly analyzed type of networks when using Gephi. 70% say that they typically analyze social networks when using Gephi. Social media and semantic network analysis are also common and typically analyzed by 46% and 43%, respectively. The rest of the networks are less common with ecological network analyzed by about 5% of users.

Despite SNA (Social Network Analysis) being the dominant use there is a large variety of other use as well. That said, networks can be analyzed only if the data are accessible and we (the community) still have work to do to ease network collection and formatting.

We always wondered if given occupations are more likely to work with specific types of networks. Based on this study, some differences exist, but they are not as prominent as we have expected. We found that people working at profit organizations are more likely to use Gephi to analyze business and financial networks. While in total 24% use Gephi to analyze business network, it is 44% among those who work in a profit organization compared to only 12% among those who do not work in a profit company. Differences for other types of networks were not conclusive.

surveyq5

Q5. What type(s) of network do you typically analyze using Gephi? n=285; multiple choice

Gephi users commonly deal with a wide range of network sizes. Although the typical network has between 100 to 10K nodes, every size from <100 nodes to 1M nodes represent at least 10% of users. In total that is more than 5 orders of magnitude difference in data size, and without taking edges in consideration!

surveyq6q7

Q6. What is/are the graph size(s) you deal with when working with Gephi? n=285; multiple choice
Q7. And what is the TYPICAL size of a graph that you manipulate with Gephi? n=285; single choice

While more than half of Gephi users have never used Gephi to analyse dynamic networks, the vast majority of the community is likely to use it in the future.  This confirms the importance of the set of features related to dynamic networks that has long been one of Gephi’s primary focus.

surveyq8

Q8. Have you ever used Gephi to work with dynamic networks (networks over time)? n=285; single choice

surveyq9

Q9. How likely are you to use Gephi to analyze dynamic networks (networks over time) in the future? n=285; single choice

Usage

Both online and offline sources are important touch points through which people learn about Gephi for the first time. While web search is the most common way how people find Gephi, word of mouth remains an important channel and is not to be underestimated.

surveyq2

Q2. How did you first learn about Gephi? n=285, single choice

The community is very diverse when it comes to usage frequency which suggests that Gephi users are likely to have diverse needs. Occasional users are likely to have different expectations from a software than regular users.  About one third uses Gephi at least once a week which confirms that there is a relatively large base of heavy users who use Gephi regularly.

surveyq3

Q3. On average, how often do you use Gephi? n=285; single choice

Online tutorials and online forums are key sources for users to learn about Gephi. This confirms the importance of creating and updating online tutorials. It also suggests that the community is well engaged to be able to provide answers one another on online forums and groups.

surveyq4

Q4. What source(s) have you used/are you using to learn how to use Gephi? n=285, multiple choice

Conclusion

This survey is a first, yet important step in understanding the Gephi user community at large. It also gives a general overview of the network visualization and analytics field and we hope this can be useful for others as well. But for us – the Gephi leadership team – this will help us in our future community management efforts. It will also help design a better tool in the future as we better understand its user community.

In addition, talking about what kinds of projects users work on also helps shape the understanding of what network analytics is used for, and ultimately bring more people to the community. In the near future we want to double-down on this topic and start a series of articles highlighting the most interesting projects. Many of the respondents indicated their willingness to share what they have worked on so there’s already plenty to choose from.

Finally, to reflect on the diversity of users we believe it simply reflects that networks are everywhere. Analyzing networks bring insights and answers to many different problems.

separator

Appendix
  • Survey was conducted among Gephi users community. While the results provide a unique view into the Gephi community it is important to clarify that they are not meant to be representative of the entire community world wide.
  • The survey invitations were distributed throughout the week of Dec 1st 2015 via email, Twitter and Facebook
  • Final data set contains responses collected between Dec 1st 2015 and Dec 23rd 2015
  • A total of 285 participants completed the survey

Plugin development gets new tools and opens-up to the community

Since the introduction of the Gephi Marketplace and tools such as the Plugins Bootcamp we’ve seen more and more plugins being developed. Even developers with little experience with Java give it a try and succeed in creating their first plugin. We want developers to be productive and make it as easy as possible to get started with plugin development and find help along the way. As the release of the 0.9 version is near, it’s time to review our plan on that matter and upcoming improvements. Here’s the summary:

  • The gephi-plugins base repository (i.e. repository plugin developers fork) is now using Maven for building and is simpler. It contains only 4 files versus 890 for the Ant-based system.
  • All Gephi modules are published on Maven central, making it very easy to inspect and extend.
  • Introduction of a custom Maven plugin designed to facilitate plugin development.
  • The submission and review of plugins will be entirely based of GitHub, making it more scalable and transparent.
  • A new online portal for plugins is coming up with an easier edit experience and new features.

From Ant to Maven

Before diving into plugins, let’s first review what has changed on how Gephi is compiled, built and packaged – as this directly affects plugins as well. Since the Gephi 0.8.2 version we have migrated our build system from Ant to Maven. This is in line with what the Netbeans Platform (i.e. which Gephi is based on) community recommends. It already has increased the level of automation we’re capable of as a result. The main benefits are (compared to Ant):

  • Maven is great at dependencies management. It’s now very clear what version of what library Gephi depends on, making it simpler to integrate. Dependencies are also downloaded automatically instead of being checked in the codebase
  • Unlike the Ant-based system, it’s independent from Netbeans. This allows developers not using Netbeans to develop Gephi and produce a build entirely from the command-line.
  • Gephi modules can now be placed on Maven Central (i.e. global repository where Maven finds its dependencies). This allows plugins to automatically find the Gephi dependencies online, reducing the manual steps at each Gephi upgrade.

Build assistant

There are a few critical steps we want to help plugin developers with and as a result started the development of a custom Maven plugin. This new tool will work behind the scenes when developers build their plugin. No installation or configuration is needed as it comes already as dependency of the gephi-plugins module. It already addresses common pain points and hope to automate more and more of the steps in the future. This is what it can do as of today:

  • Plugin validation: The assistant reviews the plugin configuration and metadata at each build. This allows for instance to check if the plugin depends on the correct Gephi version or remind the developer to define an author or license in its configuration.
  • Run Gephi with plugins: A single command allows to run Gephi with the plugins pre-installed. This makes testing faster than ever when developing plugins.
  • New plugin generator: A step-by-step command-line tool that creates the correct folder structure and configuration to get started.

In the future, we want to rely on this build assistant to further automate the process and for instance do easy migration or code generation. For instance, you could ask to generate a Layout plugin code and configuration. Afterwards, all needed would be to fill in the blanks in the code.

A new way to review and submit plugins

As the number of plugins grows, it’s important to have a clear process how plugins are reviewed and updated. We also want this process to be transparent and open to the community. So far, the process was based on the submission of the plugin binaries with a manual review done by the team. This helped us get where we are today, but we want to get it to the next level and propose to entirely move this process to GitHub – using the pull-request mechanism. This has multiple advantages, listed below:

  • Reviewing new/updated plugins can scale because any developer can read the code and contribute to the pull requests.
  • Developers are already asked to fork the gephi-plugins repository so submitting the plugin via GitHub is a natural extension to it.
  • There’s a clear history of each version, comment and what code has changed from one version to another.
  • It makes it easier to test plugins and detect issues before the plugin is approved.

As part of this migration, we’ll no longer add plugins with closed source code but all existing plugins for Gephi 0.8.2 will remain available. For security and stability reasons, it’s essential that each plugin’s code can be inspected before approval. In order for this to work, all existing plugins not already on GitHub or not forking the gephi-plugins repository will need to migrate. For those already set up, the migration will be easier but Ant-based plugins will still need to migrate to Maven.

To summarize, this is what the new 4-steps process looks like for developers:

gephi-new-plugin-development-process

In the current submission process we ask for additional information such as description, author or license as well as allow the upload of images. Going forward with GitHub, all of these data will directly be defined in the plugin’s configuration making it easier to update.

A new home for plugins (again)

Plugins are currently available online from the Gephi Marketplace, where users could also reach people providing teachings and support.  We have ideas on how to improve these community services and will be migrating them to a new architecture, starting with the plugins. We will tell you more about these changes in an upcoming post but for now our focus is on developing a new lightweight plugin portal that can directly be connected with the data source on GitHub.

Here is a preview of what it will look like for plugin pages:

new-plugin-frontend-preview

 

The content of this website will be automatically updated when plugins are published or updated. The way it works is with Travis CI (i.e. continuous integration platform) simply refreshing the JSON file after changes to the plugin repository on GitHub. Developers can even embed images and write the description in Markdown. This will remove entirely the need for plugin developers to login to the marketplace, update NBMs and metadata.

Migrating plugins

This new Maven-based repository along with the new submission process will be introduced with the Gephi 0.9 release. Let’s review what plugin developers need to know to bring their plugin to this new major version.

As with all major Gephi release, plugins compatibility needs to be evaluated as APIs may have changed. In fact, given this new version is based on an entirely redeveloped core it’s very likely code changes will be required. Hopefully, these changes will often be minor and actually simplify things (i.e less, more efficient code). Documentation will be published on these API changes and core developers will be available to answer questions as well.

Plugin developers will also get contacted regarding moving their code to GitHub with a step-by-step guide. We’re considering adding a migrate command to the new Gephi Maven plugin to facilitate the transition from Ant but that’s an unfunded project at the moment (if you’re interested contributing to that, please let us know). Stay tuned for details right after the release on the path to migration.

And again, thanks for all your hard work on bringing your ideas to life though new Gephi plugins!

 

Announcing Gephi 0.9 release date

Gephi 0.9

Gephi has an amazing community of passionate users and developers. In the past few years, they have been very dedicated creating tutorials, developing new plugins or helping out on GitHub. They also have been patiently waiting for a new Gephi release! Today we’re happy to share with you that the wait will come to an end December 20th with the release of Gephi 0.9 for Windows, MacOS X and Linux.

We’re very excited about this upcoming release and developers are hard at work to deliver its roadmap before the end of 2015. This release will resolve a serie of compatibility issues as well as improve features and performance.

Our vision for Gephi remains focused on a few fundamentals, which were already outlined in our Manifesto back in 2009. Gephi should be a software for everyone, powerful yet easy to learn. In many ways, we still have the impression that we’ve only scratched the surface and want to continue to focus on making each module of Gephi better. As part of this release, we’ve undertaken one of the most difficult project we’ve worked on and completely rewrote the core of Gephi. Although not very visible for the end-user, this brings new capabilities, better performance and a level of code quality we can be proud of. This ensure a very solid foundation for the future of this software and paves the way for a future 1.0 version.

Below is an overview of the new features and improvements the 0.9 version will bring.

Java and MacOS compatibility

This release will restore Gephi’s compatibility with the latest Java versions on all platforms. This will resolve issues our users encounter with Java 7 and 8. Compatibility issues with Mac OS will also be resolved and full support for Retina display screens added.

New redeveloped core

This release will improve performance and reliability by a large margin. The graph structure at Gephi’s core has been redeveloped from scratch and will bring multiple new features, better performance and lower memory consumption. On benchmarks, simple operations such as reading nodes or setting attributes see performance improvements ranging from 2X to 100X. This new core will make many operations in Gephi faster and push the envelope even further in large graphs exploration. Reducing memory usage has also been an area of focus and we have measured a 2X reduction compared to Gephi 0.8.2 on a medium-size graph.

New Appearance module

We’re introducing a new user module named Appearance designed to combine and replace Ranking and Partition modules. Appearance will group in one place all controls acting on the node or edge appearance. The partition capabilities will also greatly improve as part of this new module and a new palette selector is being added. In addition of the default palettes, we’re also adding a cool palette generator designed to find the optimal colors (i.e. partitions can be differentiated from each other). Moreover, it will be possible to “Auto-apply” partitions as well, which is a feature that was only available for Ranking so far.

Appearance Partition Palette selector Palette Generator

Timestamp support

This 0.9 release adds a new way to represent networks over time: timestamps. So far, users could only represent time using intervals and that was cumbersome when the underlying network data was collected at fixed time intervals (e.g. one network per day). Starting with this release, Gephi will support both intervals and timestamps to represent evolving network topology and/or evolving attribute values.

GEXF 1.3 support

The GEXF format is also evolving to its 1.3 version. This version improves the support for graphs over time and introduces the ability to represent time using timestamps rather than intervals. In addition, it’s now possible to set a timestamp or an interval for the entire graph, allowing building collections of GEXF files where each represents a “slice”. This is a common request from the community and we hope this will greatly facilitate the exploration of longitudinal networks.

Multiple files import

With the 0.9 version users will be able to import multiple files in Gephi at the same time. Once the files have been read, two choices are offered, either to import each file into a separate workspace or merge them into the same workspace. The latter is a powerful option when used with dynamic graphs. Indeed, a collection of GEXF files representing the same network over time will be imported in Gephi in a single step.

Import Report Multiple Graphs

Multi-graph support

Multi-graph exampleThis release will bring support for multi-graphs, where multiple edges can exist between two nodes each with a different relationship. Users will be able to import, filter and run algorithms on these graphs but the support for visualizing these graphs will come in a further release.

New workspace selection UI

We’ve heard users’ feedback and the workspace selection user interface will be improved. The new interface will be a “tab-style” interface where each workspace is a tab and switching from one workspace to another only requires a single click. Tabs will be located at the top just under the perspective selection. The previous interface is located at the bottom right corner and will be entirely removed.Workspace Tabs

Gephi Toolkit release

A new release of the Gephi Toolkit, based on the 0.9 version will be made soon after December 20th.

Bug fixes

We’ve done a serious bug squash and already addressed many difficult issues, more to come until the release date.

separator

Follow us on Twitter or join the Facebook group to get the latest news. If you want to know more about this upcoming release, or want to help out please send us a note.

Google Summer of Code 2013

It’s a great news, Gephi has been accepted again for the Google Summer of Code for the 5th year! The program is the best way for students around the world to start contributing to an open-source project. Since 2009, each edition is a great success and dramatically boosted Gephi’s project development.

What is Gephi?

Networks are everywhere: email systems, financial transaction systems and gene-protein interaction networks are just a few examples. Gephi began as a university student project four years ago and has quickly become an open source software leader in the visualization and analysis of large networks. It is an important contribution to the ecosystem of tools used by researchers and big data analysts to explore and extract value from the deluge of relational data and disseminate a better understanding for people to think about a “connected” world.

Gephi is a “Photoshop” for graphs: designed to make data navigation and manipulation easy, it covers the entire process from data importing to aesthetics refinements and communication. Users interact with the visualization and manipulate structures, shapes and colors to reveal the properties of complex and messy data. The goal is to help data analysts make hypotheses and intuitively discover patterns or errors in large data collections.

Gephi’s project aims at providing the perfect tool to visualize and analyze networks. We focus on usability, performance and modularity:

  • Usability: Easy to install, an UI without scripts and real-time manipulation.
  • Performance: Visualization engine and data structures are built scalable. Supporting always-larger graphs is an endless challenge!
  • Modularity: Extensible software architecture, built on top of Netbeans Platform. Add plug-ins with ease.

Learn more about Gephi, watch Introducing Gephi 0.7, download and try it by following Quick Start Tutorial.

Gephi’s project is young, the growing community is composed of engineers and scientists involved in network science, datavis and complex networks.

List of ideas

List of ideas are availabe on our wiki. They cover various skills and level of difficulties:

* Completing Legend moduleComplete the Legend module, which was started in last year GSoC.
* GraphStore benchmark and tuningOptimize and tune GraphStore based on a serie of new well-defined benchmarks.

Please also propose your ideas on the forum. They will be considered and discussed by the community. Have a look at our long-term Roadmap.

Students, join the network

Students, apply now for Gephi proposals. Join us on the forum and fill in the questionnaire. Be careful, deadline for submitting proposals is May 3 (timeline)!

Hélder Suzuki, student for Gephi in 2009 and now software engineer at Google, wrote:
At Gephi students will have the opportunity to produce high impact work on a rapidly growing area and be noted for it.

View our previous Google Summer of Code projects here and read former students interviews.

Follow Gephi on Twitter

A month of Gephi Marketplace

 

guillaume_old160Computer Science & Engineering Graduate from the Compiègne University of Technology, Guillaume Ceccarelli has been Gephi’s SysAdmin since 2008 and Web Developer since 2012. When he’s not by day working in the financial world, he freelances and stays close to ongoing entrepreneurship ventures.

Four weeks ago, we launched the Gephi Marketplace.

Haven’t taken a look yet? Didn’t know it existed? Take a few minutes to check it out now! It’s right here, waiting for you.

Why the Gephi marketplace?

For a long time, we felt like we lacked a central point of access when it came to what everyone was doing to improve or build upon Gephi. On one hand, our forums allowed for discussions to happen but they didn’t really enable anyone to share their work easily or to find out about one another. On the other hand, the wiki as well as our main site, served more as official portals to spread the word or to explore knowledge around Gephi, but they didn’t seem like a good fit to showcase the work our community was doing.

A little while ago, we decided to fix that.

The spirit behind the marketplace is simple. When we started, our idea was:

  1. To provide Gephi users with easy means to find and learn more about what the community was building for and around Gephi
  2. To give creators and people working with Gephi a way to publish their work and make them known to the entire community, without the overhead of a complex and lengthy process

This meant creating the central point for Gephi Plugins and what we call Gephi Services, which eventually became the Gephi Marketplace.

Before the marketplace existed, plugins were submitted and downloaded through the old plugin center, which required authors and the Gephi staff to go through a tedious process, creating a lot of work and unnecessary frustration for everyone. In addition, the system didn’t allow authors to update their own plugin pages, or to submit a different plugin for different versions of the Gephi application, a feature which was long requested before we finally became in a position to deliver it to you.

A new world of plugins

Now, every plugin download, even from the Gephi Application itself go through the marketplace! It means that as soon as a plugin goes live on our new platform, it’s directly available to every Gephi user around the world. Even if you haven’t made a plugin yourself, chances are you’ve already used the marketplace without realizing it!

Speaking of around the world, here’s a geographic breakdown of Gephi plugin downloads within the past 30 days:

Screen-Shot-2013-02-17-at-6.27.59-PM

Isn’t it impressive? 1169 plugin downloads and every continent is represented! (ok, not every single one. Props to the first Gephi user who’ll download a plugin from Antartica!)

It makes us extremely proud to help and foster such a vibrant community of Gephi users around the world. Remember that this map is for the last 30 days alone!

Today, we have 38 plugins listed on the marketplace, and we’ve made sure you can post your creations as easily as possible. It takes no more than 5 minutes to first fill in the plugin upload form, and we usually get back to you within 24 hours, so you can make your work available to thousands in record time!

A world of services

As briefly mentioned before, we’re also listing Gephi Services on the marketplace.

Over the years, we’ve noticed that a community of professionals and enthusiasts started providing services to the community. We’ve seen people doing trainings on Gephi, others integrating Gephi into data analysis workflows, or even create studies using Gephi as their tool of choice. We thought it was about time these fine folks got easier to find!

Every service provided by the community, for free or with a fee, is welcome to be listed. We already have a few, and to be honest we would love to see more of them!

Listing the services you provide on the Gephi marketplace is easy, and like with listing plugins it’s completely free, regardless of what you choose to charge for your work. We’re not here to collect fees. We just want to help other people know about your work, and to help you easily be found and easily be reached.

Following the same simple form submission process you can use for plugins, you give us all the info on what you’ve got for the community, you click ‘upload’, we get back to you, and then you have yourself a listing!… Which you can then customize to your liking, and for which you can include every single bit of information you want, and even a great copy! So not only can you let everyone know about what it is that you do, but also how truly wonderful it is to work with you and your expertise. So?

And now back to you.

Any question? Feel free to comment below! We’ll be glad to help you get started. And if you’d like to be kept in the know of what’s newly getting posted on the Marketplace, the RSS feed is this way.

In the meantime, thank you for your continued trust and your use of Gephi as your Graph Exploration and Manipulation tool of choice! It is our humble pleasure to serve you all.

See you soon on the marketplace! 🙂

rgexf: An R library to work with GEXF graph files

 

george_picture-100x100George Vega is an economist working at the Chilean Pension Supervisor and cofounder of nodoschile.org. His research interests are Statistical Computing, HPC, Complex Systems and Public Policy.

The first R library to work with GEXF files, rgexf allows both writing (exporting) and reading (importing) .gexf files.

Features:

  • Writing and reading GEXF files
  • Writing dynamic graphs
  • Writing graphs with attributes (boolean, numeric, char)
  • Writing graphs with VIZ attributes (color, size, shape)
  • Building GEXF graphs from scratch (node/edge by node/edge)

rgexf is written in such a way that it is not necessary to have knowledge about XML.

Some examples:

# Installing from CRAN and loading
install.packages("rgexf", dependencies=TRUE)
library(rgexf)

# Reading lesmiserables graph (and summarizing)
lesmiserables <- read.gexf("http://gephi.org/datasets/LesMiserables.gexf")
summary(lesmiserables)

# Building a GEXF class object (includes data frames of nodes/edges +
# XML representation of it) from two two-column data.frames
mygraph <- write.gexf(nodes=people, edges=relations)

# Exporting to some place
print(mygraph, output="mygraph.gexf", replace=TRUE)

# Creating a GEXF object from scratch (and adding a node)
mynewgraph <- new.gexf.graph()
mynewgraph <- add.gexf.nodes(mynewgraph, id=1, label="George")

The source code plus more examples can be found on the project website.

For suggestions, bug reports or support (any) ask me through Twitter @gvegayon or just write me an email to george [dot] vega [at] nodoschile [dot] org

George Vega Yon

Graph visualization meet-up in Paris

Meetup on graph visualization: join us the 24/01 in Paris

Neo4j, the leading graph database software, will be organizing a meetup on the visualization of graphs. It’s free, if you want to come you just have to register here.

Graph databases are a new way to store and access data by representing it as nodes and connections. It is particularly useful for dealing with highly connected data as social networks, recommendation engines, music discovery or anti-fraud systems do. Graph databases give data scientists exciting opportunities.

You will learn how to combine Neo4j and Gephi using the Neo4j plugin of Gephi. You will also discover Linkurious, a novel web-based application to explore graph data easily, which has been co-founded by Sébastien Heymann.

The workshop will be held in English and French.

Register on meetup.com

Date: 24th of January, from 7:30 PM to 10:15 PM
Place: Zenika office, 10 rue de Milan, Paris