Introducing the Gephi Plugins Bootcamp

We’re happy to announce a new tool for the community today: the Gephi Plugins Bootcamp. The bootcamp is a large set of plug-in examples to guide developers create Gephi plugins easily.

Gephi’s vision focuses on the platform and we want developers to be creative and successful. Gephi is built in a way it can be easily extended with plug-ins (layout, filters, io, preview, …) but it’s not always easy to know where to start. The bootcamp addresses this need and provide the environment and the examples to get started.

Want to create a new layout? The Grid Layout example shows how to read the graph and to change coordinates. A new filter? A new exporter? Check out the JPGExporter or the SQLite exporter examples. Below is the complete list of examples and we’ll add more soon upon requests.

Checkout the code on GitHub.

The README file contains the instructions to get the code and to run the examples.

Layout

	Grid Layout Place all nodes in a simple grid. Users can configure the size of the area and the speed.
	Sorted Grid Layout Same example as Grid Layout but users can sort nodes using an attribute column.

Filter

	Transform to Undirected Edge filter to remove mutual edges in a directed graph.
	Top nodes Keep the top K nodes using an attribute column.
	Remove Edge Crossing Example of a complex filter implementation which removes edges until no crossing occurs.

Tool

	Find with autocomplete Tool with a autocomplete text field to find any node based on labels and zoom by it.
	Add Nodes Listen to mouse clicks and adds nodes. Also adds edges if selecting other nodes.

Export

	JPG Export Vectorial export to the JPG image format. Contains a settings panel to set the width and height.
	SQLite Database Export Current graph export to a SQLite Database file. A new sub-menu is added in the Export menu and an example of a custom exporter is shown.

Preview

	Highlight Mutual Edges Colors differently mutual edges. Overwrites and extends the default edge renderer.
	Glow Renderer Adds a new renderer for node items which draws a glow effect around nodes.
	Node Z-ordering Extends the default node builder by reordering the node items by size or any number columns. Also shows how to create complex Preview UI.

Importer

Matrix Market File Importer File importer for the Matrix Market format. Large set of matrix file examples on Yifan Hu’s gallery.

Statistics

	Count Self-Loop Example of a statistics result at the global level. Simply counts the number of self-loop edges in the graph.
	Average Euclidean Distance Example of a per-node calculation. For a given node it calculates the average distance to others.

Plugins sub-menu

	Test action Simple action which displays a message and a dialog.
	Remove self loops Action which accesses the graph and removes self-loops, if any.
	Using Progress and Cancel Action which creates a long task and executes it with progress and cancel support.

Execute at startup

	When UI is ready Do something when the UI finished loading.
	Workspace select events Do something when a workspace is selected.

Processor

Initial Position Set up the nodes’ initial position always the same. It calculates a hash with all nodes so the X/Y position is randomized always in the same way.

Panels

New panel Example of a new panel plugin set up at the layout position.

If you have any questions please send an email to the gephi-plugins [at] lists.gephi.org mailing list or stop by on the forum.

Gephi migrates to GitHub

We are happy to announce we finished the migration of our code from Launchpad to GitHub. All the code and bugs have been successfully transfered with the complete history. We can now profit from the best platform out-there and use Git, the fastest revision control system.

We hope you’ll find GitHub faster and easier to use than Launchpad. The team is already appreciating how easy it is to report issues and work together on the code. GitHub has more than a million users and will make the project more visible and ease external contributions.

Technically, we migrated our branches from Bazaar to Git (thanks to git-bzr) so the history is entirely kept. We also moved all our bugs with a simple script. We are still working on the details. If you see something wrong or missing on GitHub, please contact us or create an issue on GitHub. If you had some branches on Launchpad, you can find them on the GitHub repository. Let us know if you have questions. Contributors simply fork the repository and get started. We updated the documentation on the wiki. Consult the Developer Handbook.

Checkout code

Run
git clone git://github.com/gephi/gephi.git

Report issues

Simply go to the Issues tab.

Build in one step

Simply run ant at the root of the repository to build Gephi. The executable are located in the dist folder.

We made some improvements on the building process. Previously, Netbeans was required to build Gephi. We now integrates the platform directly in the source code so it’s not necessary anymore. It’s literally a one step process.

Please let us know your feedbacks and questions as usual on the forum.

New Gephi Toolkit release, based on 0.8 beta

A new release of the Gephi Toolkit arrived, based on the 0.8 beta version. Download the latest package, including Javadoc and demos by clicking on the link below.

It includes all features and bugfixes the 0.8 beta version has. Check out the Release Notes

Demos available on the Toolkit Portal have been adapted when necessary and tested. If you are interested in using plug-ins from the Toolkit, checkout How to use plug-ins with the Toolkit.

One of the exciting new feature released in the 0.8 beta is the new Preview. Because it has been completely rewritten with a much simpler and powerful API it’s now possible to reuse the Processing-based applet in other applications. You can now reproduce the zoom-and-pan experience in Gephi’s Preview tab in other Java applications easily. Checkout this demo to learn more.

Links you may be interested:

• Gephi Toolkit Tutorial (updated with new Ranking and Preview)
• Tips to use the Gephi Toolkit with other JVM languages

First Gephi Plugin Developers Workshop on October, 6

This is an announcement for the first Gephi Plugins Developers Workshop October 6, 2011 in Mountain View, California. Come and learn how to write your first Gephi plugin and ask questions. The workshop is organized by Mathieu Bastian, Gephi Architect and will be gratefully hosted by IMVU.

Gephi is a modular software and can be extended with plug-ins. Plug-ins can add new features like layout, filters, metrics, data sources, etc. or modify existing features. Gephi is written in Java so anything that can be used in Java can be packaged as a Gephi plug-in! Visit the Plugins Portal on the wiki and follow the tutorials to get started.

The workshop will start with a presentation of Gephi’s architecture and the different types of plugins that can be written with examples. Details about Gephi’s APIs, code examples and best practices will be presented in an interactive “live coding” way. The Gephi Toolkit will also be covered in details. The second part of the workshop will be dedicated to help individuals with their projects and answer questions.

Some of the best projects using or extending Gephi are developed in the Silicon Valley and we are looking forward helping the developer community. Please don’t hesitate to send us your ideas to maximize efficiency.

RSVP here

GSoC mid-term: Automated build & Maven

My name is Keheliya Gallaba and during this Google Summer of Code I am working on the Automated build system for Gephi. The goal of this project is to add Maven build support to Gephi and set up a continuous integration system to fasten the release process. The Netbeans Platform, which Gephi is built upon, natively uses Apache Ant to compile, build and package the application. But now there is also a variant of NetBeans which uses Apache Maven as the build system. There are several reasons that make moving into a Maven based system worthwhile.

Maven vs Ant

The existing Ant build system for building NetBeans Platform-based applications which is called Ant Build Harness is very intuitive, and needs almost no initial setup. The set of standard Ant scripts and tasks can be easily triggered by the IDE or by the command line. But there are reasons that Ant might not suite a rapidly growing, multi-module project like Gephi. The Gephi project consists of a team of developers who work on dependent modules and plugins. These modules have to be composed to the application regularly. With a large number of modules, with many small packages, and with multiple projects with many inter-dependencies and external dependencies, its essential to manage different versions and branches with their dependencies. And reusing modules with the Ant build harness is not that intuitive.

But Apache Maven is introduced as a standard, well defined build system that can be customized. It uses a construct known as a Project Object Model (POM) to describe the software project being built, its dependencies on other external modules and components, and the build order. It comes with pre-defined targets for performing certain well-defined tasks such as compilation of code and its packaging. It makes dependency management very easy and efficient with the concept of repositories. Most importantly in maven unique coordinates: groupId, artifactId, version, packaging, classifier identifies an artifact which can be uploaded or retrieved from a repository. This helps to easily build modules which depend on other modules.

Work completed so far

This project involves digging deeper in to the Gephi’s architecture and understanding dependencies, building and packaging. Gephi includes 100+ submodules categorized into Core, UI, Libraries and Plugins sections. NBM, which stands for “NetBeans module”, is the deployment format of modules in NetBeans. It is a ZIP archive, with the extension .nbm, containing the JARs in the module, and their configuration files. NBM files can be manually installed using the Update Center and choosing the option for installing manually downloaded modules, or they can be downloaded and installed directly from netbeans.org or another update server.

I’m happy to say that I was able to successfully mavenize 75 modules and continuing to complete the rest. I primarily used the NetBeans Module Maven Plugin for this, which now comes built in with NetBeans 6.9 and 7.0 IDEs. Currently NBM handles the tasks like defining the ‘nbm’ packaging by registering a new packaging type “nbm” so that any project with this packaging will be automatically turned into a netbeans module project, creating nbm artifacts and managing branding. It is also capable of populating the local maven repository with module jars and NBM files from a given NetBeans installation.

Some third party libraries used in Gephi are not maintained in any public Maven Repositories. So I had set up a local Sonatype Nexus Repository to store and serve these dependencies. Basic functionalities of a repository manager like Sonatype are:

• managing project dependencies,
• artifacts & metadata,
• proxying external repositories
• and deployment of packaged binaries and JARs to share those artifacts with other developers and end-users.

We are in the process of setting up a Sonatype Nexus Repository in official Gephi server as well, so not only these third party jars, but the Gephi releases such as the Gephi Toolkit can be served as a maven dependency to maven-based projects all over the world.

Challenges faced during the process

• Researching on existing large scale applications using NetBeans RCP and Maven
• Finding documentation on handling Netbeans specific ant tasks, now in Maven
• Managing transitive dependencies and versioning (specially with slight defferences of Maven and NetBeans difinitions)
• Compilation and Test Failures.

Continuous Integration

While Maven migration is going on, I also looked in to the other aspect of the project, setting up of a continuous integration server. Main benefits of such a system are:

• checking out source from source control,
• running clean build,
• deploying the artifacts in a repository
• and running unit tests.

Furthermore it can notify developers via Email, IM or IRC on Success, Failure, Error and Warning in a build or simply a Source Code Management Failure. What this means is that when a project gets updated during development, the continuous integration system will try to build the project and will notify the developers if it ran into any issues. This is very useful when working on a multi-module project with many developers, like Gephi since a developer may unintentionally, by accident break the build since they are working concurrently on code and they may have unique configurations to their development environment that isn’t shared by other developers. I looked at the options of Apache Continuum, Hudson and Jenkins (A fork of Hudson) considering the criteria, being open source, supporting Ant & Maven and better integration with Java based projects.

Hudson is an extensible Continuous Integration Server built by Sun Microsystem’s Kohsuke Kawaguchi. Since the design of Hudson includes well thought-out extension points, developers have written plugins to support all of the major version control systems and many different notifiers, and many other options to customize the build process for example the Amazon EC2 plugin to use the Amazon “cloud” as the build cluster.

Continuum is described as a fast, lightweight, and undemanding continuous integration system built by Apache Maven team. It is built on the Plexus component framework, and comes bundled with its own Jetty application server. Like Maven, it is built on the Plexus component framework, and comes bundled with its own Jetty application server. It uses Apache Derby, a 100% Java, fully embedded database for its persistence needs. All these reasons make Continuum self-reliant, and also particularly easy to install in almost any environment.

After considering all of these reasons I settled on Apache Continuum because of the ease of setting it up, configuration and out-of-the-box support for Bazaar. Bazaar is the distributed version control system used in Launchpad for managing the source code, when lot of developers work together on software projects like Gephi. I have set up a local instance of Apache Continuum to check out and build the ant-based Gephi hourly. In the future we can host this in the Gephi server to notify the developers and administrators.

Future Work

Since the initial foundation has been laid out, it will be quite convenient to complete the rest of the planned work. These will include completion of mavenizing rest of the modules, creating .zip distribution, properly running the final project being developed and setting up the infrastructure at the Gephi server.

I would like to thank my mentors Julian Bilcke, Mathieu Bastian and Sébastien Heymann for providing all the guidelines and support for making this project a success. You can find my ongoing work at this repository: https://code.launchpad.net/~keheliya-gallaba/Gephi/maven-build

References

• NetBeans Module plugin
• List of Gephi Dependencies
• List of Gephi Netbeans Modules
• NetBeans Platform Quick Start Using Maven
• Maven NBM comments
• Efficient development of large NetBeans Platform applications with Maven by Fabrizio Giudici
• Continuous_integration
• Repository Management with Nexus

New Gephi Toolkit release, based on 0.8alpha

A new release of the Gephi Toolkit arrived, based on the 0.8alpha version. Download the latest package, including Javadoc and demos by clicking on the link below.

It includes all features and bugfixes the 0.8alpha version has, notably:

• GEXF 1.2 support (partial)
• Add Neighbour Filter
• Improve support of meta-edges in Statistics and Filters
• Edge weight option in PageRank, which can now be used by the algorithm
• VNA Import (Thanks to Vojtech Bardiovsky)
• Label Adjust algorithm 3 times faster
• Saving/Loading projects is faster and use less memory

Demos available on the Toolkit Portal have been adapted when necessary and tested. If you are intrested in using plug-ins from the Toolkit, checkout How to use plug-ins with the Toolkit.

Links you may be interested:

• Gephi Toolkit Tutorial
• Tips to use the Gephi Toolkit with other JVM languages
• Scripting plugin

This summer, the student Luiz Ribeiro is working on GSoC Scripting Plugin, a project to bring advanced scripting features in Gephi, using Python. This project will work with the Gephi Toolkit, and greatly facilitate its usage.

Gephi Toolkit released, based on 0.7beta

The 0.7beta version of Gephi has been released last week. It is today the Gephi Toolkit release, based on the latest codebase. Download the latest package, including Javadoc and demos by clicking on the link below.

It includes all features and bugfixes the 0.7beta version has. Therefore it is possible to use dynamic networks and new Data Laboratory features from the Gephi Toolkit.

Two new demos are available from the Toolkit Portal:

• Import Dynamic – How to import several static files and transform them into longitudinal network
• Dynamic Metric – How to execute a metric (ex: Average Degree) for each slice of a dynamic network

Links you may be interested:

• Gephi Toolkit Tutorial
• Tips to use the Gephi Toolkit with other JVM languages

Since it’s launch in July, the Gephi Toolkit has been used in various use cases for graph visualization. Recently, the Indiana University launched Truthy, a system to analyze and visualize the diffusion of information on Twitter. Truthy uses the Gephi Toolkit for layout.

Announcing the Gephi Toolkit

We are announcing today the first release of the Gephi Toolkit. The Toolkit project packages essential modules (Graph, Layout, Filters, IO…) in a standard Java library, which any Java project can use for getting things done. The toolkit is just a single JAR that anyone could reuse in a Java program and script Gephi features.

The toolkit is the counterpart of the desktop application. Gephi’s user interface aims to be simple, intuitive and without command-line or scripting needed. The toolkit is made for people who want to:

• Script, automate features & reproduce the same procedure over and over
• Reuse Gephi features and algorithms in other projects and softwares
• Develop all types of mashups or web-services that deals with networks

A lot of new content is coming with the release of the Toolkit. A new portal appeared on the wiki, with documentation. Above all we provide demos and examples and a tutorial for newcomers. The cool thing is that it is very easy to use and this is all compatible with Gephi plugins. What is done for Gephi desktop can be reused in the toolkit.

Gephi is designed in a modular way and splitted into different modules. All features are wrapped into separated modules, for instance a module for the graph structure, a module for the layout algorithms and so on. Moreover business modules are separated from user interfaces modules. That allows to keep only business modules and remove UI without any problems. That is the purpose of the toolkit, which wraps only core modules and removes all the UI layer. So the toolkit is just taking what already exists in Gephi and packages it.

That is all thanks to the power of Java and Netbeans Platform. The way modular development is encouraged and the ability to manually extract modules from the Netbeans Platform is all thanks to the way they designed the architecture and use standards like ‘ant’ and plain Java. It’s a good occasion to say Kudos to them!

With the release of the toolkit, we are also moving to the AGPL license, as announced earlier. The GNU Affero General Public License is a modified version of the ordinary GNU GPL version 3. It has one added requirement: if you run the program on a server and let other users communicate with it there, your server must also allow them to download the source code corresponding to the program that it’s running. If what’s running there is your modified version of the program, the server’s users must get the source code as you modified it.