We recently finished to deploy a continuous integration environment at Gephi and I’m excited to share some of the highlights.
The Gephi developer team has been hard at work to change the way we iterate and create releases at Gephi. Developer productivity has been an important theme for this year’s focus and we already made several improvements. At the end of last year we migrated our code to GitHub and improved the documentation. We then focused on plugin developers and made it really easy to create new plugins with the Plugin Bootcamp and the new gephi-plugins repository. Finally, we’re now introducing a completely automated build and release production system.
Our objective was to automate the way releases are created and tested. Previously, creating a release was a manual process and included error prone tasks like updating configuration files, unzipping translations in the right folder or creating installers. Open-source tools like Maven, Jenkins and Nexus can help to make this process seamless and always have the latest deliverables available.
We migrated our code base from Ant to Maven. Gephi is based on the Netbeans Platform and has more than 80 different modules with dependencies and third-party librairies. Maven makes it easy to manage a large number of dependencies and put all configuration parameters in one place. Maven has also a large number of plugins and is very well integrated in Netbeans and Eclipse IDE.
- A full application package, all Javadocs and sources are now produced and uploaded online with a single command.
- Dependencies are all defined in one place. It is also much easier to update to the latest version of the Netbeans Platform.
- All library JARs are dependencies to Maven Central or 3rd party repositories. No library JARs are directly included in the sources anymore.
- The Gephi project is now a standard multi-module Maven project. It can therefore be opened and built in Eclipse or IntelliJ, as well as Netbeans out of the box
- It facilitates module reuse in other projects like the Gephi Toolkit. Any other project can easily depend on any (or all) Gephi modules.
Jenkins is the continuous integration server we chose to automate building and testing Gephi. It is configured to build and test Gephi every night based on the latest version of the code on GitHub. If the build fails, developers are informed something needs to be fixed.
- Fully automated build in a stable environment. If something is wrong, it must be the code.
- In addition of Gephi itself, we’re also building the Gephi Toolkit every night. Eventually, we’ll be able to build and test plugins as well.
- Artifacts produced are uploaded to Nexus.
Nexus is a repository for artifacts, which could either be librairies Gephi is using or release binaires like the latest release. At any time, beta testers can download the nightly build and test new features. We just announced a new beta testing program, which couldn’t be possible without the availability of the nightly build.
- All 3rd party librairies have been uploaded to Nexus. Maven is using Nexus as a source for librairies.
- The nightly build packages are available for download.
- It also hosts the latest set of NBMs and Javadocs.
We learnt a lot during this project and will continue to strengthen the developer and beta-tester environment to scale up Gephi development. So far, we’ve done the Maven migration on a separate GitHub repository but we’ll soon convert the main repository and soon after release a 0.8.2 Gephi version. We’ve created a new Continuous Integration section on the Dev Portal and documented this project.
Plugin development remains the same for now and all plugins should be compatible with the new code base. In the next few months we would like to bring continuous integration to plugin developers as well. Testing at scale a large number of plugins at each new Gephi version remains a challenge and we would like to improve that. Also, we’ve seen issues where different plugins use different version of the same library and eventually cause crashes. Stay tuned for some news on that.
In the next few weeks we’ll update the documentation at various places how to build Gephi and work with the code. Developers interested to try this new system out should follow the instructions on GitHub or reach to us on the developer mailing-list.
Last but not least, we would like to say kudos to Maven, Jenkins and Nexus contributors for their huge and excellent work!