What issue tracking system is best for you?

It is an eternal truth that newly written software packages will contain bugs. To track bugs, many organizations still rely on Word documents and Excel spreadsheets, but these tactics are inefficient and error-prone to say the least. A good automated issue-tracking solution should streamline the process of raising, managing and fixing issues.

In this story, we look at three issue-tracking solutions: Bugzilla, Trac, and JIRA. According to some unofficial polling done by yours truly (see Resources), these products appear to be among the most widely used issue-tracking tools in the Java community today.

Bugzilla is probably the most well-known of the open source issue-tracking solutions and is used by many high-profile open source projects such as Mozilla, Apache and Eclipse. It is a mature, feature-rich open source issue-management solution well-adapted for use in large projects.

Trac is another open source issue-tracking system, but with a different approach. Trac is a lightweight, minimalistic solution, designed to allow effective issue management with as little overhead as possible. It also boasts excellent integration with Subversion. And JIRA is a well-regarded commercial product widely used in the Java community, especially among open source products.

Bugzilla: The original Web-based issue-management solution

Bugzilla is probably the most well-known of the open source issue-management tools. It is used on many open source projects such as Mozilla, Eclipse, and many Linux distributions, and is well-adapted to large, open projects.

Bugzilla is typically installed in a Unix or Linux environment, though it can work on Windows with a bit of extra work. Bugzilla runs on Perl, and uses either MySQL or PostgreSQL as a backing database. You also need a Web server: Apache is the typical choice. Installation occurs from the command line and basically involves installing all the necessary Perl modules, setting up the database, and scheduling external Perl scripts to collect data for bug graphs and to send notifications. The installation process is long and involved and often full of surprises; only the bravest and most intrepid will come through a full Bugzilla installation unscathed.

Bugzilla is powerful and flexible, and fits well into a multiproject environment. You can manage multiple products, optionally grouping related projects into “Classifications.” And, for a particular product, you can define components. To assist project management and quality assurance, you can define versions and development milestones and release versions.

Because of its origins as an issue-management system for open source projects, Bugzilla is, by default, quite open about security: users can usually create an account themselves and create and access bugs for any project. If need be, however, you can restrict user rights to certain projects or create groups so that certain products or bugs can only be seen by certain people.

Bugzilla Web sites are usually easy to recognize. In its standard form, Bugzilla has arguably one of the ugliest and most convoluted Web sites. However, it is functional.

As you might expect from a product that manages systems containing literally hundreds of thousands of bugs, Bugzilla has powerful, if not particularly user friendly, search features. You can search for existing issues using either a simple and convenient keyword-based search (with optional filtering by product or by status), or by using the more sophisticated Advanced Search, where you can filter on virtually any field in the database (see Figure 1).

Although the interface is rudimentary and lacks many of the niceties of more recent Web sites, entering a new issue in Bugzilla is relatively straightforward. After selecting the buggy product, you arrive at the bug details screen (see Figure 2). Out of the numerous fields you’ll see, only the Summary and Component fields are actually mandatory. You will usually also provide other details, such as a version number (the version of the product in which the bug was found), severity of bug, platform, a target milestone, and so on. You can assign a bug directly to a developer, if you know who you are working with, or just wait for it to be picked up by some willing soul. If you can’t remember the definitions of the various severity levels or how to use a particular field, convenient hyperlinks about the field names display the appropriate help pages. You can also use attachments to add screen shots or the like.

Bugzilla supports a fairly complete, albeit hard-coded workflow model (see Figure 3). The typical life of a bug goes something like this: A tester (or user) creates a new bug. New bugs are created either as unconfirmed, new, or assigned. Typically, a developer will accept a bug (or assign it to someone else). Once the developer has corrected the bug, he or she can mark it as resolved, specifying how it was resolved: fixed, invalid (not a bug), duplicate, won’t fix, and the (in)famous “works for me.” A resolved bug isn’t officially closed until someone from QA (quality assurance) checks it out. Once QA has confirmed the correction, the bug becomes verified. It remains in this state until the product release containing the fix actually ships, at which point, it is closed. Although this workflow model cannot be customized in Bugzilla, it usually proves sufficient for most organizations.

Bugzilla notifies users of any new or updated bugs by e-mail.

Bugzilla also supports a system of votes, in which users can vote for issues or features they wish to see implemented. If enough votes are made for a particular issue, it will pass automatically from “Unconfirmed” to “New.” This functionality proves useful for large projects or projects with active user bases who participate in the development process.

Bugzilla provides some reporting and charting features. You can generate tables, and simple bar or pie charts using a screen similar to the Advanced Search screen (see Figure 4). To display graphs of data over time, you need to set up special “data sets” that collect the data you want on a regular basis.

Bugzilla is a tried-and-true solution that supports large projects and user bases. Its workflow features are more than sufficient for most organizations. On the downside, Bugzilla is particularly complicated to install and maintain, and the user interface wouldn’t win any prizes for design or usability. Its reporting features will do the job, though they are not particularly user friendly.

Trac: A wiki-based solution with superb integration with Subversion

Trac is a lightweight open source wiki-based issue-tracking tool with excellent integration with Subversion. It does not provide as many features as Bugzilla, but it is easy and intuitive to use.

Installing Trac is not particularly difficult, though, like Bugzilla, you do need to work at the command-line level and make sure your system has the required dependencies. Trac uses Python, plus a few other associated libraries, and can run on MySQL, PostgreSQL, or SQLite. Trac can run as a standalone Web server or be installed on a “real” Web server such as Apache.

Compared to other issue-management solutions, Trac takes a different approach in several areas. The first, and possibly the most striking, is that Trac is in fact a wiki — any page can be modified by users with appropriate rights. Wikis are a powerful tool in today’s development world: indeed, with a minimum of communication and training, every modern development project could probably put a wiki to good use. Trac is an excellent wiki for a development team, as it provides seamless integration between issue and release management, agile team communication techniques and the source code repository.

This leads to the other big innovation in Trac: its close integration with Subversion. The wiki syntax used in Trac lets you reference Subversion revisions, change-sets and even files. For example, a comment in an issue can contain a hyperlink to a particular Subversion change-set. The Trac wiki lets you add direct links to various objects using a special syntax. For example, “#21” refers to ticket No. 21, and [124] refers to change-set (or revision) No. 124. You can also browse the source code repository directly from within Trac, displaying both the actual source code and the modification history.

Trac provides a simple but powerful full-text search functionality, which lets you search not only tickets, but also wiki pages and change-sets. The search also recognizes the Trac wiki syntax, so you can go directly to a change-set, ticket or report simply by entering its number.

Another way to find tickets is to use the View Tickets view, which contains many useful predefined reports, such as Active Tickets, My Tickets, or All Tickets by Milestone. You can also create your own custom query, which you can build using an intuitive query builder (see Figure 5). More sophisticated reporting features are limited: there are no fancy graphs or charts, nor can you track ticket data over time, for example.

Entering a new ticket in Trac is easy (see Figure 6). As mentioned earlier, Trac’s wiki-based architecture eases the insertion of links to other tickets, change-sets, or to files in the source code repository.

Once the ticket has been created, it can be viewed and updated by other users (see Figure 7).

The workflow in Trac is lightweight. Once a ticket is created, a ticket can be assigned to (or accepted by) a user. Once it has been fixed, it is marked as “resolved” (see Figure 8). There is no provision for a state between “resolved” and “closed,” where quality assurance can verify the correction, as found in Bugzilla and JIRA. A more sophisticated workflow model is currently under development.

Trac provides many views that make daily issue-management activities easier, so you can easily navigate through the project, going from tickets to revisions to source code, and so on. The Timeline view is a powerful means of keeping tabs on a project. Any activity on the project displays here, including activity involving tickets and changes to the source code repository (see Figure 9).

The Roadmap view (see Figure 10) lets you track project progress through milestones (which can be synonymous with iterations, sprints or whatever you prefer). The Roadmap view gives a graphical view of the number of tickets closed compared to the total number of tickets for each milestone. Because tickets can represent tasks as well as bugs, this can also be a useful way to coordinate and track team activity and progress.

You can customize Trac to some extent. Because it’s a wiki, you can modify the content of every page as much as you like. You can also tailor the look and feel somewhat by customizing the Cascading Style Sheets stylesheets and adding your own logos. You can also customize the main ticket fields such as priorities, severities and ticket types.

Trac is a solid lightweight issue-management solution well adapted to small teams, especially when Subversion is used.

JIRA: Nice if you can afford it

JIRA is a widely used and well-regarded commercial issue-management tool. It is well known in the open source community because Atlassian has been known to offer free licenses for open source projects.

Installing JIRA is a breeze because JIRA is implemented in Java. The JIRA installation packages come either as a standalone bundle, including a Tomcat Web server and a HSQL database, or as a war file that you can deploy to your own Java Web server. You can use it out-of-the-box with an embedded database or connect it to most any database compatible with Java Database Connectivity: MySQL, PostgreSQL, Oracle, DB2, and so on. However, according to the documentation, some databases (PostgreSQL and MySQL) do work better than others (Sybase and Oracle).

When you first start and connect to the JIRA server, a wizard helps you configure your server directories, administrator password and e-mail notification details. When you’re finished, you are ready to use the actual product.

The JIRA user interface is a pleasure to use, and is intuitive. The home screen contains numerous graphical reports designed to give you a quick overview of the project’s current status: the list of issues assigned to you, your “in-progress” issues, the number of open issues grouped by priority, and so on. A rich set of predefined issue filters and reports ease the discovery of information you are looking for.

The JIRA search functionality (see Figure 12) lets you perform full-text searches with filtering on key fields such as issue type, status, affected or fix versions, reporter, assignee, and priorities, as well as by date. The JIRA search is simpler and less cluttered than the equivalent Bugzilla Advanced Search screen. Search results are displayed in tabular form, with many visual cues.

Creating new issues is easy and requires minimum fuss (see Figure 13). The workflow resembles that used for Bugzilla issues, though with a few minor differences. An “In Progress” bug, for example, is a bug that has been assigned to someone and is being actively worked on. You can also tailor the workflow to your specific needs, adding or removing steps in the workflow.

Like Bugzilla, JIRA is well adapted to large projects. JIRA handles multiple projects and project categories with ease, and you can set up permissions and various levels of security to limit who has access to particular projects, or even particular issues.

JIRA is highly configurable — from directly within the administration screens, you can modify everything from issue types and priorities to the look and feel of the Web site.

JIRA lets you manage versions and product releases in a simple, intuitive manner. You can define versions, track version releases and release dates, and generate release notes for a particular version with the list of all fixed and unresolved bugs for that release.

JIRA has a voting feature similar to the one in Bugzilla. You can also choose to “watch” an issue, to receive notification whenever the issue is updated.

Reporting in JIRA is limited to search results: there are no graphs or bar charts, and there is no easy way of keeping track of time-related data such as the number of resolved issues over time.

As mentioned, JIRA, unlike Bugzilla and Trac, is a commercial tool. JIRA comes in three versions: standard, professional and enterprise, with some of the more advanced features such as project categories, configurable workflow and issue-level security reserved for the higher-level products. Prices range from $1,200 to $4,800 for a server license. Atlassian, the editor, is also closely involved in the open source community and offers free licenses for open source and non-profit organizations (check out their Web site for more details).

Conclusion

These products reviewed are among the most widely used in the Java community. Bugzilla, with an uninspiring user interface, is rich in features, but undeniably cumbersome to install and to maintain. Trac is a good, lightweight solution that should be seriously considered by any development team using Subversion. JIRA is a solid, powerful solution, providing almost all of the features of Bugzilla, and more, in an eminently more usable (and more productive) form — but at a cost.

When choosing an issue-management solution, many factors must be considered. Features are important, but so are other factors: How big is your organization? Who is going to host and maintain the tool? What are your security requirements? Is your development style more suited to an agile, collaborative approach or to a more structured, traditional structure?

Also, before you make your decision, don’t forget to try them out!

Would you recommend another issue-management tool not covered here? Respond in the discussion thread that appears below.