Party of one: Surviving the solo open source project

With more than 100,000 open source projects hosted on SourceForge alone, starting a new one is no small undertaking, and bringing it to the masses is hardly a sure thing. In this article reprinted from Pushing Pixels, Kirill Grouchnikov explores the challenges and pitfalls of being the sole developer on an open source project. Whether you’re thinking about starting a solo development project or struggling to maintain one, get tips for structuring your development timeline, managing development priorities, and finding a niche for your project, even in the vast sea of open source software.

JavaOne 2008 included a track on open source with a substantial number of very interesting panels, discussions, and presentations. These talks concentrated mostly on big, well-established, and very broad open source communities, however, which does not reflect the specific problems of a much vaster open source landscape. Here I’m talking about literally hundreds of thousands of projects that only have a single active developer. I call these “party of one” projects, and they pose some significant challenges to their owners, especially if the project is done as a part-time hobby.

Figure 1 shows some statistics from SourceForge, the largest repository of open source projects:

Statistics, of course, can be used to back up pretty much any point of view, but I do believe that the numbers above are a good reflection of what happens when a developer has an idea. Nowadays, it is extremely easy to find a free Web-based code repository and wait for the entire world to come. However, the absence of any entry-level barrier to creating a project quite often results in the idea being just that — an idea. Just over half of SourceForge projects have any kind of downloadable content. Worse, while it is difficult to create even a first implementation, it is even more difficult to maintain the level of commitment and arrive at a stable release. Thus, as you can see above, only one in every six SourceForge projects were deemed stable by their own developers.

When all you need to create a project is an Internet connection and enough imagination to come up with a name, you have another interesting side-effect. While every project has a potential to reach millions of developers, they all compete for the eyeballs and attention span of those few tens of thousands that are looking. You can have a great idea, you can even have a great implementation, but most certainly you can’t expect everybody to come. It is a big world out there, and the competition for attention is even bigger.

It takes determination, commitment, and perseverance to be willing to continue working on a project when nobody but you seems to be interested in it. It takes discipline and clear prioritization to understand that while coding is fun, every project has less pleasant (to us as developers) tasks, such as bug fixing, testing, documentation, and promotion. Without understanding what these tasks are, and being ready to step up and make sure that all of them get done, you might as well not begin in the first place.

Every one-man open source project has three main parts: development, maintenance, and promotion. I’ll discuss all three, beginning with development, which is where it all starts — and, if you’re not careful, also where it ends.

Developing the project

Development is the most enjoyable part of a new project, at least for some of us. This is the time where everything seems possible, where there are no implementation complexities, no bugs to fix, and no annoying support requests for esoteric environments. Even if you are doing it in your free time, it doesn’t take a lot of commitment to just sit down in front of your favorite editor and start banging away. It is also, unfortunately, a time where a sizable percentage of projects simply die.

It takes serious commitment to continue developing a new project in your own free time, at the expense of your other hobbies and sometimes even your family. This requires not only being enthusiastic about your idea, but being passionate. If you’re not passionate about the idea in particular, and about its field in general, you might as well not start at all.

The three main aspects to development are

• Tenets
• Features
• Releases

I will talk about creating features when I come to the topic of project maintenance. For now I’d like to focus on establishing tenets and setting up a release schedule. While these stages are not specific to one-man projects they are especially crucial to their success. Not planning development cycles in advance can have an extremely adverse impact on a project’s ability to survive in the wilds of today’s open source landscape.

Where do you want to go?

The majority of successful open-source projects (no matter how big the development community) have a very clear set of tenets. These are sometimes referred to as project goals, vision, direction, or guidelines. While tenets do not need to be set in stone, it is crucial that a project has a clear, concise, and simple definition of what it aims to do. A clear project vision helps immensely in deciding what goes in and what stays out. It helps users to understand whether the project fits their needs. And it ensures that the project does not end up being a “jack of all trades and master of none.”

Here is one example of a project vision:

[…] is a high performance, vastly scalable, embeddable, internet-centric client/server computing platform with a breakthrough adaptive runtime technology.

Speaking only for myself, this statement is full of buzzwords and marketing speak. Too much information is crammed in with too many generic, overused words. I had a hard time remembering it a few days after seeing it on the ‘net. Now, compare it to these very focused and plainly-worded tenets of the FormLayout project:

It makes simple things easy and the hard stuff possible, the good design easy and the bad difficult.

Here is a very simple and down-to-earth list. It is also very memorable. The entire library is driven by these four statements, which set clear rules for what goes in and what stays out. It doesn’t claim to want to be the best for everybody, and it doesn’t proclaim to provide a solution for all possible scenarios. Instead, the statement focuses FormLayout’s APIs around a simple and coherent vision, and even explains why some things (that the author considers bad) are hard to achieve.

The release schedule

Here is a real example of one of the projects hosted on SourceForge, illustrating the “anti-pattern” of release schedule planning:

On the face of it, everything looks good. There have been 11 releases over the last six years, and the project looks quite active and maintained. However, there are two big sore points with this table — the leading zero in version numbers and the ever-increasing time span between consecutive releases.

In this particular case, even the project’s developer does not consider it to be worthy of a “1.0” release label. One might say that it is a minor thing which has nothing to do with the actual quality of the project. However, outside looks count, and not only in real life, unfortunately. As cliche as it might be, “dress for success” is very good advice, and in virtual world this means stripping the alpha / beta / 0.* status from your project. You have a very short window of time to impress potential users, and a solid release number with no Greek letters is a must-have. This is the least you can do for yourself after six years of development.

The second sore point of this specific project is the ever-growing time frame for each release. It took two to three months for the first six releases, when the initial level of commitment was quite high. This then dropped to four to six months for the next two releases, and 11 to 14 months (and counting) for the last two. Potential users that seriously consider investing in your project (using, sending bug reports, or even contributing bug fixes) might be rightfully scared by such a trend.

To each his own, but I’ve found that the following release schedule works best for me:

Note that this release schedule adds up to 52 weeks in between the major releases. This means that the project will have a major release once a year, further strengthening the outward appearance of both stability and progress.

Maintaining the project

While development requires commitment, maintenance takes concentration and focus. It is my belief that if you want your project to become and stay successful, you must maintain it with utmost care and ensure that it receives all the necessary resources — possibly even more resources than you give to development. The three primary aspects of maintenance are:

• Bug fixing
• Measuring and repairing code health
• Task scheduling

Now let’s consider each of these in detail.

Bug fixing

Many developers consider bug reports a sometimes unwelcome interruption to the development flow. Some will blame users for their incorrect understanding of the APIs; some will postpone fixing existing bugs indefinitely and perhaps even half-jokingly call them features; and some will follow the existing industry trend of saving up all the bug reports until the last two weeks before a release and then handling them in a condensed and focused “bug-squashing” effort.

Unfortunately, these approaches ignore the vital part of a bug report — the submitter, or original poster (OP) in some parlance. They all overlook the simple point that the person on the other side of the bug report has gone way beyond what most of us would do. Not only has he decided to download your distributables, not only he has found enough time to play with your test applications (or even write one of his own), he has also been courteous enough to contact you and describe his experience with your product. The least you can do is to fire off an immediate and polite reply.

That said, not all bug reports are valid. Some will come from users who have no idea what are they doing. Some will not contain enough information to reproduce them. Some will come via channels that you explicitly discourage (for example, personal email instead of the official project tracker). Some will contain vague hints of threatening to switch to alternative products if the bug is not addressed immediately. Some will have nothing to do with your project and originate in other libraries. You just have to remember one thing — it is always your fault.

Bug reports: What to do

Make sure that you provide users with many ways to report a bug or raise a question. Having archived mailing lists and forums is great for the “ivory tower ego” type (whose knee-jerk instinctive reaction to user problems is “first search the forums and RTFM”), but people just don’t seem to do this. Even a simple requirement of registering to a mailing list to send a mail is a turn-off for many. Replying to a personal email by redirecting the questions to the archived mailing list or forum might not be a welcome thing for users that do not want to divulge potential IP aspects of their applications in public.

The flip-side is that responding to bug report immediately will do wonders to establish early adopters and give you extra testers. Mark every bug fix code path with bug number and simple description so that you don’t accidentally refactor it out after a few months. The worst thing that can happen to a new feature is no feedback at all. It doesn’t mean that it’s flawless; it simply means that nobody’s using it.

Priorities and code health

Before discussing code health, it is important to think about priorities. Figure 4 shows the priorities you frequently see in open source projects that don’t interact with such entities as steering committees, governance models, and incubators:

Add to this a spruce of Maven-generated goodness to create an illusion of documentation and you have the recipe for quite a few open source projects out there. If you’re fortunate, you get a little bit of code health (more on this in a moment) and real documentation. Sometimes you’re not so lucky and too much code health results in rewriting the entire project from scratch, getting back to what developers love the most — throwing out old code and writing new. In reality, your priorities should look like this:

In order to understand this priority list, consider a project’s internal implementation complexity over the cycle of a few consecutive releases.

If you have a clear project vision every new feature needs to conform to that vision and interact seamlessly with every existing one. Otherwise you end up with featuritis and feature soup. In an ideal world, adding a new feature has the same complexity no matter how many existing ones you already have. In the real world we’re all humans, and we all make mistakes (and, as we get more experienced, our mistakes tend to get more elaborate and destructive). Wouldn’t it be nice if every once in a while this graph would go down instead of climbing up?

Measuring and repairing code health

The big question, of course, is how exactly can you decrease the implementation complexity of your application over time? The smaller question, but important as well, is can you go back to the internal complexity level of the previous major release? As we’re talking about the internal implementation complexity, lowering means changing. It can be simple refactoring, or can go as far as introducing binary incompatibility and breaking existing applications. How far can you go until your users or even early adopters abandon you? And finally, what is more important to you: your users or your vision?

These are not simple questions, and they do not have simple answers. For me, all of these questions boil down to one concept: code health. Code health is the intangible metric of your confidence in the stability of your code, in your ability to find and fix bugs, and in your ability to adapt the code to new requirements. In my opinion, code health is the second most important thing after prompt bug fixes, and it has three main parts:

• Internal refactoring
• Deprecation
• External refactoring

These are the things that you do when your code becomes too complex. I would even say that these are the things that you should do before your code becomes too complex. In fact, you shouldn’t take pride in how complex your code is.

Internal refactoring is, of course, the simplest step to prevent code complexity, but sometimes it is not enough. As I mentioned already, we all make mistakes, and some of the mistakes are in the exposed API layer. It might be a bad method name, a missing parameter, an incorrect modeling of the user flows, or even too many ways to achieve the same thing. As long as you either have a better way to achieve the same functionality, or a very compelling and preferably non-technical reason to remove an existing functionality, do not be afraid to break APIs and compatibility. If you’re small, nobody will care because nobody will notice. If you’re big, your users will trust you and come with you. For the one-man project, community response to an external refactoring may be a very good indicator of whether you can get big or not.

Task scheduling

The last aspect of project maintenance is the task schedule. This is how you break up those 10 weeks and allocate time to different tasks. To decide when each task should be done, you should follow the sources of instability in your project and weigh the risks against the benefits of each task. In general, instability comes from new features, internal rewrites, and bug fixes, and you should minimize each stage and schedule it to end as early as possible in the release cycle.

• Adding new features is the biggest instability risk and should be done in the first four weeks of the cycle. Here, you can hope that your early adopters and testers will be kind enough to test these features and provide timely feedback. If they don’t, and you make a mistake that is exposed only after the release is done, you will need to wait for the next major release to remove or replace it. If you follow the 52-week schedule you won’t have to wait too long, however.
• Internal rewrites should not (in principle) have much effect on the user’s code. The reality, of course, is not ideal. Users can become dependent on undocumented incidental behaviors. They can decide to use internal classes or even resort to hacks without ever asking for a proper way to achieve the required functionality in the project forums and mailing lists. Still, internal rewrites generally have less impact than new features, and as such can continue until two weeks before the release candidate. The recommended release candidate date is two weeks before the final release.
• Bug fixes should be done all the time. In fact, the earlier you do them, the better it is for your project. If you engage in this cycle the very same day as the bug report arrives, you have much greater chance of getting an actual confirmation from the user that it works. And if you are lucky enough, every once in a while this user will turn into an early adopter. And once the release candidate is out, you should fight your every desire to add a new seemingly minuscule feature or do any sort of refactoring.

Because you have only four weeks to add new features, you’ll need to sharpen your skills in making compromises about what goes in and what waits until the next release. If your users threaten to switch to an alternative product because of “just this one feature” that is missing — it might be a good thing for your project in the long run. However, just because you “need” something to show to warrant a new release doesn’t mean that you must force a solution to the specific user request or a new feature that you want to add. Let your unconscious mind work on it for a while.

Promoting the project

If you write it they will come. Open source software has thousands of eyeballs monitoring it for flaws. Let the code speak for itself. The code is the best documentation.

Haven’t we all heard this before, and haven’t we all had this type of wishful thinking? Alas, the harsh reality of hundreds of thousands of languishing open-source projects is in a stark contrast with the rosy promises and expectations that you might nourish when you start your own one-man open source project. Your pour countless hours into developing and maintaining your project, only to see the weekly download count stuck at single digits, and community at large blissfully ignorant of your labor. There is only one thing that you’re missing, and for some it might evoke an uneasy association with “those marketing suits.” In a one-man project, you are the developer and the maintainer, but you are also the promoter, evangelist, and marketer.

You need to stay passionate to continue developing your project even when nobody comes. You need to stay focused to maintain your project and properly prioritize different tasks. When it comes to promotion, you need to stay committed.

Most developers are introverts by nature, preferring to let the code speak for itself. The only problem is that there is too much code out there, and your potential users need to be gently and constantly reminded just how awesome your code is. Obviously, you’re not one of those “marketing suits,” and your target audience is your fellow developers. All you need to do is to think about how to “sell” your project to a guy just like yourself. That would mean highlighting the capabilities of your project, objectively comparing it to the competition (even when you’re lagging behind in some areas), not digressing into marketese and buzzword speak, and, perhaps most importantly, constantly staying on the radar of those who need to know about your project.

A realistic approach

There are three main facets to project promotion:

• Documentation
• Test applications
• Blogging about the project

Take a break from your frantic coding and ask yourself a simple question: How easy it is to understand your library for a user that has absolutely no knowledge in this specific area? Do you have a simple and intuitive index to guide novice users around basic concepts? Do you have a quick-start guide? Is it kept in sync with your latest release/development version? Do you have simple examples that show how to use every single API method? Do you have in-depth tutorials for advanced users? You might be a star coder that can download a new library, load the sources in your text editor, and start using it right away, but most of us are not. Do your project a favor and document every single thing that you can think of.

The same principles apply to the test applications. While you should have a few big test applications that showcase your library as a whole, you should also write a very simple and complete example for each one of your public APIs. Keep them always in sync with the latest release (or the latest development version), interlink heavily and organize in an intuitive manner (grouping by business / related functionality).

Finally, have a blog. Get into the habit of writing two or three times a week. You might think that you don’t have anything valuable to say, but you will be surprised. If people want to download your code, they will also want to read your blog. It doesn’t have to be exclusively about your project. On the other hand, it doesn’t have to be about your favorite types of food. Once you get into the habit of writing, it will be much easier to continue doing so, even if English is not your native language.

In conclusion

There are a lot of big projects out there with dozens of active developers and hundreds of people actively involved behind the scenes. Sometimes, developers of these projects have the luxury to concentrate on what they love to do the most — coding. The majority of projects, however, tend to languish because their developers fail to recognize that certain menial and tedious tasks are crucial for the long-term health of their projects. In this article I’ve talked about the most important issues that face single-developer projects:

• Development and staying passionate
• Maintenance and staying focused
• Promotion and staying committed

Even when you’ve aced these aspects of your project, you as a solo developer might face an unreasonably high level of expectation from your target user sector, as well as from yourself. While being the best in your field is an understandable goal, I would advise focusing on other goals instead. How about admitting the following three simple things:

• You know you’ll make mistakes.
• You’re ready to learn from them.
• You hope to not repeat the same mistakes in the future.

Adhering to these principles means that you will not be afraid to make mistakes or admit them, and you will also be very quick to fix them when they occur.

Editor: This article is an excerpt from Kirill’s short series “Party of one: Surviving a hobby open-source project.” See Resources for a link to the complete series and other articles related to open source software development.

Kirill Grouchnikov has been writing software since he was in a junior high school. Since finishing his BSc in computer science, he has happily continued doing it for a living. He is currently working in the Amdocs product development group as a Java senior software engineer. His main fields of interest are desktop applications, imaging algorithms, and advanced UI technologies. See Kirill’s technical blog Pushing Pixels for more of his thoughts on Java programming.