11 questions for MongoDB’s CTO

NoSQL is the darling of the millennial tech community — and MongoDB is the leading NoSQL database, with a valuation of more than $1.2 billion.

By most accounts, Web developers have been a major driver of MongoDB’s success, because as a document database, it’s easier to write to for many applications than an RDBMS is, and you get very fast reads. Plus, you get dynamic, flexible schemas, and you can scale out using commodity hardware, rather tending a powerful database server beast.

But many potential enterprise adopters wonder if MongoDB has been overhyped. Some believe it’s still an immature solution applicable only to certain Web apps and risky for transactions.

To answer these and other objections, I sat down with Eliot Horowitz, co-founder of MongoDB and the company’s CTO. The following is an edited version of the interview:

InfoWorld: I think there’s a perception in the enterprise community that MongoDB may be great for startups, where you need to scale at the drop of a hat. But why bother with that capability for less dynamic enterprise applications?

Eliot Horowitz: When I talk to the CIOs of enterprises there are certain things that drive them crazy. One is the number of developers they need to get projects done. Two is the projects they want to get done but can’t do because they don’t have the right resources or it’s going to take too long.

What happens more often than not is that when they try one of these projects on MongoDB they get it done faster — or it’s actually possible. Enterprises are getting better and better at splitting up their systems into lots and lots of small systems that communicate or work together, then each system actually becomes sort of like a little startup.

InfoWorld: What sort of projects are they doing?

Horowitz: Often, they want a single view of something, like a single view of a customer. They want to collect data from tons of different data sources, put it together, and have a good clean view of the whole thing.

InfoWorld: When you’re talking about a single view of the customer, isn’t that the value proposition of CRM?

Horowitz: What happens is they have 72 different CRMs and there’s no way to actually combine them. It becomes an integration nightmare. Or take risk, for an example. You have 20 different risk systems that don’t really talk exactly the same way and you need something that can actually bridge all the different gaps. In the car industry, it’s a single view of a vehicle.

InfoWorld: There’s still a hesitancy, I think, to use NoSQL databases for transactions. How do you address that?

Horowitz: MongoDB is adding more and more features that you need to trust it as your transaction system. Also, because Mongo is inherently a distributed system, the way we think about transactions is a little bit different. In a distributed system, you don’t want to think of a transaction as being committed when it’s on disk, you think about it as committed when it’s on a certain number of servers such that it can’t be destroyed.

That might be something that’s actually in two separate physical data centers. You actually have much stronger reliability than if it were sitting on a physical disk, but it only makes sense in a modern distributed environment. The whole paradigm has shifted a little bit. And when people really understand that, they like it: Data in two data centers is better than being on a disk in one data center.

Then there’s simply maturity. MongoDB has been on the market for less than five years. When Oracle was five years old, it was not what it is today. The database space is a pretty long-term space. It takes a long time to get your products to maturity, and I think we’re moving quite fast — much faster than even we thought possible because there is such a need. Enterprise customers are going to try it for one use case, and they’re going to want to see it in production for a year and a half before they go more mission-critical. Now we’re seeing big waves of mission-critical applications starting to go to MongoDB.

InfoWorld: What sort of mission-critical apps?

Horowitz: One area is user-facing data. In the case of Adobe, when people are using Photoshop on their laptops or desktops, everything they do is saved in Adobe. If that goes down, then it’s quite bad for them. If you’re looking at banks and risk systems and if a risk system for a bank goes down, they can’t trade.

InfoWorld: Really? How are banks using MongoDB?

Horowitz: If you look at the banking industry, there’s so much changing all the time. The regulatory requirements with how they operate business, the way they do business, things are changing very rapidly. And Mongo does adapt faster and move faster; they can’t do that in other systems, and for them that’s really important right now. It’s a huge impact on their business. Mongo is able to get in there even though it’s young and new and open source. That’s where having a commercial company behind an open source product is so important. They’re working with us very closely. We’re in there helping them, we’re in there working with them, so it’s a pretty good relationship, as opposed to them just using some open source product off the street.

InfoWorld: Would you agree, though, that it doesn’t make much sense to move legacy enterprise systems to MongoDB? What you’re really talking about is building the new.

Horowitz: If a legacy system is working, we expect no one to move to Mongo. It doesn’t make sense. Where we see the most usage is in new applications and where they’re building new applications all the time. Or when their legacy applications are breaking and they have to re-architect and rebuild. But if a legacy system is working, they’re not going to move to Mongo for cost savings.

The only times you see it moving are one, if they have to rebuild it, or two, people say: You know what? We are so far behind in our product road map because we can’t move quickly enough that we’re going to bite the bullet for the next six months and actually migrate. It’s going to be painful, then we’re going to be able to catch up faster. But it has to be a pretty big event for someone to actually migrate.

InfoWorld: Who in organizations has been driving adoption of MongoDB?

Horowitz: Developers are probably the No. 1 group. Architects bring it in a lot because it’s all sort of architectural problems. In some cases operations teams bring it in to solve their headaches. And in fewer cases you have a VP or CIO type bring it in because they think it can really transform what they’re building. But I think more often than not, it’s grassroots. People use it, they like it, and they go up the chain to their VP and say: Look, I’m using this one product, and I think we can use it for a much broader set of projects where it will work really well.

InfoWorld: On the operations side, I’ve heard objections that scaling MongoDB isn’t necessarily as easy as advertised. What have been some of those objections and how have you answered them?

Horowitz: I think the biggest objection that we are addressing right now is the underlying system for MongoDB is designed for the greatest possible horizontal scalability and the most obvious operations. Now the most obvious operations are not always the simplest. What we’ve done is made sure that Mongo is built out of a lot of small components that you put together in these little pieces. Each one is independent and you can have a lot of wherever it gets too hard on scalability.

The problem is that now instead of having this one modern cluster you have lots of little pieces to manage. From an operations standpoint, that can be frustrating.

This year we have our automation systems coming out, which will let people actually manage a Mongo cluster very, very simply and make scaling much easier. So, for example, if I want to build a cluster, I can now do it with a few clicks. If I want to operate a cluster, I can do it with four clicks.

Operating large distributed clusters is a challenge, but if you have 200 MongoDB servers and want to upgrade them — and you want to do it all and completely roll in with zero down time — doing that manually is going to be painful. You really need a tool that does it as fast as possible, but also as safely as possible. So this is what we’ve built. It’s in beta right now and we’re releasing it later this year. I think this will be the single thing that people really need to make sure they can operate MongoDB at scale very easily.

InfoWorld: What are some of the most common misuses of MongoDB?

Horowitz: It’s not really misuse, but the biggest challenge of MongoDB is understanding the right way to model data with MongoDB because it is so flexible and you can just start throwing data in so quickly. Often people don’t take the time up front to actually design the right schema, so you end up sort of shooting yourself in the foot eventually.

One good thing with relational databases is that the model and options available to you are pretty simple and there are hard rules so it’s pretty easy not to paint yourself into a corner. With Mongo it’s a little bit easier to paint yourself into a corner, and this is sort of where education and documentation and books and articles and all that stuff are going to be so important, where you really have to make sure you’re using the right model.

There are other use cases where it doesn’t make sense. I think a lot of people have data that looks relational and they think MongoDB is interesting and they put their relational-looking data into Mongo and try to use it like relational databases and in fact it’s a terrible relational database. Those things tend to go awry pretty quickly. If you treat it like relational databases it’s not going to work very well.

InfoWorld: What keeps you up at night?

Horowitz: The biggest challenge I have that scares me the most is the product is good today. People like it. The gap between where we are today and where we want to be five years from now on the technology side is very large. But frankly, the number of users we have at this stage in our lifecycle is very high.

If you compare Mongo to other popular databases at their same lifecycle stage, MongoDB is very, very popular with a very high number of use cases — a very high number of enterprise use cases. So there’s this balance: We need to move quickly because there is this big gap we need to cover, but we also need to be incredibly rigorous because there are all these mission-critical applications.

We can’t break it. It has to work. At the same time, people have given me massive wish lists of features that they all want as soon as possible. The hardest part for me is balancing the speed and the reliability, making sure we are always being as rigorous and as reliable as we possibly can be while we make it as fast as possible.

This article, “11 questions for MongoDB’s CTO,” originally appeared at InfoWorld.com. Read more of Eric Knorr’s Modernizing IT blog. And for the latest business technology news, follow InfoWorld on Twitter.