Martin Heller
Contributing Writer

Programming Collective Intelligence

analysis
Sep 5, 20073 mins

I receive dozens of computer books to review every month. Many of them are pretty good, but they tend to blend together. That's understandable, since so many of them cover the same products and languages. In one of my favorite Newhart episodes (that's the 1982-1990 series in which Bob plays Dick, an innkeeper and author of how-to books) Dick is sued for plagiarism by a rival author of how-to books. One of the po

In one of my favorite Newhart episodes (that’s the 1982-1990 series in which Bob plays Dick, an innkeeper and author of how-to books) Dick is sued for plagiarism by a rival author of how-to books. One of the points of similarity cited in the trial is the sentence “Remove the two screws.” How many ways are there of saying that? How many ways are there of explaining the precedence of operators in C++?

Once in a while a book comes in that breaks the mold. Programming Collective Intelligence: Building Smart Web 2.0 Applications, by Toby Segaran (O’Reilly, 2007, 358 pp., ISBN 9780596529321, $39.99) is one of them.

Basically, Programming Collective Intelligence is a book about analyzing Web data using statistical and AI methods in Python. It’s more interesting than that sounds, however, at least partly because it’s very practical and pragmatic.

What does it cover? An interactive TOC is online here; the summary (from the back cover copy) follows:

Want to tap the power behind search rankings, product recommendations, social bookmarking, and online matchmaking? This fascinating book demonstrates how you can build Web 2.0 applications to mine the enormous amount of data created by people on the Internet. With the sophisticated algorithms in this book, you can write smart programs to access interesting datasets from other web sites, collect data from users of your own applications, and analyze and understand the data once you’ve found it.

Programming Collective Intelligence takes you into the world of machine learning and statistics, and explains how to draw conclusions about user experience, marketing, personal tastes, and human behavior in general — all from information that you and others collect every day. Each algorithm is described clearly and concisely with code that can immediately be used on your web site, blog, Wiki, or specialized application. This book explains:

  • Collaborative filtering techniques that enable online retailers to recommend products or media
  • Methods of clustering to detect groups of similar items in a large dataset
  • Search engine features — crawlers, indexers, query engines, and the PageRank algorithm
  • Optimization algorithms that search millions of possible solutions to a problem and choose the best one
  • Bayesian filtering, used in spam filters for classifying documents based on word types and other features
  • Using decision trees not only to make predictions, but to model the way decisions are made
  • Predicting numerical values rather than classifications to build price models
  • Support vector machines to match people in online dating sites
  • Non-negative matrix factorization to find the independent features in a dataset
  • Evolving intelligence for problem solving — how a computer develops its skill by improving its own code the more it plays a game
Each chapter includes exercises for extending the algorithms to make them more powerful. Go beyond simple database-backed applications and put the wealth of Internet data to work for you.
Martin Heller

Martin Heller is a contributing writer at InfoWorld. Formerly a web and Windows programming consultant, he developed databases, software, and websites from his office in Andover, Massachusetts, from 1986 to 2010. From 2010 to August of 2012, Martin was vice president of technology and education at Alpha Software. From March 2013 to January 2014, he was chairman of Tubifi, maker of a cloud-based video editor, having previously served as CEO.

Martin is the author or co-author of nearly a dozen PC software packages and half a dozen Web applications. He is also the author of several books on Windows programming. As a consultant, Martin has worked with companies of all sizes to design, develop, improve, and/or debug Windows, web, and database applications, and has performed strategic business consulting for high-tech corporations ranging from tiny to Fortune 100 and from local to multinational.

Martin’s specialties include programming languages C++, Python, C#, JavaScript, and SQL, and databases PostgreSQL, MySQL, Microsoft SQL Server, Oracle Database, Google Cloud Spanner, CockroachDB, MongoDB, Cassandra, and Couchbase. He writes about software development, data management, analytics, AI, and machine learning, contributing technology analyses, explainers, how-to articles, and hands-on reviews of software development tools, data platforms, AI models, machine learning libraries, and much more.

More from this author