Leafsnap, an ingenious new app for the iPhone, combines AI techniques with the smartphone's ability to capture sharp images. Could it signal a new breed of apps for collectors?

Leafsnap is an electronic field guide that allows users to identify tree species simply by taking a photograph of the tree's leaves.

In addition to the species name, Leafsnap - like its traditional printed book counterparts - provides photographs and information about the tree's flowers, fruit, seeds and bark. Currently Leafsnap's database includes the trees of the Northeast U.S., but it will soon expand to cover the trees of the entire continental United States.

The app was devised by experts from Columbia University, the University of Maryland and the Smithsonian Institution.

"We wanted to use mathematical techniques we were developing for face recognition and apply them to species identification,"

 said Peter Belhumeur, professor of computer science at Columbia and leader of the Columbia team working on Leafsnap.

"Traditional field guides can be frustrating—you often do not find what you are looking for. We thought we could redesign them using today's smartphones and visual recognition technology."

Belhumeur and his collaborator David Jacobs of the University of Maryland who worked on the algorithms and software need for the app relied on the expertise of   research botanist, John Kress at the Smithsonian's National Museum of Natural History for the identification and collection of leaves required to create the database. Kress was digitizing the botanical specimens at the Smithsonian when first contacted by Jacobs and Belhumeur, so the match between a botanist and computer scientists came at a perfect time.

Kress commented:

"Leafsnap was originally designed as a specialized aid for scientists and plant explorers to discover new species in poorly known habitats. Now Smithsonian research is available as an app for the public to get to know the plant diversity in their own backyards, in parks and in natural areas. This tool is especially important for the environment, because learning about nature is the first step in conserving it."

Users of Leafsnap will not only be learning about the trees in their communities and on their hikes—they will also be contributing to science. As people use Leafsnap, the free mobile app automatically shares their images, species identifications and the tree's location with a community of scientists who will use the information to map and monitor population growth and decline of trees.

The free app is available for the iPhone, with iPad and Android versions to be released later this summer.

For the amateur naturalist and other hobbyists the idea of a smartphone field guide is certainly an attractive one - the same idea could be applied to stamps, flowers, fungi, butterflies, birds, rocks - well just look at the list of Field Guides to complete the list. OK, in some cases just having a photograph might not be enough and the observer might have to answer supplementary questions to get to a definitive identification but the combination of visual recognition, expert systems and the smartphone has a lot going for it. 

From the scientific point of the prospect is equally appealing, opening up the collection of data to a huge number of people who can make good use of their enthusiasm without having first to be trained.