Coming to attention by winning the ImageNet contest in 2012, the AlexNet neural network can be seen as being responsible for many of the subsequent breakthroughs in AI. Now the Computer History Museum, in partnership with Google, has released its source code.

AlexNet, is an artificial neural network which can recognize the contents of photographic images. It was created at the University of Toronto by graduate students Alex Krizhevsky, after whom it came to be named, and Ilya Sutskever and their faculty advisor, Geoffrey Hinton. 

As Hinton recently summed it up: 

“Ilya thought we should do it, Alex made it work, and I got the Nobel prize.”

Another person who deserves credit for the emergence of AlexNet is Standard University Professor Fei-Fei Li, instigator of the ImageNet dataset and the ImageNet Large Scale Visual Recognition Challenge. Completed in 2009, ImageNet was larger than any previous image dataset by several orders of magnitude. Li hoped its availability would spur new breakthroughs, and she started the ImageNet competition in 2010 to encourage research teams to improve their image recognition algorithms. After the first two years, the best systems were only making marginal improvements, so in 2012 AlexNet's performance was seen as a breakthrough. 

Krizhevsky, Sutskever and Hinton documented their approach in the paper "ImageNet Classification with Deep Convolutional Neural Networks" which states:

"The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called “dropout” that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry."

The paper was presented by Krizhevsky at a computer vision conference in Florence, Italy, in October. While veteran computer vision researchers weren’t convinced of its usefulness, Yann LeCun, a former postdoctoral student of Hinton's and a proponent of Machine Learning, was at the meeting and pronounced the approach as a turning point for AI. It turned out that he was right in that before AlexNet, almost none of the leading computer vision papers used neural nets. After it, almost all of them would.

In 2020, Hansen Hsu, Curator of the CHM Software History Center reached out to Alex Krizhevsky to ask about the possibility of allowing the Computer History Museum to release the AlexNet source code, due to its historical significance.

As Google had acquired DNNresearch, the company formed by Hinton, Sutskever, and Krizhevsky in 2013, the AlexNet source code belonged to Google. The request was passed on to Hinton, who at the time was still working at Google who in turn  got the ball rolling by connecting Hsu and the CHM to the right team at Google, headed by David Beiber.

Releasing the 2012 version of the source code of Alex Net was the culmination of five years of group effort to make this historic software accessible on CHM’s GitHub Repo.  

More Information

CHM Releases AlexNet Source Code

AlexNet Source Code

Related Articles

Can Google DeepMind Win At AI?

Deep Learning Researchers To Work For Google 

Geoffrey Hinton Shares Nobel Prize For Physics 2024 

Neural Networks

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info