| Microsoft Invents A Fast CryptoNet |
| Written by Mike James |
| Monday, 15 February 2016 |
|
A CryptoNet is a neural network that works like a standard neural network but with encrypted data. It can make a prediction based on the data without revealing it, or the prediction, to a third party. A CryptoNet is where a neural network meets homomorphic encryption.
The problem that Microsoft Research is trying to solve is how to provide the facilities of a neural network as a hosted service - Machine Learning As a Service MLAS - in a secure way. A potential user might well value the result of applying a neural network to their data but be much less happier about making it visible to a remote server. There are a few possible ways of keeping the data secure but the most desirable is to use homomorphic encryption. Suppose you want to send two numbers to an "addition" server, it adds them and sends back the result. You could use standard encryption to send the two items of data, but then the server would have to have a decryption key to decrypt the values, add them and then encrypt the result. This is workable but clearly potentially leaky. Homomorphic encryption allows the user to send the two values encrypted, for the server to add the encrypted values and send the result back to the user. The user can then decrypt the result to get the sum of the original values. In other words in homomorphic encryption the sum of the encrypted values is the encryption of the sum of the original values. Homomorphic encryption is almost magic but some schemes have been invented for addition and multiplication but they are very slow. A CryptoNet is a neural network that works with homomorphically encoded data to produce an encoded result. As we can create homomorphic codes that preserve addition and multiplication and we can arrange for a neural network to work with just these operations. The problem is that training and using a neural network is already time consuming. The breakthrough is to carefully match the characteristics of the homomorphic encryption to the neural network and vice versa. The homomorphic encryption actually works with polynomials - that is, it encrypts a polynomial - and as long as the operation of the neural network only involves polynomials than we can obtain a speed up using SIMD instructions. In this case polynomials of degree up to 8191 were used to pack images into batches of 8192. The MNIST database of hand written digits was used to train the network. A training set of 50,000 images was used and the network achieved a 99% accuracy on the remaining 10,000 test cases. Each image is 28x28 and 8191 images can be encoded in one 122-second operation. The network can be applied in 570 seconds on a PC with 16GB of RAM and a Xeon E5 running at 3.5GHz. The decryption of the result takes 5 seconds. This gives a total throughput of 77236 images per hour which is practical enough for a commercial service. The idea of mixing the computational complexity of neural networks and homomorphic encryption seems like a non-starter, but apparently it can be done. You can train and use a neural network with encrypted data without decrypting it first. At the moment it is difficult to say if this technology is going anywhere. It is unclear how big the demand for machine learning as a service is, or could be. It is also unclear how often issues of data security would make this approach desirable.
More InformationCryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy Related ArticlesMicrosoft Wins ImageNet Using Extremely Deep Neural Networks Google Gears Up To Sell AI Vision Azure Machine Learning Service Goes Live Open Source Homomorphic Cryptography DARPA spends $20 million on homomorphic encryption
To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, Facebook, Google+ or Linkedin.
Comments
or email your comment to: comments@i-programmer.info |
| Last Updated ( Tuesday, 16 February 2016 ) |
