What is the simplest HTTPS client you can create using lwIP and mbedtls? The answer is very simple indeed. This is an extract from our latest book on the Pico in C.

Master the Raspberry Pi Pico in C:
WiFiwith lwIP & mbedtls

By Harry Fairhead & Mike James

Buy from Amazon.

Contents

       Preface

1. The Pico WiFi Stack
2. Introduction To TCP
         Extract: Simplest HTTP Client
3. More Advanced TCP
4. SSL/TLS and HTTPS
         Extract: Simplest HTTPS Client
   
5. Details of Cryptography
         Extract: Random Numbers
   
6. Servers
         Extract: HTTP Server NEW!!
7. UDP For Speed
         Extract:  Basic UDP
   
8. SNTP For Time-Keeping
9. SMTP For Email
10. MQTT For The IoT
    
    

    Appendix 1 Getting Started In C

<ASIN:B0C247QJLK>

SSL/TLS and HTTPS

HTTP using TCP/IP is a simple and relatively efficient way to transport data, but it isn’t secure. Today many web servers will only deliver data if it is protected by encryption and identity confirmation using HTTPS. This, and the general concern for security, causes problems for IoT programmers. Security, and encryption in particular, are not cheap in terms of the resources needed to implement them, but if you are planning to make your code available to the wider world you really don’t have a choice but to implement security. There are many ways of doing this but the best advice is, if possible, don’t do it from scratch. If you are not a security expert you are likely to get it wrong and make all of the effort you put into building something “secure” a waste of time. Users also like to know that your application is secure by recognizing well known standards. In short, there is a lot of pressure to use security standards even if they might be more than is actually required for an IoT device.

What this means in practice is that you most likely need to support HTTPS for web-based interactions and SSL/TLS (Secure Sockets Layer/Transport Layer Security), which is the basis of HTTPS, for other interactions. Fortunately, lwIP has a degree of integration with a very standard SSL/TLS library, mbedTLS.

In this chapter we look at how to use mbedTLS to create an HTTPS client, but this is just a step in using mbedTLS to implement a range of secure data transfers.

Public Key Certificates

The basis of modern security is Public Key Infrastructure, PKI. This in turn is based on the use of public key, or asymmetrical, cryptography. The simplest approach to cryptography is to use the same key for encryption as decryption. In this case, the key is private and has to be known to both the sender and the receiver and has to be kept secret from everyone else. This creates the problem of “key exchange”. How can you get a key securely from the sender to the receiver of the data. In transit the key is vulnerable.

Public key cryptography is amazing because it works with two keys – a public key that anyone can know and use to encrypt a message and a private key that only the receiver knows and can be used to decrypt the message. This is asymmetrical cryptography because different keys are used for encryption and decryption. Its big advantage is that it removes the need to exchange keys. Its big disadvantage is that it is very slow. Because it is so slow and inefficient public key cryptography is used to exchange symmetric keys. That is, most of the time you are using symmetric key cryptography with keys provided by public key cryptography. Keys are generally stored in a special format that associates the keys with an identity – a certificate.

What happens is that, when a client connects to a server, the private and public keys are used to create a shared encryption key which is then used for all further communication. There are two general situations. If the server has a certificate but the client doesn’t then the client generates a random number and uses the server’s public key to securely send this to the server. Of course, as only the server has the private key only the server can decrypt the message and so the client and the server now have a shared secret that they can use to generate a symmetric key. The second situation is where both parties have a certificate and in this case the private and public keys available at each end of the connection are used to transport a random number to act as a shared secret.

What this means in practice is that a client doesn’t need a certificate to establish secure communication. A client only needs a certificate if it needs to prove its identity. What is more, a client IoT device is less secure with a certificate that contains a private key as this has to be stored in the device and most don’t have the necessary hardware to stop someone from connecting to the device and reading the key. Servers can keep their private keys secure because they are physically secure and protecting the key is a matter of software security. For most IoT devices no amount of software security can keep the key safe from a physical attack.

It is also important to know that key exchange can occur more than once in the lifetime of a connection to ensure the maximum security. The actual symmetric key encryption algorithm that is used can also be negotiated between the client and server. This all makes the connection more complicated.

Encryption solves the privacy issue, but it doesn’t solve the identity problem. How does the client know that the server it is connecting to is the real server and not an impostor? This is where digital certificates come into the picture. At the simplest level, a certificate is simply a container for a pair of public and private keys. More commonly the certificate only contains the public key – the private key being stored in a separate key file – this allows the certificate to be shared. Certificates also contain other information about the entity that the certificate is issued to, such as name, address, domain name and so on. Obtaining a certificate is one way for a client to gain information about a server including its public key which allows key exchange to take place.

Certificates are also digitally signed to indicate their validity. Of course, how much trust you can put in the certificate depends on who signed it. You can easily create a certificate that you also sign – a self-signed certificate - and while this is useful as a way to supply a public key, it does nothing to prove that you are who you claim to be. To be convincing evidence of identity a certificate has to be digitally signed by one of a number of well-known certificate-issuing authorities. The certificate-issuing authority will make checks and ask for documentation that proves who you are and for this it makes a charge.

How can you know that a certificate-issuing authority is authentic? It is usually the case that the certificate-issuing authority has a certificate that proves who they are, signed by another, higher, authority. This leads to the idea of a chain of certificates which ultimately end in a signature from a trusted final authority with a certificate that is installed in most browsers and used to give the final verdict on the validity of a certificate. Following a certificate chain and validating an identity is time-consuming and resource heavy.

At the time of writing, the Pico uses certificates for encryption, but doesn’t validate certificates to confirm identities.