SSL Socket-Based HTTPS Client

Many websites now refuse to serve unencrypted data and insist on HTTPS. Fortunately it is very easy to create an HTTPS client as you don’t need to create a digital certificate. If you need to prove to a server that it is indeed you trying to connect then you should install a new certificate and use it. This is exactly the same procedure as installing a new server certificate, see later.

MicroPython has a very minimal implementation of the Python ssl module, but it is enough to do what you need to make an HTTPS client or server. It only has a single function wrap_socket which adds encryption, and in some cases authentication, to an existing socket. At the time of writing, MicroPython for the ESP32 doesn’t support authentication.

The wrap_socket function is:

ssl.wrap_socket(sock, keyfile=None, certfile=None,
                server_side=False, cert_reqs=CERT_NONE,
                ca_certs=None,
                do_handshake_on_connect=True,               
               )

This takes an existing socket specified by sock and turns it into an encrypted SSL socket. Apart from sock, all its parameters, including keyfile and certfile which specify files containing the certificate and/or key, are optional.

All certificates have to be in PEM (Privacy-Enhanced Mail) format. If the key is stored in the certificate then you only need to use certfile. If the key is stored separately from the certificate you need both certfile and keyfile. When True, the server_side parameter sets the behavior appropriate to a server and when False to a client and cert_reqs determines the level of certificate checking, from CERT_NONE, CERT_OPTIONAL to CERT_REQUIRED. The validity of the certificate is only checked if you select CERT_REQUIRED and currently the ESP32 never checks the certificate for validity. ca_certs is a bytes object containing the certificate change to be used to validate the client’s certificate. server_hostname is used to set the server’s hostname so that the certificate can be checked to ascertain that it does belong to the website and to allow the server to present the appropriate certificate if it is hosting multiple sites. do_handshake should be used with non-blocking sockets and when True defers the handshake whereas when False, the function doesn’t return until the handshake is complete.

Notice that not all of the full Python wrap_socket parameters are supported and not all TLS and encryption levels work. This means that you will discover that, at the moment, connecting to some websites is difficult, if not impossible. However, in many case it should simply work, as long as the website concerned is not using a cutting edge implementation.

The only modification that the previous HTTP client needs to work with an HTTPS website is:

import network
import socket
import ssl
from machine import Pin, Timer
from time import sleep_ms
def setup(country, ssid, key):
    .   .   .
wifi=setup(country, ssid, key)
print("Connected")
ai = socket.getaddrinfo("example.com", 443,
                                  socket.AF_INET)
addr = ai[0][-1]
s = socket.socket(socket.AF_INET)
s.connect(addr)
sslSock=ssl.wrap_socket(s)
request = b"GET / HTTP/1.1\r\nHost:example.com\r\n\r\n"
sslSock.write(request)
print(sslSock.read(1024))

where the setup function has been omitted.

Notice that all we need to do is perform a default wrap_socket after making the connection. The downloaded HTML is now fetched using HTTPS. The sslSock returned by the wrap_socket function only has stream read and write methods.

In chapter but not in this extract

• Socket Server
• A Socket Temperature Server
• An SSL HTTPS Server
• Non-Blocking Sockets
• The Connection Queue

 Summary

• If you want to do anything beyond HTTP, or you want to implement an HTTP server, then you need to make use of sockets.

• Sockets are completely general and you can use them to implement an HTTP client or a server.

• To allow for different types of connection, sockets can be used with a range of different types of addresses. In the case of the ESP32, however, we only need to use IP addresses.

• A socket client has to handle the details of the data transferred. In particular, you have to handle the details of HTTP headers.

• Implementing a socket server is easy, but it can be difficult to ensure that both clients and internal services are both attended to.

• The simplest solution to the server problem is to implement a polling loop and to do this you need to make use of non-blocking sockets.

• Simple sockets work with unencrypted data. If you want to use encryption you have to “wrap” the socket using the ssl module.

• HTTPS clients don’t need a certificate to implement encryption, but servers do.

• For testing you can generate your own “self-signed” certificates although most browsers will complain that this isn’t secure.

• The connection queue allows you to handle more than one client connection at a time.

Programming the ESP32in MicroPython

By Harry Fairhead & Mike James

Buy from Amazon.

Contents

       Preface

1. The ESP32 – Before We Begin
2. Getting Started
3. Getting Started With The GPIO 
4. Simple Output
5. Some Electronics
6. Simple Input
7. Advanced Input – Interrupts
8. Pulse Width Modulation
      Extract: PWM And The Duty Cycle
   
9. Controlling Motors And Servos
10. Getting Started With The SPI Bus
11. Using Analog Sensors
       Extract: Analog Input
    
12. Using The I2C Bus
       Extract: I2C, HTU21D And Slow Reading 
13. One-Wire Protocols
14. The Serial Port
15. Using WiFi
     Extract: WiFi 
16. Sockets
     Extract: Client Sockets
     Extract: SSL Client Sockets
17. Asyncio And Servers
     Extract: Asyncio ***NEW!
18. Direct To The Hardware
     Extract: Using Hardware Registers 
    

<ASIN:187196282X>