A Socket Web Client

Using what we know so far about sockets, we can easily connect to a server and send and receive data. What data we actually send and receive depends on the protocol in use. Web servers use HTTP, which is a very simple text-based protocol. Using the urequests module we could ignore the nature of the protocol as it implemented most of it for us. When it comes to using sockets we have to work out what to send in detail.

The HTTP protocol is essentially a set of text headers of the form:

headername: headerdata \r\n

that tell the server what to do, and a set of headers that the server sends back to tell you what it has done. You can look up the details of HTTP headers in the documentation – there are a lot of them.

The most basic transaction the client can have with the server is to send a GET request for the server to send back a particular file. Thus the simplest header is:

"GET /index.html HTTP/1.1\r\n\r\n"

which is a request for the server to send index.html. In most cases we need one more header, HOST, which gives the domain name of the server. Why do we need it? Simply because HTTP says you should and many websites are hosted by a single server at the same IP address. Which website the server retrieves the file from is governed by the domain name you specify in the HOST header.

This means that the simplest set of headers we can send the server is:

"GET /index.htm HTTP/1.1\r\nHOST:example.org\r\n\r\n";

which corresponds to the headers:

GET /index.html HTTP/1.1
HOST:example.org

An HTTP request always ends with a blank line. If you don't send the blank line then you will get no response from most servers. In addition, the HOST header has to have the domain name with no additional syntax - no slashes and no http: or similar.

request = b"GET /index.html HTTP/1.1\r\n
                         Host:example.org\r\n\r\n"

Now we are ready to send our request to the server but first we need its address and we need to connect the socket:

ai = socket.getaddrinfo("www.example.com",
                              80,socket.AF_INET)
addr = ai[0][-1]
s = socket.socket(socket.AF_INET)
s.connect(addr)

Now we can send the headers which constitute the GET request:

request = b"GET /index.html HTTP/1.1\r\n
                       Host:example.org\r\n\r\n"
s.send(request)

Finally we can wait for the response from the server and display it:

print(s.recv(512))

Notice that all of the methods are blocking in the sense that they don’t return until the operation is complete.

The complete program, with the setup function, given earlier, omitted is:

import network
import socket
import rp2
from machine import Pin, Timer

from time import sleep_ms
def setup(country, ssid, key):

 . . .
wifi = setup("country", "ssid", "key")
ai = socket.getaddrinfo("www.example.com",
                                  80,socket.AF_INET)
addr = ai[0][-1]
s = socket.socket(socket.AF_INET)
s.connect(addr)
request = b"GET /index.html HTTP/1.1\r\n
                             Host:example.org\r\n\r\n"
s.send(request)
print(s.recv(512))

You can see that the urequests module is much easier to use. Of course, it makes use of sockets to do its job.