The Program

The time has come to program a Hopfield network. There is nothing like building a program to prove that you understand what is happening.

The Python program listed implements an 8x8 grid of neurons but you can change this by setting n to a different value. The state of each neuron is recorded in the list s[] and weights are stored in w[][] and these are initalized using comprehensions:

    s = [-1 for x in range(n*n)]
    w = [[0 for x in range(n*n)] for y in range(n*n)]

The most difficult part of the program is the user interface implemented in Tkinter. This consists of a frame with a set of buttons organized on an x,y grid. This isn't efficient, but it is very simple:

root = Tk()
root.title("Hopfield")

menu_bar = Menu(root)
root.config(menu=menu_bar)
menu_bar.add_command(label="learn", command=learn)
menu_bar.add_command(label="recall", command=recall)

grid_frame = Frame(root)
grid_frame.pack(side=TOP)
for r in range(8):
   for c in range(8):
       button = Button(grid_frame, text="   ", 
                        borderwidth=1, bg="white")
       button.n = c+r*8
       button.grid(row=r, column=c)
       button.bind('<Button-1>', click)
root.mainloop()

A menu is set up with two simple options - learn or recall. The learn option learns the current state of the system and the recall iterates to convergence starting from the current state of the system. The buttons have no text and are set to white as a background color. A custom attribute, n, is added to the button and this gives the button's number in the state vector. That is, if n=8 the buttons are numbered 0 to 63.

We also need a click event handler for each button:

def click(e):
    but = e.widget
    i=but.n
    if s[i] == -1:
        s[i] = 1
        but.configure(bg="black")
    else:
        s[i] = -1
        but.configure(bg="white")

This finds the number of the button that has been clicked and toggles its state from 1 to -1 and vice versa. The button's color is also updated to give feedback to the user.

Finally we need to implement the Hopfield part of the program. The first is the learn function:

def learn():
    m = n*n
    for i in range(m):
        for j in range(i+1, m):
            w[i][j] = w[i][j]+s[i]*s[j]
            w[j][i] = w[i][j]

This simply makes one pass though the state vector updating the weight matrix. Notice that we only compute the upper triangle of the matrix and enforce symmetry. We also avoid updating the diagonal.

The second is the recall function:

def recall():
    m = n*n
    c=1
    while c!=0:
        c=0
        for i in range(m):
            a = 0
            for j in range(m):
                a = a+s[j]*w[i][j]
            if a<0 and s[i]>0:
                s[i]=-1
                c=1
            if a>0 and s[i]<0:
                s[i]=1
                c=1                      
    update()

This is only complicated from the basic equations by the need to detect a change in the state. The inner for loop computes the update to neuron i i.e. s[i] and as Python doesn't have a sign function we have to test for positive or negative changes.

The update function called at the end of the recall function simply updates the display:

def update():
    for key, but in grid_frame.children.items():
        i=but.n
        if s[i]==-1:
            but.configure(bg="white")
        else:
            but.configure(bg="black")

This scans though all of the buttons in the frame and updates their color based on the value in the state vector s.

Trying It Out

It can be a little difficult to figure out what the network is doing when you first interact with it. Your first task is to use it to remember some patterns. The simplest say is a horizontal line and then a vertical line:

Click the learn menu option after entering each pattern. Next you can erase the grid. Then place a single pixel on the grid and press recall. What will appear depends on how close to either line the single pixel is. You can move on from this to see what happens if you input patterns close to the horizontal or vertical line.

Once you have seen this, try learning some simple digits or letters and recalling them with slightly noisy or distorted patterns.

The Complete Program

from tkinter import *
def click(e):
    but = e.widget
    i=but.n
    if s[i] == -1:
        s[i] = 1
        but.configure(bg="black")
    else:
        s[i] = -1
        but.configure(bg="white")
def learn():
    m = n*n
    for i in range(m):
        for j in range(i+1, m):
            w[i][j] = w[i][j]+s[i]*s[j]
            w[j][i] = w[i][j]
def recall():
    m = n*n
    c=1
    while c!=0:
        c=0
        for i in range(m):
            a = 0
            for j in range(m):
                a = a+s[j]*w[i][j]
            if a<0 and s[i]>0:
                s[i]=-1
                c=1
            if a>0 and s[i]<0:
                s[i]=1
                c=1                      
    update()

def update():
    for key, but in grid_frame.children.items():
        i=but.n
        if s[i]==-1:
            but.configure(bg="white")
        else:
            but.configure(bg="black")
 
if __name__ == "__main__":
    n = 8
    s = [-1 for x in range(n*n)]
    w = [[0 for x in range(n*n)] for y in range(n*n)]
    root = Tk()
    root.title("Hopfield")
    menu_bar = Menu(root)
    root.config(menu=menu_bar)
    menu_bar.add_command(label="learn", command=learn)
    menu_bar.add_command(label="recall", command=recall)
    grid_frame = Frame(root)
    grid_frame.pack(side=TOP)
    for r in range(8):
        for c in range(8):
            button = Button(grid_frame,
              text="   ", borderwidth=1, bg="white")
            button.n = c+r*8
            button.grid(row=r, column=c)
            button.bind('<Button-1>', click)
    root.mainloop()

The program could be improved by the addition of a menu option to Clear the grid.

Related Articles

Geoffrey Hinton Shares Nobel Prize For Physics 2024

Neural Turing Machines Learn Their Algorithms

Neural Networks

McCulloch-Pitts Neuron

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info