Contents

Hopfield model for associative memory (letter recognition)

PICSCieE Matlab Usergroup

Author: Melanie Weber

Date: 03/29/2017

Literature: W. Gerstner: Neuronal Dynamics. (http://neuronaldynamics.epfl.ch/online/index.html).

clear all;close all;

Parameters and Patterns (Concepts)

N1=10;
N2=10;
N=N1*N2; % #neurons
N_patterns=3; % #concepts
time=5; % time range

% concepts
 patt1=[-1 -1 -1 -1 -1 -1 -1 -1 -1 -1;-1 1 1 1 1 1 1 1 1 -1;...
     -1 1 1 1 1 1 1 1 1 -1;-1 -1 -1 -1 1 1 -1 -1 -1 -1;...
     -1 -1 -1 -1 1 1 -1 -1 -1 -1; -1 -1 -1 -1 1 1 -1 -1 -1 -1;...
     -1 -1 -1 -1 1 1 -1 -1 -1 -1;-1 -1 -1 -1 1 1 -1 -1 -1 -1;...
     -1 -1 -1 -1 1 1 -1 -1 -1 -1;-1 -1 -1 -1 -1 -1 -1 -1 -1 -1];
 patt1_r=reshape(patt1,N,1); % letter 'T'

 patt2=[-1 -1 -1 -1 -1 -1 -1 -1 -1 -1;-1 1 1 -1 -1 -1 -1 1 1 -1;...
     -1 1 1 -1 -1 -1 -1 1 1 -1;-1 1 1 -1 -1 -1 -1 1 1 -1;...
     -1 1 1 1 1 1 1 1 1 -1;-1 1 1 1 1 1 1 1 1 -1;...
     -1 1 1 -1 -1 -1 -1 1 1 -1;-1 1 1 -1 -1 -1 -1 1 1 -1;...
     -1 1 1 -1 -1 -1 -1 1 1 -1;-1 -1 -1 -1 -1 -1 -1 -1 -1 -1];
 patt2_r=reshape(patt2,N,1); % letter 'H'

 patt3=[-1 -1 -1 -1 -1 -1 -1 -1 -1 -1;-1 1 1 1 1 1 1 1 1 -1;...
     -1 1 1 1 1 1 1 1 1 -1;-1 1 1 -1 -1 -1 -1 1 1 -1;...
     -1 1 1 -1 -1 -1 -1 1 1 -1;-1 1 1 -1 -1 -1 -1 1 1 -1;...
     -1 1 1 -1 -1 -1 -1 1 1 -1;-1 1 1 1 1 1 1 1 1 -1;...
     -1 1 1 1 1 1 1 1 1 -1;-1 -1 -1 -1 -1 -1 -1 -1 -1 -1];
 patt3_r=reshape(patt3,N,1); % letter 'O'

patterns=[patt1_r patt2_r patt3_r];

Plots and Visualization

    %plot patterns
    figure(1)

    subplot(3,1,1)
    imagesc(1,(1:N1),patt1);
    colormap(gray);
    axis equal
    axis off

    subplot(3,1,2)
    imagesc(1,(1:N1),patt2);
    colormap(gray);
    axis equal
    axis off

    subplot(3,1,3)
    imagesc(1,(1:N1),patt3);
    colormap(gray);
    axis equal
    axis off

Weight matrix

Weight matrix $W=(w_{ij})_{1<=i,j<=N}$ describing conncetions between neurons:

Neuron activity: $p_{i} =1$ (active) or $p_{i}=-1$ (inactive). Connections between neurons i and j: $w_{ij}=p_{i} p_{j}$.

W=patterns*patterns';

Initial network state: Noisy input

noise=0.3;

for i=1:N
   if rand > noise
      x_states(i)=patterns(i,2); % assign x_i equal to pattern 2
   else                          % assign random value for x_i
      if rand < .5
         x_states(i)=1;
      else
         x_states(i)=-1;
      end
   end
end

initial = x_states;

% plot perturbed pattern (input)
%figure(2)
%imagesc(1,(1:N),reshape(x_states,N1,N2));
%colormap(gray);

Overlap function

Initialize overlap function $m^{\mu}(t)$.

overlap = zeros(1,time);
overlap(1) = x_states*patterns(:,2)/N;

Dynamics

Evolution of collective dynamics and overlap between stored pattern (fixed point) and noisy input over time:

$S(t+ dt)=g(W S(t))$

with gain function $g(t)=0$ for $g<0.5$ and $g(t)=1$ for $g>0.5$.

(for details in overlap see http://neuronaldynamics.epfl.ch/online/Ch17.S2.html

figure(3)
subplot(time,1,1)
imagesc(1,(1:N1),reshape(x_states,N1,N2));
colormap(gray);
axis equal
axis off

for t=2:time
    x_states = (2*(W*x_states'>=0)-1)'; % time step
    overlap(t) = x_states*patterns(:,2)/N; % overlap (updated)

    subplot(time,1,t)
    imagesc(1,(1:N1),reshape(x_states,N1,N2));
    colormap(gray);
    axis equal
    axis off

end

Plots and Visualization of final result

Time evolution of overlap function, perturbed image and recognized image.

x_states_r=reshape(uint8(x_states),N1,N2);
initial_r=reshape(uint8(initial),N1,N2);

figure(4)

subplot(3,1,1)
plot(0:time-1,overlap,'o-');
xlabel('Time t')
ylabel('Overlap m^{\mu}(t)')
axis equal

subplot(3,1,2)
imagesc(1,(1:N1),patt2);
colormap(gray);
axis equal
axis off

subplot(3,1,3)
imagesc(1,(1:N1),x_states_r);
colormap(gray);
axis equal
axis off

Probability of moving away from right pattern

Probability pf moving away from "right" pattern (i.e. from pattern with biggest overlap).

p=0.5*(1-erf(sqrt(N/(2*N_patterns))));


Published with MATLAB® R2016b