I''m having problems with a simple hopfield network. Basically, I''m storing 3 patterns in a network of 63 neurons. The network will converge to patterns 1 and 2, but will never converge to pattern 3. Even if I set the initial state of the network identical to pattern 3, the network will always converge to pattern 2. I''m aware of the problems of storage capacity in a hopfield network. Using certain formulas you can approximate that a hopfield network with 63 neurons should be able to hold and recall upto 10 patterns. So the problem is, why is my network not able to hold and recall more than 2 patterns? I''m trying to visualise the energy landscape of the network to try and understand why it doesn''t work. It''s my understanding that local minima on the energy surface represent stored patterns. Since a hopfield network will reduce it''s energy on every iteration, it should get stuck in a local minimum and thus converge on a stored pattern. So why is it that when the network''s initial state is the same as a stored pattern it converges on another pattern? I''m assuming the reason has something to do with how similar the patterns are. Is it possible that if two patterns are similar then the local minimum of one pattern can erase the local minimum of another pattern? Does anybody have any ideas? Can anybody help me? Have I finally lost it? Thanks

You must have worked from the same book as me

I have consulted other sources on hopfield networks, so it''s not a case of a misprint in a textbook.

Have you tried implementing the task in a Bidirectional Associative Memory model to see if you get the same problem. If you do, then it''s an issue with the problem space, not the implementation. Off the top of my head I could not offer any specific answers although I would be happy to take a look at it in more detail if you cannot find a solution. Which book did the problem come from?

Cheers,

Timkin

I''ve been doing some experimenting. The problem definitely has something to do with how similar the patterns are. I could successfully recall all three patterns if I made them very different to each other.

I was intending to implement a BAM, but if they suffer from the same limitations as a hopfield network then I don''t think I''ll bother.

I''m incredibly curious. What possible use could a hopfield network have given it''s crippling limitations? Has anybody actually successfully used a hopfield network? Are they just something people stick in books for the sake of being complete? And why does buttered toast always fall buttered side down?

Cheers

From my understanding, Hopfield networks are generally included for historical reasons and because they are (generally) a good starting point for certain memory models. The BAM is more widely used - since it too performs heteroassociation - and the Hopfield is just a specific case of the BAM. I''ve been lead to believe that the BAM is more reliable than the Hopfield, but I haven''t verified this for myself.

As to the toast question it''s really an issue of minimum entropy! With buttered side up there is a rougher surface meeting the oncoming air (the unbuttered side) and thus more stirring of the air molecules, thus a higher entropy state. The universe doesn''t like higher entropy states because it''s being forced to accept them every day! So, the universe gets its own back by flipping your toast, giving a smoother, oily side to face the air stream thus reducing turbulence and minimising the entropy change. Oh, that and the fact that carpet likes butter!

Cheers,

Timkin

ROFLMAOFAVLT