🎉 Celebrating 25 Years of GameDev.net! 🎉
Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!
Articles on Neural Nets
Hi,
does anyone know some good articles on Neural Nets,
that go a little deeper than introduction articles ?
I''ve build my general neural net-class, and tested it
on pattern-recognition, and guess what ? It works !!
The thing I''m struggeling with is what values to
choose for learning rate, momenten, and when to decrease
the learning rate or the momentum and by how
much. I really need a good (mathematical) article on
this.
Thanks,
Edotorpedo
Don''t know any real good articles that focus on the mathematics .. but you really shouldn''t be looking for them anyway.
The fact at the moment is that neural nets are basically a "Black art" (to quote my NN lecturer) and all there is at present are heuristics on how to do it .. but none of them are general across all applications ..
so don''t bother looking for hard and fast rules .. for any specific application you need to test out different values and decrease rates until you hit an acceptable set. It''s just experience ..
but basically all I''ve read says that the learning rate and the momentum should be pretty small to start with, though they can be a little bigger if they decrease during training.
The point of momentum is to make sure that the network doesn''t get stuck on local minima while it''s travelling downslope, and the learning rate is to prevent the overshooting of the golbal minima once you get there.
sorry if I''ve repeated the obvious .. and good luck
The fact at the moment is that neural nets are basically a "Black art" (to quote my NN lecturer) and all there is at present are heuristics on how to do it .. but none of them are general across all applications ..
so don''t bother looking for hard and fast rules .. for any specific application you need to test out different values and decrease rates until you hit an acceptable set. It''s just experience ..
but basically all I''ve read says that the learning rate and the momentum should be pretty small to start with, though they can be a little bigger if they decrease during training.
The point of momentum is to make sure that the network doesn''t get stuck on local minima while it''s travelling downslope, and the learning rate is to prevent the overshooting of the golbal minima once you get there.
sorry if I''ve repeated the obvious .. and good luck
not sure but this is one that I heard was good:
http://www.btinternet.com/~fup/nnt1.html
I haven't had time to read it
A CRPG in development...
Need help? Well, go FAQ yourself.
Edited by - Nazrix on October 29, 2001 9:46:57 PM
http://www.btinternet.com/~fup/nnt1.html
I haven't had time to read it
A CRPG in development...
Need help? Well, go FAQ yourself.
Edited by - Nazrix on October 29, 2001 9:46:57 PM
nazrix:
I think the whole point of fup's article is that it should not be mathematical. Edotorpedo is looking for a mathematical text.
/Mankind gave birth to God.
Edited by - silvren on October 30, 2001 5:09:47 AM
I think the whole point of fup's article is that it should not be mathematical. Edotorpedo is looking for a mathematical text.
/Mankind gave birth to God.
Edited by - silvren on October 30, 2001 5:09:47 AM
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement