Connectionist Modelling Exercise 1
This exercise is to be your own effort, without the input or collaboration of your fellow students. It is open book, i.e. you may use books, notes or the web in doing it, but you may not confer or consult with any other person.
The exercise is due on Friday, Feb 23rd, 2018. You should hand up a written (printed) answer to all questions below. Please also submit an electronic copy to Fred.
Before attempting this exercise, you should have completed the first three labs...
Q1: Simple Training
Construct a simple 2*1 network with no hidden layer. Load in the patterns in ex1.pat.
Investigate the role of learning rate and momentum in training this network. For any given pair of values, you will have to train the network several times to be sure that the training results (success and time to train) are typical.
Your answer to this question should clearly and neatly summarize your experience, and your understanding of both learning rate and momentum. In particular, do NOT submit pages of numbers, or graphs that occupy half a page and convey a single number. Your goal here is to first explore, and then to communicate the result of your exploration. Also, even though there is a quantitative element to these problems, you are not excused from using proper English, writing in full sentences contained within well-organized paragraphs. Please take care in formulating an informative and well-organized submission.
Q2: A Greater Challenge?
Now try the same thing with the patterns in ex2.pat. Verify that the 2*1 network can not learn these patterns.
Why can the network not learn this set, although it is very similar to the last one? It will help if you plot the patterns in input space. Rough pen and paper plots are all you need.
Can you train any network to learn this set? Provide details of the architecture(s) you try, and how robust your solution is.