Any questions are welcomed¡I
Please leave your message*^_^*
Q&A of the year(2004)
Previous Q&A(For reference only)
Q:
What exactly are we supposed to do for Exercise 2 of
the written homework?
A:
Exercise 2. introduces one kind of Moore model, what
the question asks is similar to finding the derivation of
the Backpropagation algorithm. In other words, how does each
training step update the network to achieve the goal.
Q:
How do I read the test data for exercise 2 in homework 2?
A:
The test data in homework 2 contains 10 sets of random pattern numbers.
For example: Input_053.dat corresponds to the data set for 5 neurons, 3 patterns.
Therefore the data format is 3 pattern numbers * 10 trials
= 30 random pattern numbers.
The pattern is the binary representation of the pattern number. It is possible that
two out of the three patterns in the test data are the same, but we can still get
results using the dataset.
Q:
How to compare exercise 2 in homework 2 to the lecture notes?
A:
It is similar to the comparison shown in the lecture notes, you can compare the
number of patterns stored, number of stable states, number of limit cycles and statistics.
Q:
Which book is the Appendix A mentioned in exercise 2 in? There is
no appendix in the textbook.
A:
The Appendix A is in the lecture notes.
Q:
Does the ANN in the homework mean "a neural network"? What is DNF?
A:
ANN is the acronym of Artificial Neural Network.
DNF is Disjunctive Normal Form, which means to write boolean functions
in the form of (a&b) | (a&c) | (b&d).
Q:
Is there a limit on the number of neurons used for exercise 1 (a)(b)(e)
of the written homework?
A:
Please use only 1 neuron for (a)(b) and you can use a 3 layer multilayer
network for (e).
Q:
There is infinite number of ©wÅÜ¼Æ in exercise 1 (c)(d) of handwritten
homework?
A:
For exercise 1 (c)(d), please find a formulation for any finite N
inputs. Start from few inputs and try to find its characteristics.
Q:
I am confused about the second formula in the backward computation
at bottom of page a1 of lecture notes. In page a2, it defines i to
be in level l-1. Shouldn't we define k to be the neuron in the next
(upper) level and let k propagate the error back first?
A:
That formula uses Wij where i is the index of neuron in
the next (upper) level of neuron j. In other words, if neuron i is
in layer L, then neuron j is in layer L-1. It means the same thing
as the Wji defined in page a2 but with a different notation.
Q:
The resulting weight matrix from storage algorithm has 0 diagonal entries,
if we use EBPTA to train instead of storage algorithm, will the diagonal
entries still be 0? If it's not 0, can we use Hopfield model?
A:
Hopfield originally has 0 diagonal weight, but the professor asked
you to train again with BP, it's an enhancement
of the original Hopfield network. Of course you can still use the
original Hopfield network and it shouldn't be worse than the
pure original one. You should be able to see it from programming
homework 2.
Q:
When is the homework deadline?
A:
Please hand in all homework no later than January 16th.
No late submission will be accepted. The TA will be in 501 from
9:00am~5:00pm every Tuesday. You can submit the homework in person
or by email. Thanks for your cooporation.
Q:
For programming homework 2, I remember that the professor told us to do
the training using BP with the initial weights given in the class.
After training, use the network weights as Hopfield network instead
of the storage algorithm in page 328. Is that right?
Also, Section 5.2 in the textbook says that when updating
Hopfield network, only one neuron can update its output at
a time. Page 257 example 1 illustrates the different result
of one update at once and multiple updates at once. However,
professor did not talk about this issue in class.
A:
The storage algorithm in the textbook should be the initial weights
the professor mentioned. Just use the values given by professor.
Synchronous updating will create cycles (won't converge to one state),
so we should use asynchronous updating.
Q:
About programming homework 2.
A:
Refer to p.325~329 in the textbook. Someone asks what the weights in
the second layer should be. Isn't there only one level ?_? The feedback
weights are 1, you don't need to train those. The point is at the
storage algorithm in page 328. You have to understand it. If not, you
can refer to p.687~692 in the second textbook. That one might be
easier to understand. Good luck.
Q:
For handwriting homework, we would like to have some hints.
A:
Let me try to state the question more clearly. All 4 neurons can have
1044 possible combinations. When the network represents one
of the 256 boolean functions, each neuron would represent one of the
104 boolean functions. The question asks which of the 256 functions the
network represnets would cause the maximum number of boolean function
combinations (within 1044)to be made by the 4 neurons. That
should be clear enough. I recommend to write a program to find the
result if you can't get the answer by thinking. It would be helpful
to think the reason behind it when you have the answer in hand. Good
luck.
Q:
Is there a paper copy of professor Liou's lecture notes or overhead
slides? Can we borrow it to photocopy because we sometimes get
confused in class.
A:
You can borrow the lecture notes and overhead slides after the class.
If you have any questions in class, don't hesitate to ask ^o^, professor
would be more than happy to answer your questions.
Q:
What is the difference between learning and training? I don't quite
understand what says in the textbook. Some people say they are the same.
Can you clearify that? Thanks!
A:
According to the professor, training and learning are not much different.
The supervised way are apt to be called training and the unsupervised
way are apt to be called learning, that's about it.
Q:
Execuse me, the floppy in the textbook is 5.25'. Professor says all
the files are copied to 3.5' floppy already. Could you please zip it
and send it to me (and anyone who needs it)? Thanks.
A:
Urgh? The disk I get is 3.5' ?_? Anyway, I zipped it already. Email me
if you need it.
Q:
The professor said that we can get programs from the Internet or use
the programs come with the textbook.
Do we just list the input data, parameters and result for the homework?
Can you recommend some websites for demo programs?
A:
First, I have to say sorry to some classmates. I assumed that the
program has to be written on your own, but I checked with professor
today and confirmed that you can use other people's program for
training. What to be handed in as the homework should depend on how
you do it (I don't think everyone is going to use the same program -_-).
So put down whatever you think is appropriate and necessary. It's better
not to give only the input data and weights after training. (Input data
should be the same for everyone, you only need to explain how you use
those input patterns.) Try to explain how those weights achieve the
function you need. I haven't used other people's program for this, so
you guys have to look for it on your own ^o^. Don't worry if you can't
find any, you can learn more by writing your own program and this one
is not that difficult. I believe that you guys can finish coding this
little network pretty fast ^o^. Good luck.
Q:
In the extra two homework questions professor gave, there is one question
with 3 input and 1 neuron. What should we do with that? Also, can you give
us some hints?
A:
The professor should have mentioned the 2 input example, in other
words, when the input dimension is 2, which of the boolean functions
can be made using with only one neuron, which of them cannot? For
example, AND can be made with only one neuron, but not XOR. This
question simply changes the input dimension to 3. It shouldn't be
too difficult. Good luck ^o^.
Q:
For the EBPTA question, do we just send the patterns of both entries
being 1 into the network for training? Or all of the patterns? Do
we preset Emax or let the user input it?
A:
Training data should include all possible input patterns. The input
dimension of this question is only 8, there aren't many possible
combinations. You can try to see the differences in network performance
made by different Emax values. You can choose to let the user enter
the Emax value or not.
Q:
What language or tool should I use for the programming assignment?
A:
You can use any programming language you like, C, Fortrran, Matlab,
SAS... etc. I hope that I am not troubling anyone, but I "heard" that
the first programming homework is due at the midterm... Hurry up.
Q:
How do I hand in the programming homework?
A:
Please email the programming homework to the TA or hand in using a
floppy disc. Please zip it before you email it. Other than the source
code, please attach a short description file explaining the programming
tool and execution environment. Other than Visual C++ and Matlab (TA
is only familar with those two ^o^), please explain how to create the
executable file from your source code. Also give a short note on how
to run your program and any other things that the TA should be awared
of. Finally, don't give TA any virus, otherwise..... you'll see ^o^.
Whether to email it or to hand in a floppy disc, please remember to
write your name and student id.