assignment.txt

Part 1: Classification with MLP updated by Backpropagation following gradient 
descent (55%). 

a. Before we begin the classification, let us generate the binary class data first. The data can be generated and stored with the “Generate_data.py” file. It will create one scatter plot of the data, and “.npy” and “.mat” files that stores the data. You may also generate with your own set of codes. Show the scatter plot of the generated data. Further, if you implement your own data generation code, show your code. Note that the ground-truth output of class 1 is 0, and the ground-truth output of class 2 is 1. (10%) 

b. Next, implement a multi-layer perceptron (MLP) for classification. The MLP is a 2-layer MLP with only ONE hidden layer. Its structure should be 2 input neurons, 3 hidden layer neurons, and 1 output neuron (binary output). You may select the activation function for all the neurons (e.g., ReLU activation). Show your detailed implementation of the MLP. There are a total of 9 connections between the neurons. Initialize their weights randomly (e.g., follow a normal distribution), then plot the decision boundary of the initial MLP. With this initial MLP, what is the classification accuracy of the data? (20%) 

c. Subsequently, we update the weights of the MLP connections (9 of them) with the Backpropagation algorithm, following the spirit of gradient descent. Implement the update from scratch and update the weights over ALL 100 data entries. You may select the cost/loss function you deemed suitable (e.g., squared error). Show your codes in detail. Then plot the new decision boundary as in Part 1b. Compare the two decision boundaries and discuss the differences and the effects of your implemented update. What is the classification accuracy with the updated MLP? (25%) 

Part 2: Classification with RBF Network with Random RBF Centers (45%). 
a. With the same set of data as in Part 1, now we leverage the RBF network for this part of classification task. Assume that the Gaussian function is used for the hidden layer neurons with the width set as standard deviation = 1,Further, we set that there are 3 hidden neurons, and that the 3 RBF centers are randomly set. The randomly set RBF centers should be situated within the range of all the data. The RBF neural network also has only 1 single output neuron. Show your code that constructs this RBF network and show the plot that highlights the randomly chosen RBF center. Subsequently, code and determine the weights from the hidden layer to the output layer, using the LS estimation algorithm. Next, plot the decision boundary of this RBF network and determine the classification accuracy. Compare and discuss the performance of this RBF network compared to the updated MLP after Part 1c. (30%) 

b. Repeat Part 2a, now with 6 RBF centers that are randomly set. The hidden neurons still leverage the Gaussian function with the same width of standard deviation = 1. Determine the new weights between the hidden layer and the output neuron. Plot the decision boundary for this RBF network and determine the classification accuracy. Compare with the RBF network in Part 2a and discuss the effects of the numbers of hidden neuron on the performance of the RBF network. (15%)