Main page for course

CS 470   Artificial Intelligence,  Instructor:  Jeffrey Horn

FINAL PROJECT

You will need the files in the final project folder "Final_Project_on_Palindromes" here (and the compressed version here).

Everyone gets one data point which means one single set of parameter settings but ten different runs, each with a different random number seed.   Here is what to record.   In the spreadsheet below (there is a spreadsheet file in the project folder here), each of us needs to fill in the bright yellow cells below.   (First, upper left, comes our name and the two unique parameter settings:  # of examples, and # of training repetitions.)   The RAW DATA section has four columns and ten rows that we need to fill in.   These are the four outputs in the system console window for each run, and there are ten runs, one for each random seed:  102, 112, 122, 132, ... , 192.   You only need to enter the scores (% correct) for the four cases:  training set positive examples, training set negative, test set positive, and test set negative.   The COMPOUNDED DATA (blue cells) have formula that calculate their values from your raw data.  I want you to EMAIL me your spreadsheets with your raw data.

Everyone gets a different data point, which means a particular # OF EXAMPLES, and # OF TRAINING REPETITIONS.   The two values assigned to you can be found in the table below the spreadsheet.   You will need to edit the source code file "Backprop.java" and set the integer constants RANDOM_SEED and NUM_TRAINING_REPETITIONS in lines 13 and 16 (respectively), see figure below.   You will also need to set up the four data files:  trainPOS.txt. trainNEG.txt, testPOS.txt, testNEG.txt, with the number of examples you are assigned.   You have a choice:  (1)  get the files from the folder "Set_of_6000_examples".  The four files in there have 6000 examples earh, so you can delete any that you do not need.   In the interest of making these experiments as controlled as possible, please take your examples from the TOP of each file.   For example, if you have been assigned 500 as the number of examples, then take the first 500 examples from each of the four files, deleting the last 5500 examples from each.  (2)  If you'd like, you can generate the four example files yourself using the program Generator_palindromes.java.   Just edit line 8 of the file to change the value assigned to the int constant NUM_EXAMPLES to the number of examples you actually want.  Then compile and run.   The program will create/overwrite the four input files with NUM_EXAMPLES examples.   Since it is seeded with the same random seed (don't change it!) that was used in generating the examples in "Set_of_6000_examples" we will all be running our NNs on the same examples.  

NAME:			Random # seed	RAW DATA				COMPOUNDED DATA
# EXAMPLES:		in pos. train. set						TRAINING	TEST	POSITIVE	NEGATIVE	ALL
# train. rep.s:				TRAINING SET		TEST SET		SET	SET	EXAMPLES	EXAMPLES	EXAMPLES
# input units:	40			POS	NEG	POS	NEG	OVERALL	OVERALL	OVERALL	OVERALL	OVERALL
# hidden units:	10		102	0%	0%	0%	0%	0%	0%	0%	0%	0%
learning rate:	0.25		112	0%	0%	0%	0%	0%	0%	0%	0%	0%
			122	0%	0%	0%	0%	0%	0%	0%	0%	0%
CONCEPT:	binary palindromes		132	0%	0%	0%	0%	0%	0%	0%	0%	0%
			142	0%	0%	0%	0%	0%	0%	0%	0%	0%
			152	0%	0%	0%	0%	0%	0%	0%	0%	0%
			162	0%	0%	0%	0%	0%	0%	0%	0%	0%
			172	0%	0%	0%	0%	0%	0%	0%	0%	0%
			182	0%	0%	0%	0%	0%	0%	0%	0%	0%
			192	0%	0%	0%	0%	0%	0%	0%	0%	0%
			AVERAGES OVER ALL RANDOM SEED RUNS	0%	0%	0%	0%	0%	0%	0%	0%	0%

ISSUES:

1. Random Numbers:

   • For the sake of repeatability (i.e., to be able to reproduce our results) with a non-deterministic (i.e., randomized) algorithm, we added a RANDOM SEED.  But is this truly working?

   • Here is a test program.   If run without any changes (i.e., random seed of 122)  this is what I see in MY console window (running Java 8 build 40 runtime environment):

   •