Getting Started: 1. Root your galaxy nexus 2. Put files in mod_tiny into /system/bin directory in your galaxy nexus 3. Run this app ===== Update Log =============== Nov. 9: 1.Touch screen can sometimes be regarded as a keystroke by this program. Nov. 11: Basic function is established. Require 5 inputs of each key in training stage.� Current code is to detecting key A B C D if you want to train and test all keys, remove comment on line 36 in MainActivity.java AtoZ//, NUM, LEFT, RIGtextHT, BOTTOM and change the private final int[] ExpectedInputNum = { 4, 12, 4, 11, 6 }; to private final int[] ExpectedInputNum = { 26, 12, 4, 11, 6 }; Todo: 1. Add clear/restart button (2 buttons) on training activity to clear training for current key/restart training, in case input error in training stage 2. Enhance keystroke detection algorithm a. Move energy window b. Compare number of sample points that are greater than threshold in two window c. Threshold + Energy.... If find a sample point that is greater than threshold, then calculate energy of next several samples and energy of previous several samples. Compare the two energy 3. Online Training--Train the FFN when testing Every input in testing, if not corrected by user, use this input to train the KNN Replacement Policy:FIFO 4. Self Correction pseudo code: clickTimes = 0; onClickBackSpace(){ clickTimes ++; } //when get new key input if(clickTimes == 1) { newkey = KNN.nearest(current input feature); if( newkey != previousKey) { show newkey; // we think it is users input error clickTimes = 0; //clear the click time }else{ newkey = KNN.second_nearest(current input feature); KNN.removeNearestPoint(); show newkey; } }else if(clickTimes == 2) // { let the user input the correct corresponding key through screen; clickTimes = 0; } Nov 13: Problems: 1.click backspace on screen can possibly be regarded as keystroke 2. 1. Implement online correction policy, See function: private void dealwithBackSpace(double[] features) and OnclickButtonBackSpace 2. Halt the key recognition when user input 3. Add clear and restart button on training screen 4. Add online training: the policy is simple: if(input is after a backSpace) we don't train this key; else use our prediction to train that key; if(preinput = recognize(user input) then user click backSpace then recognize(new input) == preinput then user clickbackSpace) remove the latest training; Also I modified a little in BasicKNN.java to implement the policies 5. Modify BasicKNN.java a.Modified the key training function in KNN Add a replacement policy: when training set is full, remove the most far node from new node, and insert new node b. add 1 value: trainingSize---indicate the training set size for each key c. add 3 functons: setTrainingSize(int num), getTrainingSize and removeLatestItem() Nov 15th 1. Try to block audio processing immediately after user click backspace on screen. But failed. 2. disable screen sleep 3. Add a textview to show what is in the mKNN 4. Set the RecBuffer size to 10000. Dec 4th 1. dictionary and online adaptation of training set is disabled 2. FREQ_FEATURE_SZ=300 This parameter is very important. I found with FFT size equal to 4096, only the first 150 samples on each microphone are consistent for each key. The remaining are fairly random. 3. STROKE_CHUNKSIZE = 2000 4. Training set size and initial training num set to 10 5. Hanning window disabled 6. FFT size set to 4096, with zero patching (since keystroke duration is shorter than fft size) 7. smooth the curve before detecting 8. euclidian distance instead of standard Euclidian