I've implemented first machine which recognizes digits. I have used U.S. Post Database of digits to train the machine, and I have also used their database to test the machine. For each row, first number represents the digit and the next 256 numbers represent the input matrix (16x16).
First impression is that ActionScript is too weak for numeric computation. In the implemented machine I've got 256 input neurons, 20 hidden neurons on a single layer and of course 10 output neurons. Of course I've tried to increase the number of neurons on hidden layer but this seemed to be unreliable to actionscript 'cause AIR application just blocked after few seconds of computation. All after all, with the previous configuration I have succeded to train the machine to recognize ~1800 of 2007 digits from tests (the best case was 1815, but in general, over 1800) after a few trainings (generally after the second train). See the example here. It would take more than 10 seconds to train the machine but test would run great (works on my machine).
I must admit that the computation in getting results (even in many tests like those from this file) is done fast enough in ActionScript so I am sure that a component developed in Flex would be capable to run in normal conditions if it had the weight matrix computed previously. I am afraid that a learning machine would be ineffcient if it has been implemented in ActionScript so this is the reason why I think matrix of weight computation in backpropagation algorithm should be done in another layer using another programming language (e.g. Java and OpenAMF).
For those who are sceptic, an example of application for digit recognition: a replacement for captcha texts (draw a digit and we're sure you're not a robot).
Here you have the source code for NeuralNetwork class. Suddenly, comments are lost when the file was uploaded on ftp :D (coming up)
Subscribe to:
Post Comments (Atom)
2 comments:
Very interesting results. I too think that a precomputed matrix would do well in a client application. But indeed, an application with active training would be too much for flash to handle. If only we could mutithread with AS3. Thanks for sharing your network and your findings!
Also, I read somewhere that instead of using array.length in your for loops, it is more efficient to assign the value to a local variable and use that in the for condition. The reason being that array.length results in a function call to get length(), which is quite an overhead for a for condition with many iterations. It may not solve the active learning performance issues, but it can't hurt the client performance. So instead of doing something like:
for( i = 0; i < matrix.length; i++)
We would instead do something like:
var n:uint = matrix.length;
for( i = 0; i < n; i++)
Thanks, Ben.
Ben,
Thank you for your feedback. You are right about array.length and I am sure that there are other optimizations that could be done in order to improve the application performance.
Even though an AS3 cannot handle with a learning machine algorithm with training support because of its limits, you can assign this task to a server, and the application could be written in almost any programming language (for instance JAVA is one of the most used, with JOONE package).
I am going to post a code written in Java for OCR support as soon as possible.
Post a Comment