Real-time optical character recognition for advanced driver assistance systems using neural networks on FPGA

Submission note: A thesis submitted in total fulfilment of the requirements for the degree of Doctor of Philosophy to the School of Engineering and Mathematical Sciences, College of Science, Health and Engineering, La Trobe University, Victoria.

Scientists and engineers are continuously inventing and improving artificial intelligence (AI) to assist humans in many daily tasks. One of the most common technologies in automotive applications is the vision system that provides guidance to drivers to mitigate their lack of focus on road sign information. There is much vital information of which drivers need to be aware when negotiating through traffic. Drivers need to obey all road regulatory signs and pay attention to advisory signs. This presents a requirement for a vision system that can interpret text-based road signs and advise the drivers in a timely manner. The focus of this thesis is to develop a novel methodology to perform real-time Optical Character Recognition (OCR) on road signs as viewed by a forward facing camera to deliver necessary traffic sign information to drivers. Fundamentally, an field-programmable gate array (FPGA) with its concurrent architecture is an ideal platform to implement the system on. A feature extraction method using Haar-like features with an artificial neural network as classification has been developed to detect block letters on road signs. This neural network was trained using a comprehensive synthetic positive database formulated from a model letter set subjected to geometric mutations. Furthermore, a collection of optimization algorithms is proposed, implemented and demonstrated to facilitate the direct implementation of such a neural network on an FPGA platform. Finally, a fully working system is implemented and tested on an FPGA confirming full frame rate performance using live streaming video data.