Introducing UniView – A unipen file reader || And Approaching Recognition Problem

My project is Thamizh handwriting recognition Android IME. My project involves pattern recognition. Any pattern recognition problem essentially involves three steps;

  1. Preprocessing
  2. Feature extraction
  3. Recognition

My project involves Android development, one thing that makes my development speed to stammer is, configuring my development environment and establishing a workflow (inception -> algo development -> testing). I find this hard in Android. Usually Android emulator takes ages to start and I do not have an Android device with me.

So to simulate my project I used Processing libraries. Processing is a development platform, for non-technical programming (research scholars use it too!) enthusiasts. It was developed to teach essential programming in a visual context. Artist use it too! It has Arduino library as well.

I use Processing, since my project involves a little bit of Computer Graphics (like Affine Transform and Size Normalization).

UniView is a small Unipen file reader, developed using Processing. I have made it opensource in my Github page – https://github.com/aravindc26/UniView

The user inteface is not snazzy, still needs loads of improvement, I believe this project is unique, since there is no java app for Unipen file viewing. Hope it would be of some use to someone !

May be in my next post I will explain how recognition works, using artificial neural networks.

Tamil Handwriting Recognition Android App

I have planed to make Tamil Handwriting Recognition app. There is a strong reason for choosing this title; I am a founder member of a NGO called Maatram Thaedi, one of our mission is to motivate engineering students to make socially purposeful projects. India has a strong base of talented coders who unfortunately go behind outsourced jobs instead of engaging in entrepreneurial activities targeting the market here.

My college is an Aakash tablet project centre. To brief you about it, Aakash tablet is a visionary project initiated by Indian HRD ministry, with an aim to provide cheap (RS. 2500) tablets. These tablets are produced by UK based UbiSlate. Though much hyped, in my opinion Aakash 1 was not a huge hit because it runs primitive Android Froyo. Ubislate did not manage to get into Android Market place, which makes it almost devoid of application.

Aakash 2 is going to spread in the market now it has promising features now, with Android 4.0 ( Ice cream sandwich), I guess Ubislate has taken feedback of Aakash 1 seriously.

Now coming back to my project, I have modified the android robot to motivate my team mates.Arndroid barathiyar

Apparently this picture looks like Mahakavi Barathiyar (Renowned Indian freedom fighter and a Tamil poet)

Indian languages are hard to “keyboardify” and already existing keyboards for Indian languages are not intuitive and requires rigorous training before achieving practical typing speed. I hope my project solves the issue for all Indic languages.

Corrected my colour tracker program

There are two problems in my previous version of colour tracker program,

  • The display freezes when I remove the object of interest from the view of camera. This issue was due to silly indentation of the codeĀ  :P , which inserted the code into if block.
  • “SyntaxWarning: name ‘posX’ is assigned to before global declaration
    global posX”; which I get for usage of global variable inside if block, nested inside a while block. It seems the usage of global keyword should not be contained in a nested block, this can be rectified by using global keyword in the parent block.

    #wrong usage of global keyword
    x = 0
    while True:
        #some code
        if some-condition:
            global x
            #some other code
    #correct usage of global keyword
    x = 0
    while True:
        global x
        #some code
        if some-condition:
            #use x here

Finally I have included the second revision of the colour tracker, which runs without any bugs.


from cv import * #bad practice, its just that am bit lazy

def GetThresholdedImage(img):
#returns thresholded image of the blue bottle
imgHSV = CreateImage(GetSize(img), 8, 3)
#converts a BGR image to HSV
CvtColor(img, imgHSV, CV_BGR2HSV)
imgThreshed = CreateImage(GetSize(img), 8, 1)
#InRangeS takes source, lowerbound color, upperbound color and destination
#It converts the pixel values lying within the range to 255 and stores it in
#the destination
InRangeS(imgHSV, (100, 94, 84), (109, 171, 143), imgThreshed)
return imgThreshed

posX = 0
posY = 0

def main():
color_tracker_window = "output"
thresh_window = "thresh"
capture = CaptureFromCAM(-1)
NamedWindow( color_tracker_window, 1 )
NamedWindow( thresh_window, 1 )
imgScrible = None
global posX
global posY
while True:

frame = QueryFrame(capture)
Smooth(frame, frame, CV_BLUR, 3)

if(imgScrible is None):
imgScrible = CreateImage(GetSize(frame), 8, 3)

imgBlueThresh = GetThresholdedImage(frame)

mat = GetMat(imgBlueThresh)
#Calculating the moments
moments = Moments(mat, 0)
area = GetCentralMoment(moments, 0, 0)
moment10 = GetSpatialMoment(moments, 1, 0)
moment01 = GetSpatialMoment(moments, 0,1)

#lastX and lastY stores the previous positions
lastX = posX
lastY = posY
#Finding a big enough blob
if(area > 100000):

#Calculating the coordinate postition of the centroid
posX = int(moment10 / area)
posY = int(moment01 / area)

print 'x: ' + str(posX) + ' y: ' + str(posY) + ' area: ' + str(area)
#drawing lines to track the movement of the blob
if(lastX > 0 and lastY > 0 and posX > 0 and posY > 0):
Line(imgScrible, (posX, posY), (lastX, lastY), Scalar(0, 255, 255), 5)
#Adds the three layers and stores it in the frame
#frame -> it has the camera stream
#imgScrible -> it has the line tracking the movement of the blob
Add(frame, imgScrible, frame)

ShowImage(thresh_window, imgBlueThresh)
ShowImage(color_tracker_window, frame)
c = WaitKey(10)
if(c!=-1):
break

return;
if __name__ == "__main__":
main()

Color Tracking opencv-python and my first post!

This is my first post and I decided to post about my recent obsession towards Computer Vision. I am still a noob working my way out. Most of the time I take my vision for granted, it amazes me how one needs to understand mathematical concepts to work with computer vision which is not as intuitive as that of my vision.

I have always skipped learning python and am not a great fan of script style programming languages. I worked with opencv (c++) and I found that I need to type in huge number of lines and I needed manage memory. The fact that computer vision is mind bending and writing housekeeping codes in c++ is like adding salt to the wound.

OpenCV comes with python interface(duh!). I gave python a try. I was cursing myself for not having learnt it. No need of memory management, no need of typecasting variables *bliss*.

I would recommend you to learn python from this book from Swaroop. It is neat and simple.

Now coming back to business, I have included the program for tracking a blue coloured object below. The program follows these steps for tracking the blue coloured object

  • Find the approximate HSV colour range of the object of interest using gimp or ms paint.
  • Block the pixels which are not falling in the range and shoot up the pixel values to 255 which are in the range, so that we get a binary image.
  • Calculate the centroid of the blob formed in the binary image using image moments.
  • Track the motion of the centroid and draw a line trailing its path.

I have used the logic from Aishack and there is detailed explanation there.

(Please make sure that the indentation is correct, in case you copy the program, gist is sometimes buggy :( )

[gist id="4379792" file="colortrack.py"]

<pre>from cv import *  #bad practice, its just that am bit lazy

def GetThresholdedImage(img):
#returns thresholded image of the blue bottle
	imgHSV = CreateImage(GetSize(img), 8, 3)
#converts a BGR image to HSV
	CvtColor(img, imgHSV, CV_BGR2HSV)
	imgThreshed = CreateImage(GetSize(img), 8, 1)
#InRangeS takes source, lowerbound color, upperbound color and destination
#It converts the pixel values lying within the range to 255 and stores it in
#the destination
    InRangeS(imgHSV, (100, 94, 84), (109, 171, 143), imgThreshed)
	return imgThreshed

def main():
	color_tracker_window = "output"
	thresh_window = "thresh"
	capture = CaptureFromCAM(-1)
	NamedWindow( color_tracker_window, 1 ) 
	NamedWindow( thresh_window, 1 ) 
	imgScrible = None
	posX = 0
	posY = 0
	while True:

		frame = QueryFrame(capture)
		Smooth(frame, frame, CV_BLUR, 3)

		if(imgScrible is None):
			imgScrible = CreateImage(GetSize(frame), 8, 3)

		imgBlueThresh = GetThresholdedImage(frame)

		mat = GetMat(imgBlueThresh)
#Calculating the moments
        moments = Moments(mat, 0) 
      	area = GetCentralMoment(moments, 0, 0)
		moment10 = GetSpatialMoment(moments, 1, 0)
		moment01 = GetSpatialMoment(moments, 0,1)

#lastX and lastY stores the previous positions
		lastX = posX
		lastY = posY
#Finding a big enough blob
		if(area > 100000): 
			global posX
			global posY
#Calculating the coordinate postition of the centroid
			posX = int(moment10 / area)
			posY = int(moment01 / area)

			print 'x: ' + str(posX) + ' y: ' + str(posY) + ' area: ' + str(area)
#drawing lines to track the movement of the blob
			if(lastX > 0 and lastY > 0 and posX > 0 and posY > 0):
				Line(imgScrible, (posX, posY), (lastX, lastY), Scalar(0, 255, 255), 5)
#Adds the three layers and stores it in the frame
#frame -> it has the camera stream
#imgScrible -> it has the line tracking the movement of the blob
			Add(frame, imgScrible, frame)

            ShowImage(thresh_window, imgBlueThresh)
			ShowImage(color_tracker_window, frame)
			c = WaitKey(10)
			if(c!=-1):
				break

	return;
if __name__ == "__main__":
	main()

There are a few bugs in the program, the execution freezes once the object of interest is out of the frame andĀ  some warnings due to sloppy handling of global variables. ( Need to get my python right, learning! The problem is that there is no static variables in python, need to find a workaround)

Note that softwares like Gimp and MS paint uses Hue(H) value ranging from 0-360 degrees and 0-100% for Saturation(S) and Value(V). But OpenCV uses 0-180 degs for Hue and 0-255 for Saturation and Value. So you need convert to OpenCV’s HSV convention.

ocvH = (H/360)*180
ocvS = (S/100)*255
ocvV = (V/100)*255
Next post I will correct those bugs!
UPDATE: Corrected those bugs.