Mezzofanti – Augmented reality through text-recognition.
(Android phone – application)
What is Mezzofanti: it is an application for the Android Dev1 phone, that allows reality augmentation (as seen in camera mode) with information about any text in view. The augmentation is done through text-recognition.
After the augmentation, the digital text (or parts of it) may be used in various modes:
- it may be translated in one of the 40 available languages
- it may be searched on Google or Wiki
- it can be checked against a dictionary
Technologies:
- the OCR engine is a modified version of Tesseract 2.03, currently developed by Google under Apache License, ver 2.0. – there are 5 available languages available: English, German, Spanish, Italian and French. Never the less the OCR will work with any other language that uses the Latin alphabet (the results will be a bit less accurate).
- the interface between the OCR engine (C++) and Java is done in JNI
- the main core is programmed in Java
- the translation engine is based on Google-translate
Licenses:
- all code is available for free under the Apache License, ver 2.0 and it is posted at google code. Read the install tutorial.
Dialogue:
- we encourage the dialogue, so please do feedback us. (feedback may include, but it is not limited to: observations and questions about code, wish-list of new technical tutorials, contributions to the project, collaboration opportunities etc.)
- a brief documentation of the Mezzofanti user interface and the System architecture.
Who was the real Mezzofanti:
Giuseppe Caspar Mezzofanti (19 September 1774 – 15 March 1849) was an Italian cardinal and famed linguist and hyperpolyglot. Mezzofanti was well-known for being a hyperpolyglot who fluently spoke thirty-eight languages and forty dialects, despite never traveling outside of Italy. (more about the cardinal on Wikipedia)
A funny video with the usage of Mezzofanti:
Hey guys… great work. Thanks a lot. Was looking for a way to use the tesseract engine on android.
I can’t seem to take a picture with my G2 (Mytouch 3g). I tried touching everywhere on my screen, volume buttons, home, menu, back, search, trackball, even start / end call buttons. Nothing.
Is this a G1 only app? or did I miss something?
I like the idea. It’s one of the few apps I didn’t skip on the adc2 app.
Dear user,
we are very happy you like Mezzofanti.
If there is no camera button on the screen than the application you hold is for G1 only. Never the less there is one app for G2 (and other phones without the photo button) – please download the app from http://code.google.com/p/mezzofanti/ (version 1_0_3 featured downloads).
Thank you and looking forward for your feedback.
Enjoy using Mezzofanti
Best regards,
Vlad
Hi
Great work… thanks so much for the application and tutorials… all very helpful and well done.
The application did not run on my G1 with 16 MB flash when I first attempted to run it. I saw an error message “not enough space for installation on external storage” with a negative value for total external storage capacity.
To fix this I modified AssetsManager.GetFreeSpaceB() to cast stat.getAvaialableBocks() and stat.getBlockSize() to longs before multiplying. After this change everything worked on my setup.
Thanks again for the helpful application, source code and tutorials.
Hi, first of all great work and thank you very much for sharing the code with everyone!
I would like to reuse your code into my application. My task is a little bit simpler as I want to recognize short alpha numeric strings like HB-134 directly form camera preview.
I’ve alreay set up my android app so that it uses the accelerometer sensors to detect when you move the camera and when you stop moving. Once you stop moving, it triggers autofocus and on autofocus completion callback I then capture the preview image to be sent for OCR. Looking at android documentation and on some forums, the image format sent to preview callback is YUV. The good thing is that the frist width * height bytes of the array received by the callback represent the grey scale version of the image. SO here my first question came:
1) Can I use pass those width * height bytes to OCR.ImgOCRAndFilter(byte[] bw_img, int width, int height, int bpp) for successfull text recognition? (in that case, bpp is 1 right?)
2) Although I’m not trying to recognize a real word but only a generic alpha-numeric string, I guess I don’t really need a dictionary file, right?
3) Do I need to train Tesseract2 as stated here? http://code.google.com/p/tesseract-ocr/wiki/TrainingTesseract2
4) Can you show me where within Mezzofanti code you do that training? Can I reuse the language pack from Mezzofanti?
Thank you very much in advance for your support.
Cheers, Luca.
hi this great job..
I want make something like this for my colegge task project, can I know what algorithm for this OCR (back propagation, thinning or what ) ?
and algorithm for translator too ?
I got error while downloading a language through an android phone.
How to fix it? pls do help.
Hi
I was able to install and use Mezzofanti easily from source without problems. The only trouble I am having is I can not seem to get any words to be recognized by OCR ever. I have tried to adjust the various settings and using different NA English text from various print sources/lighting conditions. Still no luck. The results are always garbled text. Any ideas please?
Thanks.
Ken
@dwalkes: You’re a freakin genius! Thanks for saving me an hour of debugging
hi..
mezzofanti’s a great app..
i tried to make an application seems like this..
but, i had a difficulty when i try to make form (layout) to capture the text..
can u help me or share the tutorial to me.?
thanks before..:)
I have same request …. please I you could help me
thanks and regards