AttentionLipi: A Hybrid CNN–BiLSTM (CRNN) Framework with CTC for Kannada Palm Leaf Manuscript Recognition

Authors

  • Mahaveer Department of Studies in Computer Science, Davangere University, Davangere, Karnataka, India
  • Basavanna Mahadevappa Department of Studies in Computer Science, Davangere University, Davangere, Karnataka, India
  • D Monika Sharma Department of Electronics and Communication Engineering, B.M.S. College of Engineering, Bengaluru, Karnataka, India

DOI:

https://doi.org/10.14738/tmlai.1401.19862

Keywords:

Kannada Character Recognition, Manuscript Digitization, CRNN, OCR, Historical Document Analysis, Deep Learning, Palm Leaf Manuscripts, Low Resource Languages, CTC Decoding, Heritage Preservation

Abstract

Character recognition in historical Kannada palm leaf manuscripts presents significant challenges due to degraded document quality, non-uniform character spacing, and the absence of publicly available annotated datasets. In this paper, we present AttentionLipi, an end-to-end Convolutional Recurrent Neural Network (CRNN) architecture combined with Connectionist Temporal Classification (CTC) loss for recognizing Kannada characters from palm leaf manuscripts without explicit character segmentation. The CRNN architecture consists of seven convolutional layers (64–512 channels) with batch normalization and ReLU activation for hierarchical feature extraction, followed by two bidirectional LSTM layers with 256 hidden units each for temporal sequence modeling, and CTC decoding for transcription. Trained on a custom dataset of 3,500 Kannada character samples including vowels, consonants, and compound characters manually extracted from 25 historical palm leaf manuscripts through a 280-hour annotation process, the model achieves 72.6% character recognition accuracy despite severe data constraints and document degradation. The results demonstrate the feasibility of applying deep learning to low-resource manuscript digitization tasks and provide a baseline for scalable OCR systems for Kannada heritage archiving.

References

Houben, E. M. and Saraju Rath. “Introduction: Manuscript Culture and Its Impact in ‘India’: Contours and Parameters.” in Aspects of Manuscript Culture in South India, edited by Saraju Rath, 1-53. Boston: Brill, 2012.

Kumar, D. Udaya, G. V. Sreekumar, and U. A. Athvankar. "Traditional Writing System in Southern India - Palm Leaf Manuscripts." Design Thoughts (July, 2009): 2-7.

Freeman, Rachel. "Turning Over Old Leaves: Palm Leaves used in South Asian Manuscripts." The Book and Paper Group Annual 24, (2005): 99-102 .

Li, M., Lv, T., Chen, J., Cui, L., Lu, Y., Florencio, D., Zhang, C., Li, Z., & Wei, F. (2021). TrOCR: Transformer-Based Optical Character Recognition with Pre-Trained Models. arXiv preprint arXiv:2109.10282.

Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006). Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks. In Proceedings of the 23rd International Conference on Machine Learning (pp. 369–376).

Shi, B., Bai, X., & Yao, C. (2017). An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(11), 2298–2304.

S. Eom, E. Yoon, H. S. Yoon, C. Kim, M. Hasegawa-Johnson, and C. D. Yoo, “AdaMER-CTC: Connectionist Temporal Classification with Adaptive Maximum Entropy Regularization for Automatic Speech Recognition,” in *Proc. 2024 IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP)*, Seoul, Korea, Apr. 2024, pp. 12707–12711. Scholar Publishing

Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber,“Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,”in Proceedings of the 23rd International Conference on Machine Learning, New York, NY, USA, 2006, ICML ’06, p. 369–376, Association for Computing Machinery.

Kalchbrenner, N.; Grefenstette, E.; Blunsom, P. A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Baltimore, MD, USA, 24–27 pp. 1003–1006.

Rahman, Z.; Jobson, D.J.; Woodell, G.A. Multi-scale retinex for color image enhancement. In Proceedings of the International Conference on Image Processing, Lausanne, Switzerland, 19 September 1996; IEEE: Piscataway, NJ, USA, 1996; Volume 3, pp. 1003–1006.

Shi, B.; Bai, X.; Yao, C. Detecting Oriented Text in Natural Images by Linking Segments. In Proceedings of the Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 2550–2559.

N. Zhang and J. Kim, “A Survey on Attention mechanism in NLP,” in 2023 International Conference on Electronics, Information, and Communication (ICEIC), 2023, pp. 1–4, IEEE. Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Springer International Publishing: Cham, Switzerland, 2016; pp. 21–37.

Zhou, X.; Yao, C.; Wen, H.; Wang, Y.; Zhou, S.; He, W. EAST: An efficient and accurate scene text detector. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2017, Honolulu, HI, USA, 21–26 July 2017; pp. 5551–5560. Gildardo Mendoza Achee, Michelle Wilber, Mohammad Heidari Kapourchali, Masoumeh Heidari Kapourchali, "Day-Ahead Residential Electric Load Forecasting Using Bi-Directional LSTM Recurrent Neural Networks", 2025 IEEE Electrical Power and Energy Conference (EPEC), pp.354-358, 2025

Kalchbrenner, N.; Grefenstette, E.; Blunsom, P. A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Baltimore, MD, USA, 24–27 June 2014; pp. 655–665.

Liu, Y.; Chen, H.; Shen, C.; He, T.; Jin, L.; Wang, L. Real-time scene text detection with differentiable binarization. In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–12 February 2020; Volume 34, pp. 11474–11481. T.-G. Marchitan, C. Creanga, and L. P. Dinu, “Transformer and Hybrid Deep Learning Based Models for Machine-Generated Text Detection,” arXiv:2405.17964 [cs.CL], May 2024.

J. Puigcerver, “Are Multidimensional Recurrent Layers Really Necessary for Handwritten Text Recognition?” in Proc. 14th IAPR Int. Conf. on Document Analysis and Recognition (ICDAR), 2017, pp. 67–72.

A. D et al., "Image Text Detection and Documentation Using OCR," 2024 International Conference on Smart Systems for Electrical, Electronics, Communication and Computer Engineering (ICSSEECC), Coimbatore, India, 2024, pp. 410-414, doi: 10.1109/ICSSEECC61126.2024.10649443.

R. Avros and Z. Volkovich,“Detection of Computer-Generated Papers Using One-Class SVM and Cluster Approaches,” in Proc. Machine Learning and Data Mining in Pattern Recognition (MLDM 2018), New York, NY, USA, Jul. 2018, pp. 42–55. Springer, Lecture Notes in Computer Science, vol. 10935.

K. M. Abuali, L. Nissirat and A. Al-Samawi, “Advancing Network Security with AI: SVM-Based Deep Learning for Intrusion Detection,” Sensors, vol. 23, no. 21, p. 8959, Nov. 2023, doi:10.3390/s23218959. Scholar Publishing

T. M. Breuel, A. Ul-Hasan, M. A. Al-Azawi, and F. Shafait, “High-Performance OCR for Printed English and Fraktur Using LSTM Networks,” in Proc. Int. Conf. on Document Analysis and Recognition (ICDAR), 2013.

K. Vani and D. Gupta, "Using K-means cluster based techniques in external plagiarism detection," 2014 International Conference on Contemporary Computing and Informatics (IC3I), Mysore, India, 2014, pp. 1268-1273, doi: 10.1109/IC3I.2014.7019659.

J. Sauvola and M. Pietikäinen, “Adaptive Document Image Binarization,” Pattern Recognition,

Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998, doi: 10.1109/5.726791.

Zou, L., et al. (2023). Text Recognition Model Based on Multi-Scale Fusion CRNN. PMC, National Center for Biotechnology Information.

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 10012–10022).

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale. arXiv preprint arXiv:2010.11929.

N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2005, pp. 886–893.

C. Cortes and V. Vapnik, “Support-Vector Networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995.

R. Smith, “An Overview of the Tesseract OCR Engine,” in Proc. Int. Conf. on Document Analysis and Recognition (ICDAR), 2007.

A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber, “A Novel Connectionist System for Unconstrained Handwriting Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009.

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778).

Hochreiter S & Schmidhuber J. Long Short-Term Memory. Neural Computation, 9(8), 1735–1780.

J. Sauvola, T. Seppanen, S. Haapakoski and M. Pietikainen, "Adaptive document binarization," Proceedings of the Fourth International Conference on Document Analysis and Recognition, Ulm, Germany, 1997, pp. 147-152 vol.1, doi: 10.1109/ICDAR.1997.619831.

Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv preprint arXiv:1409.0473.

Cui M., Wang W., Zhang J., Wang L. Representation and correlation enhanced encoderdecoder framework for scene text recognition; Proceedings of the Document Analysis and Recognition–ICDAR 2021: 16th International Conference; Lausanne, Switzerland. 5–10 September 2021; pp. 156–170. Scholar Publishing

Chollampatt S., Ng H.T. A multilayer convolutional encoder-decoder neural network for grammatical error correction. Proc. AAAI Conf. Artif. Intell. 2018;32:1220–1230. doi: 10.1609/aaai.v32i1.12069.

Zheng T., Chen Z., Fang S., Xie H., Jiang Y. Cdistnet: Perceiving multi-domain character distance for robust text recognition. arXiv. 20212111.11011.

A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber,“A Novel Connectionist System for Unconstrained Handwriting Recognition,”IEEE Transactions on Pattern Analysis and Machine Intelligence,vol. 31, no. 5, pp. 855–868, 2009.

V. I. Levenshtein,“Binary Codes Capable of Correcting Deletions, Insertions, and Reversals,”Soviet Physics Doklady, vol. 10, no. 8, pp. 707–710, 1966.

Downloads

Published

2026-01-18

How to Cite

Mahaveer, Mahadevappa, B., & Monika Sharma, D. (2026). AttentionLipi: A Hybrid CNN–BiLSTM (CRNN) Framework with CTC for Kannada Palm Leaf Manuscript Recognition. Transactions on Engineering and Computing Sciences, 14(01), 32–46. https://doi.org/10.14738/tmlai.1401.19862