Deep Learning based Semantic Similarity Detection using Text Data

Authors

  • Muhammad Mansoor
  • Zahoor ur Rehman
  • Muhammad Shaheen
  • Muhammad Attique Khan Lecturer, HITEC University, Taxila
  • Mohamed Habib College of Computing and Informatics, Saudi Electronic University, Riyadh, Saudi Arabia Faculty of Engineering, Port Said University, Port Fuad City, Egypt

DOI:

https://doi.org/10.5755/j01.itc.49.4.27118

Keywords:

Deep Learning, Semantics, Similarity, Quora, question duplication, LSTM and CNN.

Abstract

Similarity detection in the text is the main task for a number of Natural Language Processing (NLP) applications. As textual data is comparatively large in quantity and huge in volume than the numeric data, therefore measuring textual similarity is one of the important problems. Most of the similarity detection algorithms are based upon word to word matching, sentence/paragraph matching, and matching of the whole document. In this research, a novel approach is proposed using deep learning models, combining Long Short Term Memory network (LSTM) with Convolutional Neural Network (CNN) for measuring semantics similarity between two questions. The proposed model takes sentence pairs as input to measure the similarity between them. The model is tested on publicly available Quora’s dataset. The model in comparison to the existing techniques gave 87.50 % accuracy which is better than the previous approaches.

Downloads

Published

2020-12-19

Issue

Section

Articles