Unleashing the Potential of Deep Learning Methods for Detecting Defective Expressions Using Hyperparameter Optimization Techniques

Abstract

Natural Language Processing (NLP) has emerged remarkable progress in the field of deep learning studies. Not only a superior alternative to rule-based NLP methods, deep learning-based techniques have also succeeded more accurate performances in various NLP tasks such as text classification, sentiment analysis or document clustering. Since the performance of a deep learning model undoubtedly depends on adjusting its hyperparameters ideally, tuning the most optimum hyperparameters determines the capability of the model learning in terms of meaningful pattern extraction from the input data. In this paper, hyperparameter optimization techniques of Bayesian Optimization, Random Search and Grid Search have been applied on the deep learning models of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) for the purpose of detecting defective expressions in Turkish sentences. The hyperparameters of previously implemented LSTM and CNN models for this purpose have been adjusted using trial-and-error approach, which is time-consuming and cannot guarantee the most ideal model in general. After these hyperparameters have been adjusted using optimization techniques, the performances in terms of accuracy have been increased from 87.94% to 92.82% and from 84.33% to 89.79% for the models of LSTM and CNN respectively.