A Novel Approach for Annotation-based Image Retrieval Using Deep Architecture
Guanwen Zhang, Jien Kato, Yu Wang and Kenji Mase
Due to the increasing of the digital images, it imposes a great challenge on how to effectively manage image data. Image retrieval system, searching and finding images according to specific requirement, is naturally emerging. In this paper, we present a novel approach for annotation-based image retrieval. The two recent advances of deep architectures in the computer vision and natural language processing are utilized to try to bridge the semantic gap of the image contents and human perception. Given the test images, different regions and scales of image are densely sampled as the candidates. The two strategies, saliency reduction and non-maximum suppression integration, are performed to remove the redundancy and retain the convincing candidates. We employ the convolutional neural network for candidate annotations. Based on the word embedding, a deep language architecture is trained to calculate the relatedness of the input query text and annotations. We also formulate a transition matrix to measure relative importance and transfer the references among the annotations. The experimental results on a nursery school dataset demonstrate effectiveness of the proposed approach.
Keywords: Annotation-based image retrieval, convolutional neural network, Word2Vec, saliency computation, non-maximum suppression integration, region proposals, transition matrix.