LPG: Locality-preserving graph convolutional network for aerial image

Introduction

Scene graph generation (SGG) aims to understand the visual objects and their semantic relationships from one given image. Until now, lots of SGG datasets with the eyelevel view are released but the SGG dataset with the overhead view is scarcely studied. By contrast to the multiple objects occlusion problem in the eyelevel view, the overhead view helps to clearly perceive the spatial relationships of objects in the ground scene. To fill in the gap of the overhead view dataset, this paper constructs and releases an urban aerial image scene graph generation dataset (UASG). Images from UASG are captured with the low-attitude overhead view. In UASG, 25,594 objects, 16,970 relationships and 27,175 attributes are manually annotated. To address aerial image scene graph generation, this paper proposes one new locality-preserving graph convolutional network (LPG). Different from the traditional graph convolutional network, the convolutional layer in LPG embeds the non-destructive initial features of the objects with the scene information for relationship prediction. To address the problem that there exists an extra-large number of potential object relationship pairs but only a small part of them is meaningful, we propose the adaptive bounding box scaling factor to intelligently prune the meaningless relationship pairs. Extensive experiments on UASG show that our LPG can significantly outperform the state-of-the-art methods and the effectiveness of the proposed locality-preserving strategy.