There are often residual images of the camera tripod in panoramic images, which may reduce the image quality and deteriorate the post-processing speed. To address this problem, a camera tripod removal network (TRNet) based on generative adversarial network is proposed. As an end-to-end model, the generator is designed to include recognition and reconstruction branches, which reduce the number of parameters and improve the training efficiency by sharing the encoder and correspond to scaffold recognition and texture reconstruction respectively. The recognition branch based on the U-Net structure can effectively identify the tripod area, while the reconstruction branch can brilliantly reconstruct the texture details through an intermediate layer formed by stacking dilated convolution residual blocks. Furthermore, spectral normalized Markov discriminator and multiple combined loss function are adopted to promote global texture consistency and thus result in a better texture filling effect. Finally, a data set of 400 panoramic images is constructed and experimental results on this data set demonstrate the better repair ability of TRNet against other state-of-the-art methods.