| Citation: | LI Haiyan, HUANG Hefu, GUO Lei, et al. Image inpainting method based on incomplete image samples in generative adversarial network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2021, 47(10): 1949-1958. doi: 10.13700/j.bh.1001-5965.2020.0374(in Chinese)
|
A model of Double Generator Deep Convolutional Generative Adversarial Network (DGDCGAN), which uses the incomplete or noisy sample image as the training set, is proposed, in order to solve the problem of serious distortion of large area image inpainting, complete and high-quality training samples are frequently required, which is hard to acquire. Furthermore, the convergence of single generator is slow. Therefore, two generators and a discriminator are constructed. The incomplete image training set is used to cross calculate and search the image information similar to the loss area as the sample of training generation model, which achieves faster convergence speed. The loss function of the discriminator is improved to be the Wasserstein distance of the output. The adaptive estimation algorithm is used to optimize the model parameters for generating network loss function and identifying network loss function. Finally, the distance difference between two sets of images is calculated, and the reconstructed image is optimized by discriminating model and minimizing mean square error of the total distance change of a group of repaired images. Experiments are performed on four public dataset, the subjective and objective experimental results show that the proposed method that uses incomplete samples as training data can restore large area of distortion in images with faster convergence speed and better performance compared with the existing methods in image inpainting.
| [1] |
BERTALMÍO M, BERTOZZI A L, SAPIRO G. Navier-Stokes, fluid dynamics, and image and video inpainting[C]//2001 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2001: 7176886.
|
| [2] |
LIU D, SUN X, WU F, et al. Image compression with edge-based inpainting[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2007, 17(10): 1273-1287. doi: 10.1109/TCSVT.2007.903663
|
| [3] |
BUYSSENS P, DAISY M, TSCHUMPERLE D, et al. Exemplar-based inpainting: Technical review and new heuristics for better geometric reconstructions[J]. IEEE Transactions on Image Processing, 2015, 24(6): 1809-1824.
|
| [4] |
DARABI S, SHECHTMAN E, BARNES C, et al. Image melding: Combining inconsistent images using patch-based synthesis[J]. ACM Transactions on Graphics, 2012, 31(4): 82.
|
| [5] |
HUANG J B, KANG S B, AHUJA N, et al. Image completion using planar structure guidance[J]. ACM Transactions on Graphics, 2014, 33(4): 1-10.
|
| [6] |
BARNES C, SHECHTMAN E, FINKELSTEIN A, et al. PatchMatch: A randomized correspondence algorithm for structural image editing[J]. ACM Transactions on Graphics, 2009, 28(3): 24.
|
| [7] |
WEXLER Y, SHECHTMAN E, IRANI M. Space-time completion of video[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(3): 463-476. doi: 10.1109/TPAMI.2007.60
|
| [8] |
HAYS J, EFROS A A. Scene completion using millions of photographs[J]. ACM Transactions on Graphics, 2007, 26(3): 4. doi: 10.1145/1276377.1276382
|
| [9] |
DEMIR U, UNAL G. Patch-based image inpainting with generative adversarial networks[EB/OL]. (2018-03-20)[2020-07-25].
|
| [10] |
NAZERI K, NG E, JOSEPH T, et al. EdgeConnect: Generative image inpainting with adversarial edge learning[EB/OL]. (2019-01-11)[2020-07-25].
|
| [11] |
XIE J, XU L, CHEN E. Image denoising and inpainting with deep neural networks[C]//International Conference on Neural Information Processing Systems, 2012, 1: 341-349.
|
| [12] |
YANG C, LU X, LIN Z, et al. High-resolution image inpainting using multi-scale neural patch synthesis[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 17355590.
|
| [13] |
IIZUKA S, SIMO-SERRA E, ISHIKAWA H. Globally and locally consistent image completion[J]. ACM Transactions on Graphics, 2017, 36(4): 1-14.
|
| [14] |
LI Y, LIU S, YANG J, et al. Generative face completion[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 17355119.
|
| [15] |
PATHAK D, KRAHENBUHL P, DONAHUE J, et al. Context encoders: Feature learning by inpainting[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 16526723.
|
| [16] |
GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2014: 2672-2680.
|
| [17] |
BROCK A, DONAHUE J, SIMONYAN K. Large scale GAN training for high fidelity natural image synthesis[EB/OL]. (2018-09-28)[2020-07-25].
|
| [18] |
AHMADI M, NEST T, ABDELNAIM M, et al. Reproducing AmbientGAN: Generative models from lossy measurements[EB/OL]. (2018-10-23)[2020-07-25].
|
| [19] |
SALIMANS T, GOODFELLOW I, ZAREMBA W, et al. Improved techniques for training GANs[C]//International Conference on Neural Information Processing Systems, 2016: 2234-2242.
|
| [20] |
ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 17453078.
|
| [21] |
ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[EB/OL]. (2017-01-26)[2020-07-25].
|
| [22] |
ARJOVSKY M, BOTTOU L. Towards principled methods for training generative adversarial networks[EB/OL]. (2017-01-17)[2020-07-25].
|
| [23] |
CHEN Y, HU H. An improved method for semantic image inpainting with GANs: Progressive inpainting[J]. Neural Processing Letters, 2019, 49(3): 1355-1367. doi: 10.1007/s11063-018-9877-6
|
| [24] |
LIU Z, LUO P, WANG X, et al. Deep learning face attributes in the wild[C]//2015 IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2015: 3730-3738.
|
| [25] |
LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324. doi: 10.1109/5.726791
|
| [26] |
李雪瑾, 李昕, 徐艳杰. 基于生成对抗网络的数字图像修复技术[J]. 电子测量与仪器学报, 2019, 33(1): 40-46.
LI X J, LI X, XU Y J. Digital image restoration technology based on generative adversarial networks[J]. Journal of Electronic Measurement and Instrumentation, 2019, 33(1): 40-46(in Chinese).
|
| [27] |
李天成, 何嘉. 一种基于生成对抗网络的图像修复算法[J]. 计算机应用与软件, 2019, 36(12): 195-200.
LI T C, HE J. An image inpainting algorithm based on generative adversarial networks[J]. Computer Applications and Software, 2019, 36(12): 195-200(in Chinese).
|
| [1] | ZHANG K，LIU Y，HU K. RAW image reconstruction based on multi-scale attention mechanism[J]. Journal of Beijing University of Aeronautics and Astronautics，2025，51（1）：257-264 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0959. |
| [2] | DAI Xuan, MA Yunxiang, LI Chao, WANG Genye, DAI Huisheng, LIU Lei. Anti-erosionperformance tests of cement stabilized macadam base considering apparent erosion and image evolution[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0844 |
| [3] | LIU C J，QIAO Z，YAN H W，et al. Semantic segmentation network of remote sensing images based on dual path supervision[J]. Journal of Beijing University of Aeronautics and Astronautics，2025，51（3）：732-741 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2023.0155. |
| [4] | LU S Q，GUAN F X，LAI H T，et al. Two-stage underwater image enhancement method based on convolutional neural networks[J]. Journal of Beijing University of Aeronautics and Astronautics，2025，51（1）：321-332 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.1003. |
| [5] | SU Qian-xiao, QIAO De-xin, REN Yi-li, FENG Zhou, LIN Sheng-lan, HUANG Rui-qi. Inpainting of blank strips in imaging logging images based on Fourier convolution[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0754 |
| [6] | HUANG Mengdie, WANG Lufeng, HUANG Xuxing, LI Shuang. Space target collision risk analysis algorithm based on the square Mahalanobis distance[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0167 |
| [7] | CHAI G Q，BO X S，LIU H J，et al. Self-supervised scene depth estimation for monocular images based on uncertainty[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（12）：3780-3787 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0943. |
| [8] | HU Jianping, GAO Zhipeng, MOU Yang, XIE Qi. Deep learning-based image watermarking combining attack classification and multi-channel embedding[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0552 |
| [9] | FU Honglan, ZHANG Hao. Close robust rendezvous on distant retrograde orbits[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0560 |
| [10] | LI H G，WANG Y F，YANG L C. Meta-learning-based few-shot object detection for remote sensing images[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（8）：2503-2513 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0637. |
| [11] | SHI Jiliang, ZHANG Qian, ZHOU Zunfu, YANG Sihong. Face Image Inpainting Combining Semantic Segmentation and Edge Texture[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0258 |
| [12] | WU S Y，YANG D K，WANG F，et al. GNSS-R BSAR range-Doppler imaging algorithm based on synchronization of direct and echo signal[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（3）：588-596 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0310. |
| [13] | ZHANG Zhi, YI Hua-hui, ZHENG Jin. Few-Shot Object Detection of Aerial Image Based on Language Guidance Vision[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0491 |
| [14] | CHEN Y，CHEN J，TAO M F. Mural inpainting progressive generative adversarial networks based on structure guided[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（6）：1247-1259 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0440. |
| [15] | CHEN Y，CHEN J，TAO M F. Mural inpainting with generative adversarial networks based on multi-scale feature and attention fusion[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（2）：254-264 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0242. |
| [16] | ZHANG Yi-tian, LUO Xi-ling, WANG Yu-peng. Self-supervised image change detection method based on lightweight capsule network[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023-0251 |
| [17] | LI H Y，CHEN J，YU P F，et al. Bimodal text-guided image inpainting algorithm[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（10）：2547-2557 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0720. |
| [18] | LI Y H，ZHU M Y，REN J，et al. Text-to-image synthesis based on modified deep convolutional generative adversarial network[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（8）：1875-1883 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0588. |
| [19] | CHEN Guanhua, YANG Chihang, ZHANG Chen, ZHANG Hao. Distant retrograde orbits and its bifurcations in Earth-Moon system[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(12): 2576-2588. doi: 10.13700/j.bh.1001-5965.2020.0608 |
| [20] | HU Kai, ZHAO Jian, LIU Yu, NIU Yukai, JI Gang. Images inpainting via structure guidance[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(7): 1269-1277. doi: 10.13700/j.bh.1001-5965.2021.0004 |
| 1. | 濮毅. 基于感知去模糊的高分辨率破损图像修复方法. 常州工学院学报. 2024(02): 8-13 .  | |
| 2. | 代文征，余建国，唐建国. 深度生成模型下缺损图像修复方法仿真研究. 计算机仿真. 2024(08): 170-174 . | |
| 3. | 孙国娇，李长硕. 基于可拓展机器学习的失真图像视觉复原仿真. 计算机仿真. 2023(08): 240-244 . | |
| 4. | 李海燕，陈杰，余鹏飞，李海江，张榆锋. 双重模态文本引导的图像修复算法. 北京航空航天大学学报. 2023(10): 2547-2557 .  本站查看 | |
| 5. | 夏弘睿，赵静. 基于邻域一致性的数字媒体视频图像超清修复方法. 太原师范学院学报(自然科学版). 2022(03): 32-36 . | |
| 6. | 林嵩. 基于计算机视觉的视频图像关键帧提取及修复方法. 重庆科技学院学报(自然科学版). 2022(06): 65-70 . | |
| 7. | 李云红，穆兴，朱耀麟，汤汶，苏雪平，谢蓉蓉. 深度生成式对抗网络的超分辨率图像修复与重建. 西安工程大学学报. 2021(05): 56-63 . |
Figures(12) / Tables(1)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
Lü Mingyun, Sun Kangwen, Ma Yunpeng, et al. Economic affordability of aircraft conceptual/preliminary scheme based on uncertainty[J]. Journal of Beijing University of Aeronautics and Astronautics, 2010, 36(1): 118-122. (in Chinese)
DownLoad:
