基于对抗和迁移学习的灾害天气卫星云图分类

张敏靖; 白琮; 张敬林; 郑建炜

doi:10.13700/j.bh.1001-5965.2020.0459

基于对抗和迁移学习的灾害天气卫星云图分类

doi: 10.13700/j.bh.1001-5965.2020.0459

张敏靖¹,
白琮^{1, 2, ,},
张敬林³,
郑建炜^{1, 2}

1.
浙江工业大学计算机科学与技术学院, 杭州 310023
2.
浙江省可视媒体智能处理技术研究重点实验室, 杭州 310023
3.
南京信息工程大学大气科学学院, 南京 210044

基金项目:

国家重点研发计划 2018YFE0126100

国家自然科学基金 41775008

国家自然科学基金 61976192

国家自然科学基金 61702275

浙江新苗人才计划 2020R403059

详细信息

作者简介:
张敏靖  女，硕士研究生。主要研究方向：计算机视觉与多媒体信息处理

白琮  男, 博士，副教授，博士生导师。主要研究方向：计算机视觉与多媒体信息处理

张敬林  男，博士，教授，博士生导师。主要研究方向：计算机视觉与遥感信息处理

郑建炜  男，博士，副教授，博士生导师。主要研究方向：计算机视觉与遥感信息处理

通讯作者:
白琮, E-mail: congbai@zjut.edu.cn

中图分类号: TB553
计量
- 文章访问数: 757
- HTML全文浏览量: 178
- PDF下载量: 99
- 被引次数: 0
出版历程
- 收稿日期: 2020-08-25
- 录用日期: 2020-09-04
- 网络出版日期: 2021-03-20

Classification of satellite cloud images of disaster weather based on adversarial and transfer learning

1.
College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China
2.
Key Laboratory of Visual Media Intelligent Processing Technology of Zhejiang Province, Hangzhou 310023, China
3.
School of Atmospheric Science, Nanjing University of Information Science and Technology, Nanjing 210044, China

Funds:

National Key R & D Program of China 2018YFE0126100

National Natural Science Foundation of China 41775008

National Natural Science Foundation of China 61976192

National Natural Science Foundation of China 61702275

Zhejiang Xinmiao Talent Projects 2020R403059

More Information

Corresponding author: BAI Cong, E-mail: congbai@zjut.edu.cn

摘要

摘要:
针对卫星云图中的灾害天气数据存在严重不平衡问题，提出一个结合生成对抗学习（GAN）和迁移学习（TL）的卷积神经网络（CNN）框架以解决上述问题进而提高基于卫星云图的灾害天气分类精度。该框架主要包含基于GAN的数据均衡化模块和基于迁移学习的CNN分类模块。上述2个模块分别从数据和算法层面解决数据的类间不平衡问题，分别得到一个相对均衡的数据集和一个可在不同类别数据上提取相对均衡特征的分类模型，最终实现对卫星云图的分类，提高其中灾害天气的卫星云图类别分类准确率。与此同时所提方法在自建的大规模卫星云图数据上进行了测试，消融性和综合实验结果证明了所提数据均衡方法和迁移学习方法是有效的，且所提框架模型对各个灾害天气类别的分类精度都有显著提升。
- 灾害天气 /
- 卫星影像 /
- 图片分类 /
- 生成对抗网络 /
- 类间不平衡
Abstract:
Weather can be forecasted based on clouds. However, how to use deep learning technology to achieve automatic weather forecasting, especially the automatic recognition of disaster weather, is still an unexplored field. Hence, it is necessary to carry out research on the basic problem in the field of automatic identification: the classification of satellite cloud images. Satellite cloud images have serious data imbalance problems. That is, cloud image data related to severe weather accounts for a very small proportion of all cloud image data. Therefore, this paper proposes a framework combining Generative Adversarial Network (GAN) and Transfer Learning (TL) based Convolutional Neural Network (CNN) to solve the problem of low accuracy of disaster weather classification based on satellite cloud images. The framework is mainly divided into a data balancing module based on GAN and a CNN classification module based on transfer learning. The above two modules solve the data imbalance problem from the data and algorithm level respectively, and obtain a relatively balanced dataset and a classification model that can extract relatively balanced features on different types of data. Eventually, the classification of satellite cloud images is achieved and the accuracy of the classification of satellite cloud images in disaster weather is improved. The method proposed in this paper has been tested on self-built large-scale satellite cloud image data. The ablative properties and comprehensive experimental results prove that the proposed data balancing method and transfer learning method are effective, and the proposed framework model has significantly improved the classification accuracy of various disaster weather categories.
- disaster weather /
- satellite image /
- image classification /
- generative adversarial networks /
- data imbalance

HTML全文

图 1 处理气象卫星中类间不平衡的模型框架图

Figure 1. Model framework for handling imbalance between classes in meteorological satellites

下载: 全尺寸图片幻灯片

图 2 不同方法对应数据分布情况的百分占比示意图

Figure 2. Schematic diagram of data percentage proportion of data distribution corresponding to different methods

下载: 全尺寸图片幻灯片

图 3 GAN的核心思想

Figure 3. Core idea of GAN

下载: 全尺寸图片幻灯片

图 4 分类模块训练的流程图

Figure 4. Flowchart of classification module training

下载: 全尺寸图片幻灯片

图 5 LSCIDMR-S数据集的部分示意图

Figure 5. Partial schematic of LSCIDMR-S dataset

下载: 全尺寸图片幻灯片

图 6 基于GAN的数据过采样生成的图片

Figure 6. Schematic diagram of oversampling data image generated by GAN

下载: 全尺寸图片幻灯片

图 7 各个模型中各个类别对应的ROC曲线

Figure 7. ROC curve corresponding to each category in each model

下载: 全尺寸图片幻灯片

图 8 参数k对分类性能的影响

Figure 8. Influence of parameter k on classification performance

下载: 全尺寸图片幻灯片

图 9 不同k对应的数据分布

Figure 9. Data points corresponding to different k

下载: 全尺寸图片幻灯片

表 1 LSCIDMR-S处理之后的数据分布情况表

Table 1. Data distribution of LSCIDMR-S after processing

类别	数量	比例/%
温带气旋	4 985	4.78
热带气旋	3 305	3.17
降雪	8 522	8.16
锋面	634	0.61
西风急流	628	0.60
非灾害天气	86 315	82.69
总计	104 390	100

下载: 导出CSV

表 2 各方法对应的数据分布及数据不平衡系数

Table 2. Data distribution and data imbalance degree corresponding to each method

方法	数据集的分布(对应图 2)	IR
Base	1	137.25
Base_under	2	6.759
Base_under_t	2	6.759
Base_under_over	3	3.35
Base_under_over_t	3	3.35
Base_under_gan	3	3.35
Base_under_gan_t	3	3.35

下载: 导出CSV

表 3 各个模型的总精度和分类精度的统计

Table 3. Statistics of total accuracy of each model and accuracy of each category (Accuracy)

序号	方法	总精度	分类精度
序号	方法	总精度	非灾害天气	西风急流	热带气旋	降雪	锋面	温带气旋
1	Base	0.889 3	0.966 6	0.209 7	0.215 2	0.821 6	0.095 2	0.317 3
2	Base_under	0.733 1	0.743 1	0.209 7	0.454 5	0.841 5	0.127 0	0.759 0
3	Base_under_t	0.761 8	0.765 4	0.338 7	0.648 5	0.940 1	0.190 5	0.594 4
4	Base_under_over	0.703 8	0.708 5	0.387 1	0.318 2	0.843 9	0.079 4	0.757 0
5	Base_under_over_t	0.765 1	0.771 3	0.612 9	0.336 4	0.882 6	0.396 8	0.722 9
6	Base_under_gan	0.732 0	0.716 4	0.629 0	0.687 9	0.946 0	0.333 0	0.728 9
7	本文	0.774 4	0.763 6	0.709 7	0.736 4	0.929 6	0.460 3	0.769 1

下载: 导出CSV

参考文献(15)

[1]	PARMESAN C, ROOT T L, WILLIG M R. Impacts of extreme weather and climate on terrestrial biota[J]. Bulletin of the American Meteorological Society, 2000, 81(3): 443-450. doi: 10.1175/1520-0477(2000)081<0443:IOEWAC>2.3.CO;2
[2]	王会军. 把脉极端天气气候事件, 构建预测方法[J]. 大气科学学报, 2019, 42(1): 1. https://www.cnki.com.cn/Article/CJFDTOTAL-NJQX201901001.htm WANG H J. Take the pulse of extreme weather and climate events and construct prediction methods[J]. Journal of Atmospheric Sciences, 2019, 42(1): 1(in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-NJQX201901001.htm
[3]	JOHNSON J M, KHOSHGOFTAAR T M. Survey on deep learning with class imbalance[J]. Journal of Big Data, 2019, 6(1): 27. doi: 10.1186/s40537-019-0192-5
[4]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. 2012: 1097-1105.
[5]	DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2009: 248-255.
[6]	SAIYEDA A, MIR M A. Cloud computing for deep learning analytics: A survey of current trends and challenges[J]. International Journal of Advanced Research in Computer Science, 2017, 8(2): 68-72. http://www.ijarcs.info/index.php/Ijarcs/article/view/2931
[7]	LEEVY J L, KHOSHGOFTAAR T M, BAUDER R A, et al. A survey on addressing high-class imbalance in big data[J]. Journal of Big Data, 2018, 5(1): 42. doi: 10.1186/s40537-018-0151-6
[8]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems. 2014: 2672-2680.
[9]	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL]. (2016-01-07)[2020-07-23]. https://arxiv.org/abs/151/06434v2.
[10]	KARRAS T, LAINE S, AITTALA M, et al. Analyzing and improving the image quality of stylegan[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 8110-8119.
[11]	HENSMAN P, MASKO D. The impact of imbalanced training data for convolutional neural networks[EB/OL]. [2020-07-23]. http://diva-portal.org/smash/get/diva2:811111/FULLTEXT01.pdf.
[12]	WANG S, LIU W, WU J, et al. Training deep neural networks on imbalanced data sets[C]//2016 International Joint Conference on Neural Networks(IJCNN). Piscataway: IEEE Press, 2016: 4368-4374.
[13]	HE K, ZHANG X, REN S, et al, Deep Residual Learning for Image Recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2016: 770-778.
[14]	LEE H, PARK M, KIM J. Plankton classification on imbalanced large scale database via convolutional neural networks with transfer learning[C]//2016 IEEE International Conference on Image Processing (ICIP). Piscataway: IEEE Press, 2016: 3713-3717.
[15]	KANG B, XIE S, ROHRBACH M, et al. Decoupling representation and classifier for long-tailed recognition[EB/OL]. (2020-02-19)[2020-07-23]. https://arxiv.org/abs/1910.09217?context=cs.CV.