一种面向工业互联网的云存储方法

doi:10.13700/j.bh.1001-5965.2018.0174

北京航空航天大学学报 ›› 2019, Vol. 45 ›› Issue (1): 130-140.doi: 10.13700/j.bh.1001-5965.2018.0174

一种面向工业互联网的云存储方法

孟祥曦¹, 张凌², 郭皓明³, 郭黎敏³, 夏乾臣¹, 吕江花¹, 马世龙¹

1. 北京航空航天大学计算机学院, 北京 100083;
2. 中国地震应急搜救中心, 北京 100049;
3. 中国科学院软件研究所, 北京 100190

收稿日期:2018-04-02 修回日期:2018-09-03 出版日期:2019-01-20 发布日期:2019-01-28
通讯作者: 张凌 E-mail:zhangling903@163.com
作者简介:孟祥曦,男,博士研究生。主要研究方向:数据管理系统、模型管理、软件工程;张凌,男,高级工程师。主要研究方向:大数据管理与应用、自动化测试、大数据分析;郭皓明,男,博士,高级工程师。主要研究方向:软件测试、形式化方法与软件工程,大数据管理与应用。
基金资助:
国家自然科学基金（61300007，61305054）；软件开发环境国家重点实验室自主探索基金（SKLSDE-2012ZX-28，SKLSDE-2014ZX-06）

A new approach of cloud storage for industrial Internet

MENG Xiangxi¹, ZHANG Ling², GUO Haoming³, GUO Limin³, XIA Qianchen¹, LYU Jianghua¹, MA Shilong¹

1. School of Computer Science and Engineering, Beihang University, Beijing 100083, China;
2. National Earthquake Response Support Service, Beijing 100049, China;
3. Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

Received:2018-04-02 Revised:2018-09-03 Online:2019-01-20 Published:2019-01-28
Supported by:
National Natural Science Foundation of China (61300007,61305054); Foundation of the Key Lab of Software Development Environment (SKLSDE-2012ZX-28,SKLSDE-2014ZX-06)

摘要/Abstract

摘要： 工业互联网是工业信息化进程中最受关注的热点，海量异构数据管理是其中的重点之一。传统的关系数据库（RDB）对海量多源异构数据的读写和检索都存在性能瓶颈，而近年来兴起的云数据管理方法主要是针对“键-值”（K-V）模式，无法依靠主键以外的数据属性对数据进行快速查找。提出了一种面向工业互联网的云存储方法——StoreCDB，在异构采样数据统一表达数据模型基础上，实现非结构化存储管理，同时，利用两级索引实现海量数据的快速检索。通过实验，在分布式集群实验平台上，采用海量高铁列车运行模拟数据，验证了StoreCDB具有良好的异构数据存储和检索性能，为工业互联网提供了一种新的数据管理方法。

关键词: 工业互联网, 异构数据, 海量数据管理, 分布式文件系统, 分级索引

Abstract: With the development of industrial informatization, industrial Internet has attracted many attentions, and massive heterogeneous data management is one of the most important issues. However, traditional relational database (RDB) limits the performance of access and retrieval of massive and heterogeneous data, while cloud data management mainly focuses on key-value (K-V) queries, which cannot quickly search data by using any data property other than the prime key. In this paper, a cloud storage framework-StoreCDB is proposed for data management in the industrial Internet. In StoreCDB, the heterogeneous data are represented by a uniform data model firstly and then stored in a distributed file and parallel architecture as unstructured data. In addition, a double-level index is proposed to support both key-value queries and RDB queries. This paper adopts a distributed cluster experimental platform and massive high-speed train operation simulation data to verify the framework. The experimental results show that StoreCDB has satisfactory heterogeneous data access and retrieval performance and provides a good solution for industrial Internet data management.

Key words: industrial Internet, heterogeneous data, massive data management, distributed file system, hierarchical index

中图分类号:

V219
TP333

孟祥曦, 张凌, 郭皓明, 郭黎敏, 夏乾臣, 吕江花, 马世龙. 一种面向工业互联网的云存储方法[J]. 北京航空航天大学学报, 2019, 45(1): 130-140.

MENG Xiangxi, ZHANG Ling, GUO Haoming, GUO Limin, XIA Qianchen, LYU Jianghua, MA Shilong. A new approach of cloud storage for industrial Internet[J]. JOURNAL OF BEIJING UNIVERSITY OF AERONAUTICS AND A, 2019, 45(1): 130-140.

参考文献

[1] LI D,WU H,LI S C. Internet of things in industries:A survey[J].IEEE Transactions on Industrial Informatics,2014,10(4):2233-2243.
[2] 李晓娟. 物联网中海量数据管理技术研究[D].广州:广东工业大学,2015.LI X J.Huge amounts of data management technology for internet of things[D].Guangzhou:Guangdong University of Technology,2015(in Chinese).
[3] 孙鹏.动车组维修物联网及其关键技术研究[D].北京:中国铁道科学研究院,2013.SUN P.Study on EMU maintenance in the internet of things and its key technologies[D].Beijing:China Academy of Railway Sciences,2013(in Chinese).
[4] SANCHEZ L,MUÑOZ L,GALACHE J A,et al.Smartsantander:IoT experimentation over a smart city testbed[J].Computer Networks,2014,61:217-238.
[5] 宁焕生,徐群玉.全球物联网发展及中国物联网建设若干思考[J].电子学报,2010,38(11):2590-2599.NING H S,XU Q Y.Research on global internet of things' developments and it's lonstruction in China[J].Acta Electronica Sinica,2010,38(11):2590-2599(in Chinese).
[6] ATZORI L,IERA A,MORABITD G.The internet of things:A survey[J].Computer Networks,2010,54(15):2787-2805.
[7] 康世龙,杜中一,雷咏梅,等.工业物联网研究概述[J].物联网技术,2013,3(6):88-90.KANG S L,DU Z Y,LEI Y M,et al.Over view of industrial internet of things[J].Internet of Things Technologies,2013,3(6):88-90(in Chinese).
[8] GUBBI J,BUYYA R,MARUSIC S,et al.Internet of things (IoT):A vision,architectural elements,and future directions[J].Future Generation Computer Systems,2013,29(7):1645-1660.
[9] ÖZSU M T,VALDURIEZ P.Principles of distributed database systems[M].3rd ed.New York:Springer Science & Business Media,2011:16.
[10] STONEBRAKER M.SQL databases v.NoSQL databases[J].Communications of the ACM,2010,53(4):10-11.
[11] GILBERT S,LYNCH N.Brewer's conjecture and the feasibility of consistent,available,partition-tolerant web services[J].ACM SIGACT News,2002,33(2):51-59.
[12] HUANG H,WU Q H,YU D.Robust distributed control of robot formations with parameter uncertainty[J].Journal of Control Theory and Applications,2011,9(4):51-58.
[13] XU H G,HAN J H,PAN S P,et al.The research on data consistency of distributed storage system based on two-hop DHT[C]//Procceedings of IEEE ICITIS 2011.Piscataway,NJ:IEEE Press,2011:131-132.
[14] PETER M,GRANCE T.The NIST definition of cloud computing:SP 800-145[R].Gaithersbury:NIST,2011.
[15] SHEPLER S,CALLAGHAN B,ROBINSON D,et al.Network file system(NFS)version 4 protocol[R].Reston:The Internet Society,2003.
[16] GHEMAWAT S,HOWARD G,LEUNG S T.The Google file system[J].ACM SIGOPS Operating Systems Review,2003,37(5):29-43.
[17] BORTHAKUR D.HDFS architecture guide[EB/OL].(2013-10-10)[2018-03-20].http://hadoop.apache.org/docs/r1.0.4/hdfs_design.html.
[18] CHAIKEN R,JENKINS B,LARSON P A,et al.SCOPE:Easy and efficient parallel processing of massive data sets[J].PVLDB,2008,1(2):1265-1276.
[19] BEAVER D,KUMAR S,LI H C,et al.Finding a needle in haystack:Facebook's photo storage[C]//Proceedings of OSDI 2010.Berkeley,CA:USENIX Association,2010:47-60.
[20] DECANDIA G,HASTORUN D,JAMPANI M,et al.Dynamo:Amazon's highly available key-value store[C]//Proceedings of SOSP 2007.New York:ACM,2007:205-220.
[21] CHANG F,DEAN J,GHEMAWAT S,et al.Bigtable:A distributed storage system for structured data[J].ACM Transactions on Computer Systems,2008,26(2):1-26.
[22] BAKER J.Megastore:Providing scalable,highly available storage for interactive services[C]//Biennial Conference on Innovative Data Systems Research,2011:223-234.
[23] CORBETT J C,DEAN J,EPSTEIN M,et al.Spanner:Google's globally-distributed database[C]//Usenix Conference on Operating Systems Design and Implementation,2012:251-264.

一种面向工业互联网的云存储方法

A new approach of cloud storage for industrial Internet

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价