作者：吴永旺¹, 饶银辉², 庄伟涛², 子文江¹, 杨捍东², 余蓉¹, 洪晓斌¹

作者单位：1. 华南理工大学机械与汽车工程学院，广东 广州 510640;
2. 广船国际有限公司，广东 广州 511462

关键词：无人船;Hadoop;HDFS

摘要：

针对无人船测试过程中数据量大、数据源多、数据记录分散等问题，提出一种面向无人船测试的多源数据HDFS存储优化方法。首先设计基于Hadoop的无人船测试云平台架构，分析无人船测试云平台的信息流；接着提出无人船测试多源数据容错机制及重复数据删除策略，优化无人船测试多源数据HDFS存储；最后对无人船测试多源数据进行HDFS存储实验。结果表明，Hadoop集群优化后比默认系统存储效率提升约20%，可满足无人船测试多源数据的存储需求。

HDFS storage optimization for multi-source data of USV testing
WU Yongwang¹, RAO Yinhui², ZHUANG Weitao², ZI Wenjiang¹, YANG Handong², YU Rong¹, HONG Xiaobin¹
1. School of Mechanical and Automotive Engineering, South China University of Technology, Guangzhou 510640, China;
2. Guangzhou Shipyard International Company Limited, Guangzhou 511462, China
Abstract: Aiming at the problems of large amount of data, multiple data sources and scattered data records in the testing process of unmanned surface vessel (USV), a HDFS storage optimization method for multi-source data of USV testing was proposed. Firstly, the architecture of USV testing cloud platform based on Hadoop was designed, and the information flow of USV testing cloud platform was analyzed. Then, the strategy of multi-source fault tolerance and duplicate data deletion was proposed to optimize the HDFS storage of USV testing multi-source data. Finally, an experiment was designed to test the HDFS storage method of USV testing multi-source data. The results show that the efficiency of the optimized Hadoop cluster is 20% higher than that of default system, which meets the storage requirement of USV testing multi-source data.
Keywords: USV;Hadoop;HDFS
2022, 48(5):123-127  收稿日期: 2020-12-18;收到修改稿日期: 2021-05-08
基金项目: 广东省科技计划项目(2019B151502057)；广东省自然资源厅科技项目(GDoE[2019]A13)；广州市科技计划项目(201902010024)
作者简介: 吴永旺（1995-），男，江西南昌市人，硕士研究生，专业方向为无人智能测控技术及应用
参考文献
[1] 曹娟, 王雪松. 国内外无人船发展现状及未来前景[J]. 中国船检, 2018(5): 94-97
[2] RIGATOS G, BUSAWON K. Unmanned surface vessels[M]. Robotic Manipulators and Vehicles. Springer, Cham, 2018: 501-591.
[3] WANG N, GAO Y, WENG Y, et al. Implementation of an integrated navigation, guidance and control system for an unmanned surface vehicle[C]//Advanced Computational Intelligence (ICACI), 2018 Tenth International Conference on. IEEE, 2018: 717-722.
[4] 苏士斌, 刘英策, 林洪山, 等. 无人驾驶运输船发展现状与关键技术[J]. 船海工程, 2018, 47(5): 56-59
[5] STERGIOU C, PSANNWAS K E, KIM B G, et al. Secure integration of IoT and cloud computing[J]. Future Generation Computer Systems, 2018, 78: 964-975
[6] LEE H W, ROH M I. Review of the multibody dynamics in the applications of ships and offshore structures[J]. Ocean Engineering, 2018, 167: 65-76
[7] 库波. 云计算在舰船远程实时监控系统中的应用[J]. 舰船科学技术, 2016, 38(4): 157-159
[8] 王宁, 张聪沛. 舰船云存储数据中心密文访问控制机制的优化技术[J]. 舰船科学技术, 2018, 40(8): 118-120
[9] 刘新海, 马彦恒, 侯建强. 基于云和频繁项集的认知测试性诊断方案权衡优化[J]. 中国测试, 2018, 44(3): 11-15
[10] 卢艳艳. 基于Hadoop的大数据存储关键技术研究[D]. 北京: 华北电力大学, 2016.
[11] 余蓉. 基于Hadoop的大型钢结构无损云检测技术研究[D]. 广州: 华南理工大学, 2018.