面向集群环境的虚拟化GPU计算平台

doi:10.13700/j.bh.1001-5965.2015.0731

北京航空航天大学学报 ›› 2016, Vol. 42 ›› Issue (11): 2340-2348.doi: 10.13700/j.bh.1001-5965.2015.0731

面向集群环境的虚拟化GPU计算平台

杨经纬, 马凯, 龙翔

北京航空航天大学计算机学院, 北京 100083

收稿日期:2015-11-09 修回日期:2016-01-15 出版日期:2016-11-20 发布日期:2016-03-15
通讯作者: 龙翔,Tel.:010-82339685,E-mail:long@buaa.edu.cn E-mail:long@buaa.edu.cn
作者简介:杨经纬,男,博士研究生。主要研究方向:计算系统结构、嵌入式系统、多核实时操作系统与实时调度。E-mail:yaungjw@buaa.edu.cn;马凯,男,硕士研究生。主要研究方向:计算机系统结构、并行与分布式计算。E-mail:makai@buaa.edu.cn;龙翔,男,博士,教授,博士生导师。主要研究方向:计算系统结构、并行与分布式系统、实时系统。Tel.:010-82339685,E-mail:long@buaa.edu.cn

Virtualized GPU computing platform in clustered system environment

YANG Jingwei, MA Kai, LONG Xiang

School of Computer Science and Engineering, Beijing University of Aeronautics and Astronautics, Beijing 100083, China

Received:2015-11-09 Revised:2016-01-15 Online:2016-11-20 Published:2016-03-15

摘要/Abstract

摘要： 针对集群系统的多节点多GPU环境，提出一种新型虚拟化GPU计算平台。该平台实现对集群系统所有节点上GPU资源的统一抽象与管理，构建公共GPU资源池。原有GPU应用程序可以不经任何修改而迁移到虚拟化GPU计算平台，并具备访问资源池内任何GPU的能力，编程人员无需显式针对多节点多GPU应用展开MPI编程。应用程序摆脱了单个节点上GPU资源的限制，并具备无差别地访问集群系统中任何可用GPU资源的能力，能有效提高系统总体资源利用率以及吞吐量。采用流水化通信技术，实现对虚拟化GPU计算平台的运行时开销以及节点间数据传输延迟的隐藏。实验表明：与非流水化通信相比，系统总体数据传输延迟降低了50%~70%，具备与节点机本地数据传输等同的通信性能。

关键词: GPU, MPI, CUDA, 集群系统, 硬件加速, 并行计算, 高性能计算

Abstract: A virtualized GPU computing platform is proposed for clustered systems, which are often equipped with GPUs in some nodes. All GPUs in system are uniformly abstracted as virtualized ones in a commonly accessed resource pool. Legacy GPU programs can execute on the virtualized GPU computing platform without any modification and any free virtualized GPU in the common resource pool is available to it, which relieves the burden of MPI programming. The platform frees programs with the limit of GPUs in local node and makes it possible for them to access any available GPU in distributed nodes, leading to higher system utilization and throughput. Based on pipelined communication, the run-time overhead and inter-node transmitting latency in virtualized GPU computing platform are hidden by intra-node memory copying and GPU computing. Compared with the non-pipelined communication, the total transmission latency is decreased by approximately 50%-70%. It results in a comparable performance with intra-node local data transmission.

Key words: GPU, MPI, CUDA, clustered systems, hardware acceleration, parallel computing, high performance computing

中图分类号:

TP315

杨经纬, 马凯, 龙翔. 面向集群环境的虚拟化GPU计算平台[J]. 北京航空航天大学学报, 2016, 42(11): 2340-2348.

YANG Jingwei, MA Kai, LONG Xiang. Virtualized GPU computing platform in clustered system environment[J]. JOURNAL OF BEIJING UNIVERSITY OF AERONAUTICS AND A, 2016, 42(11): 2340-2348.

参考文献

[1] KIVITY A,KAMAY Y,LAOR D,et al.KVM:The Linux virtual machine monitor[EB/OL].Proceedings of the Linux Symposium,Ottawa[2015-11-01].https://www.kernel.org/doc/ols/2007/ols2007v1-pages-225-230.pdf.
[2] BARHAM P,DRAGOVIC B,FRASER K,et al.Xen and the art of virtualization[C]//Proceedings of the 19th ACM Symposium on Operating Systems Principles.New York: ACM,2003:164-177.
[3] NextION2800-ICA-Flexible and manageable I/O expansion and virtualization[EB/OL].Austin:NEXTIO[2015-11-01].http://www.nextio.com/docs/NextIO20N2800-ICAIOConsolidationApplianceProductBriefv0.18.pdf.
[4] SHREINER D.OpenGL programming guide[M].7th ed.Boston:Addison-Wesley Professional,2009:1-28.
[5] BLYTHE D.The direct 3D 10 system[J].ACM Transactions on Graphics,2006,25(3):724-734.
[6] NVIDIA.CUDA C programming guide[EB/OL].Santa Clara:NVIDIA[2015-11-01].https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html.
[7] KHRONOS OpenCL Working Group.OpenCL 2.0 specification[EB/OL].OR:Khronos[2015-11-01].http://www.khronos.org/registry/cl/specs/opencl-1.2.pdf.
[8] DOWTY M,SUGERMAN J.GPU virtualization on VMware's hosted I/O architecture[J].ACM SIGOPS Operating Systems Review,2009,43(3):73-82.
[9] LAGAR-CAVILLA H A,TOLIA N,SATYANARAYANAN M,et al.VMM-independent graphics acceleration[C]//VEE'07:Proceedings of the 3rd International Conference on Virtual Execution Environments.New York:ACM,2007:33-43.
[10] SHI L,CHEN H,SUN J.vCUDA:GPU accelerated high performance computing in virtual machines[C]//23rd IEEE International Symposium on Parallel and Distributed Processing (IPDPS'09).Piscataway,NJ:IEEE Press,2009:418-428.
[11] GUPTA V,GAVRILOVSKA A,SCHWAN K,et al.GViM:GPU-accelerated virtual machines[C]//Proceedings of the 3rd ACM Workshop on System-level Virtualization for High Performance Computing.New York:ACM,2009:17-24.
[12] GIUNTA G,MONTELLA R,AGRILLO G,et al.A GPGPU transparent virtualization component for high performance computing clouds[C]//16th International Euro-Par-Conference on Parallel Processing.Berlin:Springer,2010:379-391.
[13] KEGEL P,STEUWER M,GORLATCH S.dOpenCL:Towards a uniform programming approach for distributed heterogeneous multi-/many-core systems[C]//2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum (IPDPSW).Piscataway,NJ:IEEE Press,2012:174-186.
[14] BARAK A,BEN-NUN T,LEVY E,et al.A package for OpenCL based heterogeneous computing on clusters with many GPU devices[C]//2010 IEEE International Conference on Cluster Computing Workshops and Posters (CLUSTER WORKSHOPS).Piscataway,NJ:IEEE Press,2010:1-7.
[15] DUATO J,PEA A J,SILLA F,et al.rCUDA:Reducing the number of GPU-based accelerators in high performance clusters[C]//2010 International Conference on High Performance Computing and Simulation (HPCS).Piscataway,NJ:IEEE Press,2010,6:224-231.
[16] PEA A J,REAO C,SILLA F,et al.A complete and efficient CUDA-sharing solution for HPC clusters[J].Parallel Computing,2014,40(10):574-588.

面向集群环境的虚拟化GPU计算平台

Virtualized GPU computing platform in clustered system environment

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 14

编辑推荐

Metrics

本文评价

[1]	宋波, 李威, 廉国选. 基于CUDA的超声二维声场EFIT仿真[J]. 北京航空航天大学学报, 2019, 45(7): 1322-1328.
[2]	鲍军鹏, 杨科, 周静. 卫星时序数据挖掘节点级并行与优化方法[J]. 北京航空航天大学学报, 2018, 44(12): 2470-2478.
[3]	葛志闪, 鲜宁, 王津申, 李阳. 二维海面上三维电大尺寸舰船目标电磁散射仿真[J]. 北京航空航天大学学报, 2018, 44(11): 2299-2304.
[4]	李超, 赵长海, 晏海华, 刘超, 文佳敏, 王增波. 一种在复杂环境中支持容错的高性能规约框架[J]. 北京航空航天大学学报, 2018, 44(10): 2115-2124.
[5]	冯圆, 代小霞, 唐晓斌, 龚晓燕. 基于分布式平台的FDTD并行算法[J]. 北京航空航天大学学报, 2016, 42(9): 1874-1883.
[6]	黄宇, 阎超, 袁武. 适用于混合网格的改进雅可比迭代法及其应用[J]. 北京航空航天大学学报, 2016, 42(3): 551-561.
[7]	付子豪, 龚光红. 矩的显式积分算法研究与应用[J]. 北京航空航天大学学报, 2015, 41(4): 727-731.
[8]	何芒, 刘金波, 王斌斌, 徐晓文, 章传芳, 孙厚军. 接收模式下加罩天线系统特性的快速电磁仿真[J]. 北京航空航天大学学报, 2015, 41(10): 1792-1800.
[9]	郭咏新, 毛剑琴. 超磁致伸缩作动器的率相关建模与跟踪控制[J]. 北京航空航天大学学报, 2013, 39(10): 1360-1365.
[10]	贺东京, 宋晓, 张霖, 李潭. 基于扩充Modelica的复杂系统多层次并行仿真[J]. 北京航空航天大学学报, 2012, (9): 1240-1244.
[11]	郑红, 李俊. 基于行程累加的目标标记硬件加速算法[J]. 北京航空航天大学学报, 2012, (11): 1425-1429.
[12]	温建华,朱自强,吴宗成,陈泽民. 基于N-S方程串并行计算的机翼优化设计[J]. 北京航空航天大学学报, 2008, 34(02): 127-130.
[13]	胡凯. 网络分布式并行计算的负载平衡[J]. 北京航空航天大学学报, 2004, 30(11): 1121-1124.
[14]	黄宁, 金茂忠. PVM系统的统一并行调试对象模型[J]. 北京航空航天大学学报, 1998, 24(4): 462-465.