基于高分辨率网络的单声道歌声分离

doi:10.13700/j.bh.1001-5965.2019.0491

北京航空航天大学学报 ›› 2020, Vol. 46 ›› Issue (8): 1555-1563.doi: 10.13700/j.bh.1001-5965.2019.0491

基于高分辨率网络的单声道歌声分离

张阳¹, 牛之贤¹, 牛保宁¹, 常艳²

1. 太原理工大学信息与计算机学院, 晋中 030600;
2. 中国科学院软件研究所, 北京 100190

收稿日期:2019-09-09 发布日期:2020-08-27
通讯作者: 牛之贤 E-mail:niuniurose63@163.com
作者简介:张阳女,硕士研究生。主要研究方向:音乐信息检索。
牛之贤女,硕士,副教授,硕士生导师。主要研究方向:信息检索、数据挖掘、软件理论与算法。
牛保宁男,博士,教授,博士生导师。主要研究方向:大数据、数据库系统的自主计算与性能管理。
常艳女,硕士研究生。主要研究方向:操作系统安全。
基金资助:
国家重点研发计划（2017YFB1401001-01）；国家自然科学基金（61572345）

Monaural singing voice separation based on high-resolution network

ZHANG Yang¹, NIU Zhixian¹, NIU Baoning¹, CHANG Yan²

1. College of Information and Computer, Taiyuan University of Technology, Jinzhong 030600, China;
2. Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

Received:2019-09-09 Published:2020-08-27
Supported by:
National Key R & D Program of China (2017YFB1401001-01); National Natural Science Foundation of China (61572345)

摘要/Abstract

摘要： 单声道歌声分离是指将单声道歌曲中的伴奏和歌声分离，在旋律提取、歌词识别、卡拉OK伴奏等方面有重要应用。针对当前时频谱图预测精度受限的问题，利用高分辨率网络具有并行结构及特征充分交互提高模型性能的优势，提出基于高分辨率网络的单声道歌声分离算法。设计并构建适合单声道歌声分离的高分辨率网络，输入歌曲的时频谱图到网络，得到预测的伴奏和歌声时频谱图。结合歌曲相位进行重构，得到伴奏和歌声的时域信号。实验表明，在公开数据集MIR-1K上，所提算法的SNR、SIR、SAR指标均优于当前代表性算法，提高了分离后伴奏和歌声的质量。

关键词: 单声道歌声分离, 深度学习, 时频谱图, 高分辨率网络, 频域模型

Abstract: Monaural singing voice separation separates singing voice and accompaniment from a song, which can be used for applications such as melody extraction, lyrics recognition, karaoke, etc. To resolve the limited accuracy of predicted spectrogram, this paper proposes a monaural singing voice separation algorithm based on high-resolution neural network, which has the advantages of parallel structure and sufficient features interaction for improving the performance of the model. Firstly, the high-resolution network suitable for singing voice separation is designed and constructed. Then, the spectrogram of the origin song is input to the network in order to get the predicted spectrograms of accompaniment and singing voice. Finally, the time-domain signals are reconstructed by combining the song phases with the separated spectrograms. Experiments conducted on the MIR-1K dataset show that SNR, SIR and SAR indicators of the proposed algorithm are better than those of the state-of-the-art algorithm, and the proposed algorithm improves the quality of the separated accompaniment and singing voice.

Key words: monaural singing voice separation, deep learning, spectrogram, high-resolution network, frequency-domain model

中图分类号:

TP391

张阳, 牛之贤, 牛保宁, 常艳. 基于高分辨率网络的单声道歌声分离[J]. 北京航空航天大学学报, 2020, 46(8): 1555-1563.

ZHANG Yang, NIU Zhixian, NIU Baoning, CHANG Yan. Monaural singing voice separation based on high-resolution network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(8): 1555-1563.

参考文献

[1] 李伟,李子晋,高永伟.理解数字音乐——音乐信息检索技术综述[J].复旦学报(自然科学版),2018,57(3):5-47.LI W,LI Z J,GAO Y W.Understanding digital music-A review of music information retrieval technology[J].Journal of Fudan University(Natural Science),2018,57(3):5-47(in Chinese).
[2] SIMPSON A J R,ROMA G,PLUMBLEY M D.Deep karaoke:Extracting vocals from musical mixtures using a convolutional deep neural network[C]//International Conference on Latent Variable Analysis and Signal Separation.Berlin:Springer,2015:429-436.
[3] HUANG P S,KIM M,HASEGAWA-JOHNSON M,et al.Joint optimization of masks and deep recurrent neural networks for monaural source separation[J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2015,23(12):2136-2147.
[4] UHLICH S,PORCH M,GIRON F,et al.Improving music source separation based on deep neural networks through data augmentation and network blending[C]//2017 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Piscataway:IEEE Press,2017:261-265.
[5] JANSSON A,HUMPHREY E,MONTECCHIO N,et al.Singing voice separation with deep U-Net convolutional networks[C]//18th International Society for Music Information Retrieval Conference(ISMIR),2017:745-751.
[6] PARK S,KIM T,LEE K,et al.Music source separation using stacked hourglass networks[C]//19th International Society for Music Information Retrieval Conference(ISMIR),2018:289-296.
[7] STOLLER D,EWERT S,DIXON S.Wave-U-Net:A multi-scale neural network for end-to-end audio source separation[C]//19th International Society for Music Information Retrieval Conference(ISMIR),2018:334-340.
[8] SUN K,XIAO B,LIU D,et al.Deep high-resolution representation learning for human pose estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2019:5693-5703.
[9] SUN K,ZHAO Y,JIANG B R,et al.High-resolution representations for labeling pixels and regions[EB/OL].(2019-04-09)[2019-09-01].https://arxiv.org/abs/1904.04514.
[10] VIRTANEN T.Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria[J].IEEE Transactions on Audio,Speech,and Language Processing,2007,15(3):1066-1074.
[11] HUANG P S,CHEN S D,SMARAGDIS P,et al.Singing-voice separation from monaural recordings using robust principal component analysis[C]//2012 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Piscataway:IEEE Press,2012:57-60.
[12] HSU C L,WANG D L,JANG J S R,et al.A tandem algorithm for singing pitch extraction and voice separation from music accompaniment[J].IEEE Transactions on Audio,Speech,and Language Processing,2012,20(5):1482-1491.
[13] IKEMIYA Y,ITOYAMA K,YOSHⅡ K.Singing voice separation and vocal F0 estimation based on mutual combination of robust principal component analysis and subharmonic summation[J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2016,24(11):2084-2095.
[14] RAFⅡ Z,PARDO B.Music/voice separation using the similarity matrix[C]//13th International Society for Music Information Retrieval Conference(ISMIR),2012:583-588.
[15] ZHU B L,LI W,LI R J,et al.Multi-stage non-negative matrix factorization for monaural singing voice separation[J].IEEE Transactions on Audio,Speech,and Language Processing,2013,21(10):2096-2107.
[16] ZHANG X,LI W,ZHU B L.Latent time-frequency component analysis:A novel pitch-based approach for singing voice separation[C]//2015 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Piscataway:IEEE Press,2015:131-135.
[17] DEIF H,WANG W,GAN L,et al.Local discontinuity based approach for monaural singing voice separation from accompanying music with multi-stage non-negative matrix factorization[C]//2015 IEEE Global Conference on Signal and Information Processing(GlobalSIP).Piscataway:IEEE Press,2015:93-97.
[18] HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2016:770-778.

基于高分辨率网络的单声道歌声分离

Monaural singing voice separation based on high-resolution network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李欣致, 董胜波, 崔向阳, 刘志哲, 郭广浩. 应用于智能芯片的可视化反馈系统研究[J]. 北京航空航天大学学报, 2020, 46(8): 1494-1502.
[2]	李承昊, 茹乐, 何林远, 迟文升. 一种可变锚框候选区域网络的目标检测方法[J]. 北京航空航天大学学报, 2020, 46(8): 1610-1617.
[3]	吕娜, 周家欣, 陈卓, 刘鹏飞, 高维廷. 一种鲁棒性增强的机载网络流量分类方法[J]. 北京航空航天大学学报, 2020, 46(7): 1237-1246.
[4]	郑磊, 胡维多, 刘畅. 基于深度学习的大型陨石坑识别方法研究[J]. 北京航空航天大学学报, 2020, 46(5): 994-1004.
[5]	边亮, 罗霄阳, 李硕. 基于深度学习的图像拼接篡改检测[J]. 北京航空航天大学学报, 2020, 46(5): 1039-1044.
[6]	杨思晨, 王华锋, 王月海, 李锦涛, 王赟豪. 深度学习机制与小波融合的超分辨率重建算法[J]. 北京航空航天大学学报, 2020, 46(1): 189-197.
[7]	张克明, 蔡远文, 任元. 基于生成对抗网络的航天异常事件检测方法[J]. 北京航空航天大学学报, 2019, 45(7): 1329-1336.
[8]	魏宏喜, 张越. 基于生成对抗网络的零样本图像分类[J]. 北京航空航天大学学报, 2019, 45(12): 2345-2350.
[9]	张新峰, 郭宇桐, 蔡轶珩, 孙萌. 基于DCNN和全连接CRF的舌图像分割算法[J]. 北京航空航天大学学报, 2019, 45(12): 2364-2374.
[10]	赵国, 秦世引. PMMWI与VI优势互补的人体隐蔽违禁物检测与定位[J]. 北京航空航天大学学报, 2019, 45(10): 2011-2025.
[11]	郭琳, 秦世引. 遥感图像飞机目标高效搜检深度学习优化算法[J]. 北京航空航天大学学报, 2019, 45(1): 159-173.
[12]	韩京冶, 许福, 陈志泊, 刘辉. 一种基于深度学习的交互式电话号码识别方法[J]. 北京航空航天大学学报, 2018, 44(5): 1074-1080.
[13]	王鑫, 吴际, 刘超, 杨海燕, 杜艳丽, 牛文生. 基于LSTM循环神经网络的故障时间序列预测[J]. 北京航空航天大学学报, 2018, 44(4): 772-784.
[14]	车畅畅, 王华伟, 倪晓梅, 洪骥宇. 基于深度学习的航空发动机故障融合诊断[J]. 北京航空航天大学学报, 2018, 44(3): 621-628.
[15]	许轲, 吴凤鸽, 赵军锁. 基于深度强化学习的软件定义卫星姿态控制算法[J]. 北京航空航天大学学报, 2018, 44(12): 2651-2659.