计算机科学 ›› 2023, Vol. 50 ›› Issue (3): 238-245.doi: 10.11896/jsjkx.230100064

• 计算机图形学&多媒体 • 上一篇    下一篇

基于深度学习的可视化仪表板生成技术研究

陈亮1, 王璐1, 李生春2, 刘昌宏2   

  1. 1 西安工程大学计算机科学学院 西安 710048
    2 重庆中烟工业有限责任公司黔江卷烟厂 重庆 409000
  • 收稿日期:2023-01-12 修回日期:2023-02-13 出版日期:2023-03-15 发布日期:2023-03-15
  • 通讯作者: 王璐(1466756411@qq.com)
  • 作者简介:(chenliang@xpu.edu.cn)
  • 基金资助:
    国家自然科学基金(51675108);陕西省教育厅重点科学研究计划(22JS021)

Study on Visual Dashboard Generation Technology Based on Deep Learning

CHEN Liang1, WANG Lu1, LI Shengchun2, LIU Changhong2   

  1. 1 School of Computer Science,Xi’an Polytechnic University,Xi’an 710048,China
    2 China Tobacco Chongqing Industrial Co.Ltd. Qianjiang Cigarette Factory,Chongqing 409000,China
  • Received:2023-01-12 Revised:2023-02-13 Online:2023-03-15 Published:2023-03-15
  • About author:CHEN Liang,born in 1977,Ph.D,associate professor,master's supervisor,is a member of China Computer Federation.His main research interests include industrial big data,big data analysis and visualization.
    WANG Lu,born in 1996,postgraduate.Her main research interests include data visualization and image identification.
  • Supported by:
    National Natural Science Foundation of China(51675108) and Key Scientific Research Program of the Education Department of Shaanxi Province,China(22JS021).

摘要: 仪表板是支持制造企业开展数据可视化分析和经营决策的重要手段。为解决可视化仪表板设计与实现过程中用户对专业知识依赖性强及流程迭代繁杂的问题,提出了一种基于深度学习技术YOLOv5s算法的可视化仪表板自动识别与生成方法。首先,基于YOLOv5s算法对仪表板图像及手绘草图中包含的可视化图表组件进行检测,并针对在检测过程中出现的手绘草图中不规则线条对识别图表的干扰及误检等问题,引入CA注意力机制来增强模型对重要特征的关注及目标精确定位能力,从而提高模型的识别精度;其次,将图表检测模型部署在Web中,服务器根据模型检测结果调用封装好的可视化图表组件代码,生成多组件组合的初始仪表板;最后,基于此Web设计,开发了一款数据可视化仪表板构建平台,为用户提供可修改配置仪表板样式及数据的详细选项,以方便用户快速构建完整的仪表板。通过收集Tableau,Power BI等可视化工具产生的仪表板图像及企业应用过程中手绘仪表板草图形成数据集,基于该数据集进行实验验证,改进的模型识别精度比原YOLOv5s模型提升了2.1%,mAP为98.4%,并通过系统部署应用验证了图表识别方法及开发的平台可有效识别及生成相应图表组件,满足用户的基本需求。

关键词: 可视化, 深度学习, 目标检测, 仪表板生成

Abstract: Dashboard is an important tool to support manufacturing enterprises in data visualization analysis and business decision-making.In order to solve the problems of users’ strong dependence on professional knowledge and complicated process iteration in the design and implementation of visual dashboards,a method for automatic recognition and generation of visual dashboards based on the YOLOv5s algorithm in deep learning technology is proposed.Firstly,based on the YOLOv5s algorithm,the visual chart components contained in the dashboard images and hand-drawn sketches are detected,and in order to address the problems of interference and false detection caused by irregular lines in hand-drawn sketches during the detection process,the CA attention mechanism is introduced to enhance the ability of the model to focus on important features and accurately locate the target,so as to improve the recognition accuracy of the model.Secondly,deploy the chart detection model in the web,and the server calls the encapsulated visual chart component code according to the model detection results to generate the initial dashboard of multi-component combination.Finally,a data visualization dashboard building platform is developed based on this web design,which provides users with detailed options to modify and configure the dashboard style and data,so that users can quickly build a complete dashboard.Through the collection of dashboard images generated by visualization tools such as Tableau and Power BI and hand-drawn dashboard sketches during the enterprise application process to form a dataset for experimental validation,the improved model increases the recognition accuracy by 2.1% compared to the original YOLOv5s model,and the mAP is 98.4%.The system deployment application verifies that the chart recognition method and the developed platform can effectively identify and generate the corresponding chart components to meet the basic needs of users.

Key words: Visualization, Deep learning, Object detection, Dashboard generation

中图分类号: 

  • TP391.41
[1]COOK K A,THOMAS J J.Illuminating the path:The research and development agenda for visual analytics[R].Richland,WA(United States):Pacific Northwest National Lab,2005.
[2]SARIKAYA A,CORRELL M,BARTRAM L,et al.What do we talk about when we talk about dashboards? [J].IEEE Transactions on Visualization and Computer Graphics,2018,25(1):682-692.
[3]KEIM E D,KOHLHAMMER J,ELLIS G.Mastering the Information Age:Solving Problems with Visual Analytics[M]// Goslar:Eurographics Association,2010.
[4]GRAMMEL L,TORY M,STOREY M A.How information visualization novices construct visualizations [J].IEEE Transactions on Visualization and Computer Graphics,2010,16(6):943-952.
[5]VÁZQUEZ-INGELMO A,GARCÍA-PEEÑALVO F J,THERÓN R,et al.Connecting domain-specific features to source code:Towards the automatization of dashboard generation [J].Cluster Computing,2020,23(3):1803-1816.
[6]DA COL S,CIUCANU R,SOARE M,et al.DashBot:An ML-Guided Dashboard Generation System[C]//Proceedings of the 30th ACM International Conference on Information & Know-ledge Management.New York:Association for Computing Machinery,2021:4696-4700.
[7]AKSU Ü,DEL-RÍO-ORTEGA A,RESINAS M,et al.An approach for the automated generation of engaging dashboards[C]//Proceedings of OTM Confederated International Confe-rences on the Move to Meaningful Internet Systems.Cham:Springer,2019:363-384.
[8]ELIAS M,BEZERIANOS A.Exploration views:understanding dashboard creation and customization for visualization novices[C]//Proceedings of IFIP Conference on Human-Computer Interaction.Berlin:Springer,2011:274-291.
[9]BIGELOW A,DRUCKER S,FISHER D,et al.Iterating be-tween tools to create and edit visualizations [J].IEEE Transactions on Visualization and Computer Graphics,2016,23(1):481-490.
[10]ELSHEHALY M,RANDELL R,BREHMER M,et al.QualDash:Adaptable generation of visualisation dashboards for healthcare quality improvement [J].IEEE Transactions on Visualization and Computer Graphics,2020,27(2):689-699.
[11]SAVVA M,KONG N,CHHAJTA A,et al.Revision:Automated classification,analysis and redesign of chart images[C]//Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology.New York,United States:Association for Computing Machinery,2011:393-402.
[12]LIU C,LAI C F,JIANG R K,et al.Visualization driven by deep learning [J].Journal of Computer Aided Design and Graphics,2020,32(10):1537-1548.
[13]MA R,MEI H,GUAN H,et al.Ladv:Deep learning assisted authoring of dashboard visualizations from images and sketches [J].IEEE Transactions on Visualization and Computer Gra-phics,2020,27(9):3717-3732.
[14]AZIZ L,SALAM M S B H,SHEIKH U U,et al.Exploring deep learning-based architecture,strategies,applications and current trends in generic object detection:A comprehensive review [J].IEEE Access,2020,8:170461-170495.
[15]JIAO L,ZHANG F,LIU F,et al.A survey of deep learning-based object detection [J].IEEE Access,2019,7:128837-128868.
[16]REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:Unified,real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:779-788.
[17]LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature pyramidnetworks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:2117-2125.
[18]LIU S,QI L,QIN H,et al.Path aggregation network for instance segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2018:8759-8768.
[19]HOU Q,ZHOU D,FENG J.Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2021:13713-13722.
[20]REZATOFIGHI H,TSOI N,GWAK J Y,et al.Generalized intersection over union:A metric and a loss for bounding box regression[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2019:658-666.
[21]ZHOU X,WANG D,KRÄHENBÜHL P.Objects as points [J].arXiv:1904.07850,2019.
[22]LIU W,ANGUELOV D,ERHAN D,et al.Ssd:Single shotmultibox detector[C]//Proceedings of European Conference on Computer Vision.Cham:Springer,2016:21-37.
[23]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:2980-2988.
[24]REN S,HE K,GIRSHICK R,et al.Faster R-CNN:TowardsReal-Time Object Detection with Region Proposal Networks [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137-1149.
[1] 董永峰, 黄港, 薛婉若, 李林昊.
融合IRT的图注意力深度知识追踪模型
Graph Attention Deep Knowledge Tracing Model Integrated with IRT
计算机科学, 2023, 50(3): 173-180. https://doi.org/10.11896/jsjkx.211200134
[2] 华晓凤, 冯娜, 于俊清, 何云峰.
基于规则推理的足球视频任意球射门事件检测
Shooting Event Detection of Free Kick in Soccer Video Based on Rule Reasoning
计算机科学, 2023, 50(3): 181-190. https://doi.org/10.11896/jsjkx.220300062
[3] 梅鹏程, 杨吉斌, 张强, 黄翔.
一种基于三维卷积的声学事件联合估计方法
Sound Event Joint Estimation Method Based on Three-dimension Convolution
计算机科学, 2023, 50(3): 191-198. https://doi.org/10.11896/jsjkx.220500259
[4] 白雪飞, 马亚楠, 王文剑.
基于特征融合的边缘引导乳腺超声图像分割方法
Segmentation Method of Edge-guided Breast Ultrasound Images Based on Feature Fusion
计算机科学, 2023, 50(3): 199-207. https://doi.org/10.11896/jsjkx.211200294
[5] 刘航, 普园媛, 吕大华, 赵征鹏, 徐丹, 钱文华.
极化自注意力约束颜色溢出的图像自动上色
Polarized Self-attention Constrains Color Overflow in Automatic Coloring of Image
计算机科学, 2023, 50(3): 208-215. https://doi.org/10.11896/jsjkx.220100149
[6] 张卫良, 陈秀宏.
跨层融合和感受野扩增的SSD目标检测算法
SSD Object Detection Algorithm with Cross-layer Fusion and Receptive Field Amplification
计算机科学, 2023, 50(3): 231-237. https://doi.org/10.11896/jsjkx.211100281
[7] 张译, 吴秦.
特征增强损失与前景注意力人群计数网络
Crowd Counting Network Based on Feature Enhancement Loss and Foreground Attention
计算机科学, 2023, 50(3): 246-253. https://doi.org/10.11896/jsjkx.220100219
[8] 应宗浩, 吴槟.
深度学习模型的后门攻击研究综述
Backdoor Attack on Deep Learning Models:A Survey
计算机科学, 2023, 50(3): 333-350. https://doi.org/10.11896/jsjkx.220600031
[9] 邹芸竹, 杜圣东, 滕飞, 李天瑞.
一种基于多模态深度特征融合的视觉问答模型
Visual Question Answering Model Based on Multi-modal Deep Feature Fusion
计算机科学, 2023, 50(2): 123-129. https://doi.org/10.11896/jsjkx.211200303
[10] 王鹏宇, 台文鑫, 刘芳, 钟婷, 罗绪成, 周帆.
基于数据增强的自监督飞行航迹预测
Self-supervised Flight Trajectory Prediction Based on Data Augmentation
计算机科学, 2023, 50(2): 130-137. https://doi.org/10.11896/jsjkx.211200016
[11] 郭楠, 李婧源, 任曦.
基于深度学习的刚体位姿估计方法综述
Survey of Rigid Object Pose Estimation Algorithms Based on Deep Learning
计算机科学, 2023, 50(2): 178-189. https://doi.org/10.11896/jsjkx.211200164
[12] 李俊林, 欧阳智, 杜逆索.
基于改进区域候选网络的场景文本检测
Scene Text Detection with Improved Region Proposal Network
计算机科学, 2023, 50(2): 201-208. https://doi.org/10.11896/jsjkx.211000191
[13] 华杰, 刘学亮, 赵烨.
基于特征融合的小样本目标检测
Few-shot Object Detection Based on Feature Fusion
计算机科学, 2023, 50(2): 209-213. https://doi.org/10.11896/jsjkx.220500153
[14] 商迪, 吕彦锋, 乔红.
受人脑中记忆机制启发的增量目标检测方法
Incremental Object Detection Inspired by Memory Mechanisms in Brain
计算机科学, 2023, 50(2): 267-274. https://doi.org/10.11896/jsjkx.220900212
[15] 梁佳利, 华保健, 苏少博.
融合循环划分的张量指令生成优化
Tensor Instruction Generation Optimization Fusing with Loop Partitioning
计算机科学, 2023, 50(2): 374-383. https://doi.org/10.11896/jsjkx.220300147
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!