基于深度学习的可视化仪表板生成技术研究

doi:10.11896/jsjkx.230100064

Abstract

Abstract: Dashboard is an important tool to support manufacturing enterprises in data visualization analysis and business decision-making.In order to solve the problems of users’ strong dependence on professional knowledge and complicated process iteration in the design and implementation of visual dashboards,a method for automatic recognition and generation of visual dashboards based on the YOLOv5s algorithm in deep learning technology is proposed.Firstly,based on the YOLOv5s algorithm,the visual chart components contained in the dashboard images and hand-drawn sketches are detected,and in order to address the problems of interference and false detection caused by irregular lines in hand-drawn sketches during the detection process,the CA attention mechanism is introduced to enhance the ability of the model to focus on important features and accurately locate the target,so as to improve the recognition accuracy of the model.Secondly,deploy the chart detection model in the web,and the server calls the encapsulated visual chart component code according to the model detection results to generate the initial dashboard of multi-component combination.Finally,a data visualization dashboard building platform is developed based on this web design,which provides users with detailed options to modify and configure the dashboard style and data,so that users can quickly build a complete dashboard.Through the collection of dashboard images generated by visualization tools such as Tableau and Power BI and hand-drawn dashboard sketches during the enterprise application process to form a dataset for experimental validation,the improved model increases the recognition accuracy by 2.1% compared to the original YOLOv5s model,and the mAP is 98.4%.The system deployment application verifies that the chart recognition method and the developed platform can effectively identify and generate the corresponding chart components to meet the basic needs of users.

Key words: Visualization, Deep learning, Object detection, Dashboard generation

CLC Number:

TP391.41

CHEN Liang, WANG Lu, LI Shengchun, LIU Changhong. Study on Visual Dashboard Generation Technology Based on Deep Learning[J].Computer Science, 2023, 50(3): 238-245.

References

[1]COOK K A,THOMAS J J.Illuminating the path:The research and development agenda for visual analytics[R].Richland,WA(United States):Pacific Northwest National Lab,2005.
[2]SARIKAYA A,CORRELL M,BARTRAM L,et al.What do we talk about when we talk about dashboards? [J].IEEE Transactions on Visualization and Computer Graphics,2018,25(1):682-692.
[3]KEIM E D,KOHLHAMMER J,ELLIS G.Mastering the Information Age:Solving Problems with Visual Analytics[M]// Goslar:Eurographics Association,2010.
[4]GRAMMEL L,TORY M,STOREY M A.How information visualization novices construct visualizations [J].IEEE Transactions on Visualization and Computer Graphics,2010,16(6):943-952.
[5]VÁZQUEZ-INGELMO A,GARCÍA-PEEÑALVO F J,THERÓN R,et al.Connecting domain-specific features to source code:Towards the automatization of dashboard generation [J].Cluster Computing,2020,23(3):1803-1816.
[6]DA COL S,CIUCANU R,SOARE M,et al.DashBot:An ML-Guided Dashboard Generation System[C]//Proceedings of the 30th ACM International Conference on Information & Know-ledge Management.New York:Association for Computing Machinery,2021:4696-4700.
[7]AKSU Ü,DEL-RÍO-ORTEGA A,RESINAS M,et al.An approach for the automated generation of engaging dashboards[C]//Proceedings of OTM Confederated International Confe-rences on the Move to Meaningful Internet Systems.Cham:Springer,2019:363-384.
[8]ELIAS M,BEZERIANOS A.Exploration views:understanding dashboard creation and customization for visualization novices[C]//Proceedings of IFIP Conference on Human-Computer Interaction.Berlin:Springer,2011:274-291.
[9]BIGELOW A,DRUCKER S,FISHER D,et al.Iterating be-tween tools to create and edit visualizations [J].IEEE Transactions on Visualization and Computer Graphics,2016,23(1):481-490.
[10]ELSHEHALY M,RANDELL R,BREHMER M,et al.QualDash:Adaptable generation of visualisation dashboards for healthcare quality improvement [J].IEEE Transactions on Visualization and Computer Graphics,2020,27(2):689-699.
[11]SAVVA M,KONG N,CHHAJTA A,et al.Revision:Automated classification,analysis and redesign of chart images[C]//Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology.New York,United States:Association for Computing Machinery,2011:393-402.
[12]LIU C,LAI C F,JIANG R K,et al.Visualization driven by deep learning [J].Journal of Computer Aided Design and Graphics,2020,32(10):1537-1548.
[13]MA R,MEI H,GUAN H,et al.Ladv:Deep learning assisted authoring of dashboard visualizations from images and sketches [J].IEEE Transactions on Visualization and Computer Gra-phics,2020,27(9):3717-3732.
[14]AZIZ L,SALAM M S B H,SHEIKH U U,et al.Exploring deep learning-based architecture,strategies,applications and current trends in generic object detection:A comprehensive review [J].IEEE Access,2020,8:170461-170495.
[15]JIAO L,ZHANG F,LIU F,et al.A survey of deep learning-based object detection [J].IEEE Access,2019,7:128837-128868.
[16]REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:Unified,real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:779-788.
[17]LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature pyramidnetworks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:2117-2125.
[18]LIU S,QI L,QIN H,et al.Path aggregation network for instance segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2018:8759-8768.
[19]HOU Q,ZHOU D,FENG J.Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2021:13713-13722.
[20]REZATOFIGHI H,TSOI N,GWAK J Y,et al.Generalized intersection over union:A metric and a loss for bounding box regression[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2019:658-666.
[21]ZHOU X,WANG D,KRÄHENBÜHL P.Objects as points [J].arXiv:1904.07850,2019.
[22]LIU W,ANGUELOV D,ERHAN D,et al.Ssd:Single shotmultibox detector[C]//Proceedings of European Conference on Computer Vision.Cham:Springer,2016:21-37.
[23]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:2980-2988.
[24]REN S,HE K,GIRSHICK R,et al.Faster R-CNN:TowardsReal-Time Object Detection with Region Proposal Networks [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137-1149.

Related Articles 15

[1]	DONG Yongfeng, HUANG Gang, XUE Wanruo, LI Linhao. Graph Attention Deep Knowledge Tracing Model Integrated with IRT [J]. Computer Science, 2023, 50(3): 173-180.
[2]	HUA Xiaofeng, FENG Na, YU Junqing, HE Yunfeng. Shooting Event Detection of Free Kick in Soccer Video Based on Rule Reasoning [J]. Computer Science, 2023, 50(3): 181-190.
[3]	MEI Pengcheng, YANG Jibin, ZHANG Qiang, HUANG Xiang. Sound Event Joint Estimation Method Based on Three-dimension Convolution [J]. Computer Science, 2023, 50(3): 191-198.
[4]	BAI Xuefei, MA Yanan, WANG Wenjian. Segmentation Method of Edge-guided Breast Ultrasound Images Based on Feature Fusion [J]. Computer Science, 2023, 50(3): 199-207.
[5]	LIU Hang, PU Yuanyuan, LYU Dahua, ZHAO Zhengpeng, XU Dan, QIAN Wenhua. Polarized Self-attention Constrains Color Overflow in Automatic Coloring of Image [J]. Computer Science, 2023, 50(3): 208-215.
[6]	ZHANG Weiliang, CHEN Xiuhong. SSD Object Detection Algorithm with Cross-layer Fusion and Receptive Field Amplification [J]. Computer Science, 2023, 50(3): 231-237.
[7]	ZHANG Yi, WU Qin. Crowd Counting Network Based on Feature Enhancement Loss and Foreground Attention [J]. Computer Science, 2023, 50(3): 246-253.
[8]	YING Zonghao, WU Bin. Backdoor Attack on Deep Learning Models:A Survey [J]. Computer Science, 2023, 50(3): 333-350.
[9]	ZOU Yunzhu, DU Shengdong, TENG Fei, LI Tianrui. Visual Question Answering Model Based on Multi-modal Deep Feature Fusion [J]. Computer Science, 2023, 50(2): 123-129.
[10]	WANG Pengyu, TAI Wenxin, LIU Fang, ZHONG Ting, LUO Xucheng, ZHOU Fan. Self-supervised Flight Trajectory Prediction Based on Data Augmentation [J]. Computer Science, 2023, 50(2): 130-137.
[11]	GUO Nan, LI Jingyuan, REN Xi. Survey of Rigid Object Pose Estimation Algorithms Based on Deep Learning [J]. Computer Science, 2023, 50(2): 178-189.
[12]	LI Junlin, OUYANG Zhi, DU Nisuo. Scene Text Detection with Improved Region Proposal Network [J]. Computer Science, 2023, 50(2): 201-208.
[13]	HUA Jie, LIU Xueliang, ZHAO Ye. Few-shot Object Detection Based on Feature Fusion [J]. Computer Science, 2023, 50(2): 209-213.
[14]	SHANG Di, LYU Yanfeng, QIAO Hong. Incremental Object Detection Inspired by Memory Mechanisms in Brain [J]. Computer Science, 2023, 50(2): 267-274.
[15]	LIANG Jiali, HUA Baojian, SU Shaobo. Tensor Instruction Generation Optimization Fusing with Loop Partitioning [J]. Computer Science, 2023, 50(2): 374-383.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Study on Visual Dashboard Generation Technology Based on Deep Learning

PDF (PC)