计算机科学 ›› 2020, Vol. 47 ›› Issue (6): 66-73.doi: 10.11896/jsjkx.191000072

• 数据库&大数据&数据科学 • 上一篇    下一篇

大数据分解-融合及其智能获取

刘纪芹1, 史开泉2   

  1. 1 山东财经大学数学与数量经济学院 济南250014
    2 山东大学数学学院 济南250100
  • 收稿日期:2019-10-12 出版日期:2020-06-15 发布日期:2020-06-10
  • 通讯作者: 史开泉(shikq@sdu.edu.cn)
  • 作者简介:sdfiljq@126.com
  • 基金资助:
    国家社会科学基金项目(71663010);山东省自然科学基金项目(zr2013aq019)

Big Data Decomposition-Fusion and Its Intelligent Acquisition

LIU Ji-qin1, SHI Kai-quan2   

  1. 1 School of Mathematics and Quantitative Economics,Shandong University of Finance and Economics,Jinan 250014,China
    2 School of Mathematics,Shandong University,Jinan 250100,China
  • Received:2019-10-12 Online:2020-06-15 Published:2020-06-10
  • About author:IU Ji-qin,born in 1968,Ph.D,professor.Her main research interests include big data intelligent analysis and application and rough system theory and application.
    SHI Kai-quan,born in 1945,professor,Ph.D supervisor.His main research interests include big data theory and application,information intelligent system.He proposed S-rough sets,function S-rough sets,P-sets,inverse P-sets,function P-sets and function inverse P-sets.
  • Supported by:
    This work was supported by the National Social Science Foundation of China (71663010) and Natural Science Foundation of Shandong Province, China (zr2013aq019)

摘要: 文中给出了通过大数据分解、融合生成的大数据分解-融合以及大数据距离;利用这些概念,给出了大数据并-交分解定理以及大数据交-并分解定理与它们的属性合取关系、大数据融合的智能生成定理与大数据融合的距离关系、大数据分解-融合的识别准则与大数据分解-融合获取的智能算法与算法的过程,以及这些理论结果在大数据分解-融合智能获取的应用。文中给出了∧型大数据新的特征,∧型大数据是利用P-集合模型得到的。

关键词: 大数据, 分解-融合, 识别准则, 智能算法

Abstract: The concepts of big data decomposition-fusion and big data distance generated by big data decomposition and fusion are given.By using these concepts,an union-intersection decomposition theorem of big data,an intersection-union decomposition theorem of big data and their attribute conjunction relation are given.Intelligent generation theorems and the distance relationship of big data fusion are given.A recognition criterion of big data decomposition-fusion,an intelligent algorithm and algorithm process of big data decomposition-fusion acquisition are given.The application of these theoretical results in big data decomposition-fusion intelligent acquisition is presented.In this paper,the new characteristics of ∧-type big data are given,∧-type big data is obtained by using P-sets model.

Key words: Big data, Decomposition-Fusion, Intelligent algorithm, Recognition criterion

中图分类号: 

  • O144
[1]SHI K Q.Big data structure-logic characteristics and big data law [J].Journal of Shandong University (Natural Science),2019,54(2):1-29.
[2]WANG F Y,CARLEY K M,ZENG D,et al.Social computing:from social informatics to social intelligence [J].IEEE Intelligent systems,2007,22(2):79-83.
[3]ZHANG N,WANG F Y,ZHU F,et al.DynaCAS:Computational experiments and decision support for ITS [J].IEEE Intelligent Systems,2008,23(6):19-23.
[4]WANG F Y.Parallel control and management for intelligent transportation systems:Concepts,architectures,and applications[J].IEEE Transactions on Intelligent Transportation Systems,2010,11(3):630-638.
[5]WANG F Y.Scanning the issue and beyond:Parallel driving with software vehicular robots for safety and smartness [J].IEEE Transactions on Intelligent Transportation Systems,2014,15(4):1381-1387.
[6]LV Y,DUAN Y,KANG W,et al.Traffic flow prediction with big data:A deep learning approach [J].IEEE Trans.Intelligent Transportation Systems,2015,16(2):865-873.
[7]LV Y,ZHANG X,KANG W,et al.Managing emergency traffic evacuation with a partially random destination allocation strategy:a computational-experiment-based optimization approach[J].IEEE Trans.Intelligent Transportation Systems,2015,16(4):2182-2191.
[8]YUAN Y,WANG F Y,ZENG D.Developing a cooperative bidding framework for sponsored search markets-An evolutionary perspective [J].Information Sciences,2016,369:674-689.
[9]WANG F Y.The emergence of intelligent enterprises:From CPS to CPSS [J].IEEE Intelligent Systems,2010,25(4):85-88.
[10]GOODHOPE K,KOSHY J,KREPS J,et al.Building LinkedIn’s real-time activity data pipeline [J].IEEE Data Eng.Bull.,2012,35(2):33-45.
[11]MCKUSICK K,QUINLAN S.GFS:evolution on fastforward [J].Communications of the ACM,2010,53(3):42-49.
[12]COOPER B F,RAMAKRISHNAN R,SRIVASTAVA U,et al.PNUTS:Yahoo!’s hosted data serving platform [J].Procee-dings of the VLDB Endowment,2008,1(2):1277-1288.
[13]MELNIK S,GUBAREV A,LONG J J,et al.Dremel:interactive analysis of web-scale datasets [J].Proceedings of the VLDB Endowment,2010,3(1/2):330-339.
[14]BU Y,HOWE B,BALAZINSKA M,et al.HaLoop:efficient iterative data processing on large clusters [J].Proceedings of the VLDB Endowment,2010,3(1/2):285-296.
[15]LAM W,LIU L,PRASAD S T S,et al.Muppet:MapReducestyle processing of fast data [J].Proceedings of the VLDB Endowment,2012,5(12):1814-1825.
[16]CHEN S.CHEETAH:a high performance,custom data warehouse on top of MapReduce [J].Proceedings of the VLDB Endowment,2010,3(1/2):1459-1468.
[17]CANIM M,MIHAILA G A,BHATTACHARJEE B,et al.SSD bufferpool extensions for database systems [J].Proceedings of the VLDB Endowment,2010,3(1/2):1435-1446.
[18]LUO T,LEE R,MESNIER M,et al.Storage-DB:heterogeneity-aware data management to exploit the full capability of hybrid storage systems [J].Proceedings of the VLDB Endowment,2012,5(10):1076-1087.
[19]WONG P C,SHEN H W,JOHNSON C R,et al.The top 10 challenges in extreme-scale visual analytics[J].IEEE computer graphics and applications,2012,32(4):63-67.
[20]PIKE R,DORWARD S,GRIESEMER R,et al.Interpreting the data:parallel analysis with Sawzall [J].Scientific Programming,2005,13(4):277-298.
[21]SHI K Q.P-sets [J].Journal of Shandong University (Natural Science),2008,43(11):77-84.
[22]SHI K Q.P-sets and its applications [J].Advances in Systems Science and Applications,2009,9(2):209-219.
[23]SHI K Q.P-sets and its applied characteristics [J].Computer Science,2010,37(8):1-8.
[24]SHI K Q.P-reasoning and P-reasoning discoveryidentification of information [J].Computer Science,2011,38(7):1-9.
[25]SHI K Q.P-sets,inverse P-sets and the intelligent fusionfilter identification of information [J].Computer Science,2012,39(4):1-13.
[26]FAN C X,LIN H K.P-sets and the reasoningidentification of disaster information [J].International Journal of Convergence Information Technology,2012,7(1):337-345.
[27]LIN H K,FAN C X.The dual form of P-reasoning and identification of unknown attribute [J].International Journal of Digital Content Technology and its Applications,2012,6(1):121-131.
[28]ZHANG L,CUI Y Q,SHI K Q.Outer P-sets and data internal recovery [J].Systems Engineering and Electronics,2010,32(6):1233-1238.
[29]WANG Y,GENG H Q,SHI K Q.P-sets and dependencediscovery of dynamic information [J].Systems Engineering and Electronics,2011,33(9):2035-2038.
[30]LI Y Y,LIN H K,SHI K Q.Characteristics of data discrete interval and data discovery-application [J].Systems Engineering and Electronics,2011,33(10):2258-2262.
[31]ZHANG L,TANG J H,SHI K Q.The fusion of internal packet information and its feature of attribute conjunction [J].Journal of Shandong University(Natural Science),2014,49(2):93-97.
[32]SHI K Q,ZHANG L.Internal P-set and data outerrecovery[J].Journal of Shandong University (Natural Science),2009,44(4):8-14.
[33]ZHANG G Y,ZHOU H Y,SHI K Q.P-sets and the recoveryidentification double [J].Systems Engineering and Electronics,2010,32(9):1919-1924.
[34]ZHANG L,REN X F.P-sets and (f,f)-heredity [J].Quantitative Logic and Soft Computing,2010,2(1):735-743.
[35]ZHANG L,XIU M,SHI K Q.P-sets and application of internal-outer data circle [J].Quantitative Logic and Soft Computing,2010,2(1):581-591.
[36]QIU Y F,CHEN B H.f-model generated by P-sets [J].Quantitative Logic and Soft Computing,2010,2(1):613-620.
[37]LI Y Y,ZHANG L,SHI K Q.Generation and recovery of compressed data and redundant data[J].Quantitative Logic and Soft Computing,2010,2(1):661-671.
[38]XIU M,SHI K Q,ZHANG L.P-sets and F-data selection-discovery [J].Quantitative Logic and Soft Computing,2010,2(1):791-799.
[39]ZHAO S L,FAN C X,SHI K Q.Outer P-information generation and its reasoning-searching discovery[J].Journal of Shandong University(Natural Science),2012,47(1):99-104.
[40]ZHAO S L,WU S L,SHI K Q.Internal Preasoning information recovery and attribute hiding searching discovery [J].Computer Science,2013,40(4):209-213.
[41]SHI K Q.Function P-sets[J].Journal of Shandong University (Natural Science),2011,46(2):62-69.
[42]SHI K Q.Function P-sets[J].International Journal of Machine Learning and Cybernetics,2011,2(4):281-288.
[43]SHI K Q.P-information law intelligent fusion and soft information image intelligent generation[J].Journal of Shandong University(Natural Science),2014,49(4):1-17.
[44]TANG J H,ZHANG L,SHI K Q.Intelligent fusion of information law and its inner separation[J].Computer Science,2015,42(2):204-209.
[45]LIN R,FAN C X.Function P-sets and dynamic characteristics of information law[J].Journal of Shandong University (Natural Science),2012,47(1):121-126.
[46]REN X F,ZHANG L,SHI K Q.Two types of dynamic information law models and their applications in information camouflage and risk identification[J].Computer Science,2018,45(9):230-236.
[47]CHEN B H,ZHANG L,SHI K Q.Intelligent dynamic fusion of packet information and the intelligent state recognition of information law[J].Journal of Shandong University (Natural Science),2018,53(2):83-87.
[48]TANG J H,ZHANG L,SHI K Q,et al.Outer Pinformation law reasoning andits application in intelligent fusion and separating of information law[J].Microsystem Technologies,2018,24(10):4389-4398.
[49]SHI K Q.P-augmented matrix and dynamic intelligent discovery-identification of information[J].Journal of Shandong University (Natural Science),2015,50(10):1-12.
[50]LIN H K,FAN C X.Embedding camouflage of inverse P-information and application[J].International Journal of Convergence Information and Technology,2012,7(20):471-480.
[51]FAN C X,HUANG S L.Inverse P-reasoning discovery identification of inverse P-information[J].International Journal of Di-gital Content Technology and its Applications,2012,6(20):735-744.
[52]SHI K Q.Function inverse P-sets and the hiding information generated by function inverse P-information law fusion[C]// Proceedings of the 13th IFIP WG 6.11 Conference on e-Business,e-Services,and e-Society,Sanya,China,2014:224-237.
[53]GUO H L,REN X F,ZHANG L.Relationships between dynamic data mining and P-augmented matrix[J].Journal of Shandong University (Natural Science),2016,51(8):105-110.
[54]GUO H L,CHEN B H,TANG J H.Inverse P-sets and intelligent fusion mining-discovery of information[J].Journal of Shandong University (Natural Science),2013,48(8):97-103,110.
[55]ZHANG L,REN X F,SHI K Q.The dynamic segmentation characteristics of P-augmented matrix and the dynamic intelligent acquisition of P-information[J].International Journal of Applied Decision Sciences,2016,9(4):413-425.
[56]ZHANG L,REN X F,SHI K Q,et al.Inverse packet matrix reasoning model-based the intelligent dynamic separation and acquisition of educational information[J].Microsystem Technologies,2018,24(10):4415-4421.
[57]REN X F,ZHANG L,SHI K Q,et al.Inverse P-augmented matrix method-based the dynamic findings of unknown information[J].Microsystem Technologies,2018,24(10):4187-4192.
[58]ZHANG L,REN X F,SHI K Q.Inverse P-information law models and the reality-camouflage intelligent transformations of information image[C]//Proceedings of the 2016 International Conference on Network and Information Systems for Compu-ters,Washington:IEEE,2016:337-341.
[59]ZHANG L,REN X F,SHI K Q.Inverse P-data models and data intelligent separation[C]//Proceedings of the 2016 International Conference on Electronic Information Technology and Intellectualization,DOI:10.12783/dtcse/iceiti2016/6107,2016.
[60]REN X F,ZHANG L,SHI K Q.Surplusdeficiency of cardinal number and inverse P-augmented matrices[J].Journal of Shandong University (Natural Science),2015,50(10):13-18,26.
[61]ZHANG L,REN X F.Surplus-deficient theorem of cardinal number and data internal-outer miningseparation [J].Journal of Shandong University (Natural Science),2015,50(8):90-94.
[62]GUO H L,ZHANG L.Data separation and its attribute state characteristics [J].Journal of Shandong University (Natural Science),2017,52(12):89-94.
[63]TANG J H,CHEN B H.Intelligent fusion of internal inverse packet information and expansion relationship of attribute disjunction[J].Journal of Shandong University (Natural Science),2014,49(2):89-92,97.
[64]ZHANG L,REN X F,SHI K Q.Intelligent switchcamouflage of information laws and P-law augmented matrices [J].Journal of Shandong University (Natural Science),2016,51(8):90-97.
[65]REN X F,ZHANG L,SHI K Q.Boundary Characteristics of Inverse P-sets and System Condition Monitoring[J].Computer Science,2016,43(10):211-213,255.
[66]REN X F,ZHANG L.Perturbation theorems of inverse P-sets and perturbation-based data mining[J].Journal of Shandong University (Natural Science),2016,51(12):54-60.
[67]LIU J Q,ZHANG H Y.Information P-dependence and P-dependence mining-sieving [J].Computer Science,2018,45(7):202-206.
[68]SHI K Q,YAO B X.Function S-rough sets and law identification[J].Science in China E:Information Science,2008,38(4):553-564.
[69]SHI K Q,ZHAO J L.Function S-rough sets and security-authentication of hiding law[J].Science in China E:Information Science,2008,38(8):1234-1243.
[70]SHI K Q,YAO B X.Function S-rough sets and law identification[J].Science in China F:Information Science,2008,51(5):499-510.
[71]SHI K Q,ZHAO J L.Function S-rough sets and security-authentication of hiding law[J].Science in China F:information Sciences,2008,51(7):924-935.
[1] 陈晶, 吴玲玲.
多源异构环境下的车联网大数据混合属性特征检测方法
Mixed Attribute Feature Detection Method of Internet of Vehicles Big Datain Multi-source Heterogeneous Environment
计算机科学, 2022, 49(8): 108-112. https://doi.org/10.11896/jsjkx.220300273
[2] 何强, 尹震宇, 黄敏, 王兴伟, 王源田, 崔硕, 赵勇.
基于大数据的进化网络影响力分析研究综述
Survey of Influence Analysis of Evolutionary Network Based on Big Data
计算机科学, 2022, 49(8): 1-11. https://doi.org/10.11896/jsjkx.210700240
[3] 周天清, 岳亚莉.
超密集物联网络中多任务多步计算卸载算法研究
Multi-Task and Multi-Step Computation Offloading in Ultra-dense IoT Networks
计算机科学, 2022, 49(6): 12-18. https://doi.org/10.11896/jsjkx.211200147
[4] 孙轩, 王焕骁.
政务大数据安全防护能力建设:基于技术和管理视角的探讨
Capability Building for Government Big Data Safety Protection:Discussions from Technologicaland Management Perspectives
计算机科学, 2022, 49(4): 67-73. https://doi.org/10.11896/jsjkx.211000010
[5] 王美珊, 姚兰, 高福祥, 徐军灿.
面向医疗集值数据的差分隐私保护技术研究
Study on Differential Privacy Protection for Medical Set-Valued Data
计算机科学, 2022, 49(4): 362-368. https://doi.org/10.11896/jsjkx.210300032
[6] 王俊, 王修来, 庞威, 赵鸿飞.
面向科技前瞻预测的大数据治理研究
Research on Big Data Governance for Science and Technology Forecast
计算机科学, 2021, 48(9): 36-42. https://doi.org/10.11896/jsjkx.210500207
[7] 余乐章, 夏天宇, 荆一楠, 何震瀛, 王晓阳.
面向大数据分析的智能交互向导系统
Smart Interactive Guide System for Big Data Analytics
计算机科学, 2021, 48(9): 110-117. https://doi.org/10.11896/jsjkx.200900083
[8] 王立梅, 朱旭光, 汪德嘉, 张勇, 邢春晓.
基于深度学习的民事案件判决结果分类方法研究
Study on Judicial Data Classification Method Based on Natural Language Processing Technologies
计算机科学, 2021, 48(8): 80-85. https://doi.org/10.11896/jsjkx.210300130
[9] 张蔷, 黄樟灿, 谈庆, 李华峰, 湛航.
基于动态近邻套索算子的金字塔演化策略
Pyramid Evolution Strategy Based on Dynamic Neighbor Lasso
计算机科学, 2021, 48(6): 215-221. https://doi.org/10.11896/jsjkx.200400115
[10] 王雪岑, 张昱, 刘迎婕, 于戈.
基于表示学习的在线学习交互质量评价方法
Evaluation of Quality of Interaction in Online Learning Based on Representation Learning
计算机科学, 2021, 48(2): 207-211. https://doi.org/10.11896/jsjkx.201000042
[11] 滕建, 滕飞, 李天瑞.
基于3D卷积和LSTM编码解码的出行需求预测
Travel Demand Forecasting Based on 3D Convolution and LSTM Encoder-Decoder
计算机科学, 2021, 48(12): 195-203. https://doi.org/10.11896/jsjkx.210400022
[12] 张育龙, 王强, 陈明康, 孙静涛.
图像去雨算法在云物联网应用中的研究综述
Survey of Intelligent Rain Removal Algorithms for Cloud-IoT Systems
计算机科学, 2021, 48(12): 231-242. https://doi.org/10.11896/jsjkx.201000055
[13] 曹萌, 于洋, 梁英, 史红周.
基于区块链的大数据交易关键技术与发展趋势
Key Technologies and Development Trends of Big Data Trade Based on Blockchain
计算机科学, 2021, 48(11A): 184-190. https://doi.org/10.11896/jsjkx.210100163
[14] 刘亚臣, 黄雪莹.
卫星监测时空大数据蠕变特征提取及预警算法
Research on Creep Feature Extraction and Early Warning Algorithm Based on Satellite MonitoringSpatial-Temporal Big Data
计算机科学, 2021, 48(11A): 258-264. https://doi.org/10.11896/jsjkx.201000071
[15] 王梓强, 胡晓光, 李晓筱, 杜卓群.
移动机器人全局路径规划算法综述
Overview of Global Path Planning Algorithms for Mobile Robots
计算机科学, 2021, 48(10): 19-29. https://doi.org/10.11896/jsjkx.200700114
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!