Computer Science ›› 2020, Vol. 47 ›› Issue (6): 85-91.doi: 10.11896/jsjkx.190500085

• Databωe & Big Data & Data Science • Previous Articles     Next Articles

Data Composition View Positioning Update Approach with Incremental Logs

ZHANG Yuan-ming, LI Meng-ni, HUANG Lang-you, LU Jia-wei, XIAO Gang   

  1. College of Computer Science and Technology,Zhejiang University of Technology,Hangzhou 310023,China
  • Received:2019-05-17 Online:2020-06-15 Published:2020-06-10
  • About author:ZHANG Yuan-ming,born in 1977,Ph.D, associate professor.His main research interests include service computing,cloud computing and big data processing.
  • Supported by:
    This work was supported by the Major Science and Technology Projects of Zhejiang Province (2018C01064) and Natural Science Foundation of Zhejiang Province(LY19F020034)

Abstract: Data resources stored in different units and departments in cloud environment are cross-domain,heterogeneous and complex.As a unified data model for cross-origin and heterogeneous data sources,data service can publish data sources in the form of services,and generate data composition view by composing several data services according to users’ data requirements.Since the data sources are autonomous,it becomes a key issue to update data composition view in real time with minimal cost.This paper proposes a data composition view positioning update approach based on incremental logs.The latest data changes of data sources are captured according to incremental logs,and then the attributes and tuples in data composition view are indexed.The index numbers of different tuples can be calculated with positioning attributes.The corresponding tuple update operations can be performed according to data changes’ type.A log-based update data acquisition algorithm and a data composition view positioning update algorithm are presented.The proposed approach has been evaluated in a cross-origin heterogeneous elevator data service system by using datasets from multiple departments.When the proportion of the number of changed tuples is much smaller than the total number of tuples,the update efficiency of positioning update approach is much higher than existing methods.When the number of attributes of the data composition view is larger,the update efficiency of the positioning update approach is much higher than existing methods.

Key words: Data composition view, Data service, Incremental logs, Positioning update, Real time

CLC Number: 

  • TP311
[1]LEMOS A L,DANIEL F,BENATALLAH B.Web service composition[J].ACM Computing Surveys,2016,48(3):1-41.
[2]CAREY M J,ONOSE N,PETROPOULOS M.Data services [J].Communications of the ACM,2012,55(6):86.
[3]KURZADKAR S,BAJPAYEE A.Anatomization of miscellaneous approaches for selection and maintenance of Materialized view[C]//2015 IEEE 9th International Conference on Intelligent Systems and Control (ISCO).IEEE,2015:1-5.
[4]TERZO O,RUIU P,BUCCI E,et al.Data as a service (DaaS) for sharing and processing of large data collections in the cloud[C]//2013 Seventh International Conference on Complex,Intelligent,and Software Intensive Systems.IEEE,2013:475-480.
[5]ZHANG Y M,YE C L,HUANG L Y,et al.Research on data service dependency graph model and automatic composition[J].Journal of Chinese Computer Systems,2018,39(3):450-456.
[6]WEN Y,LIU C,HAN Y B.iViewer:service-based view construction method for just-in-time sharing business data across organizations[J].Journal of Frontiers of Computer Science & Technology,2012,6(3):221-236.
[7]GUPTA A,MUMICK I S.Materialized views[M].The MIT Press,1999.
[8]ZHANG P,HAN Y B,WANG G L.Implementing dynamic nested view update based on data service[J].Chinese Journal of Computers,2013,36(2):226-237.
[9]RAGOTHAMAN P,PANDA B.Analyzing transaction logs for effective damage assessment[M]//Research Directions in Data and Applications Security.Boston,MA:Springer US,2003:89-101.
[10]ZOU X X,JIA W J,PAN J H.Research of log-based change data capture[J].Journal of Chinese Computer Systems,2012,33(3):531-536.
[11]CAREY M,REVELIOTIS P,THATTE S,et al.Data service modeling in the AquaLogic data services platform[C]//2008 IEEE Congress on Services.IEEE,2008:78-80.
[12]ZHANG Y,ZHU L M,XU X W,et al.Data service API design for data analytics[M]//Services Computing-SCC 2018.Cham:Springer International Publishing,2018:87-102.
[13]ALTINEL M,BROWN P,CLINE S,et al.Damia:A Data Mashup Fabric for Intranet Applications[C]//Proceedings of the 33rd International Conference on Very Large Data Bases.Vienna:VLDB Endowment,2007:1370-1373.
[14]ZHANG P,WANG G L,JI G,et al.Optimization update for data composition view based on data service[J].Chinese Journal of Computers,2011,34(12):2344-2354.
[15]CAO B,BADIA A.SQL query optimization through nested relational algebra[J].ACM Transactions on Database Systems,2007,32(3):18.
[16]HAN Y B,WANG G L,JI G,et al.Situational data integration with data services and nested table[J].Service Oriented Computing and Applications,2013,7(2):129-150.
[17]AMDOUNI S,BARHAMGI M,BENSLIMANE D,et al.Handling uncertainty in data services composition[C]//2014 IEEE International Conference on Services Computing.IEEE,2014:653-660.
[18]MALKI A,BARHAMGI M,BENSLIMANE S M,et al.Composing data services with uncertain semantics[J].IEEETran-sactions on Knowledge and Data Engineering,2015,27(4):936-949.
[19]VU Q H,PHAM T V,TRUONG H L,et al.DEMODS:a description model for data-as-a-service[C]//2012 IEEE 26th International Conference on Advanced Information Networking and Applications.IEEE,2012:605-612.
[20]LIU X,HU C,LI Y,et al.The advanced data service architecture for modern enterprise information system[C]//2014 International Conference on Information Science & Applications (ICISA).IEEE,2014:1-4.
[21]ZHANG Z J,ZHANG Y M,LU J W,et al.CMfgIA:a cloud manufacturing application mode for industry alliance[J].The International Journal of Advanced Manufacturing Technology,2018,98(9/10/11/12):2967-2985.
[1] LI Jia-rui, LING Xiao-bo, LI Chen-xi, LI Zi-mu, YANG Jia-hai, ZHANG Lei, WU Cheng-nan. Dynamic Network Security Analysis Based on Bayesian Attack Graphs [J]. Computer Science, 2022, 49(3): 62-69.
[2] QIAN Guang-ming, YI Chao. Real Time Wireless Connection Scheme for Multi-nodes [J]. Computer Science, 2021, 48(11A): 446-451.
[3] LIU Yu-hong,LIU Shu-ying,FU Fu-xiang. Optimization of Compressed Sensing Reconstruction Algorithms Based on Convolutional Neural Network [J]. Computer Science, 2020, 47(3): 143-148.
[4] ZENG Lei, LI Hao, LIN Yu-fei, ZHANG Shuai. Study on Simulation Optimization of Gazebo Based on Asynchronous Mechanism [J]. Computer Science, 2020, 47(11A): 593-598.
[5] ZHANG Zhou, HUANG Guo-rui, JIN Pei-quan. Task Scheduling on Storm:Current Situations and Research Prospects [J]. Computer Science, 2019, 46(9): 28-35.
[6] PANG Yu, LIU Ping, LEI Yin-jie. Realization of “Uncontrolled” Object Recognition Algorithm Based on Mobile Terminal [J]. Computer Science, 2019, 46(6A): 153-157.
[7] GUANJian, WANG Jing-bin, BIAN Qian-hong. Multi-keyword Streaming Parallel Retrieval Algorithm Based on Urban Security Knowledge Graph [J]. Computer Science, 2019, 46(2): 35-41.
[8] TAI Ying-peng, XING Ke-xin, LIN Ye-gui and ZHANG Wen-an. Research of Path Planning in Multi-AGV System [J]. Computer Science, 2017, 44(Z11): 84-87.
[9] HUANG Zhong-ping, BAI Guang-wei, SHEN Hang, CHENG Xiao and HUA Zhi-xiang. Speculative Execution Optimization Algorithm with MapReduce [J]. Computer Science, 2017, 44(4): 193-196.
[10] TANG Xiao-chun and TIAN Kai-fei. Validity Protection Strategy for Real Time Data in CPS Based on Semantics [J]. Computer Science, 2017, 44(12): 11-16.
[11] LIU Chun-yao and ZHANG Li-chen. Dynamic Multi-priority Scheduling for Cyber-physical Systems [J]. Computer Science, 2015, 42(1): 28-32.
[12] SONG Cui-ye, DU Cheng-lie and LI Gang. Indicator-effect Modeling and Evaluation for CPS System Based on Extended DPN [J]. Computer Science, 2015, 42(1): 23-27.
[13] WANG Gui-ling,CAO Bo,ZHANG Sai,GENG Mei-zhen and ZHANG Feng. User-steered Application Building Environment for Situational Data Integration [J]. Computer Science, 2014, 41(9): 96-100.
[14] DONG Chang-cheng,WU Wei,LI Jun-lin and ZHAO Heng. Research on the QoS Control of the Real-time Data Transmission Based on the Tactical Data Link [J]. Computer Science, 2013, 40(Z6): 283-286.
[15] WU Qian and ZHOU Qing. Implementation of Message Transmission of IEC61850Standard Based on Embedded Real Time Operating System of VxWorks [J]. Computer Science, 2013, 40(8): 86-89.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!