Computer Science ›› 2025, Vol. 52 ›› Issue (1): 131-141.doi: 10.11896/jsjkx.231200079

• Database & Big Data & Data Science • Previous Articles     Next Articles

Study on Collaborative Data Persistence in NewSQL Databases

ZUO Shun1, LI Yongkun2, XU Yinlong2   

  1. 1 School of Computer Science and Technology,University of Science and Technology of China,Hefei 230026,China
    2 Anhui Province Key Laboratory of High Performance Computing,Hefei 230026,China
  • Received:2023-12-12 Revised:2024-05-22 Online:2025-01-15 Published:2025-01-09
  • About author:ZUO Shun,born in 1999,postgraduate.His main research interests include distributed databases and data persistence.
    LI Yongkun,born in 1986,professor,is a member of CCF(No.33184M).His main research interests include data storage and file systems,especially storage and system problems in application scenarios such as cloud computing and big data,virtualized memory management,key-value systems,flash,NVRAM.
  • Supported by:
    National Natural Science Foundation of China(62172382).

Abstract: To ensure high availability of data,modern NewSQL databases often create several copies of data so that it can be accessed from other copies in case one copy is not available.With multiple data copies,it is essential to consider data consistency between them.This means that the results should be the same when different clients read the same data at a particular moment.Therefore,a transaction processing mechanism is introduced.In the interactive transactional process with multiple write operations,each write operation must be performed on both the primary and backup copies of the data,since there are multiple copies.However,the primary and backup replicas are typically located on different machines,resulting in increased latency when writing to remote replicas,which in turn can ultimately lead to an increase in the processing latency of the entire transaction.In this paper,we present a collaborative data persistence scheme where the client caches the transaction write logs locally.When the transaction is finally committed,the client firstly persists the write logs of the transaction and sends the logs to the coordinator node of the transaction to allow the coordinator to distribute the log data,so as to achieve the purpose of the two cooperating in persistence of the transaction data.Experimental results show that in comparison to the synchronous persistence scheme,cooperative persistence scheme can not only reduce the latency of interactive transaction processing,but also improve the system limit throughput rate by roughly 38%.

Key words: Distributed database, Concurrency control, Data persistence, Data consistency, High-contention workload

CLC Number: 

  • TP311
[1]DANIELSEN A.The evolution of data models and approachesto persistence in database systems[J/OL].https://www.fing.edu.uy/inco/grupos/csi/esp/Cursos/cursos_act/2000/DAP_DisAvDB/documentacion/OO/Evol_DataModels.html.
[2]FATIMA H,WASNIK K.Comparison of SQL,NoSQL and NewSQL databases for internet of things[C]//2016 IEEE Bombay Section Symposium(IBSS).IEEE,2016:1-6.
[3]BINANI S,GUTTI A,UPADHYAY S.SQL vs.NoSQL vs.New-SQL-a comparative study[J].Database,2016,6(1):1-4.
[4]MONIRUZZAMAN A,HOSSAIN S.Nosql database:New era of databases for big data analytics-classification,characteristics and comparison[J].arXiv:1307.0191,2013.
[5]TAFT R,SHARIF I,MATEI A,et al.Cockroachdb:The resi-lient geo-distributed sql database[C]//Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data.2020:1493-1509.
[6]HUANG D,LIU Q,CUI Q,et al.TiDB:a Raft-based HTAPdatabase[J].Proceedings of the VLDB Endowment,2020,13(12):3072-3084.
[7]CORBETT J C,DEAN J,EPSTEIN M,et al.Spanner:Google’s globally distributed database[J].ACM Transactions on Computer Systems(TOCS),2013,31(3):1-22.
[8]Hewlett Packard Corporation.The machine:A new kind of computer[EB/OL].https://www.hpl.hp.com/research/systems-research/themachine/.
[9]Intel Corporation.Intel rack scale design architecture[EB/OL].https://www.intel.com/content/www/us/en/architecture-and-technology/rack-scale-design-overview.html,2021.
[10]MARCOS K A,NADAV A,IRINA C,et al.Remote regions:a simple abstraction for remote memory[C]//2018 USENIX Annual Technical Conference.2018:775-787.
[11]DU X Y,LI T,LU W,et al.Cross-domain Data Management[J].Computer Science,2024,51(1):4-12.
[12]YAN X,YANG L,ZHANG H,et al.Carousel:Low-latencytransaction processing for globally-distributed data[C]//Proceedings of the 2018 International Conference on Management of Data.2018:231-243.
[13]MU S,NELSON L,LLOYD W,et al.Consolidating concurrency control and consensus for commits under conflicts[C]//12th USENIX Symposium on Operating Systems Design and Implementation(OSDI 16).2016:517-532.
[14]REN K,LI D,ABADI D J.Slog:Serializable,low-latency,geo-replicated transactions[J].Proceedings of the VLDB Endowment,2019,12(11):1747-1761.
[15]ONGARO D,OUSTERHOUT J.In search of an understandable consensus algorithm[C]//2014 USENIX Annual Technical Conference(USENIX ATC 14).2014:305-319.
[16]LAMPORT L.Paxos made simple[J/OL].https://lamport.azurewebsites.net/pubs/paxos-simple.pdf.
[17]YIZHOU S,YUTONG H,YILUN C,et al.Legoos:A disseminated,distributed OS for hardware resource disaggregation[C]//13th USENIX Symposium on Operating Systems Design and Implementation.2018:69-87.
[18]PENGFEI Z,JIAZHAO S,LIU Y,et al.One-sided rdma-con-scious extendible hashing for disaggregated memory[C]//2021 USENIX Annual Technical Conference.2021:15-29.
[19]Futurewei Cloud.Chogori-Platform[EB/OL].https://github.com/futureweicloud/chogori-platform.
[20]General HBase tuning[EB/OL].https://www.ibm.com/docs/en/db2-big-sql/5.0.2?topic=performance-general-hbase-tu-ning.
[21]Cockroachlabs.How Pipelining consensus writes speeds up distributed SQL transactions[EB/OL].https://www.cockroach-
labs.com/blog/transaction-pipelining/.
[22]CARLSON J.Redis in action[M].Simon and Schuster,2013.
[23]DONG S,KRYCZKA A,JIN Y,et al.Rocksdb:Evolution of development priorities in a key-value store serving large-scale applications[J].ACM Transactions on Storage(TOS),2021,17(4):1-32.
[24]LevelDB[EB/OL].https://github.com/google/leveldb.
[25]THOMSON A,DIAMOND T,WENG S C,et al.Calvin:fastdistributed transactions for partitioned database systems[C]//Proceedings of the 2012 ACM SIGMOD International Confe-rence on Management of Data.2012:1-12.
[26]FunnaDB[EB/OL].https://fauna.com/blog/inside-faunas-distributed-transaction-engine-dte.
[27]CHEN Y,YU X,KOUTRIS P,et al.Plor:General transactions with predictable,low tail latency[C]//Proceedings of the 2022 International Conference on Management of Data.2022:19-33.
[28]ZHOU X,YU X,GRAEFE G,et al.Lotus:scalable multi-partition transactions on single-threaded partitioned databases[J].Proceedings of the VLDB Endowment,2022,15(11):2939-2952.
[1] YU Xin-yi, WANG Xu-yan, YING Hao-zhe, OU Lin-lin. Design of Low-latency Remote Serial Communication System [J]. Computer Science, 2021, 48(6A): 432-437.
[2] LING Fei, CHEN Shi-ping. Shared Digital Credits Management Mechanism of Enterprise Alliance Based on Blockchain [J]. Computer Science, 2021, 48(11A): 533-539.
[3] LIAO Bin, ZHANG Tao, LI Min, YU Jiong, GUO Bing-lei, LIU Yan. Consistency Checking Algorithm for Distributed Key-Value Database Based on Operation History Graph [J]. Computer Science, 2019, 46(12): 213-219.
[4] WANG Huan, ZHANG Yun-feng and ZHANG Yan. Rapid Decision Method for Repairing Sequence Based on CFDs [J]. Computer Science, 2018, 45(3): 311-316.
[5] SUN Zhi-long, Edwin H-M Sha, ZHUGE Qing-feng, CHEN Xian-zhang and WU Kai-jie. Research on Data Consistency for In-memory File Systems [J]. Computer Science, 2017, 44(2): 222-227.
[6] XIONG Zhong-min, ZHU Ji-guang, LI Chang-ji and YUAN Hong-chun. Confluence Decision Method for Active Rule Set Based on Condition Conflict Analysis [J]. Computer Science, 2016, 43(5): 169-173.
[7] FAN Bi-jian and ZHUANG Yi. Adaptive Concurrency Control Algorithm Based on Conflict-rate Prediction [J]. Computer Science, 2016, 43(11): 280-283.
[8] CHEN Yi-rui and ZHUANG Yi. Concurrency Control Algorithm Based on Dynamic Decision [J]. Computer Science, 2015, 42(Z6): 1-4.
[9] . Data Consistency Trust Evaluation of Wireless Sensor Networks Based on Multi-event Concurrent [J]. Computer Science, 2013, 40(3): 163-166.
[10] . Cloud Data Storage Security and Privacy Protection Policies under IoT Environment [J]. Computer Science, 2012, 39(5): 62-65.
[11] LIU Lin,XIONG Qi,WU Shi-zhong. Survey of Data Consistency Insurance Technologies for Continuous Data Protection [J]. Computer Science, 2011, 38(Z10): 124-127.
[12] LI Jie. Research and Application of Lightweight Data Persistence Technology Based on ORM [J]. Computer Science, 2010, 37(9): 190-193.
[13] . [J]. Computer Science, 2009, 36(4): 172-174.
[14] WU Hai, CHEN Wei, LU Yan-sheng (Computer College, Huazhong University of Science and Technology, Wuhan 430074, China). [J]. Computer Science, 2009, 36(2): 155-157.
[15] LI Guo-Hui, YANG Bing ,XIANG Jun ,CHEN Hui (School of Computer Science ~ Technology, Huazhong University of Science & Technology,Wuhan 430074). [J]. Computer Science, 2008, 35(4): 54-59.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!