Computer Science ›› 2019, Vol. 46 ›› Issue (6A): 412-416.

• Big Data & Data Mining • Previous Articles     Next Articles

Common Issues and Case Analysis of System Data Migration

LU Ye-shan   

  1. School of Computer and Electronic Information,Guangxi University,Nanning 530001,China
  • Online:2019-06-14 Published:2019-07-02

Abstract: With the development of society and the rapid change of technical framework,it has become a trend for the daily system to replace the old system with the new one.The replacement of the old system with the new system will inevitably involve the data docking between the old system and the new system.In the system construction of an organization in a city,the project needs to migrate all business data of the old system to the new system.Due to the inconsistency of table space,table structure and table field between the old and new systems,in order to ensure the consistency and integrity of data,ensure that data before and after migration is not missing,and ensure that dirty data does not migrate to affect the operation of the new system,how to migrate data between the old and new systems has become a top priority in the project.In order to solve the problem of data migration,this paper designs a data migration process based on ETL tools,and obtains a complete data migration process line through combination and series connection,thus realizing data migration to complete data docking between old and new systems.This paper elaborates the following problems and solutions for data migration:1)Common errors and solutions in data flow.2)Data migration problems and solutions with inconsistent data types.3)Inconsistent length of field in target database of data migration and solutions.4)How to re-change the original data when data migration is completed Problems and solutions of adjusting migration measurement.Based on this,this paper makes a brief analysis and summary of the problems in the process of data migration and the countermeasures to solve these problems.

Key words: Data migration, Data transfer, ETL tools, SQL statement

CLC Number: 

  • TP311
[1]亓开元,赵卓峰,房俊,等.针对高速数据流的大规模数据实时处理方法.计算机学报,2012,35(3):477-490.
董媛香.基于软集合的不完备不一致数据分析及决策方法研究.重庆:重庆大学,2014:18.
郭迟,刘经南,方媛,等.位置大数据的价值提取与协同挖掘方法.软件学报,2014,25(4):713-730.
张晓冉,舒昝.面向ETL数据处理过程的优化研究.自动化与仪器仪表,2016(7):109-110,109.
李云松.小微企业数据整合中ETL工具Kettle的应用.菏泽学院学报,2014,36(2):19-22.
金翰伟.基于Spark的大数据清洗框架设计与实现.杭州:浙江大学,2016.
郭志懋,周傲英.数据质量和数据清洗研究综述.软件学报,2002(11):2076-2082.
王元卓,靳小龙,程学旗.网络大数据:现状与展望.计算机学报,2013,36(6):1125-1138.
张宁,贾自艳,史忠植.数据仓库中ETL技术的研究.计算机工程与应用,2002(24):213-216.
伍江瑶.广东电网数据回流系统ETL的设计与实现.自动化与仪器仪表,2017(6):80-83.
张瑞.ETL数据抽取研究综述.软件导刊,2010,9(10):164-165.
徐俊刚,裴莹.数据ETL研究综述.计算机科学,2011,38(4):15-20.
车燕京,张月清,韩宪忠.ETL转换功能的设计与实现.河北农业大学学报,2010,33(3):132-135.
章水鑫,徐宏炳,于立.增量式ETL工具的研究与实现.现代计算机(专业版),2005(3):6-10.
[1] CAO He-xin, ZHAO Liang, LI Xue-feng. Technical Research of Graph Neural Network for Text-to-SQL Parsing [J]. Computer Science, 2022, 49(4): 110-115.
[2] HAN Ye-fei, BAI Guang-wei, ZHANG Gong-xuan. Routing Optimization Algorithm of Wireless Sensor Network Based on Improved SVM [J]. Computer Science, 2018, 45(8): 131-133.
[3] ZHANG Yong, ZHANG Jie-hui and LIU Bin. Big Data Dynamic Migration Method Based on Global Load Balancing in Cloud Environment [J]. Computer Science, 2018, 45(1): 196-199.
[4] YANG Peng, ZHAO Hui and BAO Zhong-gui. Analysis on Fitting Model of Network Covert Timing Channel [J]. Computer Science, 2017, 44(1): 145-148.
[5] ZHANG Wu-neng, LI Hong-wei, SHEN Li-wei and ZHAO Wen-yun. Method of Code Evolution Recommendation for Database Schema Change [J]. Computer Science, 2016, 43(2): 216-223.
[6] ZHENG Sheng and LI Tong. Data Placement Algorithm for Large-scale Storage System [J]. Computer Science, 2013, 40(Z11): 270-273.
[7] SHI Guang-yuan and ZHANG Yu. Hierarchical Storage Access Model Based on Multi-Attributes Measurement [J]. Computer Science, 2013, 40(Z11): 165-169.
[8] LUO Xiang-yu,WANG Yun and CHEN Xiao-mei. Evaluation and Analysis of Load Balancing Mechanisms in Storage Systems [J]. Computer Science, 2013, 40(9): 55-60.
[9] . Research on Evidence Collection under Cloud Computing Environment [J]. Computer Science, 2012, 39(9): 105-108.
[10] WANG Jun-he,GAO Fei,WU Jie-lin. Point-to-Point Wireless Data Acquisition Based on CC1100 [J]. Computer Science, 2011, 38(Z10): 212-214.
[11] GE Xiong-zi,FENG Dan,LU Cheng-tao,JIN Chao. Dynamic Analysis Model of Green Network Storage Systems [J]. Computer Science, 2011, 38(8): 291-296.
[12] LIU Ke,QIN Lei-hua,ZHOU Jing-li,NIE Xue-jun,ZENG Dong. Two-phrase Retrieval Strategy in Content Aware Network Storage System [J]. Computer Science, 2011, 38(5): 20-23.
[13] REN Xun-yi, WANG Ru-chuan, ZHANG Yu. NGridFTP:A Grid Data Transfer Tool Based on GridFTP [J]. Computer Science, 2009, 36(9): 148-150.
[14] DOU Jun, ZENG Hua-shen, CHEN Wen-jia (School of Information Science and Technology, Southwest Jiaotong University, Chengdu 610031, China). [J]. Computer Science, 2009, 36(4): 112-115.
[15] DOU Jun ZENG Hua-Shen WANG Hai-Ying (School of Information Science and Technology, Southwest Jiaotong University, Chengclu 610031). [J]. Computer Science, 2008, 35(3): 31-33.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!