Computer Science ›› 2016, Vol. 43 ›› Issue (Z11): 93-96.doi: 10.11896/j.issn.1002-137X.2016.11A.020

Previous Articles     Next Articles

Spark Based Large-scale Semantic Data Distributed Reasoning Framework

CHEN Heng   

  • Online:2018-12-01 Published:2018-12-01

Abstract: With the emergence of large-scale semantic data,the study of efficient parallel semantic reasoning has already become a hot topic.In terms of scalability,most of the existing reasoning frameworks still have deficiencies,so it is hard to meet the needs of large-scale semantic data.To solve this problem,a distributed reasoning framework for large-scale semantic data based on Spark was proposed in this paper,which is composed of 3 modules,including semantic modeling,rule extraction and Spark-based parallel reasoning.The result of the process analysis and reasoning instance reveals that computing performance of the proposed distributed parallel reasoning (T(n)=O(log2n)) is far better than that of sequential reasoning (T(n)=O(n)).

Key words: Spark,Parallel semantic reasoning,Distributed framework,Semantic big data

[1] W3C.Resource Description Framework[EB/OL].[2015-01-25].http://www.w3.org/RDF
[2] W3C.Linking open data[EB/OL].[2015-01-25].http://www.w3.org/wiki/SweoIG/TaskForces/CommunityProjects/Linking-OpenData
[3] Dean J,Ghemawat S.MapReduce:simplified data processing on large clusters[J].Communications of the ACM,2008,51(1):107-113
[4] 顾荣,王芳芳,袁春风,等.YARM:基于MapReduce的高效可扩展的语义推理引擎[J].计算机学报,2015,38(1):74-85
[5] 陈曦,陈华钧,顾珮嵌,等.一种基于Hadoop的语义大数据分布式推理框架[J].计算机研究与发展,2013(2):103-113
[6] Urbani J,Kotoulas S,Oren E,et al.Scalable distributed reaso-ning using mapreduce[C]∥Proceedings of the 8th International Semantic Web Conference.Washington,USA,2009:634-649
[7] Mika P,Tummarello G.Web semantics in the clouds[J].Intelligent Systems,IEEE,2008,23(5):82-87
[8] The Apache Software Foundation.Spark[EB/OL].[2016-02-01].https://spark.apache.org
[9] Urbani J,Kotoulas S,Maassen J,et al.WebPIE:A Web-scale parallel inference engine using MapReduce[J].Web Semantics:Science,Services and Agents on the World Wide Web,2012,10:59-75
[10] Broekstra J,Kampman A,Van Harmelen F.Sesame:A generic architecture for storing and querying RDF and RDF schema[C]∥Proceedings of the 1st International Semantic Web Conference on The Semantic Web.Sardinia,Italy,2002:54-68
[11] Kaoudi Z,Koubarakis M.Distributed RDFS reasoning overstructured overlay networks[J].Journal on Data Semantics,2013,2(4):189-227
[12] Fang Q,Zhao Y,Yang G,et al.Scalable distributed ontologyreasoning using DHT-based partitioning[C]∥Proceedings of the 3rd Asian Semantic Web Conf.Berlin:Springer,2008:91-105
[13] Soma R,Prasanna V K.Parallel inferencing for OWL knowledge bases[C]∥Processing ofthe 37th International Conference on Parallel Processing.Portland,USA,2008:75-82
[14] Weaver J,Hendler J A.Parallel materialization of the finite rdfs closure for hundreds of millions of triples[C]∥Proceedings of the 8th International Semantic Web Conference.Washington,USA,2009:682-697
[15] Kotoulas S,Oren E,Van Harmelen F.Mind the data skew:distributed inferencing by speeddating in elastic regions[C]∥Proceedings of the 19th International Conference on World Wide Web.ACM,2010:531-540
[16] Tsatsanifos G,Sacharidis D,Sellis T.On enhancing scalability for distributed RDF/S stores[C]∥Proceedings of the 14th International Conference on Extending Database Technology.Uppsala,Sweden,2011:141-152
[17] Muhleisen H,Dentler K.Large-scale storage and reasoning for semantic data using swarms[J].IEEE Computational Intelligence Magazine,2012,7(2):32-44
[18] Wu H,Liu J,Ye D,et al.Scalable Horn-Like rule inference of semantic data using MapReduce[C]∥Proceedings of the 7th International Conference(KSEM 2014).Sibiu,Romania,2014:270-277
[19] Wu H,Liu J,Ye D,et al.A distributed rule execution mechanism based on MapReduce in sematic web reasoning[C]∥Proceedings of the 5th Asia-Pacific Symposium on Internetware.ACM,2013:6
[20] Liu B,Huang K,Li J,et al.An incremental and distributed inference method for large-scale ontologies based on MapReduce paradigm[J].IEEE Transactions on Cybernetics,2015,45(1):53-64
[21] Gu R,Wang S,Wang F,et al.Cichlid:Efficient Large ScaleRDFS/OWL Reasoning with Spark[C]∥2015 IEEE International Parallel and Distributed Processing Symposium (IPDPS).IEEE,2015:700-709
[22] Kim J M,Park Y T.Scalable OWL-Horst ontology reasoningusing SPARK[C]∥2015 International Conference on Big Data and Smart Computing (BigComp).IEEE,2015:79-86
[23] Zaharia M,Chowdhury M,Franklin M J,et al.Spark:clustercomputing with working sets[J].HotCloud,2010(10):10

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!