A large dataset for the evaluation of ontology matching

Fausto Giunchiglia; Mikalai Yatskevich; Paolo Avesani; Pavel Shivaiko; Fausto Giunchiglia; Mikalai Yatskevich; Paolo Avesani; Pavel Shivaiko

doi:10.1017/S026988890900023X

Abstract: Recently, the number of ontology matching techniques and systems has increased significantly. This makes the issue of their evaluation and comparison more severe. One of the challenges of the ontology matching evaluation is in building large-scale evaluation datasets. In fact, the number of possible correspondences between two ontologies grows quadratically with respect to the numbers of entities in these ontologies. This often makes the manual construction of the evaluation datasets demanding to the point of being infeasible for large-scale matching tasks. In this paper, we present an ontology matching evaluation dataset composed of thousands of matching tasks, called TaxME2. It was built semi-automatically out of the Google, Yahoo, and Looksmart web directories. We evaluated TaxME2 by exploiting the results of almost two-dozen of state-of-the-art ontology matching systems. The experiments indicate that the dataset possesses the desired key properties, namely it is error-free, incremental, discriminative, monotonic, and hard for the state-of-the-art ontology matching systems.

Other Articles By Authors

A large dataset for the evaluation of ontology matching

Department of Information Engineering and Computer Science (DISI)

Via Sommarive 18, 38050 Povo, Trento

Published online: 01 June 2009

Abstract: Abstract: Recently, the number of ontology matching techniques and systems has increased significantly. This makes the issue of their evaluation and comparison more severe. One of the challenges of the ontology matching evaluation is in building large-scale evaluation datasets. In fact, the number of possible correspondences between two ontologies grows quadratically with respect to the numbers of entities in these ontologies. This often makes the manual construction of the evaluation datasets demanding to the point of being infeasible for large-scale matching tasks. In this paper, we present an ontology matching evaluation dataset composed of thousands of matching tasks, called TaxME2. It was built semi-automatically out of the Google, Yahoo, and Looksmart web directories. We evaluated TaxME2 by exploiting the results of almost two-dozen of state-of-the-art ontology matching systems. The experiments indicate that the dataset possesses the desired key properties, namely it is error-free, incremental, discriminative, monotonic, and hard for the state-of-the-art ontology matching systems.

HTML

Acknowledgments

We appreciate support from the Knowledge Web14 European Network of Excellence (IST-2004-507482) and the OpenKnowledge15 European STREP (FP6-027253). We are grateful to Jérôme Euzenat for many fruitful discussions on the topic of this paper.

http://www.google.com/Top/

http://dir.yahoo.com/

See http://www.ontologymatching.org for complete information on the topic.

http://oaei.ontologymatching.org/

The abbreviation TaxME stands for ‘TAXonomy Mapping Evaluation’, though in this paper, we use the terminology of Euzenat and Shvaiko (2007) and keep this abbreviation as originally introduced in Avesani et al. (2005) for historical reasons.

The complete results can be found as follows: OAEI-2005: http://oaei.ontologymatching.org/2005/results/ OAEI-2006: http://oaei.ontologymatching.org/2006/results/ OAEI-2007: http://oaei.ontologymatching.org/2007/results/

See also http://www.ontologymatching.org/

http://www.unspsc.org

http://www.eclass.de

http://oaei.ontologymatching.org/

http://www.cancer.gov/cancerinfo/terminologyresources/

http://www.informatics.jax.org/searches/AMA_form.shtml

http://www.atl.external.lmco.com/projects/ontology/i3con.html

http://www.knowledgeweb.semanticweb.org

http://openk.org

Rights and permissions

References (49)

About this article

Cite this article

Fausto Giunchiglia, Mikalai Yatskevich, Paolo Avesani, Pavel Shivaiko. 2009. A large dataset for the evaluation of ontology matching. The Knowledge Engineering Review. 24: doi: 10.1017/S026988890900023X

Fausto Giunchiglia, Mikalai Yatskevich, Paolo Avesani, Pavel Shivaiko. 2009. A large dataset for the evaluation of ontology matching. The Knowledge Engineering Review. 24: doi: 10.1017/S026988890900023X

{{lists.name}}

A large dataset for the evaluation of ontology matching

Abstract

Rights and permissions

References

About this article

Cite this article

Article Metrics

Access History

Other Articles By Authors