Developing a sustainable platform for entity annotation benchmarks

Michael Röder; Ricardo Usbeck; Axel Cyrille Ngonga

doi:10.1007/978-3-319-25639-9_36

Developing a sustainable platform for entity annotation benchmarks

Michael Röder^*
, Ricardo Usbeck
, Axel Cyrille Ngonga

^*Korrespondierende/r Autor/-in für diese Arbeit

Universität Leipzig

Publikation: Beiträge in Sammelwerken › Aufsätze in Konferenzbänden › Forschung › Begutachtung

1 Zitat (Scopus)

Abstract

The existing entity annotation systems that drive the extraction of RDF from unstructured data are hard to compare as their evaluation relies on different data sets and measures. We developed GERBIL, an evaluation framework for semantic entity annotation that provides developers, end users and researchers with easy-to-use interfaces for the agile, fine-grained and uniform evaluation of 9 annotation tools on 11 different data sets within 6 different experimental settings on 6 different measures. In this paper, we present the developed interfaces, data flows and data structures. Moreover, we show how GERBIL supports a better reproducibility and archiving of experimental results.

Originalsprache	Englisch
Titel	The Semantic Web : ESWC 2015 Satellite Events - ESWC 2015 Satellite Events
Redakteure/-innen	Antoine Zimmermann, Serena Villata, John Breslin, Catherine Faron-Zucker, Fabien Gandon, Christophe Guéret
Seitenumfang	7
Herausgeber (Verlag)	Springer International Publishing
Erscheinungsdatum	01.01.2015
Seiten	190-196
ISBN (Print)	978-3-319-25638-2
ISBN (elektronisch)	978-3-319-25639-9
DOIs	https://doi.org/10.1007/978-3-319-25639-9_36
Publikationsstatus	Erschienen - 01.01.2015
Extern publiziert	Ja
Veranstaltung	12th European Semantic Web Conference - ESWC 2015 - Portoroz, Slowenien Dauer: 31.05.2015 → 04.06.2015 Konferenznummer: 12 https://2015.eswc-conferences.org/index.html https://2015.eswc-conferences.org/call-challenges.html

Bibliographische Notiz

Funding Information:
Parts of this work were supported by the FP7 project GeoKnow (GA No. 318159) and the BMWi project SAKE (GA No. 01MD15006E).

Publisher Copyright:
© Springer International Publishing Switzerland 2015.

Fachgebiete und Schlagwörter

Informatik
Wirtschaftsinformatik

Zugriff auf Dokument

10.1007/978-3-319-25639-9_36Lizenz: CC BY-NC

Dieses zitieren

Röder, M., Usbeck, R., & Ngonga, A. C. (2015). Developing a sustainable platform for entity annotation benchmarks. in A. Zimmermann, S. Villata, J. Breslin, C. Faron-Zucker, F. Gandon, & C. Guéret (Hrsg.), The Semantic Web: ESWC 2015 Satellite Events - ESWC 2015 Satellite Events (S. 190-196). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Band 9341). Springer International Publishing. https://doi.org/10.1007/978-3-319-25639-9_36

Röder, Michael ; Usbeck, Ricardo ; Ngonga, Axel Cyrille. / Developing a sustainable platform for entity annotation benchmarks. The Semantic Web: ESWC 2015 Satellite Events - ESWC 2015 Satellite Events. Hrsg. / Antoine Zimmermann ; Serena Villata ; John Breslin ; Catherine Faron-Zucker ; Fabien Gandon ; Christophe Guéret. Springer International Publishing, 2015. S. 190-196 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{86701dc859244fe29d80a029d89556db,

title = "Developing a sustainable platform for entity annotation benchmarks",

abstract = "The existing entity annotation systems that drive the extraction of RDF from unstructured data are hard to compare as their evaluation relies on different data sets and measures. We developed GERBIL, an evaluation framework for semantic entity annotation that provides developers, end users and researchers with easy-to-use interfaces for the agile, fine-grained and uniform evaluation of 9 annotation tools on 11 different data sets within 6 different experimental settings on 6 different measures. In this paper, we present the developed interfaces, data flows and data structures. Moreover, we show how GERBIL supports a better reproducibility and archiving of experimental results.",

keywords = "Informatics, Business informatics",

author = "Michael R{\"o}der and Ricardo Usbeck and Ngonga, \{Axel Cyrille\}",

note = "Parts of this work were supported by the FP7 project GeoKnow (GA No. 318159) and the BMWi project SAKE (GA No. 01MD15006E). Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2015.; 12th European Semantic Web Conference - ESWC 2015, ESWC 2015 ; Conference date: 31-05-2015 Through 04-06-2015",

year = "2015",

month = jan,

day = "1",

doi = "10.1007/978-3-319-25639-9\_36",

language = "English",

isbn = "978-3-319-25638-2",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer International Publishing",

pages = "190--196",

editor = "Antoine Zimmermann and Serena Villata and John Breslin and Catherine Faron-Zucker and Fabien Gandon and Christophe Gu{\'e}ret",

booktitle = "The Semantic Web",

address = "Switzerland",

url = "https://2015.eswc-conferences.org/index.html, https://2015.eswc-conferences.org/call-challenges.html",

}

Röder, M, Usbeck, R & Ngonga, AC 2015, Developing a sustainable platform for entity annotation benchmarks. in A Zimmermann, S Villata, J Breslin, C Faron-Zucker, F Gandon & C Guéret (Hrsg.), The Semantic Web: ESWC 2015 Satellite Events - ESWC 2015 Satellite Events. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bd. 9341, Springer International Publishing, S. 190-196, 12th European Semantic Web Conference - ESWC 2015, Portoroz, Slowenien, 31.05.15. https://doi.org/10.1007/978-3-319-25639-9_36

Developing a sustainable platform for entity annotation benchmarks. / Röder, Michael; Usbeck, Ricardo; Ngonga, Axel Cyrille.
The Semantic Web: ESWC 2015 Satellite Events - ESWC 2015 Satellite Events. Hrsg. / Antoine Zimmermann; Serena Villata; John Breslin; Catherine Faron-Zucker; Fabien Gandon; Christophe Guéret. Springer International Publishing, 2015. S. 190-196 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Band 9341).

Publikation: Beiträge in Sammelwerken › Aufsätze in Konferenzbänden › Forschung › Begutachtung

TY - CHAP

T1 - Developing a sustainable platform for entity annotation benchmarks

AU - Röder, Michael

AU - Usbeck, Ricardo

AU - Ngonga, Axel Cyrille

N1 - Conference code: 12

PY - 2015/1/1

Y1 - 2015/1/1

N2 - The existing entity annotation systems that drive the extraction of RDF from unstructured data are hard to compare as their evaluation relies on different data sets and measures. We developed GERBIL, an evaluation framework for semantic entity annotation that provides developers, end users and researchers with easy-to-use interfaces for the agile, fine-grained and uniform evaluation of 9 annotation tools on 11 different data sets within 6 different experimental settings on 6 different measures. In this paper, we present the developed interfaces, data flows and data structures. Moreover, we show how GERBIL supports a better reproducibility and archiving of experimental results.

AB - The existing entity annotation systems that drive the extraction of RDF from unstructured data are hard to compare as their evaluation relies on different data sets and measures. We developed GERBIL, an evaluation framework for semantic entity annotation that provides developers, end users and researchers with easy-to-use interfaces for the agile, fine-grained and uniform evaluation of 9 annotation tools on 11 different data sets within 6 different experimental settings on 6 different measures. In this paper, we present the developed interfaces, data flows and data structures. Moreover, we show how GERBIL supports a better reproducibility and archiving of experimental results.

KW - Informatics

KW - Business informatics

UR - https://www.scopus.com/pages/publications/84952647204

UR - https://www.mendeley.com/catalogue/9e637d5b-a1d0-3e13-a985-6e1b2ff94aeb/

U2 - 10.1007/978-3-319-25639-9_36

DO - 10.1007/978-3-319-25639-9_36

M3 - Article in conference proceedings

AN - SCOPUS:84952647204

SN - 978-3-319-25638-2

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 190

EP - 196

BT - The Semantic Web

A2 - Zimmermann, Antoine

A2 - Villata, Serena

A2 - Breslin, John

A2 - Faron-Zucker, Catherine

A2 - Gandon, Fabien

A2 - Guéret, Christophe

PB - Springer International Publishing

T2 - 12th European Semantic Web Conference - ESWC 2015

Y2 - 31 May 2015 through 4 June 2015

ER -

Röder M, Usbeck R, Ngonga AC. Developing a sustainable platform for entity annotation benchmarks. in Zimmermann A, Villata S, Breslin J, Faron-Zucker C, Gandon F, Guéret C, Hrsg., The Semantic Web: ESWC 2015 Satellite Events - ESWC 2015 Satellite Events. Springer International Publishing. 2015. S. 190-196. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-25639-9_36

Developing a sustainable platform for entity annotation benchmarks

Abstract

Bibliographische Notiz

Fachgebiete und Schlagwörter

Zugriff auf Dokument

Fingerprint

Dieses zitieren