Proceedings of the BioCreative V.5 Challenge Evaluation Workshop
Last April 26-27 the BioCreative V.5 Challenge Evaluation Workshop took place in Barcelona. The goal of BioCreative V.5 was to address some of the major barriers to the adoption and use of text mining tools, related to assessment, accessibility, interoperability, robustness and integration.
To promote synergies the workshop was co-located with the ELIXIR-EXCELERATE workshop on text mining infrastructure requirements, with a particular emphasis on the role of Text Mining systems for Data Curation and Knowledge Management in the Life Sciences domain.
Topics of the workshop
The topics that were addressed during the workshop included: (1) continuous evaluation/stability of text mining tools, (2) enabling of interoperability of multiple text annotation systems at the technical level (design of compatible annotation schemas), (3) extraction of textual content from heterogeneous document sources, and (4) visualization and comparative assessment of automatic and manual annotations.
Two traditional tracks, one new track
The workshop addressed two traditional BioCreative tracks: the chemicals CEMP track and genes/proteins GPRO track. A total of 21 teams submitted results for at least one of these tasks and the BeCalm platform supported prediction submission and evaluation (http://www.becalm.eu). This time, a novel track called TIPS (Technical interoperability and performance of annotation servers) focused on the technical aspects of the evaluation of continuous text annotation web services.
The TIPS track consisted in a novel experimental task with the aim to, for the first time ever, carry out a text mining challenge with particular focus on the continuous assessment of technical aspects of text annotation web servers, specifically of biomedical online named entity recognition systems. A total of 13 teams registered annotation servers, implemented in various programming languages, supporting up to 12 different general annotation types.
Joint collaboration
The workshop was a joint collaboration between OpenMinted and the ELIXIR-EXCELERATE H2020 projects and the Encomienda MINETAD-CNIO as part of the Plan for the Advancement of Language Technology of the Spanish government.
View or download the proceedings
The Proceedings of the BioCreative V.5 Challenge Evaluation Workshop are available at the BioCreative website. You can also download a pdf version here.
Marta Villegas, Barcelona Supercomputing Center