The CNIO team will provide the Biocreative challenges and associated infrastructure to contribute to the project. The BioCreative initiative will enable to test not only the method quality, but also their stability/accessibility and more importantly how they work together to perform a given task.
Including not only standard text mining (as we do) but also other more NLP and non-biology driven components. Among the example tasks that will be tested in this context are: named entity recognition, information extraction of binary relationships and text categorization and ranking. The evaluation challenges and integration can be linked to a system developed at the CNIO, GOPHER, the core of a general purpose critical assessment continuous evaluation system (CACES), which initially was born for the evaluation of function prediction methods and servers. The system has been built around an eXist XML database instance, providing two REST APIs and an XML Schema which models both the database contents and the incoming and outgoing messages from these APIs. One of the APIs is for the coordination server itself (participant management et al) and another for the servers and evaluators subscribed to the continuous evaluation to submit evaluation jobs and receive the corresponding results, using thebio-creative infrastructure (assuming that it can be considered as such)and connecting with Jose Maria (goopher – general purpose method testing system).
Regarding the involvement in WPs, CNIO will provide the biology perspective in the life science use cases (WP4, WP9), participate in the interoperability framework specifications working groups (WP5). In WP2 and WP3, CNIO will be actively involved in community engagement and training activities. In the second WP, CNIO will also design with ARC and carries out the tender calls for 3rd party uptake.