Question: What is the best performing model benchmarking the One Billion Word dataset in terms of PPL metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "PPL" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "One Billion Word" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What are the models that have been benchmarked on the Words in Context dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Words in Context" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the Classic dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Classic" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Name the datasets that have been used for benchmarking in the Graph Embedding research problem?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "Graph Embedding" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: What is the top benchmark score and its metric on the ClueWeb09-B dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ClueWeb09-B" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: List the metrics that are used to evaluate models on the Penn Treebank (Word Level) benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Penn Treebank (Word Level)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Which model has achieved the highest F1 score on the MedNLI benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MedNLI" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: List the code links in papers that use the BERT-joint model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "BERT-joint" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you provide links to code used in papers that benchmark the BERT classifier model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "BERT classifier" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the ner_dataset_recognition dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ner_dataset_recognition" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Can you list the metrics used to evaluate models on the 20NEWS dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "20NEWS" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Can you list the metrics used to evaluate models on the SciERC dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SciERC" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What is the name of the top performing model in terms of FLOPS score when benchmarked on the FGVC Aircraft dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "FLOPS" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "FGVC Aircraft" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Which model has achieved the highest Accuracy score on the Reuters De-En benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Reuters De-En" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of papers that have utilized the Feedback Transformer (4 layers) model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Feedback Transformer (4 layers)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the name of the top performing model in terms of Accuracy score when benchmarked on the ImageNet dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ImageNet" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: List the code links in papers that use the NCBI_BERT(base) (P) model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "NCBI_BERT(base) (P)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Provide a list of benchmarked datasets related to the stochastic classification research area?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "stochastic classification" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: What models are being evaluated on the Recipe dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Recipe" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Where can I find code references in papers that have used the Intrinsic Reward Agent model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Intrinsic Reward Agent" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Indicate the model that performed best in terms of RE Micro F1 metric on the ACE 2004 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "RE Micro F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ACE 2004" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the top benchmark result (metric and value) over the dataset TempEval-3?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "TempEval-3" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What are the models that have been benchmarked on the MLDoc Zero-Shot English-to-Russian dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MLDoc Zero-Shot English-to-Russian" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What are the titles and IDs of research papers that include a benchmark for the MLDoc Zero-Shot English-to-Italian dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MLDoc Zero-Shot English-to-Italian" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What is the top benchmark score and its metric on the MUTAG dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MUTAG" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What is the top benchmark result (metric and value) over the dataset Atari 2600 James Bond?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 James Bond" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Where can I find code references in papers that have used the BigBird model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "BigBird" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the top benchmark result (metric and value) over the dataset CommonsenseQA?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CommonsenseQA" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What are the metrics of evaluation over the Atari 2600 Krull dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Krull" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What is the top benchmark result (metric and value) over the dataset ARC-PDN?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ARC-PDN" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: List the metrics that are used to evaluate models on the COPA benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "COPA" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Where can I find code references in papers that have used the AVTS model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "AVTS" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Provide a list of papers that have utilized the Dynamic Coattention Networks (ensemble) model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Dynamic Coattention Networks (ensemble)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: List the code links in papers that use the DDRL A3C model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "DDRL A3C" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the best performing model benchmarking the STL-10, 1000 Labels dataset in terms of Accuracy metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "STL-10, 1000 Labels" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the best performing model benchmarking the X-Sum dataset in terms of ROUGE-3 metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "ROUGE-3" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "X-Sum" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: List the metrics that are used to evaluate models on the Pubmed benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Pubmed" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What is the average energy generation of all energy sources considered?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT ( AVG ( ?elec_gen_value ) AS ?average_elec_gen_value ) WHERE { orkgr:R153801 orkgp:compareContribution ?contrib . ?contrib orkgp:P43135 ?energy_sources . ?energy_sources rdfs:label ?energy_sources_labels ; orkgp:P43134 ?electricity_generation . FILTER ( REGEX ( ?energy_sources_labels , "all sources" ) ) ?electricity_generation orkgp:HAS_VALUE ?value . BIND ( xsd:float ( ?value ) AS ?elec_gen_value ) }

What is the mean value of energy generation for all energy sources in the studies?


Question: What is the name of the top performing model in terms of Accuracy score when benchmarked on the BoolQ dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "BoolQ" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of research paper titles and IDs that have benchmarked models on the CoQA dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CoQA" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Can you provide the highest benchmark result, including the metric and score, for the Amazon dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Amazon" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Which model has achieved the highest PARAMS score on the CINIC-10 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "PARAMS" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CINIC-10" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Can you list the metrics used to evaluate models on the WMT2016 Romanian-English dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WMT2016 Romanian-English" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Can you provide links to code used in papers that benchmark the ApproxRepSet model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "ApproxRepSet" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the top benchmark score and its metric on the WMT2016 Russian-English dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WMT2016 Russian-English" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What are the titles and IDs of research papers that include a benchmark for the ImageNet dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ImageNet" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: List the code links in papers that use the Transformer-XL - 24 layers model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Transformer-XL - 24 layers" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the top benchmark score and its metric on the DROP Test dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "DROP Test" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What are the most commonly used benchmark datasets for the Phrase Extraction research field?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "Phrase Extraction" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: Provide a list of research paper titles and IDs that have benchmarked models on the ARC (Easy) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ARC (Easy)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Where can I find code references in papers that have used the Cluster-Former (#C=512) model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Cluster-Former (#C=512)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the top benchmark score and its metric on the iNaturalist 2018 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "iNaturalist 2018" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What models are being evaluated on the Ball in cup, catch (DMControl100k) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Ball in cup, catch (DMControl100k)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What are the metrics of evaluation over the RotoWire dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "RotoWire" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: List the metrics that are used to evaluate models on the WikiText-2 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WikiText-2" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What is the top benchmark score and its metric on the NYT24 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "NYT24" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: List the title and ID of research papers that contain a benchmark over the Atari 2600 Alien dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Alien" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Can you provide links to code used in papers that benchmark the mLSTM + dynamic eval model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "mLSTM + dynamic eval" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Indicate the model that performed best in terms of Accuracy metric on the Penn Treebank benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Penn Treebank" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: List the title and ID of research papers that contain a benchmark over the STEM-ECR v1.0 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "STEM-ECR v1.0" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Can you provide the highest benchmark result, including the metric and score, for the MedNLI dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MedNLI" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the QuAC dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "QuAC" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: List the code links in papers that use the entity and relations table model model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "entity and relations table model" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the best performing model benchmarking the FGVC Aircraft dataset in terms of Top-1 metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Top-1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "FGVC Aircraft" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Indicate the model that performed best in terms of Accuracy metric on the MPQA benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MPQA" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Can you list the metrics used to evaluate models on the GENIA - UAS dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "GENIA - UAS" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Can you provide links to code used in papers that benchmark the Transformer Big + adversarial MLE model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Transformer Big + adversarial MLE" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the Amazon dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Amazon" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What is the highest benchmark result achieved on the Yelp-2 dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Yelp-2" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: List the code links in papers that use the Sarsa-φ-EB model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Sarsa-φ-EB" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: List the metrics that are used to evaluate models on the Atari 2600 Demon Attack benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Demon Attack" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Name the datasets that have been used for benchmarking in the Humor Detection research problem?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "Humor Detection" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: What evaluation metrics are commonly used when benchmarking models on the Paper Field dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Paper Field" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Indicate the model that performed best in terms of F1 metric on the Reuters-21578 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Reuters-21578" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the highest benchmark result achieved on the Atari 2600 Private Eye dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Private Eye" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What are the most commonly used benchmark datasets for the Named entity recognition research field?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "Named entity recognition" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: What is the highest benchmark result achieved on the Classical music, 5 seconds at 12 kHz dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Classical music, 5 seconds at 12 kHz" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Can you list the models that have been evaluated on the PWC Leaderboards (restricted) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "PWC Leaderboards (restricted)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What evaluation metrics are commonly used when benchmarking models on the MedSTS dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MedSTS" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What are the titles and IDs of research papers that include a benchmark for the TDM Tagged Corpus dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "TDM Tagged Corpus" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Can you provide the highest benchmark result, including the metric and score, for the Atari-57 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari-57" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Where can I find code references in papers that have used the SpERT (without overlap) model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "SpERT (without overlap)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you provide links to code used in papers that benchmark the SMT as posterior regularization model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "SMT as posterior regularization" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the DuIE dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "DuIE" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What are the models that have been benchmarked on the AAPD dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "AAPD" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Can you provide links to code used in papers that benchmark the MPCM model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "MPCM" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Which model has achieved the highest Macro Precision score on the PWC Leaderboards (restricted) benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Macro Precision" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "PWC Leaderboards (restricted)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Where can I find code references in papers that have used the ST-MoE model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "ST-MoE" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What evaluation metrics are commonly used when benchmarking models on the OA-STM dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "OA-STM" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What is the top benchmark result (metric and value) over the dataset NLP-TDMS?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "NLP-TDMS" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Which model has achieved the highest PARAMS score on the STL-10 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "PARAMS" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "STL-10" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the best performing model benchmarking the RotoWire (Relation Generation) dataset in terms of count metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "count" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "RotoWire (Relation Generation)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Can you provide the highest benchmark result, including the metric and score, for the DRI Corpus dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "DRI Corpus" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Where can I find code references in papers that have used the e2e-coref + ELMo model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "e2e-coref + ELMo" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you provide the highest benchmark result, including the metric and score, for the TriviaQA dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "TriviaQA" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Can you provide the highest benchmark result, including the metric and score, for the SNLI dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SNLI" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What are the most commonly used benchmark datasets for the Data-to-Text Generation research field?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "Data-to-Text Generation" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: Indicate the model that performed best in terms of F1 (Short) metric on the Natural Questions benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1 (Short)" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Natural Questions" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What models are being evaluated on the CIFAR-10 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CIFAR-10" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: List the title and ID of research papers that contain a benchmark over the Open Entity dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Open Entity" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Which model has achieved the highest PARAMS score on the CIFAR-100 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "PARAMS" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CIFAR-100" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the best performing model benchmarking the RTE dataset in terms of Accuracy metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "RTE" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Can you list the metrics used to evaluate models on the Twitter dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Twitter" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What is the top benchmark score and its metric on the Barabasi-Albert dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Barabasi-Albert" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Provide a list of benchmarked datasets related to the Scientific Claim Verification research area?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "Scientific Claim Verification" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: What is the top benchmark score and its metric on the nuScenes dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "nuScenes" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the Oxford 102 Flowers dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Oxford 102 Flowers" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What is the name of the top performing model in terms of Accuracy score when benchmarked on the CommitmentBank dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CommitmentBank" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the highest benchmark result achieved on the Car speed in Liuliqiao District, Beijing dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Car speed in Liuliqiao District, Beijing" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Indicate the model that performed best in terms of Score metric on the Atari 2600 Alien benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Score" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Alien" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Which model has achieved the highest Entropy Difference score on the Barabasi-Albert benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Entropy Difference" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Barabasi-Albert" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the top benchmark result (metric and value) over the dataset ANLI test?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ANLI test" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the AESLC dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "AESLC" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Provide a list of papers that have utilized the LASER model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "LASER" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What are the models that have been benchmarked on the ANLI test dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ANLI test" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Provide a list of papers that have utilized the He et al., 2017 + ELMo model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "He et al., 2017 + ELMo" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: List the title and ID of research papers that contain a benchmark over the BUCC Russian-to-English dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "BUCC Russian-to-English" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What is the top benchmark score and its metric on the IWSLT2015 English-German dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "IWSLT2015 English-German" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Where can I find code references in papers that have used the EfficientNetV2-S (21k) model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "EfficientNetV2-S (21k)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the best performing model benchmarking the Atari 2600 Time Pilot dataset in terms of Score metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Score" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Time Pilot" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Can you provide the highest benchmark result, including the metric and score, for the Atari 2600 Ms. Pacman dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Ms. Pacman" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Could you provide a list of models that have been tested on the UCF101 (finetuned) benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "UCF101 (finetuned)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What is the top benchmark score and its metric on the SQuAD1.1 dev dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SQuAD1.1 dev" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What is the best performing model benchmarking the ACE 2005 dataset in terms of RE Micro F1 metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "RE Micro F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ACE 2005" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the top benchmark result (metric and value) over the dataset OA-STM?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "OA-STM" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Can you list the metrics used to evaluate models on the Atari 2600 Yars Revenge dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Yars Revenge" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the Atari 2600 Kangaroo dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Kangaroo" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Can you list benchmarked problems in the area of Information Science?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?problem ?problem_lbl WHERE { ?rf a orkgc:ResearchField ; rdfs:label ?rf_label . FILTER ( str ( ?rf_label ) = "Information Science" ) ?paper orkgp:P30 ?rf ; orkgp:P31 ?cont . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . ?problem rdfs:label ?problem_lbl . }


Question: List the title and ID of research papers that contain a benchmark over the Ball in cup, catch (DMControl500k) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Ball in cup, catch (DMControl500k)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Which model has achieved the highest F1 score on the GENIA - UAS benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "GENIA - UAS" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: List the code links in papers that use the word BiLSTM + char CNN + CRF model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "word BiLSTM + char CNN + CRF" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you provide links to code used in papers that benchmark the SciKG model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "SciKG" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you list the models that have been evaluated on the SciREX dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SciREX" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What are the models that have been benchmarked on the Atari 2600 Frostbite dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Frostbite" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Provide a list of papers that have utilized the DARQN hard model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "DARQN hard" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Which model has achieved the highest Accuracy score on the OpenBookQA benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "OpenBookQA" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the name of the top performing model in terms of Score score when benchmarked on the Atari 2600 Pong dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Score" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Pong" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the best performing model benchmarking the CIFAR-10 dataset in terms of Search Time (GPU days) metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Search Time (GPU days)" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CIFAR-10" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What evaluation metrics are commonly used when benchmarking models on the Reacher, easy (DMControl100k) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Reacher, easy (DMControl100k)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What are the models that have been benchmarked on the SciFACT dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SciFACT" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Can you provide links to code used in papers that benchmark the SpERT (with overlap) model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "SpERT (with overlap)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the Sequential MNIST dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Sequential MNIST" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Can you list the models that have been evaluated on the BioASQ dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "BioASQ" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What is the name of the top performing model in terms of Validation perplexity score when benchmarked on the Penn Treebank (Word Level) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Validation perplexity" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Penn Treebank (Word Level)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Which model has achieved the highest F1 score on the NYT24 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "NYT24" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of papers that have utilized the Perceiver model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Perceiver" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Which model has achieved the highest Relation F1 score on the ACE 2005 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Relation F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ACE 2005" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What are the models that have been benchmarked on the Twitter dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Twitter" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What models are being evaluated on the TDM Tagged Corpus dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "TDM Tagged Corpus" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Indicate the model that performed best in terms of A1 metric on the ANLI test benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "A1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ANLI test" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the best performing model benchmarking the DDI extraction 2013 corpus dataset in terms of F1 metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "DDI extraction 2013 corpus" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: List the code links in papers that use the ZFNet (ensemble, 6 convnets) model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "ZFNet (ensemble, 6 convnets)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }