Question: Could you provide a list of models that have been tested on the Food-101 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Food-101" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What is the name of the top performing model in terms of Percentage correct score when benchmarked on the CIFAR-100 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Percentage correct" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CIFAR-100" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What are the titles and IDs of research papers that include a benchmark for the BBCSport dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "BBCSport" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What is the highest benchmark result achieved on the BUCC Chinese-to-English dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "BUCC Chinese-to-English" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What is the name of the top performing model in terms of Perplexity score when benchmarked on the LAMBADA dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Perplexity" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "LAMBADA" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Can you list the metrics used to evaluate models on the SST-2 Binary classification dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SST-2 Binary classification" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Can you list the metrics used to evaluate models on the seel.cse.lsu.edu/data/refsq17.zip dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "seel.cse.lsu.edu/data/refsq17.zip" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Provide a list of papers that have utilized the DEQ-TrellisNet model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "DEQ-TrellisNet" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the top benchmark result (metric and value) over the dataset Finger, spin (DMControl100k)?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Finger, spin (DMControl100k)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What is the name of the top performing model in terms of Test mAP score when benchmarked on the AudioSet dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Test mAP" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "AudioSet" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the best performing model benchmarking the Reuters-21578 dataset in terms of Accuracy metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Reuters-21578" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Can you provide links to code used in papers that benchmark the TokenFuser model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "TokenFuser" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: List the metrics that are used to evaluate models on the IWSLT2014 German-English benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "IWSLT2014 German-English" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What models are being evaluated on the Cartpole, swingup (DMControl500k) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Cartpole, swingup (DMControl500k)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Provide a list of papers that have utilized the COMET - DynaGen model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "COMET - DynaGen" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: List the code links in papers that use the BiT-L (ResNet) model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "BiT-L (ResNet)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you provide links to code used in papers that benchmark the 4 layer QRNN model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "4 layer QRNN" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Which model has achieved the highest Accuracy score on the WSC benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WSC" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Could you provide a list of models that have been tested on the Winograd Schema Challenge benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Winograd Schema Challenge" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What are the metrics of evaluation over the Atari 2600 Video Pinball dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Video Pinball" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Where can I find code references in papers that have used the DEQ-Transformer (medium, adaptive embed) model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "DEQ-Transformer (medium, adaptive embed)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What evaluation metrics are commonly used when benchmarking models on the WMT2014 German-English dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WMT2014 German-English" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What is the top benchmark result (metric and value) over the dataset EBM-NLP?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "EBM-NLP" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Where can I find code references in papers that have used the RND model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "RND" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you list the metrics used to evaluate models on the BC5CDR dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "BC5CDR" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Provide a list of research paper titles and IDs that have benchmarked models on the IMDb-M dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "IMDb-M" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What are the models that have been benchmarked on the Finger, spin (DMControl500k) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Finger, spin (DMControl500k)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Provide a list of papers that have utilized the SMT + NMT (tuning and joint refinement) model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "SMT + NMT (tuning and joint refinement)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you list the metrics used to evaluate models on the RotoWire (Relation Generation) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "RotoWire (Relation Generation)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Can you list the metrics used to evaluate models on the Atari 2600 Breakout dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Breakout" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What are the metrics of evaluation over the AAN Corpus dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "AAN Corpus" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What are the metrics of evaluation over the Cornell Grasp Dataset dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Cornell Grasp Dataset" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: List the title and ID of research papers that contain a benchmark over the SQuAD2.0 dev dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SQuAD2.0 dev" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What evaluation metrics are commonly used when benchmarking models on the Atari-57 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari-57" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: List the metrics that are used to evaluate models on the Finger, spin (DMControl500k) benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Finger, spin (DMControl500k)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the UCF101 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "UCF101" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What is the highest benchmark result achieved on the HMDB51 dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "HMDB51" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: List the code links in papers that use the Commonsense Transformers (COMET) model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Commonsense Transformers (COMET)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Indicate the model that performed best in terms of F1 metric on the QuAC benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "QuAC" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: In which papers are energy system scenarios used that consider the 2050 time frame?

PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT ?paper WHERE { { orkgr:R150337 orkgp:compareContribution ?contrib . } UNION { orkgr:R153801 orkgp:compareContribution ?contrib . } ?paper orkgp:P31 ?contrib . { ?contrib orkgp:P37586 ?scenarios . ?scenarios orkgp:P37675 ?studies . ?studies orkgp:P35205 ?timeframes . } UNION { ?contrib orkgp:P37581 ?scenario . ?scenario orkgp:P43138 ?goal . ?goal orkgp:P43139 ?timeframes . } FILTER ( xsd:int ( ?timeframes ) = 2050 ) }

Which papers contain evaluation of energy system scenarious usage in the 2050 timeframe?

In which papers are mentioned energy system usage scenarious concerning the 2050 timeframe?


Question: Can you provide the highest benchmark result, including the metric and score, for the ACL Anthology dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ACL Anthology" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What is the top benchmark result (metric and value) over the dataset Reuters RCV1/RCV2 English-to-German?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Reuters RCV1/RCV2 English-to-German" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Provide a list of papers that have utilized the MAC model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "MAC" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: List the code links in papers that use the Dueling DQN model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Dueling DQN" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you provide links to code used in papers that benchmark the Sandwich Transformer (adaptive span) model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Sandwich Transformer (adaptive span)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: List the code links in papers that use the Prioritized DQN model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Prioritized DQN" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the highest benchmark result achieved on the Oxford 102 Flowers dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Oxford 102 Flowers" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Provide a list of research paper titles and IDs that have benchmarked models on the MLDoc Zero-Shot German-to-French dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MLDoc Zero-Shot German-to-French" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What is the name of the top performing model in terms of Top-1 Accuracy score when benchmarked on the iNaturalist 2019 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Top-1 Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "iNaturalist 2019" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What models are being evaluated on the PIQA dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "PIQA" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What are the models that have been benchmarked on the WOS-46985 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WOS-46985" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Indicate the model that performed best in terms of F1 metric on the Natural Questions (short) benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Natural Questions (short)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What are the models that have been benchmarked on the OntoNotes dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "OntoNotes" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Can you list the models that have been evaluated on the Hutter Prize dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Hutter Prize" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What is the top benchmark score and its metric on the Cartpole, swingup (DMControl100k) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Cartpole, swingup (DMControl100k)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What evaluation metrics are commonly used when benchmarking models on the ObjectNet (Bounding Box) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ObjectNet (Bounding Box)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: List the code links in papers that use the Poolingformer model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Poolingformer" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Provide a list of papers that have utilized the NASNet-A + c/o model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "NASNet-A + c/o" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: List the metrics that are used to evaluate models on the ARC (Easy) benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ARC (Easy)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What is the name of the top performing model in terms of Score score when benchmarked on the Atari 2600 Beam Rider dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Score" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Beam Rider" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the highest benchmark result achieved on the Finger, spin (DMControl500k) dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Finger, spin (DMControl500k)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Provide a list of papers that have utilized the GShard model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "GShard" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Which methods are used?

SELECT DISTINCT ?methods, ?methods_labels
WHERE {
  orkgr:R110361 orkgp:compareContribution ?cont.
  ?cont orkgp:HAS_METHOD ?methods.
  ?methods rdfs:label ?methods_labels.
}

What are the methods used?
Which methods are applied?


Question: Provide a list of benchmarked datasets related to the Word Embeddings research area?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "Word Embeddings" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: Can you provide the highest benchmark result, including the metric and score, for the seel.cse.lsu.edu/data/refsq17.zip dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "seel.cse.lsu.edu/data/refsq17.zip" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What are the models that have been benchmarked on the IWSLT2015 English-German dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "IWSLT2015 English-German" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: List the code links in papers that use the FS-LSTM-2 model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "FS-LSTM-2" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the highest benchmark result achieved on the STEM-ECR v1.0 dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "STEM-ECR v1.0" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What evaluation metrics are commonly used when benchmarking models on the Atari 2600 Ms. Pacman dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Ms. Pacman" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Can you list the models that have been evaluated on the MLDoc Zero-Shot English-to-Chinese dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MLDoc Zero-Shot English-to-Chinese" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Can you provide the highest benchmark result, including the metric and score, for the SciCite dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SciCite" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Can you provide links to code used in papers that benchmark the OverFeat - 7 accurate models model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "OverFeat - 7 accurate models" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What evaluation metrics are commonly used when benchmarking models on the AESLC dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "AESLC" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Which model has achieved the highest Best Score score on the Atari 2600 Frostbite benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Best Score" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Frostbite" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Does FEEM Sustainability Index consider air quality?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> ASK { ?papers rdf:type orkgc:Paper . ?papers rdfs:label ?papers_labels . FILTER ( REGEX ( ?papers_labels , "FEEM Sustainability Index" ) ) ?papers orkgp:P31 ?contrib . ?contrib ?properties ?properties_values . ?properties rdfs:label ?properties_labels . FILTER ( REGEX ( ?properties_labels , "air quality" , "i" ) ) FILTER ( ?properties_values = "yes" ) }

Is air quality accounted for in the FEEM Sustainability Index?


Question: What is the best performing model benchmarking the Winograd Schema Challenge dataset in terms of Accuracy metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Winograd Schema Challenge" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of papers that have utilized the NAT-M4 model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "NAT-M4" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Which energy system modeling papers indicate where to find their simulation software and whether there are associated simulation scenarios mentioned in ORKG?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> SELECT ?repo ?scenario ?simulation_parameters WHERE { ?paper orkgp:P31 ?contrib . ?contrib orkgp:P37586 ?model . ?model orkgp:P39010 ?code . ?code orkgp:P4077 ?repo . OPTIONAL { ?model orkgp:P43146 ?scenario . { SELECT ?scenario ?simulation_parameters WHERE { ?scenario orkgp:P31 ?contrib2 . ?contrib2 orkgp:P37586 ?model2 . ?model2 orkgp:P37583 ?data . ?data orkgp:HAS_DATASET ?dataset . ?dataset orkgp:P37343 ?type ; orkgp:P4059 ?simulation_parameters . ?type rdfs:label ?label . FILTER ( REGEX ( ?label , "Technical" ) ) } } } }

What are the simulation scenarios and their code of energy simulation papers?


Question: What is the best performing model benchmarking the Atari 2600 Private Eye dataset in terms of Score metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Score" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Private Eye" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What models are being evaluated on the DROP Test dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "DROP Test" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: List the title and ID of research papers that contain a benchmark over the Ball in cup, catch (DMControl100k) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Ball in cup, catch (DMControl100k)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Does ViCoMap support aggregation?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT ?has_aggregation_support WHERE { ?application rdfs:label "ViCoMap" ^^ xsd:string ; orkgp:P5038 ?has_aggregation_support_ . BIND ( IF ( ?has_aggregation_support_ = "T" ^^ xsd:string , "yes" , "no" ) AS ?has_aggregation_support ) }


Question: Can you provide links to code used in papers that benchmark the Random Forest model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Random Forest" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Provide a list of research paper titles and IDs that have benchmarked models on the Flowers-102 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Flowers-102" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What are the titles and IDs of research papers that include a benchmark for the MedNLI dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MedNLI" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: List the metrics that are used to evaluate models on the IMDb benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "IMDb" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Could you provide a list of models that have been tested on the ART/CoreSC benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "ART/CoreSC" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What is the highest benchmark result achieved on the Atari 2600 Road Runner dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Road Runner" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What is the best performing model benchmarking the SciCite dataset in terms of F1 metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SciCite" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What models are being evaluated on the MLDoc Zero-Shot English-to-Italian dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MLDoc Zero-Shot English-to-Italian" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the Softcite dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Softcite" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What is the highest benchmark result achieved on the Atari 2600 Robotank dataset, including the metric and its value?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Robotank" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Can you provide the highest benchmark result, including the metric and score, for the Ohsumed dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Ohsumed" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What are the models that have been benchmarked on the Atari 2600 Wizard of Wor dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Wizard of Wor" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Can you provide links to code used in papers that benchmark the SpanRel model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "SpanRel" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the MLDoc Zero-Shot English-to-French dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MLDoc Zero-Shot English-to-French" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Provide a list of papers that have utilized the Neural Network Language Model (NNLM) model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Neural Network Language Model (NNLM)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What models are being evaluated on the WOS-5736 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WOS-5736" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Which model has achieved the highest Best Score score on the Atari 2600 Space Invaders benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Best Score" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Space Invaders" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the PIQA dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "PIQA" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Can you provide links to code used in papers that benchmark the AdvSoft (+ 4 layer QRNN + dynamic eval) model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "AdvSoft (+ 4 layer QRNN + dynamic eval)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the average installed capacity of all energy sources considered?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT ( AVG ( ?installed_cap_value ) AS ?average_installed_cap_value ) WHERE { orkgr:R153801 orkgp:compareContribution ?contrib . ?contrib orkgp:P43135 ?energy_sources . ?energy_sources rdfs:label ?energy_sources_labels ; orkgp:P43133 ?installed_capacity . FILTER ( REGEX ( ?energy_sources_labels , "all sources" ) ) ?installed_capacity orkgp:HAS_VALUE ?value . BIND ( xsd:float ( ?value ) AS ?installed_cap_value ) }

What is the mean value of installed capacity for all energy sources in the studies?


Question: Where can I find code references in papers that have used the SciBERT (full data) model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "SciBERT (full data)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Where can I find code references in papers that have used the XLNet (single model) model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "XLNet (single model)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you provide links to code used in papers that benchmark the Multi-Format Contrastive model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Multi-Format Contrastive" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Provide a list of research paper titles and IDs that have benchmarked models on the IWSLT2014 German-English dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "IWSLT2014 German-English" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What is the top benchmark score and its metric on the SearchQA dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SearchQA" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What models are being evaluated on the Atari 2600 Enduro dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Enduro" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Indicate the model that performed best in terms of Top-1 Error Rate metric on the FGVC Aircraft benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Top-1 Error Rate" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "FGVC Aircraft" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of research paper titles and IDs that have benchmarked models on the Nottingham dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Nottingham" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What models are being evaluated on the CUB-200-2011 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CUB-200-2011" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What models are being evaluated on the Pubmed dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Pubmed" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Provide a list of papers that have utilized the BART Base (with text infilling) model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "BART Base (with text infilling)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Name the datasets that have been used for benchmarking in the Leaderboard extraction research problem?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "Leaderboard extraction" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: Provide a list of papers that have utilized the Longformer Small model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Longformer Small" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you list the models that have been evaluated on the Atari 2600 Amidar dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Amidar" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: List the title and ID of research papers that contain a benchmark over the FB15k dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "FB15k" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Indicate the model that performed best in terms of EM metric on the SQuAD1.1 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "EM" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SQuAD1.1" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of papers that have utilized the Hierarchical Transformer Encoder + conditional copy model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Hierarchical Transformer Encoder + conditional copy" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What models are being evaluated on the SQuAD2.0 dev dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SQuAD2.0 dev" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What is the best performing model benchmarking the CoNLL04 dataset in terms of NER Micro F1 metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "NER Micro F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CoNLL04" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of papers that have utilized the BioSentVec (PubMed + MIMIC-III) model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "BioSentVec (PubMed + MIMIC-III)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What evaluation metrics are commonly used when benchmarking models on the NYT24 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "NYT24" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What is the name of the top performing model in terms of Score score when benchmarked on the Atari 2600 Name This Game dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Score" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Name This Game" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Can you list the metrics used to evaluate models on the CoNLL04 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CoNLL04" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Give me a list of research papers along with their titles and IDs, that have performed benchmarks on the RTE dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "RTE" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: Can you list the metrics used to evaluate models on the MLDoc Zero-Shot English-to-French dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "MLDoc Zero-Shot English-to-French" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Where can I find code references in papers that have used the BBG (ResNet-18) model for benchmarking purposes?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "BBG (ResNet-18)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you provide links to code used in papers that benchmark the AWD-LSTM-DOC + Partial Shuffle model?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "AWD-LSTM-DOC + Partial Shuffle" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What are the most commonly used benchmark datasets for the Semantic Textual Similarity research field?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?dataset ?dataset_lbl WHERE { ?problem a orkgc:Problem ; rdfs:label ?problem_lbl . FILTER ( str ( ?problem_lbl ) = "Semantic Textual Similarity" ) ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:P32 ?problem . }


Question: What are the models that have been benchmarked on the DDI dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "DDI" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Indicate the model that performed best in terms of Error metric on the Kuzushiji-MNIST benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Error" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Kuzushiji-MNIST" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Which model has achieved the highest SemEval 2015 score on the Supervised: benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "SemEval 2015" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Supervised:" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of papers that have utilized the Grave et al. (2016) - LSTM model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Grave et al. (2016) - LSTM" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What models are being evaluated on the Cartpole, swingup (DMControl100k) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Cartpole, swingup (DMControl100k)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Indicate the model that performed best in terms of F1 metric on the SQuAD1.1 benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SQuAD1.1" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Indicate the model that performed best in terms of BLEU score metric on the WMT2016 English-Romanian benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "BLEU score" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WMT2016 English-Romanian" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Indicate the model that performed best in terms of Accuracy metric on the Quora Question Pairs benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Accuracy" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Quora Question Pairs" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of papers that have utilized the DocRED-Context-Aware model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "DocRED-Context-Aware" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the top benchmark result (metric and value) over the dataset IWSLT2014 German-English?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "IWSLT2014 German-English" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: What are the metrics of evaluation over the WMT2016 English-Russian dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WMT2016 English-Russian" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: Provide a list of papers that have utilized the Discrete Latent Space World Model (VQ-VAE) model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Discrete Latent Space World Model (VQ-VAE)" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Can you list the metrics used to evaluate models on the Natural Questions dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Natural Questions" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: List the code links in papers that use the CRF with term expansion model in any benchmark?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "CRF with term expansion" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What is the top benchmark score and its metric on the CIFAR-100 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl ( MAX ( ?value ) AS ?score ) WHERE { { SELECT ?metric ?metric_lbl ?value WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "CIFAR-100" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } ?cont orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?cont orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } } ORDER BY DESC ( ?value ) } } GROUP BY ?metric ?metric_lbl


Question: Which model has achieved the highest Relation F1 score on the SciERC benchmark dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Relation F1" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "SciERC" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What models are being evaluated on the Cart Pole (OpenAI Gym) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Cart Pole (OpenAI Gym)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What is the best performing model benchmarking the Text8 dataset in terms of Bit per Character (BPC) metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Bit per Character (BPC)" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Text8" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: Provide a list of papers that have utilized the PtGen model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "PtGen" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: Provide a list of research paper titles and IDs that have benchmarked models on the Atari 2600 Boxing dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?paper ?paper_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Boxing" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?paper orkgp:P31 ?cont ; rdfs:label ?paper_lbl . }


Question: What are the models that have been benchmarked on the WMT2016 English-Czech dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WMT2016 English-Czech" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What models are being evaluated on the Fashion-MNIST dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Fashion-MNIST" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: Provide a list of papers that have utilized the Longformer Large model and include the links to their code?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?code WHERE { ?model a orkgc:Model ; rdfs:label ?model_lbl . FILTER ( str ( ?model_lbl ) = "Longformer Large" ) ?benchmark orkgp:HAS_DATASET ?dataset . ?cont orkgp:HAS_BENCHMARK ?benchmark . ?cont orkgp:HAS_MODEL ?model ; orkgp:HAS_SOURCE_CODE ?code . }


Question: What models are being evaluated on the Atari-57 dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari-57" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }


Question: What is the best performing model benchmarking the Yelp Fine-grained classification dataset in terms of Error metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Error" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Yelp Fine-grained classification" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What is the best performing model benchmarking the WikiText-2 dataset in terms of Validation perplexity metric?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?metric a orkgc:Metric ; rdfs:label ?metric_lbl . FILTER ( str ( ?metric_lbl ) = "Validation perplexity" ) { SELECT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "WikiText-2" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?eval orkgp:HAS_VALUE ?value ; orkgp:HAS_METRIC ?metric . ?cont orkgp:HAS_BENCHMARK ?benchmark ; orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } ORDER BY DESC ( ?value ) LIMIT 1 } }


Question: What evaluation metrics are commonly used when benchmarking models on the Atari 2600 Kangaroo dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?metric ?metric_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Atari 2600 Kangaroo" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . OPTIONAL { ?eval orkgp:HAS_METRIC ?metric . ?metric rdfs:label ?metric_lbl . } }


Question: What models are being evaluated on the Rotowire (Content Selection) dataset?

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX orkgp: <http://orkg.org/orkg/predicate/> PREFIX orkgc: <http://orkg.org/orkg/class/> SELECT DISTINCT ?model ?model_lbl WHERE { ?dataset a orkgc:Dataset ; rdfs:label ?dataset_lbl . FILTER ( str ( ?dataset_lbl ) = "Rotowire (Content Selection)" ) ?benchmark orkgp:HAS_DATASET ?dataset ; orkgp:HAS_EVALUATION ?eval . ?paper orkgp:HAS_BENCHMARK ?benchmark . OPTIONAL { ?paper orkgp:HAS_MODEL ?model . ?model rdfs:label ?model_lbl . } }