Skip to content

Commit

Permalink
update many queries for newer elasticsearch
Browse files Browse the repository at this point in the history 
Removes 'and', 'or', 'missing', and 'filtered' queries.
Fixes #1264

Removes 'filter' field in 'nested' query.
Fixes #1265

Removes 'filter' in 'function_score' queries.
Fixes #1266

Removes use of top level 'filter' search parameter.
Fixes #1267

Removes use of negative boosts.
Fixes #1270
  • Loading branch information
@haarg
haarg committed Sep 26, 2024
1 parent 179eb32 commit 85d4f14
Show file tree
Hide file tree
Showing 29 changed files with 536 additions and 531 deletions.
188 changes: 103 additions & 85 deletions docs/API-docs.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -174,22 +174,24 @@ dependency.

```sh
curl -XPOST https://fastapi.metacpan.org/v1/release/_search -d '{
"size": 5000,
"fields": [ "distribution" ],
"filter": {
"and": [
{ "term": { "dependency.module": "MooseX::NonMoose" } },
{ "term": {"maturity": "released"} },
{ "term": {"status": "latest"} }
]
}
"size" : 5000,
"fields" : [ "distribution" ],
"query" : {
"bool" : {
"must" : [
{ "term" : { "dependency.module" : "MooseX::NonMoose" } },
{ "term" : { "maturity" : "released" } },
{ "term" : { "status" : "latest" } }
]
}
}
}'
```

_Note it is also possible to use these queries in GET requests (useful for cross-domain JSONP requests) by appropriately encoding the JSON query into the `source` parameter of the URL. For example the query above [would become](https://fastapi.metacpan.org/v1/release/_search?source=%7B%22query%22%3A%7B%22match_all%22%3A%7B%7D%7D%2C%22size%22%3A5000%2C%22fields%22%3A%5B%22distribution%22%5D%2C%22filter%22%3A%7B%22and%22%3A%5B%7B%22term%22%3A%7B%22release.dependency.module%22%3A%22MooseX%3A%3ANonMoose%22%7D%7D%2C%7B%22term%22%3A%7B%22release.maturity%22%3A%22released%22%7D%7D%2C%7B%22term%22%3A%7B%22release.status%22%3A%22latest%22%7D%7D%5D%7D%7D):_
_Note it is also possible to use these queries in GET requests (useful for cross-domain JSONP requests) by appropriately encoding the JSON query into the `source` parameter of the URL. For example the query above [would become](https://fastapi.metacpan.org/v1/release/_search?source=%7B%0A%20%20%20%20%22size%22%20%3A%205000%2C%0A%20%20%20%20%22fields%22%20%3A%20%5B%20%22distribution%22%20%5D%2C%0A%20%20%20%20%22query%22%20%3A%20%7B%0A%20%20%20%20%20%20%20%20%22bool%22%20%3A%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%22must%22%20%3A%20%5B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%7B%20%22term%22%20%3A%20%7B%20%22dependency.module%22%20%3A%20%22MooseX%3A%3ANonMoose%22%20%7D%20%7D%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%7B%20%22term%22%20%3A%20%7B%20%22maturity%22%20%3A%20%22released%22%20%7D%20%7D%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%7B%20%22term%22%20%3A%20%7B%20%22status%22%20%3A%20%22latest%22%20%7D%20%7D%0A%20%20%20%20%20%20%20%20%20%20%20%20%5D%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%7D):_

```
curl 'https://fastapi.metacpan.org/v1/release/_search?source=%7B%22query%22%3A%7B%22match_all%22%3A%7B%7D%7D%2C%22size%22%3A5000%2C%22fields%22%3A%5B%22distribution%22%5D%2C%22filter%22%3A%7B%22and%22%3A%5B%7B%22term%22%3A%7B%22release.dependency.module%22%3A%22MooseX%3A%3ANonMoose%22%7D%7D%2C%7B%22term%22%3A%7B%22release.maturity%22%3A%22released%22%7D%7D%2C%7B%22term%22%3A%7B%22release.status%22%3A%22latest%22%7D%7D%5D%7D%7D'
curl 'https://fastapi.metacpan.org/v1/release/_search?source=%7B%0A%20%20%20%20%22size%22%20%3A%205000%2C%0A%20%20%20%20%22fields%22%20%3A%20%5B%20%22distribution%22%20%5D%2C%0A%20%20%20%20%22query%22%20%3A%20%7B%0A%20%20%20%20%20%20%20%20%22bool%22%20%3A%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%22must%22%20%3A%20%5B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%7B%20%22term%22%20%3A%20%7B%20%22dependency.module%22%20%3A%20%22MooseX%3A%3ANonMoose%22%20%7D%20%7D%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%7B%20%22term%22%20%3A%20%7B%20%22maturity%22%20%3A%20%22released%22%20%7D%20%7D%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%7B%20%22term%22%20%3A%20%7B%20%22status%22%20%3A%20%22latest%22%20%7D%20%7D%0A%20%20%20%20%20%20%20%20%20%20%20%20%5D%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%7D'
```

### [The size of the CPAN unpacked](https://github.com/metacpan/metacpan-examples/blob/master/scripts/file/5-size-of-cpan.pl)
Expand All @@ -199,64 +201,68 @@ curl 'https://fastapi.metacpan.org/v1/release/_search?source=%7B%22query%22%3A%7

```sh
curl -XPOST https://fastapi.metacpan.org/v1/release/_search?size=100 -d '{
"query": {
"range" : {
"date" : {
"gte" : "2010-06-05T00:00:00",
"lte" : "2011-06-05T00:00:00"
"query" : {
"range" : {
"date" : {
"gte" : "2010-06-05T00:00:00",
"lte" : "2011-06-05T00:00:00"
}
}
}
},
"fields": ["license", "name", "distribution", "date", "version_numified"]
},
"fields": [ "license", "name", "distribution", "date", "version_numified" ]
}'
```

### Aggregate by license:

```sh
curl -XPOST https://fastapi.metacpan.org/v1/release/_search -d '{
"query": {
"match_all": {}
"query" : {
"match_all" : {}
},
"aggs": {
"license": {
"terms": {
"field": "license"
"aggs" : {
"license" : {
"terms" : {
"field" : "license"
}
}
},
"size": 0
"size" : 0
}'
```

### Most used file names in the root directory of releases:

```sh
curl -XPOST https://fastapi.metacpan.org/v1/file/_search -d '{
"query": { "filtered":{"query":{"match_all":{}},"filter":{"term":{"level":0}}}
},
"aggs": {
"license": {
"terms": {
"size":100,
"field":"name"
} } },
"size":0
"query" : {
"term" : { "level" : 0 }
},
"aggs" : {
"license" : {
"terms" : {
"size" : 100,
"field" : "name"
}
}
},
"size" : 0
}'
```

### Find all releases that contain a particular version of a module:

```sh
curl -XPOST https://fastapi.metacpan.org/v1/file/_search -d '{
"query": { "filtered":{
"query":{"match_all":{}},
"filter":{"and":[
{"term":{"module.name":"DBI::Profile"}},
{"term":{"module.version":"2.014123"}}
]}
}},
"fields":["release"]
"query" : {
"bool" : {
"must" : [
{ "term" : { "module.name" : "DBI::Profile" } },
{ "term" : { "module.version" : "2.014123" } }
]
}
},
"fields" : [ "release" ]
}'
```

Expand All @@ -277,50 +283,61 @@ Note that "size" should be the number of distributions you are looking for.
```sh
lynx --dump --post_data https://fastapi.metacpan.org/v1/release/_search <<EOL
{
"query" : { "terms" : { "distribution" : [
"Mojolicious",
"MetaCPAN-API",
"DBIx-Class"
] } },
"filter" : { "term" : { "status" : "latest" } },
"query" : {
"bool" : {
"must" : [
{
"terms" : {
"distribution" : [
"Mojolicious",
"MetaCPAN-API",
"DBIx-Class"
]
}
},
{ "term" : { "status" : "latest" } }
]
}
},
"fields" : [ "distribution", "version" ],
"size" : 3
"size" : 3
}
EOL
```

### Get a list of all files where the directory is false and the path is blank
```sh
curl -XPOST https://fastapi.metacpan.org/v1/file/_search -d '{
"query": {
"match_all": {}
},
"size": 1000,
"fields": [ "name", "status", "directory", "path", "distribution" ],
"filter": {
"and": [
{ "term": { "directory": false } }, { "term" : { "path" : "" } }
]
}
"query" : {
"bool" : {
"must" : [
{ "term" : { "directory" : false } },
{ "term" : { "path" : "" } }
]
}
},
"size" : 1000,
"fields" : [ "name", "status", "directory", "path", "distribution" ],
}'
```

### List releases which have an email address for a bugtracker, but not an url
```sh
curl -XPOST https://fastapi.metacpan.org/v1/release/_search -d '{
"query": {
"match_all": {}
},
"size": 10,
"fields": [ "name", "resources.bugtracker.mailto" ],
"filter": {
"and": [
{ "term": {"maturity": "released"} },
{ "term": {"status": "latest"} },
{ "exists" : { "field" : "resources.bugtracker.mailto" } },
{ "missing" : { "field" : "resources.bugtracker.web" } }
]
}
"query" : {
"bool" : {
"must" : [
{ "term" : {"maturity" : "released"} },
{ "term" : {"status" : "latest"} },
{ "exists" : { "field" : "resources.bugtracker.mailto" } }
],
"must_not" : [
{ "exists" : { "field" : "resources.bugtracker.web" } }
]
}
},
"size": 10,
"fields": [ "name", "resources.bugtracker.mailto" ],
}'
```

Expand All @@ -330,18 +347,19 @@ curl -XPOST https://fastapi.metacpan.org/v1/release/_search -d '{
### Search the current PDL documentation for the string `axisvals`
```sh
curl -XPOST https://fastapi.metacpan.org/v1/file/_search -d '{
"query" : { "filtered" : {
"query" : {
"query_string" : {
"query" : "axisvals",
"fields" : [ "pod.analyzed", "module.name" ] }
},
"filter" : { "and" : [
{ "term" : { "distribution" : "PDL" } },
{ "term" : { "status" : "latest" } }
]}
}},
"query" : {
"bool" : {
"must" : [
"query_string" : {
"query" : "axisvals",
"fields" : [ "pod.analyzed", "module.name" ]
},
{ "term" : { "distribution" : "PDL" } },
{ "term" : { "status" : "latest" } }
]
}
},
"fields" : [ "documentation", "abstract", "module.name" ],
"size" : 20
}'
}'
```
2 changes: 1 addition & 1 deletion lib/MetaCPAN/Document/Distribution.pm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ has river => (
sub releases {
my $self = shift;
return $self->index->type("release")
->filter( { term => { "distribution" => $self->name } } );
->query( { term => { "distribution" => $self->name } } );
}

sub set_first_release {
Expand Down
Loading

0 comments on commit 85d4f14

Please sign in to comment.