Geoshape Example

[swill]

I have spent many hours the last couple days trying to figure out how to use the geoshape functionality provided by GeoJSON with an associated query.

So far, this is what have. I have a working implementation without using the geoshape components.

What I currently have:

// setup bleve
geoFieldMapping := bleve.NewGeoPointFieldMapping()
geoFieldMapping.Type = "geoshape"
docMapping := bleve.NewDocumentMapping()
docMapping.AddFieldMappingsAt("Geo", geoFieldMapping)
indexMapping := bleve.NewIndexMapping()
indexMapping.DefaultMapping = docMapping
bleveIndex, err = bleve.New(blevePath, indexMapping)
if err != nil {
    bleveIndex, err = bleve.Open(blevePath)
    if err != nil {
        log.Fatal(err.Error())
    }
}

The "Geo" field is a geoshape created with the following constructor.

ad.Geo = geojson.NewGeoJsonPoint([]float64{lon, lat})

I know that the GeoJSON stuff works because I am able to do some post query processing. For example, if I do a standard query (without geo data involved) which has pagination, it looks like this:

query := bleve.NewQueryStringQuery(strings.TrimSpace(post.Text))
bleveSearch := bleve.NewSearchRequest(query)
bleveSearch.Size = *pageSize
bleveSearch.From = post.From
bleveSearch.Fields = []string{"*"} // get all fields
switch post.SortBy {
case "star-price":
	bleveSearch.SortBy([]string{"-Starred", "Price", "-_score", "_id"})
case "age-price":
	bleveSearch.SortBy([]string{"-PostedDate", "Price", "-_score", "_id"})
case "auction-price":
	bleveSearch.SortBy([]string{"AuctionEnd", "-PostedDate", "Price", "-_score", "_id"})
default:
	bleveSearch.SortBy([]string{"Price", "-_score", "_id"})
}
result, err := bleveIndex.Search(bleveSearch)

I get back search results that include the "Geo" field. I can then filter the results of the query by the GeoJSON polygon functionality. So I can create a bounding polygon with something like this.

var polygon index.GeoJSON
filterByPolygon := false
if post.Polygon != "" {
	polygon, err = geojson.ParseGeoJSONShape([]byte(post.Polygon))
	if err == nil {
		filterByPolygon = true
	} else {
		log.Println("Polygon Error:", err.Error())
	}
}

Then, with this polygon, I can check if each of the query results are within that polygon using:

// while looping through the results
if filterByPolygon {
    if ad.Geo != nil {
        inside, err := polygon.Contains(*ad.Geo)
        if err != nil || !inside {
            continue
        }
        // if the result is inside the polygon, it continues to populate the objects to be returned
    }
}

In this case, I am able to reduce the results based on the result being within a polygon, but it totally messes up the pagination of the query results and how many results there are, etc... I know this is not the "right" way to do this, and instead, I should be able to add to my search query if the document is within the bounding polygon.

I am looking for a way to incorporate the filtering of the results based on if the document has a "Geo" field within the polygon specified in the query, rather than doing a different query and then filtering the results manually. The only documentation I can find on the geoshape is here (https://github.com/blevesearch/bleve/blob/master/geo/README.md), but it doesn't actually provide details on how to use the query in golang.

I have combed through the different examples available, but they are all written for the geopoint type, which are limited to searching in a bounding box or a distance from some point. I have tried to review all the different implementation tests I could find, but didn't find any for the querying of geoshapes. Additionally, I have started trying to dig into the implementation code to try to figure it out, but I still haven't found a good reference to know how to do what I am trying to do.

I am looking to do a "Conjunction" query with my search terms, and the results falling within a specified polygon, and with the results of the query only being the documents matching both conditions.

Any help would be appreciated. Thank you...

[abhinavdangeti]

@swill Correct me if I'm wrong in the assumption that you want to index a "point" and in the query you wish to specify a polygon to see if the point index lies within the polygon.

You can simple use bleve.NewGeoPointFieldMapping() and get rid of this override - geoFieldMapping.Type = "geoshape".
So your code should simply be -

// setup bleve
geoFieldMapping := bleve.NewGeoPointFieldMapping()
docMapping := bleve.NewDocumentMapping()
docMapping.AddFieldMappingsAt("Geo", geoFieldMapping)
indexMapping := bleve.NewIndexMapping()
indexMapping.DefaultMapping = docMapping
bleveIndex, err = bleve.New(blevePath, indexMapping)
if err != nil {
    bleveIndex, err = bleve.Open(blevePath)
    if err != nil {
        log.Fatal(err.Error())
    }
}

The summary from above - you can simply use the geopoint type while indexing your data.

Your query will then be bounding box query specifying the polygon coordinates. Here's a JSON example that you could unmarshal to bleve.SearchRequest -

{
  "query": {
    "field": "Geo",
    "polygon_points": [
      "37.79393211306212,-122.44234633404847",
      "37.77995881733997,-122.43977141339417",
      "37.788031092020155,-122.42925715405579",
      "37.79026946582319,-122.41149020154114",
      "37.79571192027403,-122.40735054016113",
      "37.79393211306212,-122.44234633404847"
    ]
  },
  "sort": [
    ...
  ],
  "from": ..,
  "size": ..
}

I'm thinking this should work for your use case.

---

If you wish to index some other shape (not a point) as supported here to do querying over, I'd recommend using this method to define the rules for indexing the field -

bleve/mapping.go

Line 73 in 135ba33

func NewGeoShapeFieldMapping() *mapping.FieldMapping {

Here's a unit test that can help you a bit in that situation -

bleve/search_test.go

Line 1938 in 135ba33

func TestGeoShapePolygonContainsPoint(t *testing.T) {

[swill]

You are absolutely right about what I wanted to be able to do. In looking at the references you pointed me at (which I somehow had not found yet), I realise that I completely missed this fundamental piece, being the query.NewGeoShapeQuery.

q, err := query.NewGeoShapeQuery(
	[][][][]float64{{{test.coordinates}}},
	geo.PointType,
	"contains",
)

I also didn't realise I could unmarshal JSON into a map[string]interface{} type and bleve would know how to work with it.

I think I have found the missing pieces that I was looking for to get this working. Thank you so much for taking the time to point me in the right direction. I will post back with my final solution once I get it working so others can benefit from this.