はじめに
自社でElasticsearchを使っているがあまり触ったことがなくて詳しくわからない、、導入したいと思っているがイマイチ勝手がわからない、、という方いるのではないでしょうか？
検索において、検索結果の並び順を最適化するのは非常に重要です。プロダクトによってはCTRやCVRが利益に直結します。今回はElasticsearchでどうやって並び順を制御しているのか？どうやって調整すればよいのかを実験したいと思います。
Elasticsearchはどうやって並び替えをしている？
基本的には関連度スコアというものを計算して並び順を決定しています。デフォルトではこの関連度スコアの高い順に並べられます。
他にもスクリプトで計算したスコアで並べ替えたり、複数のスコアを組み合わせて並びを決めることもできます。特定のfieldの値の単純ソートも可能です。
下記では具体例を交えながら実際に動かしてみたいと思います！
具体例を交えて実際に試してみる
前回はファッションサイトを例にフィルタリングと簡単な並べ替えについての記事を書きました。今回もファッションサイトを例にランキングの調整について実験してみたいと思います。
まずはindex作成とデータ投入を行います。
PUT clothes
{
"mappings": {
"properties": {
"name": {
"type": "text",
"analyzer": "kuromoji"
},
"description": {
"type": "text",
"analyzer": "kuromoji"
},
"category": {
"type": "keyword"
},
"size": {
"type": "keyword"
},
"color": {
"type": "keyword"
},
"price": {
"type": "integer"
},
"stock_quantity": {
"type": "integer"
},
"review_rating": {
"type": "float"
},
"number_of_reviews": {
"type": "integer"
}
}
}
}
POST _bulk
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"キレイめなパーカー","category":"トップス","size":["S","M","L","XL"],"color":["赤","紫","青"],"price":5000,"stock_quantity":12,"review_rating":4.2}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"キレイめなパーカー","category":"トップス","size":["M","L","XL"],"color":["白","黒","赤","緑","青"],"price":6000,"stock_quantity":3,"review_rating":3.7}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"キレイめなパーカー","category":"トップス","size":["M","L","XL"],"color":["白","黒","赤","緑","青"],"price":7000,"stock_quantity":58,"review_rating":2.1}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"キレイめなパーカー","category":"トップス","size":["S","M","L","XL"],"color":["白","黒","赤","緑","青"],"price":5000,"stock_quantity":22,"review_rating":4.0}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"キレイめなパーカー","category":"トップス","size":["M","L","XL"],"color":["白","緑","青"],"price":3000,"stock_quantity":9,"review_rating":2.5}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"キレイめなパーカー","category":"トップス","size":["S","M","L"],"color":["白","黒","赤","緑"],"price":5000,"stock_quantity":41,"review_rating":3.9}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"キレイめなパーカー","category":"トップス","size":["M","L"],"color":["白","緑","青"],"price":13000,"stock_quantity":10,"review_rating":5.0}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"キレイめなパーカー","category":"トップス","size":["S","M","L"],"color":["白","黒","赤","緑","青"],"price":5000,"stock_quantity":0,"review_rating":3.7}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"カジュアルなパーカー","category":"トップス","size":["M","L","XL"],"color":["白","黒","赤","緑","青"],"price":7000,"stock_quantity":10,"review_rating":1.2}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"カジュアルなパーカー","category":"トップス","size":["S","M","L"],"color":["白","黒","赤","緑","青"],"price":5000,"stock_quantity":46,"review_rating":3.2}
{"index": {"_index": "clothes"}}
{"name":"パーカー","description":"カジュアルなパーカー","category":"トップス","size":["M","L","XL"],"color":["白","黒","赤","緑"],"price":12000,"stock_quantity":21,"review_rating":4.5}
{"index": {"_index": "clothes"}}
{"name":"デニムパンツ","description":"キレイめなデニムパンツ","category":"ボトムス","size":["S","M","L","XL"],"color":["白","緑","青"],"price":5000,"stock_quantity":3,"review_rating":2.1}
{"index": {"_index": "clothes"}}
{"name":"デニムパンツ","description":"カジュアルなデニムパンツ","category":"ボトムス","size":["S","M","L","XL"],"color":["白","緑","青"],"price":7000,"stock_quantity":23,"review_rating":4.0}
{"index": {"_index": "clothes"}}
{"name":"スラックス","description":"カジュアルなスラックス","category":"ボトムス","size":["S","M","L","XL"],"color":["白","緑","青"],"price":7200,"stock_quantity":17,"review_rating":3.3}
{"index": {"_index": "clothes"}}
{"name":"スラックス","description":"キレイめなスラックス","category":"ボトムス","size":["S","M","L","XL"],"color":["白","緑","青"],"price":5000,"stock_quantity":24,"review_rating":3.9}
それでは1万円以下で青くてキレイめのXLサイズのパーカーを探してみましょう
GET clothes/_search
{
"size": 20,
"query": {
"bool": {
"filter": [
{
"range": {
"price": {
"lte": 10000
}
}
}
],
"should": [
{
"term": {
"category": {
"value": "トップス"
}
}
},
{
"match": {
"description": "キレイめ"
}
},
{
"term": {
"size": {
"value": "XL"
}
}
},
{
"term": {
"color": {
"value": "青"
}
}
}
],
"minimum_should_match": 3
}
}
}
↓検索結果
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 9,
"relation": "eq"
},
"max_score": 1.8559989,
"hits": [
{
"_index": "clothes",
"_id": "qGmN8pMBQoI2sJ2kY1PU",
"_score": 1.8559989,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"赤",
"紫",
"青"
],
"price": 5000,
"stock_quantity": 12,
"review_rating": 4.2
}
},
{
"_index": "clothes",
"_id": "qWmN8pMBQoI2sJ2kY1PU",
"_score": 1.8559989,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 6000,
"stock_quantity": 3,
"review_rating": 3.7
}
},
{
"_index": "clothes",
"_id": "qmmN8pMBQoI2sJ2kY1PU",
"_score": 1.8559989,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 7000,
"stock_quantity": 58,
"review_rating": 2.1
}
},
{
"_index": "clothes",
"_id": "q2mN8pMBQoI2sJ2kY1PU",
"_score": 1.8559989,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 22,
"review_rating": 4
}
},
{
"_index": "clothes",
"_id": "rGmN8pMBQoI2sJ2kY1PU",
"_score": 1.8559989,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"緑",
"青"
],
"price": 3000,
"stock_quantity": 9,
"review_rating": 2.5
}
},
{
"_index": "clothes",
"_id": "tmmN8pMBQoI2sJ2kY1PU",
"_score": 1.5257572,
"_source": {
"name": "スラックス",
"description": "キレイめなスラックス",
"category": "ボトムス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"白",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 24,
"review_rating": 3.9
}
},
{
"_index": "clothes",
"_id": "s2mN8pMBQoI2sJ2kY1PU",
"_score": 1.4240088,
"_source": {
"name": "デニムパンツ",
"description": "キレイめなデニムパンツ",
"category": "ボトムス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"白",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 3,
"review_rating": 2.1
}
},
{
"_index": "clothes",
"_id": "r2mN8pMBQoI2sJ2kY1PU",
"_score": 1.3932399,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"S",
"M",
"L"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 0,
"review_rating": 3.7
}
},
{
"_index": "clothes",
"_id": "sGmN8pMBQoI2sJ2kY1PU",
"_score": 1.0374895,
"_source": {
"name": "パーカー",
"description": "カジュアルなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 7000,
"stock_quantity": 10,
"review_rating": 1.2
}
}
]
}
}
パーカーが9件取得されており、上位5件が1万円以下で青くてキレイめのXLサイズのパーカーになっています。
しかし同じ条件のパーカーが5件もあって、この中で優劣を付けたいです。例では5件しかないですが、実際だと同じ条件のものが数百件も出てくることもあり、この中で如何に優劣をつけるかでCTR,CVRが大きく変わり売上に直結します。
データの中にレビュー評価という項目があるのでこちらを使ってみましょう。
filedの値をスコアに反映する方法としてfield_value_factorというものがあります。
GET clothes/_search
{
"size": 20,
"query": {
"function_score": {
"functions": [
{
"field_value_factor": {
"field": "review_rating",
"missing": 1
}
}
],
"query": {
"bool": {
"filter": [
{
"range": {
"price": {
"lte": 10000
}
}
}
],
"should": [
{
"term": {
"category": {
"value": "トップス"
}
}
},
{
"match": {
"description": "キレイめ"
}
},
{
"term": {
"size": {
"value": "XL"
}
}
},
{
"term": {
"color": {
"value": "青"
}
}
}
],
"minimum_should_match": 3
}
}
}
}
}
こちらを実行します。
↓検索結果
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 9,
"relation": "eq"
},
"max_score": 7.795195,
"hits": [
{
"_index": "clothes",
"_id": "qGmN8pMBQoI2sJ2kY1PU",
"_score": 7.795195,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"赤",
"紫",
"青"
],
"price": 5000,
"stock_quantity": 12,
"review_rating": 4.2
}
},
{
"_index": "clothes",
"_id": "q2mN8pMBQoI2sJ2kY1PU",
"_score": 7.4239955,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 22,
"review_rating": 4
}
},
{
"_index": "clothes",
"_id": "qWmN8pMBQoI2sJ2kY1PU",
"_score": 6.867196,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 6000,
"stock_quantity": 3,
"review_rating": 3.7
}
},
{
"_index": "clothes",
"_id": "tmmN8pMBQoI2sJ2kY1PU",
"_score": 5.9504533,
"_source": {
"name": "スラックス",
"description": "キレイめなスラックス",
"category": "ボトムス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"白",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 24,
"review_rating": 3.9
}
},
{
"_index": "clothes",
"_id": "r2mN8pMBQoI2sJ2kY1PU",
"_score": 5.1549873,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"S",
"M",
"L"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 0,
"review_rating": 3.7
}
},
{
"_index": "clothes",
"_id": "rGmN8pMBQoI2sJ2kY1PU",
"_score": 4.639997,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"緑",
"青"
],
"price": 3000,
"stock_quantity": 9,
"review_rating": 2.5
}
},
{
"_index": "clothes",
"_id": "qmmN8pMBQoI2sJ2kY1PU",
"_score": 3.8975976,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 7000,
"stock_quantity": 58,
"review_rating": 2.1
}
},
{
"_index": "clothes",
"_id": "s2mN8pMBQoI2sJ2kY1PU",
"_score": 2.9904184,
"_source": {
"name": "デニムパンツ",
"description": "キレイめなデニムパンツ",
"category": "ボトムス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"白",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 3,
"review_rating": 2.1
}
},
{
"_index": "clothes",
"_id": "sGmN8pMBQoI2sJ2kY1PU",
"_score": 1.2449875,
"_source": {
"name": "パーカー",
"description": "カジュアルなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 7000,
"stock_quantity": 10,
"review_rating": 1.2
}
}
]
}
}
評価が高いパーカーが上位に来るようになりました。
しかし、評価の高いスラックスがパーカーよりも上位に来てしまいました！いくら評価が高いとは言えボトムスがパーカーより上位にくるのは不適切でしょう。なぜパーカーよりもデニムが上位に来てしまったのでしょうか？
最初に叩いたクエリの実行結果の_scoreを見ると、上位5件はいずれも1.8559989となっています。今回field_value_factorを導入したことでレビュー評価の値が乗算されることになります。
式で書くとこうなっています。
レビュー評価 * 関連度スコア(1.8559989)
レビュー評価は1~5、となるので高評価の3.9のスラックスがパーカーを超えてしまっています。
パーカーがボトムスを超えないように、つまりカテゴリは超えないようにクエリを設計してみましょう。
GET clothes/_search
{
"size": 20,
"query": {
"function_score": {
"functions": [
{
"field_value_factor": {
"field": "review_rating",
"missing": 1
}
}
],
"score_mode": "sum",
"boost_mode": "sum",
"query": {
"bool": {
"must": [
{
"constant_score": {
"filter": {
"range": {
"price": {
"lte": 10000
}
}
},
"boost": 10
}
}
],
"should": [
{
"constant_score": {
"filter": {
"term": {
"category": "トップス"
}
},
"boost": 100
}
},
{
"match": {
"description": {
"query": "キレイめ",
"boost": 60
}
}
},
{
"constant_score": {
"filter": {
"term": {
"size": "XL"
}
},
"boost": 10
}
},
{
"constant_score": {
"filter": {
"term": {
"color": "青"
}
},
"boost": 10
}
}
],
"minimum_should_match": 3
}
}
}
}
}
↓検索結果
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 9,
"relation": "eq"
},
"max_score": 183.31056,
"hits": [
{
"_index": "clothes",
"_id": "qGmN8pMBQoI2sJ2kY1PU",
"_score": 183.31056,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"赤",
"紫",
"青"
],
"price": 5000,
"stock_quantity": 12,
"review_rating": 4.2
}
},
{
"_index": "clothes",
"_id": "q2mN8pMBQoI2sJ2kY1PU",
"_score": 183.11057,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 22,
"review_rating": 4
}
},
{
"_index": "clothes",
"_id": "qWmN8pMBQoI2sJ2kY1PU",
"_score": 182.81056,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 6000,
"stock_quantity": 3,
"review_rating": 3.7
}
},
{
"_index": "clothes",
"_id": "rGmN8pMBQoI2sJ2kY1PU",
"_score": 181.61057,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"緑",
"青"
],
"price": 3000,
"stock_quantity": 9,
"review_rating": 2.5
}
},
{
"_index": "clothes",
"_id": "qmmN8pMBQoI2sJ2kY1PU",
"_score": 181.21057,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 7000,
"stock_quantity": 58,
"review_rating": 2.1
}
},
{
"_index": "clothes",
"_id": "r2mN8pMBQoI2sJ2kY1PU",
"_score": 172.81056,
"_source": {
"name": "パーカー",
"description": "キレイめなパーカー",
"category": "トップス",
"size": [
"S",
"M",
"L"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 0,
"review_rating": 3.7
}
},
{
"_index": "clothes",
"_id": "sGmN8pMBQoI2sJ2kY1PU",
"_score": 131.2,
"_source": {
"name": "パーカー",
"description": "カジュアルなパーカー",
"category": "トップス",
"size": [
"M",
"L",
"XL"
],
"color": [
"白",
"黒",
"赤",
"緑",
"青"
],
"price": 7000,
"stock_quantity": 10,
"review_rating": 1.2
}
},
{
"_index": "clothes",
"_id": "tmmN8pMBQoI2sJ2kY1PU",
"_score": 83.01057,
"_source": {
"name": "スラックス",
"description": "キレイめなスラックス",
"category": "ボトムス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"白",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 24,
"review_rating": 3.9
}
},
{
"_index": "clothes",
"_id": "s2mN8pMBQoI2sJ2kY1PU",
"_score": 75.10566,
"_source": {
"name": "デニムパンツ",
"description": "キレイめなデニムパンツ",
"category": "ボトムス",
"size": [
"S",
"M",
"L",
"XL"
],
"color": [
"白",
"緑",
"青"
],
"price": 5000,
"stock_quantity": 3,
"review_rating": 2.1
}
}
]
}
}
今回は5件ある1万円以下で青くてキレイめのXLサイズのパーカーの中でレビュー評価順で並べることができました。
式で言うと、、、
レビュー評価(1~5) + カテゴリの関連度スコア(0~100) + その他の関連度スコア(0~90)
このような形になっています。足し算に変えてあることに注目してください。掛け算の場合は係数をつけると全体にかかってしまって、各検索条件個別に調整することが難しい状態でした。
レビュー評価は一の位、カテゴリの関連度スコアは百の位、その他のスコア合計は十の位でスコア付けするようにしています。カテゴリ(トップス、ボトムスなど)が百の位、レビュー評価を一の位にしているのでレビュー評価の値でカテゴリを超えてしまうことはない状態になっています。さらにサイズや色などの他のスコアは一致していれば一致しているほど加点となるがカテゴリのスコアは超えることがない設計となっています。
こうすることで理解しやすいかつ調整しやすくなります。
また検索順位を変えるということは開発者個人の判断ではなく、POなどの責任者の承認だったりします。
人への説明が必要だったりするので理解しやすさは重要になります。
いかがでしたでしょうか？
Elasticsearchでスコアを調整する方法と注意点などについて説明しました。Elasticsearchを触るきっかけとなれば幸いです！
最後に
スペースマーケットでは、一緒にサービスを成長させていく仲間を探しています。
ビジネスサイド、エンジニアメンバー共に話しやすいメンバーが多く非常に働きやすい環境だと思います！
ご興味ある方ぜひ見てみて下さい！
スペースを簡単に貸し借りできるサービス「スペースマーケット」のエンジニアによる公式ブログです。 弊社採用技術スタックはこちら -> whatweuse.dev/company/spacemarket
Discussion