ES的dense vector字段类型

PUT my-index
{
  "mappings": {
    "properties": {
      "my_vector": {
        "type": "dense_vector",
        "dims": 3
      },
      "my_text" : {
        "type" : "keyword"
      }
    }
  }
}

PUT my-index/_doc/1
{
  "my_text" : "text1",
  "my_vector" : [0.5, 10, 6]
}

PUT my-index/_doc/2
{
  "my_text" : "text2",
  "my_vector" : [-0.5, 10, 10]
}

Parameters for dense vector fields

The following mapping parameters are accepted:

element_type

(Optional, string) The data type used to encode vectors. The supported data types are float (default) and byte. float indexes a 4-byte floating-point value per dimension. byte indexes a 1-byte integer value per dimension. Using byte can result in a substantially smaller index size with the trade off of lower precision. Vectors using byte require dimensions with integer values between -128 to 127, inclusive for both indexing and searching.

dims

(Optional, integer) Number of vector dimensions. Can’t exceed 4096. If dims is not specified, it will be set to the length of the first vector added to the field.

index

(Optional, Boolean) If true, you can search this field using the kNN search API. Defaults to true.

similarity

(Optional*, string) The vector similarity metric to use in kNN search. Documents are ranked by their vector field’s similarity to the query vector. The _score of each document will be derived from the similarity, in a way that ensures scores are positive and that a larger score corresponds to a higher ranking. Defaults to cosine.

* This parameter can only be specified when index is true.

Valid values for similarity

l2_norm
Computes similarity based on the L2 distance (also known as Euclidean distance) between the vectors. The document _score is computed as 1 / (1 + l2_norm(query, vector)^2).
dot_product
Computes the dot product of two unit vectors. This option provides an optimized way to perform cosine similarity. The constraints and computed score are defined by element_type.
When element_type is float, all vectors must be unit length, including both document and query vectors. The document _score is computed as (1 + dot_product(query, vector)) / 2.
When element_type is byte, all vectors must have the same length including both document and query vectors or results will be inaccurate. The document _score is computed as 0.5 + (dot_product(query, vector) / (32768 * dims)) where dims is the number of dimensions per vector.
cosine
Computes the cosine similarity. Note that the most efficient way to perform cosine similarity is to normalize all vectors to unit length, and instead use dot_product. You should only use cosine if you need to preserve the original vectors and cannot normalize them in advance. The document _score is computed as (1 + cosine(query, vector)) / 2. The cosine similarity does not allow vectors with zero magnitude, since cosine is not defined in this case.
max_inner_product
Computes the maximum inner product of two vectors. This is similar to dot_product, but doesn’t require vectors to be normalized. This means that each vector’s magnitude can significantly effect the score. The document _score is adjusted to prevent negative values. For max_inner_product values < 0, the _score is 1 / (1 + -1 * max_inner_product(query, vector)). For non-negative max_inner_product results the _score is calculated max_inner_product(query, vector) + 1.

Although they are conceptually related, the similarity parameter is different from text field similarity and accepts a distinct set of options.

index_options

(Optional*, object) An optional section that configures the kNN indexing algorithm. The HNSW algorithm has two internal parameters that influence how the data structure is built. These can be adjusted to improve the accuracy of results, at the expense of slower indexing speed. When index_options is provided, all of its properties must be defined.

* This parameter can only be specified when index is true.

+ .Properties of index_options

Details

type

(Required, string) The type of kNN algorithm to use. Currently only hnsw is supported.

m

(Required, integer) The number of neighbors each node will be connected to in the HNSW graph. Defaults to 16.

ef_construction

(Required, integer) The number of candidates to track while assembling the list of nearest neighbors for each new node. Defaults to 100.

Synthetic _source

Synthetic _source is Generally Available only for TSDB indices (indices that have index.mode set to time_series). For other indices synthetic _source is in technical preview. Features in technical preview may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

dense_vector fields support synthetic _source .