Suggester



The suggester object contains its configuration, which for the term suggester used in the preceding command, is the field we want to use for suggestions (the field property). We can also send more than one suggestion at a time by adding multiple suggestion names. Elsevier JournalFinder helps you find journals that could be best suited for publishing your scientific article. JournalFinder uses smart search technology and field-of-research specific vocabularies to match your paper’s abstract to scientific journals. Suggester is a tool to assist in the creation of songs and chord progressions. This app will help you find chords that work together. It is efficient and fun; use it to build musical phrases that. Over 2,500 journals. Search all Springer and BMC journals to find the most suitable journal for your manuscript. Get quick Video Game recommendations by typing in a few games you enjoy, based on data collected from 300,000+ gamers.

Suggests similar looking terms based on a provided text by using a suggester.Parts of the suggest feature are still under development.

The suggest feature suggests similar looking terms based on a provided text byusing a suggester. The suggest request part is defined alongside the query partin a _search request. If the query part is left out, only suggestions arereturned.

Several suggestions can be specified per request. Each suggestion is identifiedwith an arbitrary name. In the example below two suggestions are requested. Bothmy-suggest-1 and my-suggest-2 suggestions use the term suggester, but havea different text.

The below suggest response example includes the suggestion response formy-suggest-1 and my-suggest-2. Each suggestion part containsentries. Each entry is effectively a token from the suggest text andcontains the suggestion entry text, the original start offset and lengthin the suggest text and if found an arbitrary number of options.

Each options array contains an option object that includes thesuggested text, its document frequency and score compared to the suggestentry text. The meaning of the score depends on the used suggester. Theterm suggester’s score is based on the edit distance.

Global suggest textedit
Suggester

To avoid repetition of the suggest text, it is possible to define aglobal text. In the example below the suggest text is defined globallyand applies to the my-suggest-1 and my-suggest-2 suggestions.

Suggester

The suggest text can in the above example also be specified assuggestion specific option. The suggest text specified on suggestionlevel override the suggest text on the global level.

The term suggester suggests terms based on edit distance. The providedsuggest text is analyzed before terms are suggested. The suggested termsare provided per analyzed suggest text token. The term suggesterdoesn’t take the query into account that is part of request.

Suggester

text

The suggest text. The suggest text is a required option thatneeds to be set globally or per suggestion.

field

The field to fetch the candidate suggestions from. This isa required option that either needs to be set globally or persuggestion.

analyzer

The analyzer to analyse the suggest text with. Defaultsto the search analyzer of the suggest field.

size

The maximum corrections to be returned per suggest texttoken.

sort

Defines how suggestions should be sorted per suggest textterm. Two possible values:

  • score: Sort by score first, then document frequency andthen the term itself.
  • frequency: Sort by document frequency first, then similarityscore and then the term itself.

suggest_mode

The suggest mode controls what suggestions areincluded or controls for what suggest text terms, suggestions should besuggested. Three possible values can be specified:

  • missing: Only provide suggestions for suggest text terms that arenot in the index. This is the default.
  • popular: Only suggest suggestions that occur in more docs thanthe original suggest text term.
  • always: Suggest any matching suggestions based on terms in thesuggest text.

max_edits

The maximum edit distance candidate suggestions canhave in order to be considered as a suggestion. Can only be a valuebetween 1 and 2. Any other value results in a bad request error beingthrown. Defaults to 2.

prefix_length

The number of minimal prefix characters that mustmatch in order be a candidate for suggestions. Defaults to 1. Increasingthis number improves spellcheck performance. Usually misspellings don’toccur in the beginning of terms.

min_word_length

The minimum length a suggest text term must have inorder to be included. Defaults to 4.

shard_size

Sets the maximum number of suggestions to be retrievedfrom each individual shard. During the reduce phase only the top Nsuggestions are returned based on the size option. Defaults to thesize option. Setting this to a value higher than the size can beuseful in order to get a more accurate document frequency for spellingcorrections at the cost of performance. Due to the fact that terms arepartitioned amongst shards, the shard level document frequencies ofspelling corrections may not be precise. Increasing this will make thesedocument frequencies more precise.

max_inspections

A factor that is used to multiply with theshards_size in order to inspect more candidate spelling corrections onthe shard level. Can improve accuracy at the cost of performance.Defaults to 5.

min_doc_freq

The minimal threshold in number of documents asuggestion should appear in. This can be specified as an absolute numberor as a relative percentage of number of documents. This can improvequality by only suggesting high frequency terms. Defaults to 0f and isnot enabled. If a value higher than 1 is specified, then the numbercannot be fractional. The shard level document frequencies are used forthis option.

max_term_freq

The maximum threshold in number of documents in which asuggest text token can exist in order to be included. Can be a relativepercentage number (e.g., 0.4) or an absolute number to represent documentfrequencies. If a value higher than 1 is specified, then fractional cannot be specified. Defaults to 0.01f. This can be used to exclude highfrequency terms — which are usually spelled correctly — from being spellchecked.This also improves the spellcheck performance. The shard level document frequenciesare used for this option.

string_distance

Which string distance implementation to use for comparing how similarsuggested terms are. Five possible values can be specified:

  • internal: The default based on damerau_levenshtein but highly optimizedfor comparing string distance for terms inside the index.
  • damerau_levenshtein: String distance algorithm based onDamerau-Levenshtein algorithm.
  • levenshtein: String distance algorithm based on Levenshtein edit distancealgorithm.
  • jaro_winkler: String distance algorithm based on Jaro-Winkler algorithm.
  • ngram: String distance algorithm based on character n-grams.

The term suggester provides a very convenient API to access wordalternatives on a per token basis within a certain string distance. The APIallows accessing each token in the stream individually whilesuggest-selection is left to the API consumer. Yet, often pre-selectedsuggestions are required in order to present to the end-user. Thephrase suggester adds additional logic on top of the term suggesterto select entire corrected phrases instead of individual tokens weightedbased on ngram-language models. In practice this suggester will beable to make better decisions about which tokens to pick based onco-occurrence and frequencies.

In general the phrase suggester requires special mapping up front to work.The phrase suggester examples on this page need the following mapping towork. The reverse analyzer is used only in the last example.

Once you have the analyzers and mappings set up you can use the phrasesuggester in the same spot you’d use the term suggester:

The response contains suggestions scored by the most likely spelling correction first. In this case we received the expected correction 'nobel prize'.

field

The name of the field used to do n-gram lookups for thelanguage model, the suggester will use this field to gain statistics toscore corrections. This field is mandatory.

gram_size

Sets max size of the n-grams (shingles) in the field.If the field doesn’t contain n-grams (shingles), this should be omittedor set to 1. Note that Elasticsearch tries to detect the gram sizebased on the specified field. If the field uses a shingle filter, thegram_size is set to the max_shingle_size if not explicitly set.

real_word_error_likelihood

The likelihood of a term beingmisspelled even if the term exists in the dictionary. The default is0.95, meaning 5% of the real words are misspelled.

confidence

The confidence level defines a factor applied to theinput phrases score which is used as a threshold for other suggestcandidates. Only candidates that score higher than the threshold will beincluded in the result. For instance a confidence level of 1.0 willonly return suggestions that score higher than the input phrase. If setto 0.0 the top N candidates are returned. The default is 1.0.

max_errors

The maximum percentage of the termsconsidered to be misspellings in order to form a correction. This methodaccepts a float value in the range [0..1) as a fraction of the actualquery terms or a number >=1 as an absolute number of query terms. Thedefault is set to 1.0, meaning only corrections withat most one misspelled term are returned. Note that setting this too highcan negatively impact performance. Low values like 1 or 2 are recommended;otherwise the time spend in suggest calls might exceed the time spend inquery execution.

separator

The separator that is used to separate terms in thebigram field. If not set the whitespace character is used as aseparator.

size

The number of candidates that are generated for eachindividual query term. Low numbers like 3 or 5 typically produce goodresults. Raising this can bring up terms with higher edit distances. Thedefault is 5.

analyzer

Sets the analyzer to analyze to suggest text with.Defaults to the search analyzer of the suggest field passed via field.

shard_size

Sets the maximum number of suggested terms to beretrieved from each individual shard. During the reduce phase, only thetop N suggestions are returned based on the size option. Defaults to5.

text

Sets the text / query to provide suggestions for.

highlight

Sets up suggestion highlighting. If not provided thenno highlighted field is returned. If provided mustcontain exactly pre_tag and post_tag, which arewrapped around the changed tokens. If multiple tokensin a row are changed the entire phrase of changed tokensis wrapped rather than each token.

collate

Checks each suggestion against the specified query to prune suggestionsfor which no matching docs exist in the index. The collate query for asuggestion is run only on the local shard from which the suggestion hasbeen generated from. The query must be specified and it can be templated,see search templates for more information.The current suggestion is automatically made available as the {{suggestion}}variable, which should be used in your query. You can still specifyyour own template params — the suggestion value will be added to thevariables you specify. Additionally, you can specify a prune to controlif all phrase suggestions will be returned; when set to true the suggestionswill have an additional option collate_match, which will be true ifmatching documents for the phrase was found, false otherwise.The default value for prune is false.

Suggester

This query will be run once for every suggestion.

The {{suggestion}} variable will be replaced by the textof each suggestion.

An additional field_name variable has been specified inparams and is used by the match query.

All suggestions will be returned with an extra collate_matchoption indicating whether the generated phrase matched anydocument.

Suggester

The phrase suggester supports multiple smoothing models to balanceweight between infrequent grams (grams (shingles) are not existing inthe index) and frequent grams (appear at least once in the index). Thesmoothing model can be selected by setting the smoothing parameterto one of the following options. Each smoothing model supports specificproperties that can be configured.

stupid_backoff

A simple backoff model that backs off to lowerorder n-gram models if the higher order count is 0 and discounts thelower order n-gram model by a constant factor. The default discount is0.4. Stupid Backoff is the default model.

laplace

A smoothing model that uses an additive smoothing where aconstant (typically 1.0 or smaller) is added to all counts to balanceweights. The default alpha is 0.5.

linear_interpolation

A smoothing model that takes the weightedmean of the unigrams, bigrams, and trigrams based on user suppliedweights (lambdas). Linear Interpolation doesn’t have any default values.All parameters (trigram_lambda, bigram_lambda, unigram_lambda)must be supplied.

The phrase suggester uses candidate generators to produce a list ofpossible terms per term in the given text. A single candidate generatoris similar to a term suggester called for each individual term in thetext. The output of the generators is subsequently scored in combinationwith the candidates from the other terms for suggestion candidates.

Currently only one type of candidate generator is supported, thedirect_generator. The Phrase suggest API accepts a list of generatorsunder the key direct_generator; each of the generators in the list iscalled per term in the original text.

The direct generators support the following parameters:

field

The field to fetch the candidate suggestions from. This isa required option that either needs to be set globally or persuggestion.

size

The maximum corrections to be returned per suggest text token.

suggest_mode

The suggest mode controls what suggestions are included on the suggestionsgenerated on each shard. All values other than always can be thought ofas an optimization to generate fewer suggestions to test on each shard andare not rechecked when combining the suggestions generated on eachshard. Thus missing will generate suggestions for terms on shards that donot contain them even if other shards do contain them. Those should befiltered out using confidence. Three possible values can be specified:

  • missing: Only generate suggestions for terms that are not in theshard. This is the default.
  • popular: Only suggest terms that occur in more docs on the shard thanthe original term.
  • always: Suggest any matching suggestions based on terms in thesuggest text.

max_edits

The maximum edit distance candidate suggestions can havein order to be considered as a suggestion. Can only be a value between 1and 2. Any other value results in a bad request error being thrown.Defaults to 2.

prefix_length

The number of minimal prefix characters that mustmatch in order be a candidate suggestions. Defaults to 1. Increasingthis number improves spellcheck performance. Usually misspellings don’toccur in the beginning of terms.

min_word_length

The minimum length a suggest text term must have inorder to be included. Defaults to 4.

max_inspections

A factor that is used to multiply with theshards_size in order to inspect more candidate spelling corrections onthe shard level. Can improve accuracy at the cost of performance.Defaults to 5.

min_doc_freq

The minimal threshold in number of documents asuggestion should appear in. This can be specified as an absolute numberor as a relative percentage of number of documents. This can improvequality by only suggesting high frequency terms. Defaults to 0f and isnot enabled. If a value higher than 1 is specified, then the numbercannot be fractional. The shard level document frequencies are used forthis option.

max_term_freq

The maximum threshold in number of documents in which asuggest text token can exist in order to be included. Can be a relativepercentage number (e.g., 0.4) or an absolute number to represent documentfrequencies. If a value higher than 1 is specified, then fractional cannot be specified. Defaults to 0.01f. This can be used to exclude highfrequency terms — which are usually spelled correctly — from being spellchecked. This also improves the spellcheckperformance. The shard level document frequencies are used for thisoption.

pre_filter

A filter (analyzer) that is applied to each of thetokens passed to this candidate generator. This filter is applied to theoriginal token before candidates are generated.

post_filter

A filter (analyzer) that is applied to each of thegenerated tokens before they are passed to the actual phrase scorer.

The following example shows a phrase suggest call with two generators:the first one is using a field containing ordinary indexed terms, and thesecond one uses a field that uses terms indexed with a reverse filter(tokens are index in reverse order). This is used to overcome the limitationof the direct generators to require a constant prefix to providehigh-performance suggestions. The pre_filter and post_filter optionsaccept ordinary analyzer names.

pre_filter and post_filter can also be used to inject synonyms aftercandidates are generated. For instance for the query captain usq wemight generate a candidate usa for the term usq, which is a synonym foramerica. This allows us to present captain america to the user if thisphrase scores high enough.

The completion suggester provides auto-complete/search-as-you-typefunctionality. This is a navigational feature to guide users torelevant results as they are typing, improving search precision.It is not meant for spell correction or did-you-mean functionalitylike the term or phrase suggesters.

Ideally, auto-complete functionality should be as fast as a usertypes to provide instant feedback relevant to what a user has alreadytyped in. Hence, completion suggester is optimized for speed.The suggester uses data structures that enable fast lookups,but are costly to build and are stored in-memory.

To use this feature, specify a special mapping for this field,which indexes the field values for fast completions.

Mapping supports the following parameters:

analyzer

The index analyzer to use, defaults to simple.

search_analyzer

The search analyzer to use, defaults to value of analyzer.

preserve_separators

Preserves the separators, defaults to true.If disabled, you could find a field starting with Foo Fighters, if yousuggest for foof.

preserve_position_increments

Enables position increments, defaults to true.If disabled and using stopwords analyzer, you could get afield starting with The Beatles, if you suggest for b. Note: Youcould also achieve this by indexing two inputs, Beatles andThe Beatles, no need to change a simple analyzer, if you are able toenrich your data.

max_input_length

Limits the length of a single input, defaults to 50 UTF-16 code points.This limit is only used at index time to reduce the total number ofcharacters per input string in order to prevent massive inputs frombloating the underlying datastructure. Most use cases won’t be influencedby the default value since prefix completions seldom grow beyond prefixes longerthan a handful of characters.

You index suggestions like any other field. A suggestion is made of aninput and an optional weight attribute. An input is the expectedtext to be matched by a suggestion query and the weight determines howthe suggestions will be scored. Indexing a suggestion is as follows:

The following parameters are supported:

input

The input to store, this can be an array of strings or justa string. This field is mandatory.

This value cannot contain the following UTF-16 control characters:

  • u0000 (null)
  • u001f (information separator one)
  • u001e (information separator two)

weight

A positive integer or a string containing a positive integer,which defines a weight and allows you to rank your suggestions.This field is optional.

You can index multiple suggestions for a document as follows:

You can use the following shorthand form. Note that you can not specifya weight with suggestion(s) in the shorthand form.

Suggesting works as usual, except that you have to specify the suggesttype as completion. Suggestions are near real-time, which meansnew suggestions can be made visible by refresh anddocuments once deleted are never shown. This request:

Prefix used to search for suggestions

Type of suggestions

Name of the field to search for suggestions in

returns this response:

_source metadata field must be enabled, which is the defaultbehavior, to enable returning _source with suggestions.

The configured weight for a suggestion is returned as _score. Thetext field uses the input of your indexed suggestion. Suggestionsreturn the full document _source by default. The size of the _sourcecan impact performance due to disk fetch and network transport overhead.To save some network overhead, filter out unnecessary fields from the _sourceusing source filtering to minimize_source size. Note that the _suggest endpoint doesn’t support sourcefiltering but using suggest on the _search endpoint does:

Filter the source to return only the suggest field

Name of the field to search for suggestions in

Number of suggestions to return

Which should look like:

The basic completion suggester query supports the following parameters:

field

The name of the field on which to run the query (required).

size

The number of suggestions to return (defaults to 5).

skip_duplicates

Whether duplicate suggestions should be filtered out (defaults to false).

The completion suggester considers all documents in the index.See Context Suggester for an explanation of how to query a subset ofdocuments instead.

In case of completion queries spanning more than one shard, the suggestis executed in two phases, where the last phase fetches the relevant documentsfrom shards, implying executing completion requests against a single shard ismore performant due to the document fetch overhead when the suggest spansmultiple shards. To get best performance for completions, it is recommended toindex completions into a single shard index. In case of high heap usage due toshard size, it is still recommended to break index into multiple shards insteadof optimizing for completion performance.

Queries can return duplicate suggestions coming from different documents.It is possible to modify this behavior by setting skip_duplicates to true.When set, this option filters out documents with duplicate suggestions from the result.

When set to true, this option can slow down search because more suggestionsneed to be visited to find the top N.

The completion suggester also supports fuzzy queries — this meansyou can have a typo in your search and still get results back.

Suggestions that share the longest prefix to the query prefix willbe scored higher.

The fuzzy query can take specific fuzzy parameters.The following parameters are supported:

fuzziness

The fuzziness factor, defaults to AUTO.See Fuzziness for allowed settings.

transpositions

if set to true, transpositions are countedas one change instead of two, defaults to true

min_length

Minimum length of the input before fuzzysuggestions are returned, defaults 3

prefix_length

Minimum length of the input, which is notchecked for fuzzy alternatives, defaults to 1

unicode_aware

If true, all measurements (like fuzzy editdistance, transpositions, and lengths) aremeasured in Unicode code points instead ofin bytes. This is slightly slower than rawbytes, so it is set to false by default.

If you want to stick with the default values, but still use fuzzy, you can either use fuzzy: {} or fuzzy: true.

The completion suggester also supports regex queries meaningyou can express a prefix as a regular expression

The regex query can take specific regex parameters.The following parameters are supported:

flags

Possible flags are ALL (default), ANYSTRING, COMPLEMENT,EMPTY, INTERSECTION, INTERVAL, or NONE. See regexp-syntaxfor their meaning

max_determinized_states

Regular expressions are dangerous because it’s easy to accidentallycreate an innocuous looking one that requires an exponential number ofinternal determinized automaton states (and corresponding RAM and CPU)for Lucene to execute. Lucene prevents these using themax_determinized_states setting (defaults to 10000). You can raisethis limit to allow more complex regular expressions to execute.

The completion suggester considers all documents in the index, but it is oftendesirable to serve suggestions filtered and/or boosted by some criteria.For example, you want to suggest song titles filtered by certain artists oryou want to boost song titles based on their genre.

To achieve suggestion filtering and/or boosting, you can add context mappings whileconfiguring a completion field. You can define multiple context mappings for acompletion field.Every context mapping has a unique name and a type. There are two types: categoryand geo. Context mappings are configured under the contexts parameter inthe field mapping.

It is mandatory to provide a context when indexing and querying a context enabled completion field.

The following defines types, each with two context mappings for a completionfield:

Defines a category context named place_type where the categories must besent with the suggestions.

Defines a geo context named location where the categories must be sentwith the suggestions.

Defines a category context named place_type where the categories areread from the cat field.

Defines a geo context named location where the categories are read fromthe loc field.

Adding context mappings increases the index size for completion field. The completion indexis entirely heap resident, you can monitor the completion field index size using Index stats.

Category Contextedit

The category context allows you to associate one or more categories with suggestions at indextime. At query time, suggestions can be filtered and boosted by their associated categories.

The mappings are set up like the place_type fields above. If path is definedthen the categories are read from that path in the document, otherwise they mustbe sent in the suggest field like this:

These suggestions will be associated with cafe and food category.

If the mapping had a path then the following index request would be enough toadd the categories:

These suggestions will be associated with cafe and food category.

If context mapping references another field and the categoriesare explicitly indexed, the suggestions are indexed with both setof categories.

Category Queryedit

Suggestions can be filtered by one or more categories. The followingfilters suggestions by multiple categories:

If multiple categories or category contexts are set on the querythey are merged as a disjunction. This means that suggestions matchif they contain at least one of the provided context values.

Suggestions with certain categories can be boosted higher than others.The following filters suggestions by categories and additionally boostssuggestions associated with some categories:

The context query filter suggestions associated withcategories cafe and restaurants and boosts thesuggestions associated with restaurants by afactor of 2

In addition to accepting category values, a context query can be composed ofmultiple category context clauses. The following parameters are supported for acategory context clause:

context

The value of the category to filter/boost on.This is mandatory.

boost

The factor by which the score of the suggestionshould be boosted, the score is computed bymultiplying the boost with the suggestion weight,defaults to 1

prefix

Whether the category value should be treated as aprefix or not. For example, if set to true,you can filter category of type1, type2 andso on, by specifying a category prefix of type.Defaults to false

If a suggestion entry matches multiple contexts the final score is computed as themaximum score produced by any matching contexts.

Suggester Discord Bot

Geo location Contextedit

A geo context allows you to associate one or more geo points or geohashes with suggestionsat index time. At query time, suggestions can be filtered and boosted if they are withina certain distance of a specified geo location.

Internally, geo points are encoded as geohashes with the specified precision.

Geo Mappingedit

In addition to the path setting, geo context mapping accepts the following settings:

precision

This defines the precision of the geohash to be indexed and can be specifiedas a distance value (5m, 10km etc.), or as a raw geohash precision (1..12).Defaults to a raw geohash precision value of 6.

The index time precision setting sets the maximum geohash precision thatcan be used at query time.

Indexing geo contextsedit

geo contexts can be explicitly set with suggestions or be indexed from a geo point field in thedocument via the path parameter, similar to category contexts. Associating multiple geo location contextwith a suggestion, will index the suggestion for every geo location. The following indexes a suggestionwith two geo location contexts:

Suggester Bot

Geo location Queryedit

Suggestions can be filtered and boosted with respect to how close they are to one ormore geo points. The following filters suggestions that fall within the area represented bythe encoded geohash of a geo point:

When a location with a lower precision at query time is specified, all suggestionsthat fall within the area will be considered.

If multiple categories or category contexts are set on the querythey are merged as a disjunction. This means that suggestions matchif they contain at least one of the provided context values.

Suggestions that are within an area represented by a geohash can also be boosted higherthan others, as shown by the following:

The context query filters for suggestions that fall underthe geo location represented by a geohash of (43.662, -79.380)with a precision of 2 and boosts suggestionsthat fall under the geohash representation of (43.6624803, -79.3863353)with a default precision of 6 by a factor of 2

If a suggestion entry matches multiple contexts the final score is computed as themaximum score produced by any matching contexts.

Suggester Definition

Suggester

In addition to accepting context values, a context query can be composed ofmultiple context clauses. The following parameters are supported for ageo context clause:

Suggester Bot

context

A geo point object or a geo hash string to filter orboost the suggestion by. This is mandatory.

boost

The factor by which the score of the suggestionshould be boosted, the score is computed bymultiplying the boost with the suggestion weight,defaults to 1

precision

The precision of the geohash to encode the query geo point.This can be specified as a distance value (5m, 10km etc.),or as a raw geohash precision (1..12).Defaults to index time precision level.

neighbours

Accepts an array of precision values at whichneighbouring geohashes should be taken into account.precision value can be a distance value (5m, 10km etc.)or a raw geohash precision (1..12). Defaults togenerating neighbours for index time precision level.

Sometimes you need to know the exact type of a suggester in order to parse its results. The typed_keys parameter can be used to change the suggester’s name in the response so that it will be prefixed by its type.

Considering the following example with two suggesters term and phrase:

In the response, the suggester names will be changed to respectively term#my-first-suggester andphrase#my-second-suggester, reflecting the types of each suggestion:

Suggester App

The name my-first-suggester now contains the term prefix.

The name my-second-suggester now contains the phrase prefix.

Suggest Discord Bot

Most Popular