class Ferret::Analysis::LetterAnalyzer

Summary¶ ↑

A LetterAnalyzer creates a TokenStream that splits the input up into maximal strings of characters as recognized by the current locale. If implemented in Ruby it would look like;

class LetterAnalyzer
  def initialize(lower = true)
    @lower = lower
  end

  def token_stream(field, str)
    return LetterTokenizer.new(str, @lower)
  end
end

As you can see it makes use of the LetterTokenizer.

Public Class Methods

new(lower = true) → analyzer click to toggle source

Create a new LetterAnalyzer which downcases tokens by default but can optionally leave case as is. Lowercasing will be done based on the current locale.

lower: set to false if you don't want the field's tokens to be downcased

static VALUE
frb_letter_analyzer_init(int argc, VALUE *argv, VALUE self)
{
    Analyzer *a;
    GET_LOWER(true);
#ifndef POSH_OS_WIN32
    if (!frb_locale) frb_locale = setlocale(LC_CTYPE, "");
#endif
    a = mb_letter_analyzer_new(lower);
    Frt_Wrap_Struct(self, NULL, &frb_analyzer_free, a);
    object_add(a, self);
    return self;
}