Class ParagraphPreservingWhitespaceCharSequenceNormalizer

java.lang.Object
opennlp.tools.util.normalizer.ParagraphPreservingWhitespaceCharSequenceNormalizer
All Implemented Interfaces:
Serializable, opennlp.tools.util.normalizer.CharSequenceNormalizer, opennlp.tools.util.normalizer.OffsetAwareNormalizer
public class ParagraphPreservingWhitespaceCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.OffsetAwareNormalizer
An OffsetAwareNormalizer that unwraps hard-wrapped prose while keeping real paragraph breaks. Within each whitespace run, at most one logical line break (including CRLF as a single break) collapses to a single space; two or more collapse to one newline (U+000A). Leading and trailing whitespace is trimmed.

This is the form wanted before sentence detection on Gutenberg-style or other fixed-width plain text: intra-paragraph line wraps become spaces, blank-line paragraph boundaries survive as newlines. It reuses the cursor based CharClass.collapseParagraphPreserving(CharSequence, CodePointSet, int) engine, so it recognizes the full Unicode White_Space set with no regular expression.

For display-oriented text where every line break should survive, use LineBreakPreservingWhitespaceCharSequenceNormalizer instead. For raw markdown with lists, code blocks, or other single-newline structure, apply this normalizer only to extracted prose.

See Also:

  • Constructor Details

    • ParagraphPreservingWhitespaceCharSequenceNormalizer

      public ParagraphPreservingWhitespaceCharSequenceNormalizer()

  • Method Details

    • getInstance

      public static ParagraphPreservingWhitespaceCharSequenceNormalizer getInstance()
      Returns the shared, stateless instance.
      Returns:
      the shared, stateless instance

    • normalize

      public CharSequence normalize(CharSequence text)
      Specified by:
      normalize in interface opennlp.tools.util.normalizer.CharSequenceNormalizer

    • normalizeAligned

      public opennlp.tools.util.normalizer.AlignedText normalizeAligned(CharSequence text)
      Specified by:
      normalizeAligned in interface opennlp.tools.util.normalizer.OffsetAwareNormalizer