apache > cocoon

- Home

# Lexical Transformer

The lexical transformer tokenizes the content of special marked elements of a SAX stream, by using a lexicon file.

- Name: lexer
- Class: org.apache.cocoon.transformation.LexicalTransformer
- Cacheable: yes - uses the last modification date of the lexicon document for validation.

The lexer parses the following elements from the SAX stream, and replaces them through generated documents.

    <text xmlns="http://chaperon.sourceforge.net/schema/text/1.0">
     [Text, which should be parsed]
    </text>

The lexical transformer will replace these elements by a list of lexemes (tokens).

    <lexemes xmlns="http://chaperon.sourceforge.net/schema/lexemes/1.0">
     <lexeme symbol="..." text="..."/>
     <lexeme symbol="..." text="..."/>
     <lexeme symbol="..." text="..."/>
    </lexemes>

A detailed explanation of function and the lexicon format can be found at Chaperon.

Errors and Improvements? If you see any errors or potential improvements in this document please help us: View, Edit or comment on the latest development version (registration required).