User:Francis Tyers/FST

Finite-state morphological analysis is the use of finite-state approaches to the problem of computational analysis of the morphology of natural languages.

Subsequential and $$p$$-Subsequential transducers
A subsequential finite-state transducer is a finite-state transducer which can have an extra string outputted at final states. A typical subsequential transducer only allows one extra string to be outputted, this is a problem when dealing with natural languages which contain ambiguity.

Tokenisation

 * Pre-tokenisation


 * Tokenise as you analyse