Tokenizer

Tokenizer is a java-library that splits an input-stream into tokens. A parser can use this as input.

A subversion-repository with anonymous read-access is available under http://svn.mnementh.de/Tokenizer/.