BufferedTokenizer extracts token delimited entities from a sequence of
arbitrary inputs.