Natural Language Processing with Java Cookbook
上QQ阅读APP看书,第一时间看更新

How to do it...

Let's go through the following steps:

  1. Add the following import statement to your project's class:
import java.util.ArrayList;
import java.util.Scanner;
  1. Add the following statements to the main method to declare the sample string, create an instance of the Scanner class, and add a list to hold the tokens:
String sampleText = 
"In addition, the rook was moved too far to be effective.";
Scanner scanner = new Scanner(sampleText);
ArrayList<String> list = new ArrayList<>();
  1. Insert the following loops to populate the list and display the tokens:
while (scanner.hasNext()) {
String token = scanner.next();
list.add(token);
}

for (String token : list) {
System.out.println(token);
}
  1. Execute the program. You should get the following output:
In
addition,
the
rook
was
moved
too
far
to
be
effective.