Java Convert Text into Tokens or Code Points

Understand the idea

split uses a regular expression and can discard trailing empty results unless a limit changes that behaviour. toCharArray copies code units, and codePoints().toArray produces code points. Those results are not interchangeable. A delimiter-based exercise should specify whether empty fields are meaningful, and should not claim to parse full CSV when quoting and escaping rules are absent.

Run it locally

Use a local JDK 21 or newer. Save the complete example as Main.java in a separate lab folder, then compile and run it with the commands below. Java examples do not run in Bhylu’s browser playground. The -ea option enables assertion checks used by some lessons.

JDK and terminal setup

javac --release 21 -encoding UTF-8 -d out Main.java
java -ea -cp out Main

A worked example

public class Main {
    public static void main(String[] args) {
        String record = "Clay|4|";
        String[] fields = record.split("\\|", -1);
        System.out.println(fields.length);
        System.out.println(fields[2].isEmpty());
    }
}

Expected output

3
true

The regex escapes the literal pipe because pipe has a special regex meaning. The negative limit preserves the trailing empty field, giving three fields instead of silently dropping the final one. The format here has no quoted delimiters; a richer CSV or interchange format needs a suitable parser.