Tokens in Java | Types of Tokens
In this tutorial, we will understand tokens in Java in simple words. A Java program is basically made up of a group of classes and methods. A class is a container that contains a set of declaration statements and methods containing executable statements.
A statement consists of variables, constants, operators, keywords, comments, identifiers, punctuators, etc. When the program is run, comments are stripped and executable statements are executed.
The output is called translation unit. A translation unit is a sequence of tokens: symbols, numbers, and words.
What are Tokens in Java?
Tokens are the various elements in the Java program that are identified by Java compiler. A token is the smallest individual element (unit) in a program that is meaningful to the compiler.
In simple words, a java program is a group of tokens, comments, and white spaces. For example, consider the below java statements:
final double p = 3.14 // A constant.
x = a + b; // An expression.
v = Math.pow(10, 1); // An inbuilt java function.Let us consider the first statement, which is made up of six tokens: “final”, “double”, “p”, “=”, “3.14”, and “;”.
Similarly, the second statement consists of six tokens: “x”, “=”, “a”, “+”, “b”, and “;”.
Types of Tokens
Java language contains five types of tokens that are as follows:
- Reserved Keywords
- Identifiers
- Literals,
- Operators
- Separators
Key Points to Remember
(1) Tokens are the basic building blocks of a Java program, including identifiers, keywords, literals, operators, and separators.
(2) Keywords represent reserved words predefined by the Java language. For example, class, public, static, if, else, etc. You cannot use these words as identifiers for the names of variables, methods, classes, etc.
(3) Identifiers represent names given to elements, such as variables, methods, classes, and interfaces. Identifiers must start with a letter (A-Z or a-z), underscore (_), or dollar sign ($), followed by any combination of letters, digits, underscores, or dollar signs.
(4) Literals are constants or fixed values in the program that don’t change during the execution of the program. They include integer, floating-point, character, string, and boolean literals.
(5) Operators are symbols in Java or other programming languages that perform various types of operations variables and values. Examples of operators are arithmetic operators (+, -, *, /), comparison operators (<, >, ==, !=), logical operators (&&, ||), etc.
(6) Separators (also known as delimiters) are characters that separate elements within the program code. Examples of separators are parentheses (( )), braces ({ }), brackets ([ ]), semicolon (;), and comma (,).
(7) The process of breaking down the source code into valid tokens is known as tokenization.
(8) Whitespace is used to separate tokens in Java but does not affect the execution of the program. It includes spaces, tabs, and newline characters.
(9) Java is case-sensitive, meaning it distinguishes between uppercase and lowercase letters. For example, myCollege and MyCollege are different identifiers.
