package scanner

Import Path
	text/scanner (on golang.org and go.dev)

Dependency Relation
	imports 6 packages, and imported by one package

Involved Source Files
	d-> scanner.go

Exported Type Names

type Position (struct) A source position is represented by a Position value. A position is valid if Line > 0. // column number, starting at 1 (character count per line) // filename, if any // line number, starting at 1 // byte offset, starting at 0 IsValid reports whether the position is valid. ( T) String() string *T : database/sql/driver.Validator T : expvar.Var T : fmt.Stringer func (*Scanner).Pos() (pos Position)
type Scanner (struct) A Scanner implements reading of Unicode characters and tokens from an io.Reader. Error is called for each error encountered. If no Error function is set, the error is reported to os.Stderr. ErrorCount is incremented by one for each error encountered. IsIdentRune is a predicate controlling the characters accepted as the ith rune in an identifier. The set of valid characters must not intersect with the set of white space characters. If no IsIdentRune function is set, regular Go identifiers are accepted instead. The field may be changed at any time. The Mode field controls which tokens are recognized. For instance, to recognize Ints, set the ScanInts bit in Mode. The field may be changed at any time. Start position of most recently scanned token; set by Scan. Calling Init or Next invalidates the position (Line == 0). The Filename field is always left untouched by the Scanner. If an error is reported (via Error) and Position is invalid, the scanner is not inside a token. Call Pos to obtain an error position in that case, or to obtain the position immediately after the most recently scanned token. // column number, starting at 1 (character count per line) // filename, if any // line number, starting at 1 // byte offset, starting at 0 The Whitespace field controls which characters are recognized as white space. To recognize a character ch <= ' ' as white space, set the ch'th bit in Whitespace (the Scanner's behavior is undefined for values ch > ' '). The field may be changed at any time. Init initializes a Scanner with a new source and returns s. Error is set to nil, ErrorCount is set to 0, Mode is set to GoTokens, and Whitespace is set to GoWhitespace. IsValid reports whether the position is valid. Next reads and returns the next Unicode character. It returns EOF at the end of the source. It reports a read error by calling s.Error, if not nil; otherwise it prints an error message to os.Stderr. Next does not update the Scanner's Position field; use Pos() to get the current position. Peek returns the next Unicode character in the source without advancing the scanner. It returns EOF if the scanner's position is at the last character of the source. Pos returns the position of the character immediately after the character or token returned by the last call to Next or Scan. Use the Scanner's Position field for the start position of the most recently scanned token. Scan reads the next token or Unicode character from source and returns it. It only recognizes tokens t for which the respective Mode bit (1<<-t) is set. It returns EOF at the end of the source. It reports scanner errors (read and token errors) by calling s.Error, if not nil; otherwise it prints an error message to os.Stderr. ( T) String() string TokenText returns the string corresponding to the most recently scanned token. Valid after calling Scan and in calls of Scanner.Error. *T : database/sql/driver.Validator T : expvar.Var T : fmt.Stringer func (*Scanner).Init(src io.Reader) *Scanner
Exported Values
const Char = -5 The result of Scan is one of these tokens or a Unicode character.
const Comment = -8 The result of Scan is one of these tokens or a Unicode character.
const EOF = -1 The result of Scan is one of these tokens or a Unicode character.
const Float = -4 The result of Scan is one of these tokens or a Unicode character.
const GoTokens = 1012 Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to: ScanIdents | ScanInts | SkipComments With the exceptions of comments, which are skipped if SkipComments is set, unrecognized tokens are not ignored. Instead, the scanner simply returns the respective individual characters (or possibly sub-tokens). For instance, if the mode is ScanIdents (not ScanStrings), the string "foo" is scanned as the token sequence '"' Ident '"'. Use GoTokens to configure the Scanner such that it accepts all Go literal tokens including Go identifiers. Comments will be skipped.
const GoWhitespace = 4294977024 GoWhitespace is the default value for the Scanner's Whitespace field. Its value selects Go's white space characters.
const Ident = -2 The result of Scan is one of these tokens or a Unicode character.
const Int = -3 The result of Scan is one of these tokens or a Unicode character.
const RawString = -7 The result of Scan is one of these tokens or a Unicode character.
const ScanChars = 32 Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to: ScanIdents | ScanInts | SkipComments With the exceptions of comments, which are skipped if SkipComments is set, unrecognized tokens are not ignored. Instead, the scanner simply returns the respective individual characters (or possibly sub-tokens). For instance, if the mode is ScanIdents (not ScanStrings), the string "foo" is scanned as the token sequence '"' Ident '"'. Use GoTokens to configure the Scanner such that it accepts all Go literal tokens including Go identifiers. Comments will be skipped.
const ScanComments = 256 Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to: ScanIdents | ScanInts | SkipComments With the exceptions of comments, which are skipped if SkipComments is set, unrecognized tokens are not ignored. Instead, the scanner simply returns the respective individual characters (or possibly sub-tokens). For instance, if the mode is ScanIdents (not ScanStrings), the string "foo" is scanned as the token sequence '"' Ident '"'. Use GoTokens to configure the Scanner such that it accepts all Go literal tokens including Go identifiers. Comments will be skipped.
const ScanFloats = 16 // includes Ints and hexadecimal floats Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to: ScanIdents | ScanInts | SkipComments With the exceptions of comments, which are skipped if SkipComments is set, unrecognized tokens are not ignored. Instead, the scanner simply returns the respective individual characters (or possibly sub-tokens). For instance, if the mode is ScanIdents (not ScanStrings), the string "foo" is scanned as the token sequence '"' Ident '"'. Use GoTokens to configure the Scanner such that it accepts all Go literal tokens including Go identifiers. Comments will be skipped.
const ScanIdents = 4 Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to: ScanIdents | ScanInts | SkipComments With the exceptions of comments, which are skipped if SkipComments is set, unrecognized tokens are not ignored. Instead, the scanner simply returns the respective individual characters (or possibly sub-tokens). For instance, if the mode is ScanIdents (not ScanStrings), the string "foo" is scanned as the token sequence '"' Ident '"'. Use GoTokens to configure the Scanner such that it accepts all Go literal tokens including Go identifiers. Comments will be skipped.
const ScanInts = 8 Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to: ScanIdents | ScanInts | SkipComments With the exceptions of comments, which are skipped if SkipComments is set, unrecognized tokens are not ignored. Instead, the scanner simply returns the respective individual characters (or possibly sub-tokens). For instance, if the mode is ScanIdents (not ScanStrings), the string "foo" is scanned as the token sequence '"' Ident '"'. Use GoTokens to configure the Scanner such that it accepts all Go literal tokens including Go identifiers. Comments will be skipped.
const ScanRawStrings = 128 Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to: ScanIdents | ScanInts | SkipComments With the exceptions of comments, which are skipped if SkipComments is set, unrecognized tokens are not ignored. Instead, the scanner simply returns the respective individual characters (or possibly sub-tokens). For instance, if the mode is ScanIdents (not ScanStrings), the string "foo" is scanned as the token sequence '"' Ident '"'. Use GoTokens to configure the Scanner such that it accepts all Go literal tokens including Go identifiers. Comments will be skipped.
const ScanStrings = 64 Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to: ScanIdents | ScanInts | SkipComments With the exceptions of comments, which are skipped if SkipComments is set, unrecognized tokens are not ignored. Instead, the scanner simply returns the respective individual characters (or possibly sub-tokens). For instance, if the mode is ScanIdents (not ScanStrings), the string "foo" is scanned as the token sequence '"' Ident '"'. Use GoTokens to configure the Scanner such that it accepts all Go literal tokens including Go identifiers. Comments will be skipped.
const SkipComments = 512 // if set with ScanComments, comments become white space Predefined mode bits to control recognition of tokens. For instance, to configure a Scanner such that it only recognizes (Go) identifiers, integers, and skips comments, set the Scanner's Mode field to: ScanIdents | ScanInts | SkipComments With the exceptions of comments, which are skipped if SkipComments is set, unrecognized tokens are not ignored. Instead, the scanner simply returns the respective individual characters (or possibly sub-tokens). For instance, if the mode is ScanIdents (not ScanStrings), the string "foo" is scanned as the token sequence '"' Ident '"'. Use GoTokens to configure the Scanner such that it accepts all Go literal tokens including Go identifiers. Comments will be skipped.
const String = -6 The result of Scan is one of these tokens or a Unicode character.
func TokenString(tok rune) string TokenString returns a printable string for a token or Unicode character.