package scanner

Import Path
	go/scanner (on go.dev)

Dependency Relation
	imports 9 packages, and imported by 3 packages


Involved Source Files

	    errors.go
	
		Package scanner implements a scanner for Go source text.
		It takes a []byte as source which can then be tokenized
		through repeated calls to the Scan method.


Code Examples

	
		package main
		
		import (
			"fmt"
			"go/scanner"
			"go/token"
		)
		
		func main() {
			// src is the input that we want to tokenize.
			src := []byte("cos(x) + 1i*sin(x) // Euler")
		
			// Initialize the scanner.
			var s scanner.Scanner
			fset := token.NewFileSet()                      // positions are relative to fset
			file := fset.AddFile("", fset.Base(), len(src)) // register input "file"
			s.Init(file, src, nil /* no error handler */, scanner.ScanComments)
		
			// Repeated calls to Scan yield the token sequence found in the input.
			for {
				pos, tok, lit := s.Scan()
				if tok == token.EOF {
					break
				}
				fmt.Printf("%s\t%s\t%q\n", fset.Position(pos), tok, lit)
			}
		
		}




Package-Level Type Names (total 5)


	/* sort by:  |  */
	
		In an [ErrorList], an error is represented by an *Error.
		The position Pos, if valid, points to the beginning of
		the offending token, and the error condition is described
		by Msg.

		
			Msg string
			Pos token.Position
		
			
				Error implements the error interface.

		
			 Error : error


	
		An ErrorHandler may be provided to [Scanner.Init]. If a syntax error is
		encountered and a handler was installed, the handler is called with a
		position and an error message. The position points to the beginning of
		the offending token.

		
			func (*Scanner).Init(file *token.File, src []byte, err ErrorHandler, mode Mode)


	
		ErrorList is a list of *Errors.
		The zero value for an ErrorList is an empty ErrorList ready to use.

		
			
				Add adds an [Error] with given position and error message to an [ErrorList].

			
				Err returns an error equivalent to this error list.
				If the list is empty, Err returns nil.

			
				An [ErrorList] implements the error interface.

			
				[ErrorList] implements the sort Interface.

			( ErrorList) Less(i, j int) bool
			
				RemoveMultiples sorts an [ErrorList] and removes all but the first error per line.

			
				Reset resets an [ErrorList] to no errors.

			
				Sort sorts an [ErrorList]. *[Error] entries are sorted by position,
				other errors are sorted by error message, and before any *[Error]
				entry.

			( ErrorList) Swap(i, j int)
		
			 ErrorList : error
			 ErrorList : sort.Interface


	
		A mode value is a set of flags (or 0).
		They control scanner behavior.

		
			func (*Scanner).Init(file *token.File, src []byte, err ErrorHandler, mode Mode)
		
			const ScanComments


	
		A Scanner holds the scanner's internal state while processing
		a given text. It can be allocated as part of another data
		structure but must be initialized via [Scanner.Init] before use.

		
			
				public state - ok to modify
				// number of errors encountered

		
			
				Init prepares the scanner s to tokenize the text src by setting the
				scanner at the beginning of src. The scanner uses the file set file
				for position information and it adds line information for each line.
				It is ok to re-use the same file when re-scanning the same file as
				line information which is already present is ignored. Init causes a
				panic if the file size does not match the src size.
				
				Calls to [Scanner.Scan] will invoke the error handler err if they encounter a
				syntax error and err is not nil. Also, for each error encountered,
				the [Scanner] field ErrorCount is incremented by one. The mode parameter
				determines how comments are handled.
				
				Note that Init may call err if there is an error in the first character
				of the file.

			
				Scan scans the next token and returns the token position, the token,
				and its literal string if applicable. The source end is indicated by
				[token.EOF].
				
				If the returned token is a literal ([token.IDENT], [token.INT], [token.FLOAT],
				[token.IMAG], [token.CHAR], [token.STRING]) or [token.COMMENT], the literal string
				has the corresponding value.
				
				If the returned token is a keyword, the literal string is the keyword.
				
				If the returned token is [token.SEMICOLON], the corresponding
				literal string is ";" if the semicolon was present in the source,
				and "\n" if the semicolon was inserted because of a newline or
				at EOF.
				
				If the returned token is [token.ILLEGAL], the literal string is the
				offending character.
				
				In all other cases, Scan returns an empty literal string.
				
				For more tolerant parsing, Scan will return a valid token if
				possible even if a syntax error was encountered. Thus, even
				if the resulting token sequence contains no illegal tokens,
				a client may not assume that no error occurred. Instead it
				must check the scanner's ErrorCount or the number of calls
				of the error handler, if there was one installed.
				
				Scan adds line information to the file added to the file
				set with Init. Token positions are relative to that file
				and thus relative to the file set.





Package-Level Functions (only one)


	
		PrintError is a utility function that prints a list of errors to w,
		one error per line, if the err parameter is an [ErrorList]. Otherwise
		it prints the err string.




Package-Level Constants (only one)


	const ScanComments Mode = 1 // return comments as COMMENT tokens