代码拉取完成,页面将自动刷新
// Copyright (c) 2015 The golex Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package lex is a Unicode-friendly run time library for golex[0] generated
// lexical analyzers[1].
//
// Changelog
//
// 2015-04-08: Initial release.
//
// Character classes
//
// Golex internally handles only 8 bit "characters". Many Unicode-aware
// tokenizers do not actually need to recognize every Unicode rune, but only
// some particular partitions/subsets. Like, for example, a particular Unicode
// category, say upper case letters: Lu.
//
// The idea is to convert all runes in a particular set as a single 8 bit
// character allocated outside the ASCII range of codes. The token value, a
// string of runes and their exact positions is collected as usual (see the
// Token and TokenBytes method), but the tokenizer DFA is simpler (and thus
// smaller and perhaps also faster) when this technique is used. In the example
// program (see below), recognizing (and skipping) white space, integer
// literals, one keyword and Go identifiers requires only an 8 state DFA[5].
//
// To provide the conversion from runes to character classes, "install" your
// converting function using the RuneClass option.
//
// References
//
// -
//
// [0]: http://godoc.org/github.com/cznic/golex
// [1]: http://en.wikipedia.org/wiki/Lexical_analysis
// [2]: http://golang.org/cmd/yacc/
// [3]: https://github.com/cznic/golex/blob/master/lex/example.l
// [4]: http://golang.org/pkg/io/#RuneReader
// [5]: https://github.com/cznic/golex/blob/master/lex/dfa
package lex
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。