维基词典

模組:Unicode data

下列說明文檔位於Module:Unicode data/doc。[編輯]
相關链接：子頁面列表 • 链入 • 嵌入包含 • 測試用例 • 沙盒 (差異)

該模塊用於訪問Unicode字符數據庫（版本11.0），其文件來自於 http://www.unicode.org/Public/UNIDATA/ 。

函數

所有的功能都只能在模塊中使用。關於模板的可調用函數，參見Module:Unicode data/templates。

lookup_name
Look up the name of a code point. Used by Module:character info, Module:character list, Module:Template:also, and Module:mul-domino def.
lookup_image
Look up the image for a codepoint. Used by Module:character info and Module:character list.
enum_blocks
Used in a for in loop in Module:character list to iterate through the array of block ranges.
lookup_plane
Returns the name of the plane to which a codepoint belongs. Unused.
get_block_range
Given a block name, returns the range of codepoints included in the block. Used by Module:character list.
is_valid_pagename
Given a string, returns true or false depending on whether the string can be used as a page title. Used by Module:template link.
add_dotted_circle
Given a string, places a dotted circle before every combining character in the string. Used by Module:category tree/charactercat and Module:grc-utilities.
char_to_script
Given a character or a codepoint, returns the Wiktionary script code for the character, or else "None", using Module:Unicode data/scripts. Used by Module:character info, Module:Template:also, Module:character list, and Module:games.
get_entry_title
Returns the title for the entry on a character, which is a subpage of Unsupported titles for characters that cannot appear in pagenames. See Manual:Page title on the MediaWiki website for more information. Used by Module:character info and Module:character list.

The following functions relate to a codepoint's General Category. They receive a codepoint and return two values: a boolean (which expresses whether the codepoint fulfills the condition) and a string describing the General Category, if the character is in the General Categories of Separator (Z) or Other (C), or the string "assigned". They use Module:Unicode data/control.

is_assigned
Checks whether a codepoint's General Category is not Cn (Other, unassigned). Used by Module:character list.
is_printable
Checks whether a codepoint is printable: that its General Category is not "Other" (C), "Separator, line" (Zl), or "Separator, paragraph" (Zp). Used by Module:character info and Module:character list.
is_whitespace
Checks whether a codepoint's General Category is Zs ("Separator, space"). Used by Module:character list and Module:links/print data.

User:Kephir/Unicode contains scripts which may be helpful in maintaining this database.

數據模塊

The data used by functions in this module is found on subpages. Most of them were constructed by the scripts given in User:Kephir/Unicode

The name and image data modules (Module:Unicode data/names/xxx and Module:Unicode data/images/xxx) were compiled from UnicodeData.txt. Each one contains, at maximum, code points U+xxx000 to U+xxxFFF.


Unicode name 和 image 模塊，
前三碼組成以十六進位代碼點為基礎
0 1 2 3 4 5 6 7 8 9 A B C D E F
00x names names names names names           names     names   names
images images images images images images images images images images images images images images   images
01x names names names names names   names   names   names names names names names names
images images images images images   images images images   images images images images images images
02x                               names
images images images images images images images images images images images images images images images images
03x                                
images images images                          
0Ex names                              
images                              

版權

The Unicode database is released by Unicode Inc. under the following terms:

Copyright © 1991-2018 Unicode, Inc. All rights reserved. Distributed under the Terms of Use in http://www.unicode.org/copyright.html.

Permission is hereby granted, free of charge, to any person obtaining a copy of the Unicode data files and any associated documentation (the "Data Files") or Unicode software and any associated documentation (the "Software") to deal in the Data Files or Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the Data Files or Software, and to permit persons to whom the Data Files or Software are furnished to do so, provided that either (a) this copyright and permission notice appear with all copies of the Data Files or Software, or (b) this copyright and permission notice appear in associated Documentation.

THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA FILES OR SOFTWARE.

Except as contained in this notice, the name of a copyright holder shall not be used in advertising or otherwise to promote the sale, use or other dealings in these Data Files or Software without prior written authorization of the copyright holder. 

local export = {}
local udata = mw.loadData("Module:Unicode data/data")

local floor = math.floor

local function errorf(first_arg, ...)
	if type(first_arg) == "number" then
		return error(string.format(...), first_arg + 1)
	else
		return error(string.format(first_arg, ...), 2)
	end
end

local function binary_range_search(codepoint, ranges)
	local low, mid, high
	low, high = 1, ranges.length or require "Module:table".length(ranges)
	while low <= high do
		mid = floor((low + high) / 2)
		local range = ranges[mid]
		if codepoint < range[1] then
			high = mid - 1
		elseif codepoint <= range[2] then
			return range, mid
		else
			low = mid + 1
		end
	end
	return nil, mid
end
export.binary_range_search = binary_range_search

local function linear_range_search(codepoint, ranges)
	for i, range in ipairs(ranges) do
		if codepoint < range[1] then
			break
		elseif codepoint <= range[2] then
			return range
		end
	end
end

-- Load a module by indexing "loader" with the name of the module minus the
-- "Module:Unicode data/" part. For instance, loader.blocks returns
-- [[Module:Unicode data/blocks]]. If a module cannot be loaded, false will be
-- returned.
local loader = setmetatable({}, {
	__index = function (self, key)
		local success, data = pcall(mw.loadData, "Module:Unicode data/" .. key)
		if not success then
			data = false
		end
		self[key] = data
		return data
	end
})

-- For the algorithm used to generate Hangul Syllable names,
-- see "Hangul Syllable Name Generation" in section 3.12 of the
-- Unicode Specification:
-- https://www.unicode.org/versions/latest/ch03.pdf
-- For most of the name rules given here, see the subsection
-- "Unicode Name Property" in section 4.8 (Name) and the table 4-8
-- (Name Derivation Rule Prefix Strings):
-- https://www.unicode.org/versions/latest/ch04.pdf
local name_hooks = {
	{     0x00,     0x1F, "<控制字符-%04X>" }, -- C0 控制字符
	{     0x7F,     0x9F, "<控制字符-%04X>" }, -- DEL 和 C1 控制字符
	{   0x3400,   0x4DBF, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字A
	{   0x4E00,   0x9FFF, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字
	{   0xAC00,   0xD7A3, function (codepoint) -- 諺文音節
		local Hangul_data = loader.Hangul
		local syllable_index = codepoint - 0xAC00

		return ("諺文音節 %s%s%s"):format(
			Hangul_data.leads[floor(syllable_index / Hangul_data.final_count)],
			Hangul_data.vowels[floor((syllable_index % Hangul_data.final_count)
				/ Hangul_data.trail_count)],
			Hangul_data.trails[syllable_index % Hangul_data.trail_count]
		)
	end },
	-- 高半代用區、高半私人代用區、低半代用區
	{   0xD800,   0xDFFF, "<代用區-%04X>" },
	{   0xE000,   0xF8FF, "<私人使用區-%04X>" }, -- 私人使用區
	-- 中日韓相容表意文字
	{   0xF900,   0xFA6D, "中日韓相容表意文字-%04X" },
	{   0xFA70,   0xFAD9, "中日韓相容表意文字-%04X" },
	{  0x17000,  0x187F7, "西夏文表意文字-%04X" }, -- 西夏文
	{  0x18800,  0x18AFF, function (codepoint)
		return ("西夏文部件-%03d"):format(codepoint - 0x187FF)
	end },
	{  0x18D00,  0x18D08, "西夏文表意文字-%04X" }, -- 西夏文
	{  0x18B00,  0x18CD5, "契丹小字-%04X" },
	{  0x1B170,  0x1B2FB, "女書文字-%04X" }, -- 女書
	{  0x20000,  0x2A6DF, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字B
	{  0x2A700,  0x2B738, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字C
	{  0x2A740,  0x2B81D, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字D
	{  0x2B820,  0x2CEA1, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字E
	{  0x2CEB0,  0x2EBE0, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字F
	{  0x2EBF0,  0x2EE5D, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字I
	-- 中日韓相容表意文字補充區 (表意文字補充平面)
	{  0x2F800,  0x2FA1D, "中日韓相容表意文字-%04X" },
	{  0x30000,  0x3134A, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字G
	{  0x31350,  0x323AF, "中日韓統一表意文字-%04X" }, -- 中日韓統一表意文字H
	{  0xE0100,  0xE01EF, function (codepoint) -- 選擇器變化補充
		return ("變體選擇符-%d"):format(codepoint - 0xE0100 + 17)
	end},
	{  0xF0000,  0xFFFFD, "<私人使用區-%04X>" }, -- 第15平面私人使用區
	{ 0x100000, 0x10FFFD, "<私人使用區-%04X>" }  -- 第16平面私人使用區
}
name_hooks.length = #name_hooks

local name_range_cache

local function generate_name(data, codepoint)
	if type(data) == "string" then
		return data:format(codepoint)
	else
		return data(codepoint)
	end
end

--[[
-- Checks that the code point is a number and in range.
-- Does not check whether code point is an integer.
-- Not used
local function check_codepoint(funcName, argIdx, val)
	require 'libraryUtil'.checkType(funcName, argIdx, val, 'number')
	if codepoint < 0 or 0x10FFFF < codepoint then
		errorf("代碼點 %04X 超出範圍", codepoint)
	end
end
--]]

-- https://www.unicode.org/versions/latest/ch04.pdf, section 4.8
function export.lookup_name(codepoint)
	-- U+FDD0-U+FDEF and all code points ending in FFFE or FFFF are Unassigned
	-- (Cn) and specifically noncharacters:
	-- https://www.unicode.org/faq/private_use.html#nonchar4
	if 0xFDD0 <= codepoint and (codepoint <= 0xFDEF
			or floor(codepoint % 0x10000) >= 0xFFFE) then
		return ("<noncharacter-%04X>"):format(codepoint)
	end

	if name_range_cache -- Check if previously used "name hook" applies to this code point.
			and codepoint >= name_range_cache[1]
			and codepoint <= name_range_cache[2] then
		return generate_name(name_range_cache[3], codepoint)
	end
	
	local range = binary_range_search(codepoint, name_hooks)
	if range then
		name_range_cache = range
		return generate_name(range[3], codepoint)
	end

	local data = loader[('names/%03X'):format(codepoint / 0x1000)]
	
	if data and data[codepoint] then
		return data[codepoint]
	
	-- Unassigned (Cn) consists of noncharacters and reserved characters.
	-- The character has been established not to be a noncharacter,
	-- and if it were assigned, its name would already been retrieved,
	-- so it must be reserved.
	else
		return ("<reserved-%04X>"):format(codepoint)
	end
end

function export.lookup_image(codepoint)
	local data = loader[('images/%03X'):format(codepoint / 0x1000)]
	
	if data then
		return data[codepoint]
	end
end

-- Load [[Module:Unicode data/blocks]] if needed and assign it to this variable.
local blocks

local function block_iter(blocks, i)
	i = i + 1
	local data = blocks[i]
	if data then
		 -- Unpack doesn't work on tables loaded with mw.loadData.
		return i, data[3], data[1], data[2]
	end
end

-- An ipairs-type iterator generator for the list of blocks.
function export.enum_blocks()
	local blocks = loader.blocks
	return block_iter, blocks, 0
end

function export.get_block_range(name)
	local range
	
	for i, block in ipairs(loader.blocks) do
		if block[3] == name then
			range = block
		end
	end
	
	if range then
		return range[1], range[2]
	end
end

function export.lookup_plane(codepoint)
	local i = floor(codepoint / 0x10000)
	return udata.planes[i] or ("平面 %u"):format(i)
end

function export.lookup_block(codepoint)
	local blocks = loader.blocks
	local range = binary_range_search(codepoint, blocks)
	if range then
		return range[3]
	else
		return "找無區段"
	end
end

function export.get_block_info(name)
	for i, block in ipairs(loader.blocks) do
		if block[3] == name then
			return block
		end
	end
end

function export.is_valid_pagename(pagename)
	local has_nonws = false

	for cp in mw.ustring.gcodepoint(pagename) do
		if (cp == 0x0023) -- #
		or (cp == 0x005B) -- [
		or (cp == 0x005D) -- ]
		or (cp == 0x007B) -- {
		or (cp == 0x007C) -- |
		or (cp == 0x007D) -- }
		or (cp == 0x180E) -- MONGOLIAN VOWEL SEPARATOR
		or ((cp >= 0x2000) and (cp <= 0x200A)) -- spaces in General Punctuation block
		or (cp == 0xFFFD) -- REPLACEMENT CHARACTER
		then
			return false
		end

		local printable, result = export.is_printable(cp)
		if not printable then
			return false
		end

		if result ~= "space-separator" then
			has_nonws = true
		end
	end

	return has_nonws
end

local function manual_unpack(what, from)
	if what[from + 1] == nil then
		return what[from]
	end
	
	local result = {}
	from = from or 1
	for i, item in ipairs(what) do
		if i >= from then
			table.insert(result, item)
		end
	end
	return unpack(result)
end

local function compare_ranges(range1, range2)
	return range1[1] < range2[1]
end

-- Creates a function to look up data in a module that contains "singles" (a
-- code point-to-data map) and "ranges" (an array containing arrays that contain
-- the low and high code points of a range and the data associated with that
-- range).
-- "loader" loads and returns the "singles" and "ranges" tables.
-- "match_func" is passed the code point and either the data or the "dots", and
-- generates the final result of the function.
-- The varargs ("dots") describes the default data to be returned if there wasn't
-- a match.
-- In case the function is used more than once, "cache" saves ranges that have
-- already been found to match, or a range whose data is the default if there
-- was no match.
local function memo_lookup(data_module_subpage, match_func, ...)
	local dots = { ... }
	local cache = {}
	local singles, ranges

	return function (codepoint)
		if not singles then
			local data_module = loader[data_module_subpage]
			singles, ranges = data_module.singles, data_module.ranges
		end

		if singles[codepoint] then
			return match_func(codepoint, singles[codepoint])
		end

		local range = binary_range_search(codepoint, cache)
		if range then
			return match_func(codepoint, manual_unpack(range, 3))
		end
		
		local range, index = binary_range_search(codepoint, ranges)
		if range then
			table.insert(cache, range)
			table.sort(cache, compare_ranges)
			return match_func(codepoint, manual_unpack(range, 3))
		end
		
		if ranges[index] then
			local dots_range
			if codepoint > ranges[index][2] then
				dots_range = {
					ranges[index][2] + 1,
					ranges[index + 1] and ranges[index + 1][1] - 1 or 0x10FFFF,
					unpack(dots)
				}
			else -- codepoint < range[index][1]
				dots_range = {
					ranges[index - 1] and ranges[index - 1][2] + 1 or 0,
					ranges[index][1] - 1,
					unpack(dots)
				}
			end
			table.sort(cache, compare_ranges)
		end
		
		return match_func(codepoint, unpack(dots))
	end
end

-- Get a code point's combining class value in [[Module:Unicode data/combining]],
-- and return whether this value is not zero. Zero is assigned as the default
-- if the combining class value is not found in this data module.
-- That is, return true if character is combining, or false if it is not.
-- See https://www.unicode.org/reports/tr44/#Canonical_Combining_Class_Values for
-- more information.
export.is_combining = memo_lookup(
	"combining",
	function (codepoint, combining_class)
		return combining_class and combining_class ~= 0 or false
	end,
	0)

function export.add_dotted_circle(str)
	return (mw.ustring.gsub(str, ".",
		function(char)
			if export.is_combining(mw.ustring.codepoint(char)) then
				return '◌' .. char
			end
		end))
end

local lookup_control = memo_lookup(
	"control",
	function (codepoint, ccc)
		return ccc or "assigned"
	end,
	"assigned")
export.lookup_control = lookup_control

function export.is_assigned(codepoint)
	return lookup_control(codepoint) ~= "unassigned"
end

function export.is_printable(codepoint)
	local result = lookup_control(codepoint)
	return (result == "assigned") or (result == "space-separator"), result
end

function export.is_whitespace(codepoint)
	local result = lookup_control(codepoint)
	return (result == "space-separator"), result
end

export.lookup_category = memo_lookup(
	"category",
	function (codepoint, category)
		return category
	end,
	"Cn")

export.lookup_script = memo_lookup(
	"scripts",
	function (codepoint, script)
		return script
	end,
	"Zzzz")

function export.get_entry_title(codepoint)
	if udata.unsupported_title[codepoint] then
		return udata.unsupported_title[codepoint]
	end
	if lookup_control(codepoint) ~= "assigned" then
		return nil
	end
	return mw.ustring.char(codepoint)
end

return export
来自“https://zh.wiktionary.org/w/index.php?title=Module:Unicode_data&oldid=7874310”