Extension:Scribunto/New Lua version

Scribunto uses Lua 5.1, but people have requested a newer version (T178146). This page tracks specific requests for functionality available in newer Lua versions that would be beneficial to have for wiki modules and templates; please provide specific examples.

Improved multibyte string escapes

Requested by Od1n (in Phabricator).

• Lua 5.2 would let us use hexadecimal byte codes in strings, whereas we currently have to rely on decimal byte codes, which are very uncommon. Example: we could replace the very unusual \194\160 with \xC2\xA0.
  • hex and \z escapes in strings
  • \0 in patterns (addresses this issue)
• Lua 5.3 would even let us use Unicode code points in strings. Example: \u{A0}.
  • basic utf-8 support

To provide an example, on frwiki we have a Langue.nonLatin() method that works on multibyte strings. Lua 5.3 would let us change this:

local pattern = '[^'
    .. '%z'                        -- U+0000 (UTF-8 00), cas particulier qu'il faut représenter avec %z
    .. '\001-\205\175'             -- U+0001 (UTF-8 01) à U+036F (UTF-8 CDAF)
    .. '\225\180\128-\225\187\191' -- suppléments phonétique, diacritiques et latin : U+1D00 (UTF-8 E1B480) à U+1EFF (UTF-8 E1BBBF)
    .. '\226\128\128-\226\177\191' -- espace, indices, monnaies et symboles divers : U+2000 (UTF-8 E28080) à U+2C7F (UTF-8 E2B1BF)
    .. '\234\156\160-\234\159\191' -- latin étendu D : U+A720 (UTF-8 EA9CA0) à U+A7FF (UTF-8 EA9FBF)
    .. ']'

To this:

local pattern = '[^'
    .. '\u{0}-\u{36F}'     -- U+0000 (UTF-8 00) à U+036F (UTF-8 CDAF)
    .. '\u{1D00}-\u{1EFF}' -- suppléments phonétique, diacritiques et latin : U+1D00 (UTF-8 E1B480) à U+1EFF (UTF-8 E1BBBF)
    .. '\u{2000}-\u{2C7F}' -- espace, indices, monnaies et symboles divers : U+2000 (UTF-8 E28080) à U+2C7F (UTF-8 E2B1BF)
    .. '\u{A720}-\u{A7FF}' -- latin étendu D : U+A720 (UTF-8 EA9CA0) à U+A7FF (UTF-8 EA9FBF)
    .. ']'

Operator overloading

Requested by ClarFonThey

• Lua 5.2 would let you use a __len metamethod to override the # operator, which cannot be overridden on 5.1 like the pairs and ipairs methods are. This is useful because tables often include extraneous keys corresponding to internal stuff and you might want the "length" of the table to exclude these values. Additionally, it's valuable for the length to match the number of items returned by iteration, which would not be the case under Lua 5.1. It (given suitable Scribunto glue code) would also allow a module to know how many arguments it was passed without evaluating all of them, which is currently impossible.
• Lua 5.3 would let you natively use bitwise operators and override them for tables, to avoid the need for the bit32 module. (x | (x << 4)) & y is so much nicer than bit32.band(bit32.bor(x, bit32.lshift(x, 4)), y) and substantially more readable, too. Binary operators could be used as set operations for set-like tables as well.