Extension:Chart/Transforms/ja
Extension:Chartを拡張し、複雑なデータセットを扱おうとする編集者が能力を最大化できるように表のデータセット（tabular datasets）の変換、修正、切り出し、あるいは Lua コードによる生成を可能にして、それぞれに共通の Extension:Scribunto エンジンを用いて複雑なテンプレートを {{#invoke:}} パーサ関数に通すようにしました。
Chart format descriptions can include a transform which refers to a Lua Module: page and function, and passes some key-value parameters as an associative array.
ソースとなるデータセットは Lua オブジェクトに翻訳して引数とともに渡して変換、その過程は必要に応じて表形式のデータ セット全体を変更または置換できます。
Loading of additional data modules is recorded for cache invalidation purposes, so a change to the referenced module or any data pages will cause re-rendering of pages with the chart.
Developer internals
See Extension:JsonConfig/Transforms for internals documentation or to add related support to another data type / output method.
Editor usage
Invocation
Currently, transforms for Charts must be set up in the Data:.chart format description page, like so:
{
"license": "CC0-1.0",
"version": 1,
"type": "bar",
"xAxis": {
"title": { "en": "Day" }
},
"yAxis": {
"title": { "en": "Temperature" }
},
"transform": {
"module": "Weekly average temperature chart",
"function": "convert_temps",
"args": {
"units": "C"
}
},
"source": "Sample weekly temperature dataset.tab"
}
The module and function parameters are equivalent to the first two parameters to Scribunto's {{#invoke:}} and refer to the Module: page with the source code, and the specific function to invoke.
Important: like the Data: pages, the Lua Module: code will be loaded from and run in the context of the centralized data store wiki (eg, Wikimedia Commons).
This means whichever wiki you're rendering onto, you'll be running the code in a centralized place -- so modules can be shared across projects and languages, and should consider good practices for reuse and localization.
Arguments are name-value string pairs. If passing numbers around, be sure to convert them suitably if necessary.
Arguments can be overridden on the {{#chart:}} invocation by prefixing them with "arg" like arg:name=value; this allows using different parameters for each chart invocation with the same format, which we expect to be useful for a lot of cases with complex datasets.
<!-- Default Celsius -->
{{#chart:Weekly average temperature chart.chart}}
<!-- Override units to Fahrenheit -->
{{#chart:Weekly average temperature chart.chart|arg:units=F}}
Renderings of non-transformed C vs transformed F chart using same sample temperature data set:
-> 
Code layout
You can name your functions anything you like, and can include related functions in the same module or even use a module to provide both template functions and data transform functions. We're flexible!
Your transform function should take two arguments: a tabular data object converted from JSON, and an arguments list taken from the invocation which will carry key-value pairs. Return a modified tabular data object (it may be the input object after modification, or a new object in the same layout).
local p = {}
local function celsius_to_fahrenheit(val)
return val * 1.8 + 32
end
--
-- input data:
-- * tabular JSON strcuture with 1 label and 2 temp rows stored in C
--
-- arguments:
-- * units: "F" or "C"
--
function p.convert_temps(tab, args)
if args.units == "C" then
-- Stored data is in Celsius
return tab
elseif args.units == "F" then
-- Have to convert if asked for Fahrenheit
for _, row in ipairs(tab.data) do
-- first column is month
row[2] = celsius_to_fahrenheit(row[2])
row[3] = celsius_to_fahrenheit(row[3])
end
return tab
else
error("Units must be either 'C' or 'F'")
end
end
return p
データの整形
表形式のデータの全般は解説文書 Help:Tabular data を参照してください。 変換は具体的にはデータの Lua 変換によって実行、これはつまり、オブジェクトと 0 ベースの配列表示は、文字列キーまたは数値の 1 ベースの索引子を備えた Lua 表形式になります。 They will be transformed back to JSON on output for handling by the renderer.
Example small dataset:
{
"license": "CC0-1.0",
"description": {
"en": "Sample monthly temperature data in degrees C"
},
"schema": {
"fields": [
{
"name": "month",
"type": "localized",
"title": { "en": "Month" }
},
{
"name": "low",
"type": "number",
"title": { "en": "Low temp" }
},
{
"name": "high",
"type": "number",
"title": { "en": "High temp" }
}
]
},
"data": [
[ { "en": "January" }, 5, 20 ],
[ { "en": "July" }, 15, 30 ]
]
}
Beware that data format validation will be applied after your transform, so any invalid format should be called out with an error if you run through the actual transform pipeline.
Localized strings
Localizable strings in the { "lang": "string" } format are preserved in this handling, and if you return a multilingual string it will be localized as best as can for the appropriate context of the output on the rendering end.
However if it is expensive to fetch all conceivable strings, it should be acceptable to look up only the current page view language and include it.
Null/nil
Note that Lua does not allow storing a nil value in a table; this means that JSON tables containing null values may require careful handling by Lua transform code.
If your data set requires working with empty data cells for which null sounds appropriate in the JSON, be careful in iterating over rows.
For instance you can iterate using the tab.schema.fields list, which always contains every column, and use the array indexes as you got them rather than appending with table.insert:
proc sum(tab, args)
-- Append a sum field
table.insert(tab.schema.fields, {
["name"] = "sum",
["type"] = "number",
["title"] = {
["en"] = "Sum"
}
})
local sum_index = #tab.schema.fields
for i, row in ipairs(tab.data) do
local sum = 0
-- iterate over schema.fields, not row which
-- may have "holes" in it
for j, field in ipairs(tab.schema.fields) do
if row[j] then
sum = sum + row[j]
end
end
-- Do not use table.insert here!
-- It could go in the wrong column due to adjacent nils.
row[sum_index] = sum
end
end
Loading additional data sets
Additional Data: pages may be loaded via mw.ext.data.get(); note that if you pass the optional language parameter as _ you'll get the full multilingual strings, otherwise it'll pare them down to just the rendering language.
Try not to load excessive additional data, as it may increase runtime and resource usage.
You may also load additional Lua code or data modules, and they will be recorded for cache invalidation handling.
External data sources
Anything you can run from a Lua module invoked from a template, you can run here -- however be aware that this does not currently include interfaces to fetch from Wikidata Query Service or from RESTbase, so older Graphs usages that require them are not yet ready to be ported.
There is some limited interface for data fetches from Wikidata but this is likely of limited use for charts for now.
Future Lua-facing APIs are possible for other types of lookups, such as into RESTbase or via SPARQL queries, and we hope to be able to tackle multiple such features in the future through Community Wishlist projects.
パフォーマンス
Currently the transform and the chart render are run fresh on every page parse, and the resulting output saved into the parser cache. If transforms are found to be unexpectedly expensive we may need to add more aggressive caching and/or resource limits in place.
Assume that speed and memory are constrained and you should aim to conserve them for the best reader and editor experience alike. There are hard limits on memory usage and processing time, which will be enforced similarly as for template Lua code.
Remember: limits for memory and CPU time are the same as for Lua modules used in templates. Using too much memory or running too long will result in the script being canceled and an error message being shown on the page.
Note that input Data: pages are restricted in production to 2 megabytes, but the chart renderer may limit input data size further -- final size limits are to be determined, but if you find youself hitting the limits consider decimating a large data set to fewer data points.
Remember: there's a strict limit on chart input data, as data sets must not only render fast on the server, they are sent to the client for interactive rendering.
For an example if a tabular data set has 24k rows and will be rendered to a chart for mobile and desktop computers, you have many more data points than pixels -- decimating to 1 out of 10 data points either in the input data set or through a transform should reduce processing time. (See the example under #Decimating input data.)
Testing hints
レンダリングが試行された際に、Lua ランタイムエラーを返すにはパイプラインを通じて報告が来るべきです。反復処理の高速化の準備として、コードを変換レベルまたは Lua レベルでテストします。
Special:ApiSandbox
実際の JSON 変換パイプラインをテストするなら、action=jsontransform を指す Commons:Special:ApiSandbox を使います。これにより実際の入力ページから生成される出力には適切なフォーマットを与えます。
デバッグ コンソール
You can wrap a test harness around your module to call it from the Lua editor debugger console during preview, in a pinch:
local p = {}
-- Your fancy transform code here
function p.transform(tab, args)
return tab
end
-- Put p.test() in the debug console while you're editing the code to test
-- with a specific sample data set / args
function p.test(func, args, tabname, lang)
-- pass "_" for lang to get the raw multilingual source data
local tab = mw.ext.data.get(tabname or "Chart Example Data.tab", lang or "_")
local args = args or {
["key"] = "value"
}
tab = p[func or 'transform'](tab, args)
return mw.dumpObject(tab)
end
return p
実世界の適用例
Chart transform modules can be found at commons:Category:Chart transform modules.
Chart pages that include transforms can be added to commons:Category:Transformed charts.
温度の例
 | 現在、このページは草稿です。
|
commons:Data:Climate_Paris.tab には興味深いデータがたくさんありますが、残念ながらこれを直接、[レンダリングしても得られる[commons:Climate_Paris/example.chart|グラフ]]は使い物になりません。
変換を使用する場合は特定のデータ列に着目して、たとえば温度を摂氏から華氏へ、または長さをミリメートルからインチへなどの変換ができます。
Selecting curves and columns
A simple transform in commons:Module:TabUtils can be used to take the data and using the exported filter function "select" with argument "cols": "month,averageprecip" selects the first column (labeled month) of commons:Data:Climate_Paris.tab as xseries data and column 5 (averageprecip) as an yseries in commons:Data:Climate_Paris/transformed.chart:
"transform": {
"module": "TabUtils",
"function": "select",
"args": {
"cols": "month,averageprecip"
}
},
Calling {{#chart:Climate_Paris/transformed.chart}} on a wiki page results in:
Other columns can be chosen in the wikicode call by overriding the args of a .chart page that includes a TabUtils transform. Here, the two temperature columns recordhigh and meandaily are chosen as yseries. Note that the yaxis title can not be changed this way, so in this case, the yaxis title is wrong:
{{#chart:Climate Paris/transformed.chart
|arg:cols=month,recordhigh,meandaily
}}
This override results in the following chart:
If the .chart page default should show all curves, but allow selection of curves in the wikicode as in the above example, it is sufficient if the .chart json code includes the following:
"transform": {
"module": "TabUtils",
"function": "select"
}
Decimating input points
The select transform in Commons:Module:TabUtils can also decimate the input data set if you have more points than you need to produce a graph (or more than can be fed into the chart renderer successfully!)
Here's a sample that uses a 10x decimation with the select transform to make a too-large data set render:
"transform": {
"module": "TabUtils",
"function": "select",
"args": {
"decimate": "10"
}
},