View source for Citoid/uk

<languages/>
{{See also|Citoid/Enabling Citoid on your wiki}}
{{Wikimedia engineering project information
| name = Citoid
| logo = Citoid logo.svg
| description = 
| start = 2014-05-08
| group = Editing
| lead = 
| team = 
| projectpage = 
| Phabricator = citoid
| display = 
}}
[[File:Citoid results in VisualEditor Screen Shot 2015-04-02.png|thumb|360x360px|<span lang="en" dir="ltr" class="mw-content-ltr">Citoid in VisualEditor</span>]]
<span lang="en" dir="ltr" class="mw-content-ltr">The '''Citoid''' Node.js service generates citation data given a URL, DOI, ISBN, PMID, PMCID or QID.</span>
<span lang="en" dir="ltr" class="mw-content-ltr">It has a companion extension, '''{{ll|Extension:Citoid|Citoid}}''', which aims to provide use of the Citoid service to {{ll|VisualEditor}}, but also may be accessible from the wikicode editing palette, if activated.</span>

{{tmpl|0=<span lang="en" dir="ltr" class="mw-content-ltr">It is currently deployed in all VisualEditor-enabled WMF-Wikis,$2 though the extension is only configured on [[Special:MyLanguage/Citoid/Where citoid is enabled|some of them.]]</span>
|2=<ref><span lang="en" dir="ltr" class="mw-content-ltr">[[phab:diffusion/OMWC/browse/master/wmf-config/CommonSettings.php;5310c296e659f487e7ba3ba0ba628c751a7ff8db#L2080|CommonSettings.php]] on phabricator.wikimedia.org (no feature flag set)</span></ref>
}}

<span id="Public_API"></span>
== Публічний API ==
<span lang="en" dir="ltr" class="mw-content-ltr">To request metadata about a URL, DOI, PMCID, PMID, or QID, you can use the English language API endpoint.</span>
{{button|&nbsp;[[w:Special:RestSandbox/wmf-rest/v1#/Citation/getCitation|<span lang="en" dir="ltr" class="mw-content-ltr">Try it!</span>]]&nbsp;}}
<span id="Issue_tracker_and_project_management"></span>
== Трекер проблем і управління проєктами ==
<span lang="en" dir="ltr" class="mw-content-ltr">Add bugs, issues, and suggestions for improvement to the [[phab:project/view/62/|Citoid Phabricator project]].</span>
({{ll|Phabricator/Help|2=<span lang="en" dir="ltr" class="mw-content-ltr">Learn more about Phabricator.</span>}})

<span id="Installation"></span>
== Встановлення ==
{{info|1=<span lang="en" dir="ltr" class="mw-content-ltr">Best results are obtained if the URLs that are popular on your site are {{ll|Determining if a URL has a translator in Zotero|already available in Zotero}}.</span> <span lang="en" dir="ltr" class="mw-content-ltr">If they're not, performance will be better if you {{ll|Citoid/Creating Zotero translators|create Zotero translators}} for your popular sites first.</span>}}
<span lang="en" dir="ltr" class="mw-content-ltr">Citoid is a Node.js app that also requires a working installation of [https://github.com/zotero/translation-server Zotero's translation server], which uses the [https://github.com/zotero/translators Zotero translators library], and [https://developer.mozilla.org/docs/Gecko_SDK xulrunner].</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Install Node.js and npm ====
</div>
<span lang="en" dir="ltr" class="mw-content-ltr">Install nodejs and npm. When install from your OS's package manager you will likely not end up the correct version of node.js.</span>
<span lang="en" dir="ltr" class="mw-content-ltr">We recommend you install [https://github.com/nvm-sh/nvm?tab=readme-ov-file#installing-and-updating nvm] to manage nodejs installations.</span>
<span lang="en" dir="ltr" class="mw-content-ltr">Compatible versions of node are [[phab:diffusion/GCIT/browse/master/package.json$16|listed in package.json]]; as of writing this is node 20.</span>
<span lang="en" dir="ltr" class="mw-content-ltr">To install npm on linux:</span>

<syntaxhighlight lang="text">
sudo apt-get install npm
</syntaxhighlight>

Для інших систем, дивись:

* https://nodejs.org/en/download/package-manager/

<span id="Install_from_scratch"></span>
=== Встановіть з нуля ===

<span id="Install_and_configure_Zotero&#039;s_translation_server"></span>
==== Встановіть та налаштуйте сервер перекладу Zotero ====

<div lang="en" dir="ltr" class="mw-content-ltr">
For citoid versions 0.5.3 or earlier, see [https://github.com/zotero/translation-server-old Translation-server installation instructions]
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
For citoid versions 1.0.0 or higher:
</div>

# <span lang="en" dir="ltr" class="mw-content-ltr">Clone the repository <code>git clone --recursive git@gitlab.wikimedia.org:repos/mediawiki/services/zotero.git</code></span> 
## <span lang="en" dir="ltr" class="mw-content-ltr">If you forget the recursive flag, do <code>git submodule init</code> then do <code>git submodule update</code></span>
# <span lang="en" dir="ltr" class="mw-content-ltr">Change into the directory:</span> <code>cd zotero</code>
# <span lang="en" dir="ltr" class="mw-content-ltr">Install the libraries:</span> <code>npm install</code>
# <span lang="en" dir="ltr" class="mw-content-ltr">Run the server:</span> <code>npm start</code>
# <span lang="en" dir="ltr" class="mw-content-ltr">You should get the following output:</span> 

<code>(3)(+0000000): Translators initialized with 528 loaded</code>

<code>(3)(+0000003): Listening on 0.0.0.0:1969</code>

<span lang="en" dir="ltr" class="mw-content-ltr">Try a query to verify it's working:</span>  <code>curl -d 'https://www.ncbi.nlm.nih.gov/pubmed/?term=crispr' -H 'Content-Type: text/plain' http://127.0.0.1:1969/web</code>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Install and configure Citoid service ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== Get the code =====
</div>
<syntaxhighlight lang="text">
git clone https://gerrit.wikimedia.org/r/mediawiki/services/citoid
</syntaxhighlight>

If you plan to develop Citoid, then please follow [[Gerrit/Getting started|the Gerrit 'getting started' docs]] as well.

<div lang="en" dir="ltr" class="mw-content-ltr">
===== JS dependencies =====
</div>
Install the JS dependencies.
Run this command in the Citoid directory:

<syntaxhighlight lang="text">
npm install
</syntaxhighlight>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== Modify config.yaml =====
</div>
<span lang="en" dir="ltr" class="mw-content-ltr">Config.yaml contains the configuration for the Citoid service.</span>
<span lang="en" dir="ltr" class="mw-content-ltr">The defaults should work out of the box for development, however, they may need to be modified in a deployment set-up.</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== Run the translation server =====
</div>
You'll first need to run translation-server; see the directions [https://github.com/zotero/translation-server on its GitHub page], but generally from the translation-server directory you should run:

<syntaxhighlight lang="text">
git submodule init
git submodule update
npm i
npm start
</syntaxhighlight>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== Run Citoid server =====
</div>
You should be able to start the Citoid web service from the citoid directory using:

<syntaxhighlight lang="text">
node server.js
</syntaxhighlight>

This will start the Citoid service on port 1970.
To test it, navigate to http://localhost:1970 in your browser.
You'll be able to test sample queries from this page.

<div lang="en" dir="ltr" class="mw-content-ltr">
===Install Citoid extension===
</div>

See [[Extension:Citoid]] for how to download and install the Citoid MediaWiki extension. [[gerrit:c/mediawiki/extensions/Citoid/+/1131008|Versions of mediawiki 1.44.0-wmf.25 or greater require citoid >= 1.2.0]].

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Configure Citoid on a Citoid-enabled wiki ===
</div>
The Citoid extension must be configured using special TemplateData maps and a special citoid-specific message.
It is currently deployed in all VisualEditor-enabled WMF-Wikis, but it must be configured before it can be used.

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Ensure each template to be used in MediaWiki:Citoid-template-type-map.json has an 'citoid' maps value ====
</div>
{{main|Citoid/Maps TemplateData}}

{{warning|1=Each template listed in Citoid-template-type-map.json MUST have a citoid maps value! Otherwise completely empty templates will be inserted. If there is no maps value for a given template, it is better to use a different template that does in the MediaWiki message, even if the types don't match well.}}

Since Citoid has its own set of fields for each document type (for instance, the journal name is called 'publicationTitle' in citoid, but 'journal' in Template:Cite_journal), each Template must have [[TemplateData]] defined that creates a [[Extension:TemplateData|map]] between citoid's fields and the Template's field.
Calling the map 'citoid' lets the Citoid extension know which map to look for.
If the map 'citoid' doesn't suit your purposes for use with say, a userscript, you may create a citoid service related map that is called something else; an unlimited number of maps with unique keys are allowed in the maps object.
Note that you can only see TemplateData maps in edit mode; they are not visible in the TemplateData table.

Fields issn and isbn can have Arrays [] in the citoid map; using them ensures that only one ISBN is in the field.
If you do not place the parameter inside an Array (i.e. isbn: "[ISBN]"), multiple ISBNs or ISSNS will be concatenated in the field (i.e. "issn: 1234-5678, 7777-7777').
All 'person' fields, e.g. author, editor, translator, contributor etc, support 2D Arrays [[]] in the citoid map for first name and last name. It also supports a 1D array for full name fields; this will concatenate the name with the first field first, separated by a space. 
See sample templateData below for examples.

Examples of map objects on English Wikipedia that are compatible with the Citoid extension are:
* [[:en:Template:Citation/doc|https://en.wikipedia.org/wiki/Template:Citation/doc]]
* [[:en:Template:Cite_journal/doc|https://en.wikipedia.org/wiki/Template:Cite_journal/doc]]
* [[:en:Template:Cite_web/doc|https://en.wikipedia.org/wiki/Template:Cite_web/doc]]
* [[:en:Template:Cite_news/doc|https://en.wikipedia.org/wiki/Template:Cite_news/doc]]
* [[:en:Template:Cite_book/TemplateData|https://en.wikipedia.org/wiki/Template:Cite_book/TemplateData]]

<div lang="en" dir="ltr" class="mw-content-ltr">
====Configure special MediaWiki namespace Citoid message====
</div>
{{warning|It is best if you add this message after each template has a template data map as specified in the above section.
Otherwise, in the meantime empty templates will be inserted, because each template needs a template data map to be usable.}}

You'll need to configure a special [[Manual:Namespace#Built-in namespaces|MediaWiki: namespace message]].
This message maps the native citoid types (website, book, journalArticle) to the appropriate template (Cite web, Cite book, Cite journal).
You should match a template to every single citoid type; there is no default behaviour if no template is matched to a particular type.
It's better to have a bad match (there may be some fields in common between video liner notes and a book, or video liner notes and a video, for example) than none at all.

You may consider using en wiki's Template:Citation as a catch-all for types where there is no good type match as it is designed for this situation.

A sample namespace message is found here: [[Citoid/MediaWiki:Citoid-template-type-map.json]]. The simplest configuration is to use a single default template: [[Citoid/Minimal config/MediaWiki:Citoid-template-type-map.json]]

Every available citoid type is listed as a key in the sample namespace message.

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Troubleshooting Citoid VisualEditor Extension ===
</div>
Open your [[Manual:How to debug#Client side debugging (JavaScript)|browser's console]].
The Citoid extension will log errors here.

<div lang="en" dir="ltr" class="mw-content-ltr">
===== Inspector does not appear in the toolbar =====
</div>
An icon for the inspector should appear in the toolbar menu.
This means either citoid is not installed properly or not configured properly.
Verify the extension and dependent extensions are installed on the page localhost:/wiki/Special:Version.

If the extensions are installed and the icon does not appear in the toolbar, it most likely means there's a problem with [[Citoid/MediaWiki:Citoid-template-type-map.json|MediaWiki:Citoid-template-type-map.json]].
If there is no message at that location, or if the JSON is invalid, the inspector will not load.
Look at the messages logged to your browser's console to diagnose the specific problem with the message.
Alternatively, you may need to refresh your JavaScript cache.

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Get "could not make a citation for you" most or every time. ====
</div>

You may consistently get this error if the citation template being inserted contains no maps data, or if the maps data is there but not making it to the MediaWiki API. The error may be inconsistent if one or some, but not all of the templates, have broken template data. Check your browser console as it will display a warning with the name of the problem template if that's the case.
First, determine the template that the inspector is attempted to insert, for example, <code>Template:Cite web/doc</code>.
View source of the template or documentation page to verify that

<pre>
"maps": {

    "citoid": { 
</pre>
is present and contains fields.
Then verify that these data are making it to the MediaWiki API by visiting the API page, i.e. http://localhost/api.php?action=templatedata&titles=Template:Cite%20web/doc&format=jsonfm on your local installation, or https://en.wikipedia.org/w/api.php?action=templatedata&titles=Template:Cite%20web&format=jsonfm on en wiki.

If the maps object is present in TemplateData, but not in the API response, try editing the template where the TemplateData is transcluded i.e. <code>Template:Cite_web</code> (but making no changes) and saving it, a.k.a. a "[[Manual:Null edit|null edit]]".
There is a known bug with transcluded TemplateData where it can take a long time for the API to update ({{phab|T52372}}); null edits force the change. Another possible problem would be if there is template data present in both the Template page and the doc page. Empty template data present on the template page can prevent the correct template data from a documentation page from coming through. If this is the case, you can delete the template data block on the Template page so the template data on the doc page comes through. 

If the response from the API looks okay, there may be an issue with installation of the citoid service.
Open up your browser's debugger to confirm that the responses from the citoid server are OK.

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Inspector is still "pending" after a really long time following insertion ====
</div>
This typically means there is a bug.
If you open your JavaScript console, you will likely find error messages that will help you debug.

<div lang="en" dir="ltr" class="mw-content-ltr">
====Access date is formatted differently on my wiki====
</div>
The dates are in [[w:ISO 8601|ISO]] format, which is an international standard.
On the back-end, we're sticking to ISO and in the future all dates will all be in ISO, not just access date.
This is because it is an unambiguous way to present the date in any language.
If the community doesn't like the way this looks to the user, it is possible to edit the citation template to format the ISO dates to something that is standard in your language.
For instance, you can add logic to the template such that if the date is detected to be in ISO yyyy/mm/dd format, the date is reformatted *to appear* to be dd/mm/yyyy on the page.
However, if you do this, the underlying data (i.e. when you edit the wikitext, or the form in VisualEditor) will still remain the same.

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Troubleshooting the citoid service ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== My favourite site isn't recognised by citoid and only gets basic information =====
</div>
{{Main|Citoid/Determining if a URL has a translator in Zotero}}

The Citoid service relies on the brilliant Zotero community for much of the "magic", as Zotero translators need to be written for each site.
You can see a list of all Zotero translators at https://github.com/zotero/translators.
Right now, Zotero best supports English-language sources.
We need your help to improve coverage of other sites.
You can write your own Zotero translator.
Start by looking at [[Citoid/Creating Zotero translators]].

<div lang="en" dir="ltr" class="mw-content-ltr">
====Testing for the translators using the "server" option or 'v' flag====
</div>
To test with translation-server, download and install https://github.com/zotero/translation-server
Your translator will need to have the 'v' flag enabled for 'browserSupport'.
More here on that: https://www.zotero.org/support/dev/translators
As an example, see https://github.com/zotero/translators/blob/master/A%20Contra%20Corriente.js ; you will see there are a bunch of letters, one of which is ''v'', which corresponds to translation-server.
If server support is not enabled by testing it/then adding the 'v' flag, we won't be able to use the translator.

<div lang="en" dir="ltr" class="mw-content-ltr">
== Hacking citoid ==
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Installation ===
</div>
You can install Citoid with Docker. See: [[MediaWiki-Docker/Extension/Citoid]]

After following these instructions, you will further need to add wiki pages to get citoid working.
The most expedient way to do this is to [https://en.wikipedia.org/wiki/Special:Export export the following pages from en-wiki]:

<syntaxhighlight lang="text">
Template:Citation
Template:Cite_web
Template:Cite_news
Template:Cite_journal
Template:Cite_book
Template:Cite_AV_media
Template:Cite_thesis
Template:Cite_patent
MediaWiki:Citoid-template-type-map.json
MediaWiki:Cite-tool-definition.json
</syntaxhighlight>

And then import them: localhost:8080/wiki/Special:Import.

After importing the Templates, you may need to navigate to the template, hit the edit button, and then hit save (a "null" edit), in order for the templatedata to propagate to the DB.

(Unfortunately en wiki templates as of 2022 currently also require a file from commons, and if your wiki is not set up to work with commons you will get an error. You can copy this file locally and make a few changes to the CS1 Module.)

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Running tests ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== citoid service ====
</div>
<code>npm test</code> runs eslint and unit tests.

<code>mocha</code> runs the full set of tests.
(You can run <code>./node_modules/mocha/bin/mocha</code> if you only have mocha installed locally) 

<code>npm run-script coverage</code> runs the tests and reports code coverage.

In order for all tests to pass, you will need a WorldCat wskey in your config.dev.yaml file, which you can request at [https://help.oclc.org/Discovery_and_Reference/WorldCat_Discovery/Troubleshooting/How_do_I_request_a_WSKey_for_the_WorldCat_Search_API?sl=en here.]  

Another reason why some tests may fail erroneously is if your DNS redirects invalid domains to a valid IP; in some cases, this causes a 520 response instead of a 400 response to be returned.
This can be fixed by configuring your internet connection to use OpenDNS or another DNS that does not do this.

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Citoid extension ====
</div>
See: [[Manual:JavaScript unit testing]]

<div lang="en" dir="ltr" class="mw-content-ltr">
== Formatting the citation output of Citoid ==
</div>
Citoid depends on [[Extension:TemplateData]] for mapping variables from a wiki's citation templates to citoid's fields.
However, it may not be apparent that the formatting output of citations from Citoid isn't managed explicitly by Citoid but rather by TemplateData.
As such, if a particular citation format is required, you'll need to adjust the "format" in your citation templates' TemplateData.
Guidance is provided in [[Help:TemplateData#Custom formats|Help:TemplateData]].
You may, for example, see <code>"format": "inline"</code> at the bottom of many pre-existing TemplateData Citoid maps, which means the resulting citation output will appear inline.
However, this will not put any spacing in the resulting citation text.
If the output should have a space before each pipe character but no spaces before or after an equals sign in the citation, one may set <code>"format": "{{_ |_=_}}"</code>.
The code would typically be placed before closing out with the final curly brace, as such:

<syntaxhighlight lang="javascript">
<templatedata>
{
    "description": "",
    "params": {
        "1": {
            "label": "",
            "description": "",
            "type": ""
        },
        "2": {
            "label": "",
            "description": "",
            "type": ""
        }
    },
    "paramOrder": [

    ],

	"maps": {
        "citoid": {

		}
	},
	"format": "{{_ |_=_}}"
}
</templatedata>
</syntaxhighlight>

Consult [[Help:TemplateData]] for more formatting options.

It's worth noting that if you want to change this format on a large wiki project, you should probably discuss it somewhere with the community as users may have strong feelings about formatting in citations.
Additionally, note that when "format" is changed on an existing wiki, it will cause VisualEditor to update the formatting of each citation whenever someone makes changes to it.
On large wikis like Wikipedia, this can cause annoying dirty diffs and even accidental edit wars with bots that maintain citations.

<div lang="en" dir="ltr" class="mw-content-ltr">
== See also ==
</div>
* [https://github.com/zotero/translation-server-old Old Zotero translation server] -- xpcshell-based Zotero translation server, and [https://github.com/zotero/translation-server translation-server], a nodejs equivalent.
* [https://aurimasv.github.io/z2csl/typeMap.xml Valid field names for each item type] (Outdated - missing type updates)
* [https://www.zotero.org/support/kb/field_mappings Zotero field mappings], including vs. some cite templates
* [[Citoid/Maps TemplateData]]
* [[:en:Category:Citation_templates]]
* [[:en:Help:Citation tools]]
* [[VisualEditor/Citation tool]]
*[[:m:Research:Citoid support for Wikimedia references]]
*[https://www.zotero.org/support/dev/translators/testing Translators testing]
*[[Citoid/Enabling Citoid on your wiki]]
*[[Citoid/itemTypes]]
*[[Citoid/Zotero's Tech Talk]]
*[https://www.youtube.com/watch?v=1pMHSghEM7A&feature=youtu.be&t=6h50m10s James Forrester talks Citoid and Zotero] at [[:m:WikiCite 2017]]

<div lang="en" dir="ltr" class="mw-content-ltr">
== References ==
</div>
<references/>

[[Category:WMF Projects{{#translation:}}]]