Extension talk:SyntaxHighlight/Archive 2018

Previous discussion was archived at Extension talk:SyntaxHighlight/Archive 2017 on 29 March 2017.

collapsible syntaxhighlight

i have a suggestion.

i want to reduce height of syntaxhighlight element when user doubleclick, so i added some code to Mediawiki:common.js.

----

$('.mw-highlight pre').dblclick(function(event) {

    if ($(event.target).parent().attr('class') == "scrolldiv") {

        $(event.target).unwrap();

    } else {

        $(event.target).wrap('<div class="scrolldiv"/>');

        $('.scrolldiv').css({ height: '300px', overflow: 'auto' });

    }

});

---

i thought it would be useful for someone.:)

thank you! Tgds003 (talk) 18:45, 5 February 2018 (UTC)

Instead of height, I'd suggest using max-height on .scrolldiv, so that the box won't be unnecessarily tall if there's only a handful of lines of code. 「ディノ奴千？！」^{☎ Dinoguy1000} 20:34, 5 February 2018 (UTC)

yes, you are right.

but, max-height doesn't seem to work.

i think we should use the 'if' statement rather than max-height.

i edited a code, as follows

----

$('.mw-highlight pre').dblclick(function(event) {

if (parseInt($(this).css('height'), 10)>300){

if ($(this).parent().attr('class') == "scrolldiv") {

    $(this).unwrap();

} else {

    $(this).wrap('<div class="scrolldiv"/>');

    $('.scrolldiv').css({ height: '300px', overflow: 'auto' });

}

}

}); Tgds003 (talk) 03:15, 7 February 2018 (UTC)

Call to undefined method MediaWiki\Shell\Command::input()

Product	Version
MediaWiki	1.31.0-alpha
HHVM	3.21.3 (srv)
MariaDB	10.0.32-MariaDB-0+deb8u1
ICU	52.1

SyntaxHighlight

2.0 (330ac62)12:31, 25 February 2018

GPL-2.0-or-later

Provides syntax highlighting <syntaxhighlight> using Pygments - Python syntax highlighter

Brion Vibber, Tim Starling, Rob Church, Niklas Laxström, Ori Livneh and Ed Sanders

# /usr/share/nginx/html/wiki/extensions/SyntaxHighlight_GeSHi/pygments/pygmentize -V

Pygments version 2.2.0, (c) 2006-2017 by Georg Brandl. -- PS! I also tried v2.0.2 and specifying the path in LocalSettings.

[error] 683#683: *648 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Call to undefined method MediaWiki\Shell\Command::input() in /usr/share/nginx/html/wiki/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php on line 320" while reading response header from upstream, client: 1.2.3.4, server: mycoolhost.net, request: "POST /wiki/api.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "mycoolhost.net"   

                $output = $cache->getWithSetCallback(

                        $cache->makeGlobalKey( 'highlight', self::makeCacheKeyHash( $code, $lexer, $options ) ),

                        $cache::TTL_MONTH,

                        function ( $oldValue, &$ttl ) use ( $code, $lexer, $options, &$error ) {

                                $optionPairs = [];

                                foreach ( $options as $k => $v ) {

                                        $optionPairs[] = "{$k}={$v}";

                                }

                                $result = Shell::command(

                                        self::getPygmentizePath(),

                                        '-l', $lexer,

                                        '-f', 'html',

                                        '-O', implode( ',', $optionPairs )

                                )

                                        ->input( $code )

                                        ->restrict( Shell::RESTRICT_DEFAULT | Shell::NO_NETWORK )

                                        ->execute();

                                if ( $result->getExitCode() != 0 ) {

                                        $ttl = WANObjectCache::TTL_UNCACHEABLE;

                                        $error = $result->getStderr();

                                        return null;

                                }

                                return $result->getStdout();

                        }

                );

Any tips? 77.88.121.37 (talk) 10:09, 5 March 2018 (UTC)

Preview within VE does not work

On our 1.29.1-Installation we activated this plugin. The highliting works properly. Also the integration into the VE is generally working, i. e. when adding a code block it asks about language and line numbers. However, when "inserting" i only see the ugly wikitext but not a preview how it will look like after saving. Don't know if this is a bug or a missing feature. Any help appriciated. Aschroet (talk) 07:15, 7 March 2018 (UTC)

I think this is T164120, an issue that was fixed about a year ago, and was released in the 1.30 branch of the code. If I'm right, when you upgrade this should be resolved for you. If I'm not and it's a different issue, please give some more details so we can track that. Jdforrester (WMF) (talk) 14:18, 8 March 2018 (UTC)

I think it is another issue since i select a valid language and as a result i am getting a string shown in VE like <syntaxhighlight lang="java"> test </syntaxhighlight>. This string is marked in blue and there is a bubble "code block" with the edit button. Unfortunately, i cannot upgrade to 1.30 in my environment. Could you help to debug this issue. I already checked the parsoid log without any result. Aschroet (talk) 12:45, 9 March 2018 (UTC)

How to switch this off?

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

SyntaxHighlight was recently rolled out on Russian Wikivoyage, and now I see all my source text in purple, light-green, and other disturbing colors. What is a simple way to switch this off, at least for me personally through my common.css? Atsirlin (talk) 07:54, 14 June 2018 (UTC)

My question has been answered. Sorry for trouble. Atsirlin (talk) 08:00, 14 June 2018 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Pygmentize process getting TimedOut

Aside from the MediaWiki:Common.css page, any page (MediaWiki:Common.js, Module pages, pages including syntaxhighlight tags) that needs to call Extension:SyntaxHighlight throws out the following process timed out error:

Fatal exception of type "Symfony\Component\Process\Exception\ProcessTimedOutException"

The problematic process according to the logs:

The process "'.../extensions/SyntaxHighlight_GeSHi/pygments/pygmentize' '-l' 'lua' '-f' 'html' '-O' 'cssclass=mw-highlight,encoding=utf-8'" exceeded the timeout of 60 seconds.

• Tested versions: MediaWiki core (REL1_30) - SyntaxHighlight_GeSHi (REL1_30) - PHP (5.6; 7.0; 7.1; 7.2)

In the case of REL1_31 branch, the process continues to run (and the browser continues to load) indefinitely without throwing out the timed out error. Even when the browser loading has been stopped, the php-cgi command keeps on running in the background and has to be manually terminated through SSH. This could possibly be because composer update removes the "symfony" package.

• Tested versions: MediaWiki core (REL1_31) - SyntaxHighlight_GeSHi (REL1_31) - PHP (7.0; 7.1; 7.2)
• Tested versions: MediaWiki core (REL1_31) - SyntaxHighlight_GeSHi (master) - PHP (7.2)

The pygmentize file has execute permissions (755). Also updated composer.

Platform is shared hosting CentOS environment. Process used to work previously, hosting support confirmed not to have made any changes to the server and even tried to run the script after disabling all firewall protections to no avail.

Anyone has any ideas? AhmadF.Cheema (talk) 08:13, 22 June 2018 (UTC)

Does it happen on every CSS/JS page, or only some of them? Try to test with a simple CSS/JS page.

I had a similar problem with one specific page, which timed out, probably due to complex code or a bug in the syntax highlight code. Ciencia Al Poder (talk) 08:59, 25 June 2018 (UTC)

Aside from MediaWiki:Common.css, on every page.

A confusing update.

1. $wgMainCacheType (CACHE_MEMCACHED) turned Off - CSS/JS pages work fine.
2. $wgMainCacheType turned On - CSS/JS pages continue to work fine.
3. $wgMainCacheType again turned Off - CSS/JS pages start getting timed out.
   1. Comment out any extension - CSS/JS pages start working again.
   2. Re-enable the extension - CSS/JS pages continue to work fine.

Hard-refresh was done after every step. This situation was reproduce-able, however for the moment the issue with CSS/JS pages and any page containing syntaxhighlight tags code blocks appears to have stopped occurring.

However, during none of the steps above, do Module pages work. They continue to get timed out.

Update:

Module pages also appear to be working now, waiting for response from hosting provider regarding any changes on their end which might have fixed the issue.

Update II:

Hosting provider confirmed no changes on the server.

Apparently, will never get to the bottom of this.

Will close this issue in the next couple of days, unless any new information comes along. AhmadF.Cheema (talk) 15:24, 25 June 2018 (UTC)

Maybe SyntaxHighlight is slow, and it eventually gets the syntax highlight correctly and caches it, so it works correctly next time you visit the page. Of course, this doesn't make sense when you turn off $wgMainCacheType and it still works fine. Maybe the page was cached on your browser and you didn't get to the server when you tried it... Ciencia Al Poder (talk) 19:25, 25 June 2018 (UTC)

Possible, though seems doubtful, since I attempted this probably more than 10 times, with hard-refreshes and browser incognito windows. AhmadF.Cheema (talk) 19:50, 25 June 2018 (UTC)

Do you have Python version 3 installed on the server? The version of Pygments used by the extension bundled with MW 1.31 requires python3 to be installed. I have added a note at the top of this article about this issue. Tystnaden (talk) 21:40, 8 July 2018 (UTC)

I didn't before, but this particular weird issue was probably not linked to python3 as error came and went while using both MediaWiki v1.30 and 1.31 (and their corresponding extension versions). From what I understand python3 is needed when highlighting languages such as Lua, in the absence of python3, MediaWiki just outputs the non-highlighted version with a - Notice: Failed to invoke Pygments: /usr/bin/env: python3: No such file or directory - error in debugging mode. AhmadF.Cheema (talk) 07:50, 9 July 2018 (UTC)

Okay, that does sound like a different problem. Tystnaden (talk) 20:57, 9 July 2018 (UTC)

Not sure if it's the same problem, but take a look at https://phabricator.wikimedia.org/T199989. Daimona Eaytoy (talk) 13:58, 20 July 2018 (UTC)

Failed to invoke Pygments on Windows

Hello,

I've just migrated and upgraded our company's MediaWiki to a new server. Windows Server 2012 R2, Apache 2.4.33 64bit + PHP 7.1.19 64bit + Python 3.6 64bit, MediaWiki 1.31.0.

Everything works except the SyntaxHighlight extension. The error message:

Notice:  Failed to invoke Pygments: 'C:\Python36\Scripts\pygmentize.exe" "-l" "css" "-f" "html" "-O" "cssclass' is not recognized as an internal or external command, operable program or batch file.

[Called from SyntaxHighlight::highlight in C:\Apache24\htdocs\wiki\extensions\SyntaxHighlight_GeSHi\includes\SyntaxHighlight.php at line 336] in C:\Apache24\htdocs\wiki\includes\debug\MWDebug.php on line 309

The PATH is set correctly, and from a sample test php file I can call Pygments, and it works:

<?php

$output = shell_exec('C:\Python36\Scripts\pygmentize.exe -l php -f html -O cssclass C:\Apache24\htdocs\test2.php');

echo "$output2";

?>

I have tried several options, like using the exe: $wgPygmentizePath = "C:\\Python36\\Scripts\\pygmentize.exe"; or the bytecode from the cgi directory:

$wgPygmentizePath = "C:\\Apache24\\cgi-bin\\pygmentize.pyc"; but nothing helped. Please advise how to debug further this problem. Glanthor Reviol (talk) 14:32, 13 July 2018 (UTC)

I'm having the same problem, there are some system config differences (for instance, I run MediaWiki 1.32 master on XAMPP with PHP 7.2.2), but having the same identical error. Having two versions of Python in two different directories, I tried both of them (2.7 and 3.3), changed wgPygmentizePath to all possible combinations, set full user rights on pygmentize, added everything to PATH, but I'm still getting the same error. And, as Glanthor Reviol, I can indeed execute it from wiki folder using windows' cmd and also directly from SyntaxHighlight using exec. I really can't think of something more to do, sounds like there's some specific problem with Windows. BTW, SyntaxHighlight used to work correctly with an old version (can't recall which one) of the extension itself and MediaWiki, using Python 2.7. Daimona Eaytoy (talk) 19:48, 17 July 2018 (UTC)

I seem to have the same problem on a MediaWiki 1.31.0, with both Python27 and Python37.

I read somwhere that there were som changes in Shell::command from MediaWiki 1.31.0 - and I traced the problem down to somwhere in that area, but could not find a fix for it.

Hope someone can find a fix for this problem. Bar-ucholstebro (talk) 16:11, 31 July 2018 (UTC)

I wrote a short and shallow explanation on phabricator, see https://phabricator.wikimedia.org/T199989. Basically, I fear that this might be a very upstream problem, i.e. a PHP bug which won't be fixed (a couple of links on the task) since Windows doesn't actually provide the needed logic. More specifically, the problem as far as I understood it should be that on Windows you cannot set streams as not blocking, which then produces a conflict within shell pipes, entering in a vicious circle. If this is true, there's probably no easy or clean solution to this problem: we could only solve it with a workaround or by refactoring the shell code. But I guess we should really find a solution, since this problem affects a core feature (Shell) on a whole OS. Daimona Eaytoy (talk) 20:01, 31 July 2018 (UTC)

Hello everyone.

I have the same problem on my side. SyntaxHighlight isn't working since 1.32 update.

I understand the problem with PHP, sream problem etc but I'm looking for a solution, a workaround and since 2 months, I'm unable to find such solution.

Does somebody have a miracle for me ?

Regards Fanoudu62219 (talk) 10:24, 12 September 2018 (UTC)

Personally, I can't find one. Actually, I'm not completely sure about the cause, too. Any update will come on Phabricator, anyway. Daimona Eaytoy (talk) 11:35, 12 September 2018 (UTC)

Thanks for reply.

I am afraid that there is no updates on phabricator too ^^

Wait & see... Fanoudu62219 (talk) 13:16, 12 September 2018 (UTC)

Any news? I have the same problem 78.134.114.236 (talk) 00:50, 14 June 2019 (UTC)

I want to add that I have this problem as well, on the latest 1.33, latest Syntax_Highlighter from the repo, etc. I've fought with it for hours. Has anyone successfully used pygment on Windows? It seems like it'd almost be easier, at this point, just to go back to a version that worked, and figure out how to get 'json' support, which is what spawned this whole upgrade for me. 23.226.128.42 (talk) 01:04, 9 July 2019 (UTC)

To add to my last comment, after upgrading to mediawiki-1.33.0, PHP 7.1.1, on Windows 7, and trying every bloody combination suggested in this thread (including editing the Syn*.php file, adding handler to httpd.conf and just on and on), there is absolutely no combination that 'works'. The closest is specifying the exe in a path AND editing the 'fixed' Rory solution below, but that only half works. Some items are colorized, but others are flat out hidden, with no error (if there's a highlight or lines tag). This is pathetic. I've wasted almost a whole day messing with this, and it just frustrates me I can't even downgrade now. (Older versions, pre-pygmentize, show no color at all?!!!).

The HightlightJs and Highlight-Integrate extensions don't seem to support highlighting code lines, or displaying line numbers.

Is this really that hard? Can anyone come up with an actual, fully working install for this bloody thing using MW1.33.0+ with xammp? 23.226.128.66 (talk) 05:30, 9 July 2019 (UTC)

The problem has not been solved yet......I'm try to use syntaxhighlight, but no matter how the test is unsuccessful. Finally I found this article and I won't try anymore until the problem is solved.

OS: Windows 10 Professional Edition

XAMPP: Version 7.3.11 with Apache 2.4.41 and PHP 7.3.11 (VC15 X86 64bit thread safe) + PEAR

MySQL: Version 5.7.21

MediaWiki: Version 1.33.1 Sincerelywy (talk) 07:01, 2 February 2020 (UTC)

Same issue here (Windows Server, IIS8.5, PHP7.4, Python 3.8.2, mw 1.34 - see Project:Support desk/Flow/2020/03#h-Migrating_from_1.9.3_to_1.34_-_Geshi_no_longer_available-2020-03-10T15:57:00.000Z for more details 46.232.228.6 (talk) 20:30, 11 March 2020 (UTC)

just solved (MW 3.4 IIS on Windows Server) it by

• editing SyntaxHighlight.php as shown here
• 'pip install pygment' manually
• setting $wgPygmentizePath='C:/Python/Scripts/pygmentize.exe';

                Goulu (talk) 11:42, 3 September 2020 (UTC)

pencil icon on editing toolbar

the pencil icon clashes with the visual editor icon. here are some alternate icons https://thenounproject.com/term/syntax/52606/ might want to think about it. Slowking4 (talk) 14:03, 14 August 2018 (UTC)

How to add a custom lang named as 'ipl' in SyntaxHighlight_GeSHi

Hi,

In earlier version "1.0.8.12" we are having with /geshi and we able to add custom code for a new lang. But in this latest version 2.0, I don't see how to add the custom code for syntexhighlight.

Could anyone pls help in regards? Sanjay (talk) 19:39, 18 August 2018 (UTC)

Please clarify your desire.

• Do you want to add a synoymous code for an existing scheme?
• Do you want to establish a new language syntax?

At least for the second you are free to create a new formal language upstream. MediaWiki will import that after half a year, or one year, or later. The same goes for alternative identifiers of existing definitions. PerfektesChaos (talk) 09:17, 20 August 2018 (UTC)

Yes we need to have a new language 'ipl' syntax highlighting in the SyntaxHighlight_GeSHi extension for MediaWiki 1.30.0

Could you let know that how could we archive that ? Sanjay (talk) 10:43, 23 August 2018 (UTC)

• GeSHi has been an external vendor, but for continuous support it has been exchanged by pygments some years ago.
• Visit upstream: http://pygments.org/docs/lexerdevelopment/
• Feed the world with a new language definition.
• Wait approximately half a year.
• MediaWiki will swallow the upstream news including yours and update the extension.
• Then you need to upgrade at least your extension, or perhaps the entire MediaWiki installation (seems to be rather old). PerfektesChaos (talk) 11:47, 23 August 2018 (UTC)

Syntax Highlight ABAP-Code

Hi,

i want to highlight the ABAP-Coding in my Mediawiki. In the List of Supported languages ABAP is' listet, but the available lexers pygments.lexers.business.ABAPLexer

are still listet on the page http://pygments.org/docs/lexers/#lexers-for-php-and-related-languages.

What can i do, to have my highlight ABAP-Coding back???

Greetings

Meinolf Stra2senBahn (talk) 16:55, 28 October 2018 (UTC)

The ABAP lexer is the first one listed under http://pygments.org/docs/lexers/#lexers-for-business-oriented-languages. Have you actually tried using it yet? 「ディノ奴千？！」^{☎ Dinoguy1000} 21:25, 28 October 2018 (UTC)

Thank you,

i know about this ABAP lexer. But I don't know how to use it with SyntaxHighlight. I don't know how to download it (on the listet page there it no download-link), I do't know how to install this lexer and so on.

Greetings

Meinolf Stra2senBahn (talk) 06:54, 30 October 2018 (UTC)

It should be installed as long as you have this extension installed. To use it you'd use <syntaxhighlight lang="abap">code here</syntaxhighlight>, as can be seen on Wikipedia's ABAP article. 「ディノ奴千？！」^{☎ Dinoguy1000} 11:42, 30 October 2018 (UTC)

Yes, when i use it in this Kind the page get the Category:Pages_with_syntax_highlighting_errors.

And i think ist's a Problem of the Extension, because the ar a Lot of Page with syntax-Highlightling-erros. Look at the link Category:Pages with syntax highlighting errors. eg the lang php or sql doset work also as ABAP.

Greeting

Meinolf Stra2senBahn (talk) 17:28, 30 October 2018 (UTC)

Try twiddling the capitalization, e.g. Abap or ABAP instead of abap. Apparently the extension is case-sensitive, but there doesn't seem to be a 100% comprehensive list of correct values for the lang attribute. 「ディノ奴千？！」^{☎ Dinoguy1000} 21:53, 30 October 2018 (UTC)

Sorry, but this proposal does not solve the problem.

even using <syntaxhighlight> without the parameter lang generates the error category.

greeting Meinolf Stra2senBahn (talk) 11:58, 31 October 2018 (UTC)

The category is always added if you don't have a lang attribute, since the extension doesn't have a default language, and doesn't have any language auto-detection feature. However, having a lang attribute doesn't guarantee that the categoy won't be added; it is also added if the value of the lang attribute is invalid (though as I said above, there isn't any completely comprehensive list of valid values as far as I'm aware). But, one of the ways the value can be "invalid" is if it isn't the expected capitalization, even though the actual language matching doesn't seem to be case-sensitive in most cases. As long as the actual syntax highlighting works correctly, changing the lang attribute's value to prevent the category from being added isn't terribly important. So, is your code being correctly highlighted when you use <syntaxhighlight lang="abap">? 「ディノ奴千？！」^{☎ Dinoguy1000} 12:15, 31 October 2018 (UTC)

No, the code isn't highlighted! The page, the code looks like using the TAGs <pre>code</pre>

Wen i use <syntaxhighlight lang="abap"> the Page will displayed with the Category:Pages with syntax highlighting errors.

greetings Meinolf Gerling Stra2senBahn (talk) 15:17, 4 November 2018 (UTC)

Is it being highlighted if you use any other capitalization of abap? If not, I'm afraid I'm out of ideas, beyond maybe making sure your MediaWiki installation and the extension are both the latest version, and double-checking the extension's configuration (both if you're able to, of course). 「ディノ奴千？！」^{☎ Dinoguy1000} 15:39, 4 November 2018 (UTC)

Requirements for SyntaxHighLight and other extensions

The extension SyntaxHighLight uses the access to shell of the Webserver it is running on.

Most hosting provider do not give the user access to the shell.

The code of SyntaxHighLight uses the MediaWiki Class Shell which checks if the user has access to the proc_open. See:

/**
* Checks if this class is effectively disabled via php.ini config.
*
* @return bool disabled
*/
public static function isDisabled() {
  static $disabled = null;

  if ( is_null( $disabled ) ) {
    if ( ! function_exists( 'proc_open' ) ) {
      wfDebug( "proc_open() is disabled\n" );
      $disabled = true;
    } else {
      $disabled = false;
    }
  }
  return $disabled;
}

There seems not to be another solution available.

My advice to not use the shell for extensions. It won't work.

The other remark is to make a requirements chapter for every extension which gives the user this information BEFORE he is trying to use such an extension!

This requirement chapter can explain which functions and which languages are needed.

There are a number of PHP functions that are disabled in the php.ini by the hosting providers. Examples are disk_total_space, diskfreespace, exec, system, popen, proc_open, proc_nice, shell_exec, passthru, dl

Check your php.ini file or create a website with the function phpinfo();

I would also suggest to use PHP as de-facto without the need to use other languages. Harm.frielink (talk) 12:44, 30 November 2018 (UTC)

My advice to not use the shell for extensions. It won't work.

In the other direction, our advice is not to use hosting providers that don't give you shell access, as it won't work for lots of functionality.

That said, I've added a note reminding sysadmins of this. Jdforrester (WMF) (talk) 17:08, 30 November 2018 (UTC)