View source for Grants:IEG/Wikiscan multi-wiki

{{Languages|Grants:IEG/Wikiscan multi-wiki}}
{{Probox
<!--List the usernames of all project participants in the space below. You can specify more grantees, volunteers, or advisors by adding new parameters of the form "|grantee2=HerUserName ... |grantee3=". Put each new parameter on its own line.-->
|grantee1=Akeron
|volunteer1=
|advisor1=
<!--Please add an email address (or a username that can receive email) that we can use to contact you. For an email address, use {{@}} in place of "@", for example, yourname{{@}}yourdomain.com.-->
|contact=akeron.wp{{@}}gmail.com
<!--Write a short summary of your project, in 20 words or less in the space below-->
|summary=Provide daily-updated statistics for the biggest Wikimedia wikis
<!--After you complete your budget in the project plan, add the total amount of funding you are requesting below, in USD (or specify other currency)-->
|amount=6690 EUR (7632 USD)
<!--Which existing Wikimedia projects and language versions will be better served or impacted as a result of this project? Example: Italian and Spanish Wikisource.-->
|target=Wikimedia wikis with more than 100,000 edits (currently 336 wikis)
<!--Please choose 1 Wikimedia strategic goal that your project most directly aims to impact. Choices of strategic goals: Increasing Reach (more people access Wikimedia projects), Improving Quality (better quality and quantity of content on Wikimedia projects), Increasing Participation (larger and more diverse groups of people are contributing to Wikimedia projects)-->
|goal=Increase participation
<!--If you want to make it easy for people to join your project as volunteers, change below to "YES"-->
|more_participants=NO
<!--If you want to customize the image, replace below with another file name from Commons-->
|image=File:IEG_key_blue.png
<!--Delete the "YES" below if you do not want your proposal to show up in the list of IEG drafts in the IdeaLab (Grants:IdeaLab/Ideas/Drafts)-->
|idealab=
<!--When you have finished drafting your proposal and it is ready for review, change below from DRAFT to PROPOSED (all caps).-->
|status=selected

<!-- YOU DON’T NEED TO EDIT THIS SECTION, GO ON TO *PART 2* -->
|round=round 1 2016
|year=2016
|project=Wikiscan multi-wiki
|timestamp=18:14, 29 February 2016 (UTC)
|portal=IEG
|translations=Probox/IEG/Content
}}

<div style='color:#333;margin:1em 0.5em 0.5em 0.5em;'>
''This is a translation of the French version, all corrections are welcome.''
</div>

<!--*PART 2* STARTS HERE-->
==Project idea==
{{TOC_right}} <!--Please live this here and continue to the next section-->

===What is the problem you're trying to solve?===

Provide daily-updated statistics for Wikimedia wikis, allow to know the current and past activity of the wiki, its main contributors according to several metrics, detailed statistics for each user and provide a tool to assess the performance of projects which support the participation of new editors.

===What is your solution?===

Transform and improve the website http://wikiscan.org, that already computes many statistics for the French Wikipedia since 2011, into a multi-wiki and multi-language site.

The site calculates statistics by date and user :

* Display the pages and the most active users on the wiki for the past 24 hours.
* Display the pages and the most active users for each day and each month since the wiki was created.
* Show the users who contributed the most since the the wiki was created, with many statistics such as : total edits, number of articles created, number of days and months with at least one action, total number of administrator actions, number of deletions, etc.
* View detailed statistics of a user account in seconds, even if they total several millions edits.
* Provide metrics for projects which support the participation of new volunteers.

==== Introducing Wikiscan ====

The purpose of Wikiscan is to bring many useful statistics about Wikimedia projects. There are several statistics already available but they are often scattered across multiple sites and wiki pages, each source is updated at its own pace and not always with the same data sources. Some use dumps with delays from one to several months between updates (eg [//stats.wikimedia.org/EN/Sitemap.htm WikiStats]), others use MediaWiki internal data which are approximate (e.g. [//en.wikipedia.org/wiki/Wikipedia:List_of_Wikipedians_by_number_of_edits List of Wikipedians by number of edits]). Wikiscan centralizes many statistics in the same place, using the same common basis and the same update rate.

Wikiscan computes all statistics from daily data, which allows for regular updates by recalculating only the current day, and then recombine the data for the month, year, total, and for each user. Complete recalculation is also conducted periodically to include deleted and restored pages or add new statistics. This flexible system allows for pre-calculated statistics to be displayed quickly and updated regularly with minimal computation.

===== Statistics by date =====

The statistics listed below are available for the last 24 hours and for each day and each month since the creation of the wiki :
* Page statistics that show the most active pages, the number of edits, the number of users who edited the page, the number of reverts, the total difference in size over the period (diff) and the sum of absolute diff. It is possible to sort these out and multiple filters based on namespaces are also available. Examples : [//fr.wikiscan.org/date/201604/pages April 2016], [//fr.wikiscan.org/date/20160302/pages March 2, 2016], [//fr.wikiscan.org/date/201001/ pages January 2010].
* User stats : number of edits, logged actions (page moves, blocks, etc.), reverts, average contribution rate, total diff, total absolute diff and presence time. It can be filtered to display only the users, IP, bots or admins and in some cases, sorted by columns. Example : [//fr.wikiscan.org/date/201603/utilisateurs March 2016].
* General Statistics over the period such as the total number of edits and for each namespace, the number of users, IP, pages and articles creations, accumulated presence time, the total for each logged action, etc. Example : [//fr.wikiscan.org/date/201603/statistiques March 2016].

The statistics of the last 24 hours [//fr.wikiscan.org/live/24/pages] are updated several times an hour, they show where current activity is the highest, which pages are the most active, where the major edits wars are, etc. The default sorting combines multiple data to get a better indicator The data especially taken into account are the number of users and edits but also the diff and the total volume in absolute values, so an edit war adding and removing a lot of content will be more highlighted than a little edit war.

Another page called [//fr.wikiscan.org/grille grid] uses the same data but shows several blocks with different predefined filters : namespace, the name of the page or a particular sorting. This allows to see in separate blocks which are the most active pages in various fields : articles, new articles, discussions, meta, articles for deletion or the currently most viewed articles.

Statistics on page views allow to take into account the volume of visitors on the most active pages and better anticipate trends in the news. This feature has been currently disabled following a change in servers and waiting for a pending overhaul. An example can be seen on ''archive.org'' when this feature was enabled for [https://web.archive.org/web/20150416220957/http://wikiscan.org/date/201501/pages January 2015] (page views are in the first column).

''Planned improvements'' :
* Reintegrate page views with an overhaul of the old system which won't scale with multi-wikis. Use of new statistics that exclude bots.
* Improve the display of the active articles on the last 24 hours, for example view the top ten in more detail with individual graphics for each page showing recent activity.

===== User stats =====
<div style='float:right;margin:0.5em;font-size:80%'>
{| class=wikitable style='text-align:right'
! Users'<br>edits !! Users<br>tested !! Wikiscan<br>average !! Xtools-ec<br>average
|-
| {{formatnum:500}}+ || 10 || 0.5 s || 5.2 s
|-
| {{formatnum:5000}}+ || 10 || 0.6 s || 8.2 s
|-
| {{formatnum:10000}}+ || 10 || 0.6 s || 9.7 s
|-
| {{formatnum:50000}}+ || 10 || 0.7 s || 38 s
|-
| {{formatnum:100000}}+ || 10 || 0.6 s || 51 s
|-
| {{formatnum:250000}}+ || 10 || 0.7 s || 93 s
|-
| {{formatnum:500000}}+ || 8 || 0.8 s || 220 s
|-
| {{formatnum:1000000}}+ || 3 || 0.6 s || 256 s
|-
| {{formatnum:1500000}}+ || 3 || 0.8 s || 261 s
|-
| {{formatnum:2000000}}+ || 1 || 0.7 s || 467 s
|}
<div style='max-width:22em'>A quick comparison of the response time between Wikiscan and [//tools.wmflabs.org/xtools-ec/?project=fr.wikipedia.org&purge=1 Xtools-ec] on fr.wikipedia.org. Tested April 30, 2016.
</div>
</div>

Wikiscan is commonly used as an edit counter, the feature visitors use the most. Compared to other edit counters, it quickly displays the page, even if the user totals several million edits, whereas other tools may take several minutes to load, or even fail. About 30 statistics are available on editions and logged actions. The goal is to quickly give a picture of a user's contributions with their history and to provide other interesting indicators in addition to the number of edits which can vary a lot, for example by making many small changes with semi-automated tools.

Examples: [//fr.wikiscan.org/utilisateur/Polmars user with 800 k edits], [//fr.wikiscan.org/utilisateur/HAL bot with about 2 million edits], [//fr.wikiscan.org/utilisateur/HerculeBot bot with 2.5 million edits].

Some examples of additional interesting statistics :
* Estimated total time (presence time) : the estimated time spent making edits and logged actions on the wiki, for example a user making one edit every 5 minutes for 30 minutes will have the same time of presence as a user making 1 edit every minutes over the same period, while their edits number would be 5 times smaller. This indicator could be more reliable than the gross number of edits to measure the involvement of a user. It has been used in particular in a professional thesis on the value of volunteering on Wikipedia (see [http://www.amplyd.com/these/Valorisation%20du%20b%C3%A9n%C3%A9volat%20sur%20Wikip%C3%A9dia%20-%20FEB2012%20-%20Vincent%20Juhel.pdf]).
* Participated days/months : the number of different days and months with at least one action. This allows to see which users contribute over time and regularly return. These indicators are more reliable than seniority calculated according to the creation date of the account, because some accounts can remain inactive over long periods. These statistics are used to calculate average levels with other values such as the average time and the number of edits per day participated.
* Re-editing : the number of editions that are made on the same page in a short time, some users use the preview feature a lot to send all edits at once, while others can edits many times the same page in a short time, which tends to inflate the number of editions.
* Chain edits : try to detect and count fast edits on different pages for those using scripts with their account.

About thirty statistics are calculated for each user, they are available for the total and for each year and each month since the account creation which allows to see the evolution of each value. It is possible to display percentages, for example to see trends by month or year of publishing rates in articles, discussions, meta, etc.

===== Overall user statistics =====

Since all user statistics are pre-calculated, they can be displayed in a summary table [//fr.wikiscan.org/utilisateurs]. This allows to quickly view those who most contributed to the wiki. It is possible to sort this view according to most of the thirty available statistics, allowing for example to know those who spent [//fr.wikiscan.org/?usort=tot_time2&bot=0&detail=0&menu=userstats the most time] (column "Durée"i.e. "duration"), [//fr.wikiscan.org/?usort=days&bot=0&detail=0&menu=userstats the more different days] (column "Jours" i.e. "days"), [//fr.wikiscan.org/?usort=new_main&bot=0&detail=0&menu=userstats the most article creations] (column "Articles"), [//fr.wikiscan.org/?usort=block&bot=0&detail=0&menu=userstats blocks]  (column "Bloc."), [//fr.wikiscan.org/?usort=protect&bot=0&detail=0&menu=userstats protections]  (column "Protect."), [//fr.wikiscan.org/?usort=filter&bot=0&detail=0&menu=userstats abusefilter edits] (column "Filtres"), etc.

'' Planned improvements '':
* Display the same table for each year and each month since the wiki creation.
* Add graphics showing the evolution of active users based on the number of months of participation.

===== Category or user list =====

The same users table can display only users of a MediaWiki category or a list provided. This is used by projects that support the participation of new editors to measure their results. For instance the [[:fr:Project:Afripédia|Afripédia Project]] supported by [[Wikimédia France/en|Wikimedia France]] encourages participation in Francophone Africa. If participants insert a user box that adds a category, this will display all participants statistics in one page.

Examples : [//fr.wikiscan.org/?menu=userstats&userlist=Cat%C3%A9gorie%3AUtilisateur+participant+au+projet+Afrip%C3%A9dia Afripédia project participants], [//fr.wikiscan.org/?menu=userstats&userlist=Cat%C3%A9gorie%3AFormation+Afrip%C3%A9dia+Madagascar Training Afripédia in Madagascar].

It is then possible to sort the data according to the available statistics, for example we can see that 51 users of the Afripédia project contributed for 2 hours or more in total presence time [//fr.wikiscan.org/?usort=tot_time2&bot=1&detail=0&menu=userstats&userlist=Cat%C3%A9gorie%3AUtilisateur+participant+au+projet+Afrip%C3%A9dia], or that 47 users have contributed at least 5 different days [//fr.wikiscan.org/?usort=days&bot=1&detail=0&menu=userstats&userlist=Cat%C3%A9gorie%3AUtilisateur+participant+au+projet+Afrip%C3%A9dia] and 37 users at least 3 different months [//fr.wikiscan.org/?usort=months&bot=1&detail=0&menu=userstats&userlist=Cat%C3%A9gorie%3AUtilisateur+participant+au+projet+Afrip%C3%A9dia], or that 65 users have added globally over 1&nbsp;ko on articles [//fr.wikiscan.org/?usort=diff_article_no_rv&bot=1&detail=0&menu=userstats&userlist=Cat%C3%A9gorie%3AUtilisateur+participant+au+projet+Afrip%C3%A9dia], etc.

''Planned improvements'':
* Allow to specify a wiki page that contains a list of users instead of a category, with options to select a users extraction method.

===== Other improvements =====

The most important change will be the transition to a multi-wikis site that should be optimized to constantly update statistics of hundreds of wikis instead of one. A system of "workers" will allow multiple wiki updates in parallel by size without overloading the server. A maintenance page will be developed to track the status updates of all wikis.

A new global homepage will be created, it will present all the wikis by family (Wikipedia, Wiktionary, etc.). A new home page per wiki will show recent global statistics and graphs showing the historical evolution of the wiki.

Internationalization will allow to choose the interface language, an English and French version will be available at first. Other languages will be added when translation files are available.

The project will be an opportunity to review the code that has been mainly developed little by little and to document the calculated statistics. There will be restructuring in the statistical calculations and the addition of some new ones. Improvements to graphics are planed, perhaps using SVG and JavaScript to replace the current images.

==Project goals==

Main goals :
# Change the Wikiscan site that currently only supports the French Wikipedia into a multi-wikis site capable of supporting many other wikis.
# Set up a multi-language interface and produce an English version to facilitate translations into other languages.
# Improve and optimize the site, in particular add global statistics by wiki.

The current goal of the multi-wikis is to support at least all public Wikimedia wikis totalling over {{formatnum:100000}} edits, it currently represents 336 wikis.

The source code will be available under a free license, it will be technically possible to make it work with the database of any wiki using a recent version of MediaWiki.

==Project plan==
<!--Leave this section blank -->
===Activities===

Main steps of the project :

# Multi-wiki transformation :
#* Allow the site to work directly with the Wikimedia Labs database rather than a local replication. This will allow the site to operate easily with any Wikimedia wiki
#* Transform into a multi-wikis site with a subdomain and a specific database for each wiki
#* Develop tools for daily statistics updates for each wiki (workers, tracking page, configuration) and perform specific optimizations for small wikis
#* Create a new global home page and one for each wiki that will display global statistics
#Internationalization :
#* Transform the interface for multi-language support
#* Translate the interface in English
#Community report (after 3 months maximum) :
#* When the Wikiscan site for each wiki will be up and running, it will be announced on several large projects for the first feedbacks.
#Improvements and enhancements :
#* View the overall ranking of users for each year and each month
#* Add pageviews on statistics by date
#* Allow users to use a list from a wiki page instead of a category
#* Add new graphics on the evolution of active users based on the number of months of participation
#* Improve the display of the most active articles on the last 24 hours
#* Document the various statistics computed by the site
#* Improve the interface and the graphics
#* Restructure and add statistics
#* Various fixes and optimizations

===Budget===

The budget will be used to pay for the hours of programming as an independent developer, nothing is requested for the project management/administrative part. According to an online newspaper [http://www.journaldunet.com/developpeur/algo-methodes/salaire-tarif-du-developpeur-freelance-php-javascript-java-ruby.shtml], the average price of a freelance PHP developer in France is 327 euros per day (40.9 euros gross per hour). As this project represents several weeks of work, I ask for 30 euros per hour (this includes 23% professional charges payable in France).

{| class='wikitable'
! Description !! Hours !! Total<br><small>(30&nbsp;EUR/h)</small>
|-
| style="padding: 0.5em" colspan=3 | '''Multi-wikis site Transformation'''
|-
| 1 - Allow the site to calculate statistics directly with the remote Wikimedia Labs database and perform the necessary optimizations || align=right | 15 || align=right | 450
|-
| 2 - Transform the site into a multi-wiki site with a subdomain and a database for each wiki || align=right | 18 || align=right | 540
|-
| 3 - Set up a system of "workers" to update statistics for each wiki || align=right | 14 || align=right | 420
|-
| 4 - Create a new tracking page to monitor the updates status of workers of all wikis || align=right | 16 || align=right | 480
|-
| 5 - Create a new global homepage displaying the list of available wikis with some general statistics and graphs for each wiki || align=right | 20 || align=right | 600
|-
| 6 - Create a new home page for each wiki with global statistics updated regularly || align=right | 16 || align=right | 480
|-
| 7 - Adapt statistics by date for small wiki that have too few edits per day (show only months and add years, allow more than 24 hours for recent edits) || align=right | 6 || align=right | 180
|-
| align=right | Total || align=right | '''105&nbsp;h''' || align=right | '''3150&nbsp;€'''
|-
| style="padding: 0.5em" colspan=3 | '''Internationalization and documentation'''
|-
| 8 - Transform the interface for multi-language support with all texts in a single file || align=right | 18 || align=right | 540
|-
| 9 - Translate the interface in English || align=right | 8 || align=right | 240
|-
| 10 - French documentation of the various statistics computed by the site with possibility to add other languages by translating a file || align=right | 8 || align=right | 240
|-
| align=right | Total || align=right | '''34&nbsp;h '''|| align=right | '''1020&nbsp;€'''
|-
| style="padding: 0.5em" colspan=3 | '''Improvements and optimizations'''
|-
| 11 - Display the overall ranking of users for each year and each month || align=right | 6 || align=right | 180
|-
| 12 - Add page views on the statistics by date, the system must be highly optimized to work with hundreds of wikis || align=right | 20 || align=right | 600
|-
| 13 - Allow to use a list of users contained in a wiki page instead of a category || align=right | 2 || align=right | 60
|-
| 14 - Add new graphics on the evolution of active users based on the number of months of participation || align=right | 10 || align=right | 300
|-
| 15 - Improve the display of the most active articles for the last 24 hours || align=right | 12 || align=right | 360
|-
| 16 - UI improvements and graphics || align=right | 8 || align=right | 240
|-
| 17 - Restructuring and additions of statistics || align=right | 6 || align=right | 180
|-
| 18 - Optimizations and various corrections, code rewriting || align=right | 20 || align=right | 600
|-
| align=right | Total || align=right | '''84&nbsp;h '''|| align=right | '''2520&nbsp;€'''
|-
| style="padding: 1em; text-align: right" | '''Total project''' || align=right | '''223&nbsp;h''' || align=right | '''6690&nbsp;€'''
|}

'''Total project'''
: 6690.00&nbsp;EUR
: 7632.62&nbsp;USD <small>(for 1 EUR=1.1409 USD as of 8 April 2016)</small>

===Community engagement===

The site works for the French Wikipedia since 2011 where it is commonly used, there have already been several feedbacks from this community.

I will solicit francophone projects other than Wikipedia (Wiktionary, Wikisource, Wikibooks, Wikiversity, etc.) to better adapt Wikiscan to the particularities of these projects, such as their use of namespaces.

When sites for each wiki start to become operational and the interface is available in English (max 3 months after the start), I will contact the largest wiki in other languages to get feedback. I will probably open a page on meta to centralize discussions on Wikiscan.

===Sustainability===

The site is designed to operate autonomously, all statistical updates are automatic processed. I will continue to maintain the server and do bugs corrections as I have been doing for the French Wikipedia site since 2011.

Wikimedia wikis with more than {{formatnum:100000}} edits will be added automatically and their statistics will be automatically activated.

The dedicated server hosting the site is already funded, it belongs to the [[Wikimédia France/en|Wikimédia France]] association, which ensures good continuity for the project.

Many developments and improvements are possible : support of all small wikis, adding new statistics (deleted contributions in particular), making overall statistics for all users by adding all the wikis, provide custom graphics with the many available statistics, improve and allow interface customization (for example choose oneself filters for blocks displayed in the "grid" page), improve statistics for the list or category of users (calculating the total and average for all these users), export many data available in various formats (CSV, JSON...), Translatewiki support for translations, develop an API to allow external tools to easily use data (e.g. graphics embedded in wiki pages with the [[mw:extension:graph/Demo|graph extension]]), develop a dedicated MediaWiki extension...

===Measures of success===

Public Wikimedia wikis totalling at least {{formatnum:100000}} edits (currently 336 wikis) will get their own functional xxx.wikiscan.org site with the statistics explained in [[#Introducing Wikiscan]] and new developed improvements.

Wikiscan Subdomains for the 336 currently supported wiki would be :

<div class='plainlinks'>
Wikipedia (185) : [http://en.wikiscan.org en], [http://de.wikiscan.org de], [http://fr.wikiscan.org fr], [http://ru.wikiscan.org ru], [http://it.wikiscan.org it], [http://pt.wikiscan.org pt], [http://es.wikiscan.org es], [http://ja.wikiscan.org ja], [http://zh.wikiscan.org zh], [http://nl.wikiscan.org nl], [http://ar.wikiscan.org ar], [http://pl.wikiscan.org pl], [http://vi.wikiscan.org vi], [http://fa.wikiscan.org fa], [http://sv.wikiscan.org sv], [http://id.wikiscan.org id], [http://tr.wikiscan.org tr], [http://uk.wikiscan.org uk], [http://fi.wikiscan.org fi], [http://hu.wikiscan.org hu], [http://he.wikiscan.org he], [http://ro.wikiscan.org ro], [http://ca.wikiscan.org ca], [http://no.wikiscan.org no], [http://ko.wikiscan.org ko], [http://sr.wikiscan.org sr], [http://sh.wikiscan.org sh], [http://cs.wikiscan.org cs], [http://da.wikiscan.org da], [http://th.wikiscan.org th], [http://hr.wikiscan.org hr], [http://bg.wikiscan.org bg], [http://ms.wikiscan.org ms], [http://el.wikiscan.org el], [http://eo.wikiscan.org eo], [http://simple.wikiscan.org simple], [http://ceb.wikiscan.org ceb], [http://lt.wikiscan.org lt], [http://hi.wikiscan.org hi], [http://sl.wikiscan.org sl], [http://war.wikiscan.org war], [http://az.wikiscan.org az], [http://sk.wikiscan.org sk], [http://bs.wikiscan.org bs], [http://mk.wikiscan.org mk], [http://ka.wikiscan.org ka], [http://hy.wikiscan.org hy], [http://gl.wikiscan.org gl], [http://et.wikiscan.org et], [http://bn.wikiscan.org bn], [http://kk.wikiscan.org kk], [http://eu.wikiscan.org eu], [http://ml.wikiscan.org ml], [http://lv.wikiscan.org lv], [http://ta.wikiscan.org ta], [http://nn.wikiscan.org nn], [http://sq.wikiscan.org sq], [http://be.wikiscan.org be], [http://ur.wikiscan.org ur], [http://te.wikiscan.org te], [http://la.wikiscan.org la], [http://mr.wikiscan.org mr], [http://uz.wikiscan.org uz], [http://cy.wikiscan.org cy], [http://tl.wikiscan.org tl], [http://af.wikiscan.org af], [http://zh-yue.wikiscan.org zh-yue], [http://is.wikiscan.org is], [http://zh-min-nan.wikiscan.org zh-min-nan], [http://be-x-old.wikiscan.org be-x-old], [http://br.wikiscan.org br], [http://lb.wikiscan.org lb], [http://tt.wikiscan.org tt], [http://oc.wikiscan.org oc], [http://jv.wikiscan.org jv], [http://fy.wikiscan.org fy], [http://ce.wikiscan.org ce], [http://arz.wikiscan.org arz], [http://an.wikiscan.org an], [http://als.wikiscan.org als], [http://vo.wikiscan.org vo], [http://kn.wikiscan.org kn], [http://lmo.wikiscan.org lmo], [http://nds.wikiscan.org nds], [http://sco.wikiscan.org sco], [http://mg.wikiscan.org mg], [http://sw.wikiscan.org sw], [http://pms.wikiscan.org pms], [http://ast.wikiscan.org ast], [http://min.wikiscan.org min], [http://si.wikiscan.org si], [http://new.wikiscan.org new], [http://ga.wikiscan.org ga], [http://mn.wikiscan.org mn], [http://ne.wikiscan.org ne], [http://bar.wikiscan.org bar], [http://my.wikiscan.org my], [http://tg.wikiscan.org tg], [http://ckb.wikiscan.org ckb], [http://ba.wikiscan.org ba], [http://ku.wikiscan.org ku], [http://scn.wikiscan.org scn], [http://gu.wikiscan.org gu], [http://ky.wikiscan.org ky], [http://cv.wikiscan.org cv], [http://ia.wikiscan.org ia], [http://io.wikiscan.org io], [http://yi.wikiscan.org yi], [http://sa.wikiscan.org sa], [http://pa.wikiscan.org pa], [http://ht.wikiscan.org ht], [http://yo.wikiscan.org yo], [http://vec.wikiscan.org vec], [http://zh-classical.wikiscan.org zh-classical], [http://am.wikiscan.org am], [http://pnb.wikiscan.org pnb], [http://qu.wikiscan.org qu], [http://su.wikiscan.org su], [http://bpy.wikiscan.org bpy], [http://nap.wikiscan.org nap], [http://li.wikiscan.org li], [http://os.wikiscan.org os], [http://gd.wikiscan.org gd], [http://wa.wikiscan.org wa], [http://fo.wikiscan.org fo], [http://sah.wikiscan.org sah], [http://ang.wikiscan.org ang], [http://bat-smg.wikiscan.org bat-smg], [http://ksh.wikiscan.org ksh], [http://or.wikiscan.org or], [http://gan.wikiscan.org gan], [http://km.wikiscan.org km], [http://hsb.wikiscan.org hsb], [http://nah.wikiscan.org nah], [http://wuu.wikiscan.org wuu], [http://as.wikiscan.org as], [http://ilo.wikiscan.org ilo], [http://diq.wikiscan.org diq], [http://hif.wikiscan.org hif], [http://ps.wikiscan.org ps], [http://nds-nl.wikiscan.org nds-nl], [http://pam.wikiscan.org pam], [http://vls.wikiscan.org vls], [http://mzn.wikiscan.org mzn], [http://map-bms.wikiscan.org map-bms], [http://bh.wikiscan.org bh], [http://se.wikiscan.org se], [http://mt.wikiscan.org mt], [http://so.wikiscan.org so], [http://bo.wikiscan.org bo], [http://gv.wikiscan.org gv], [http://bcl.wikiscan.org bcl], [http://tk.wikiscan.org tk], [http://szl.wikiscan.org szl], [http://co.wikiscan.org co], [http://bug.wikiscan.org bug], [http://dv.wikiscan.org dv], [http://mhr.wikiscan.org mhr], [http://sc.wikiscan.org sc], [http://roa-tara.wikiscan.org roa-tara], [http://frr.wikiscan.org frr], [http://nv.wikiscan.org nv], [http://crh.wikiscan.org crh], [http://ug.wikiscan.org ug], [http://lad.wikiscan.org lad], [http://mi.wikiscan.org mi], [http://fiu-vro.wikiscan.org fiu-vro], [http://csb.wikiscan.org csb], [http://xmf.wikiscan.org xmf], [http://dsb.wikiscan.org dsb], [http://rm.wikiscan.org rm], [http://fur.wikiscan.org fur], [http://lij.wikiscan.org lij], [http://kv.wikiscan.org kv], [http://stq.wikiscan.org stq], [http://kw.wikiscan.org kw], [http://nrm.wikiscan.org nrm], [http://frp.wikiscan.org frp], [http://ext.wikiscan.org ext], [http://ie.wikiscan.org ie], [http://krc.wikiscan.org krc], [http://ln.wikiscan.org ln], [http://jbo.wikiscan.org jbo], [http://nov.wikiscan.org nov], [http://gom.wikiscan.org gom].

Wiktionary (74) : [http://enwiktionary.wikiscan.org en], [http://frwiktionary.wikiscan.org fr], [http://ruwiktionary.wikiscan.org ru], [http://dewiktionary.wikiscan.org de], [http://eswiktionary.wikiscan.org es], [http://zhwiktionary.wikiscan.org zh], [http://mgwiktionary.wikiscan.org mg], [http://plwiktionary.wikiscan.org pl], [http://itwiktionary.wikiscan.org it], [http://ptwiktionary.wikiscan.org pt], [http://nlwiktionary.wikiscan.org nl], [http://svwiktionary.wikiscan.org sv], [http://elwiktionary.wikiscan.org el], [http://trwiktionary.wikiscan.org tr], [http://fiwiktionary.wikiscan.org fi], [http://ltwiktionary.wikiscan.org lt], [http://kowiktionary.wikiscan.org ko], [http://shwiktionary.wikiscan.org sh], [http://jawiktionary.wikiscan.org ja], [http://tawiktionary.wikiscan.org ta], [http://kuwiktionary.wikiscan.org ku], [http://huwiktionary.wikiscan.org hu], [http://viwiktionary.wikiscan.org vi], [http://iowiktionary.wikiscan.org io], [http://idwiktionary.wikiscan.org id], [http://cawiktionary.wikiscan.org ca], [http://rowiktionary.wikiscan.org ro], [http://nowiktionary.wikiscan.org no], [http://bgwiktionary.wikiscan.org bg], [http://cswiktionary.wikiscan.org cs], [http://sawiktionary.wikiscan.org sa], [http://arwiktionary.wikiscan.org ar], [http://fawiktionary.wikiscan.org fa], [http://etwiktionary.wikiscan.org et], [http://mlwiktionary.wikiscan.org ml], [http://chrwiktionary.wikiscan.org chr], [http://thwiktionary.wikiscan.org th], [http://mywiktionary.wikiscan.org my], [http://tewiktionary.wikiscan.org te], [http://liwiktionary.wikiscan.org li], [http://hiwiktionary.wikiscan.org hi], [http://knwiktionary.wikiscan.org kn], [http://hywiktionary.wikiscan.org hy], [http://ukwiktionary.wikiscan.org uk], [http://simplewiktionary.wikiscan.org simple], [http://hewiktionary.wikiscan.org he], [http://glwiktionary.wikiscan.org gl], [http://orwiktionary.wikiscan.org or], [http://lowiktionary.wikiscan.org lo], [http://srwiktionary.wikiscan.org sr], [http://euwiktionary.wikiscan.org eu], [http://uzwiktionary.wikiscan.org uz], [http://iswiktionary.wikiscan.org is], [http://dawiktionary.wikiscan.org da], [http://eowiktionary.wikiscan.org eo], [http://azwiktionary.wikiscan.org az], [http://hrwiktionary.wikiscan.org hr], [http://wawiktionary.wikiscan.org wa], [http://ocwiktionary.wikiscan.org oc], [http://brwiktionary.wikiscan.org br], [http://cywiktionary.wikiscan.org cy], [http://sqwiktionary.wikiscan.org sq], [http://mnwiktionary.wikiscan.org mn], [http://scnwiktionary.wikiscan.org scn], [http://afwiktionary.wikiscan.org af], [http://lawiktionary.wikiscan.org la], [http://tlwiktionary.wikiscan.org tl], [http://fjwiktionary.wikiscan.org fj], [http://astwiktionary.wikiscan.org ast], [http://vowiktionary.wikiscan.org vo], [http://zh-min-nanwiktionary.wikiscan.org zh-min-nan], [http://slwiktionary.wikiscan.org sl], [http://fywiktionary.wikiscan.org fy], [http://swwiktionary.wikiscan.org sw].

Wikisource (21) : [http://sources.wikiscan.org (multilingue)], [http://enwikisource.wikiscan.org en], [http://frwikisource.wikiscan.org fr], [http://ruwikisource.wikiscan.org ru], [http://dewikisource.wikiscan.org de], [http://itwikisource.wikiscan.org it], [http://plwikisource.wikiscan.org pl], [http://eswikisource.wikiscan.org es], [http://zhwikisource.wikiscan.org zh], [http://hewikisource.wikiscan.org he], [http://arwikisource.wikiscan.org ar], [http://ptwikisource.wikiscan.org pt], [http://bnwikisource.wikiscan.org bn], [http://tawikisource.wikiscan.org ta], [http://svwikisource.wikiscan.org sv], [http://cswikisource.wikiscan.org cs], [http://mlwikisource.wikiscan.org ml], [http://kowikisource.wikiscan.org ko], [http://slwikisource.wikiscan.org sl], [http://nowikisource.wikiscan.org no], [http://tewikisource.wikiscan.org te].

Wikinews (12) : [http://enwikinews.wikiscan.org en], [http://srwikinews.wikiscan.org sr], [http://frwikinews.wikiscan.org fr], [http://itwikinews.wikiscan.org it], [http://ruwikinews.wikiscan.org ru], [http://eswikinews.wikiscan.org es], [http://dewikinews.wikiscan.org de], [http://plwikinews.wikiscan.org pl], [http://ptwikinews.wikiscan.org pt], [http://fawikinews.wikiscan.org fa], [http://jawikinews.wikiscan.org ja], [http://trwikinews.wikiscan.org tr].

Wikibooks (11) : [http://enwikibooks.wikiscan.org en], [http://dewikibooks.wikiscan.org de], [http://frwikibooks.wikiscan.org fr], [http://ptwikibooks.wikiscan.org pt], [http://eswikibooks.wikiscan.org es], [http://itwikibooks.wikiscan.org it], [http://huwikibooks.wikiscan.org hu], [http://nlwikibooks.wikiscan.org nl], [http://plwikibooks.wikiscan.org pl], [http://ruwikibooks.wikiscan.org ru], [http://hewikibooks.wikiscan.org he].

Wikiquote (10) : [http://enwikiquote.wikiscan.org en], [http://itwikiquote.wikiscan.org it], [http://plwikiquote.wikiscan.org pl], [http://ruwikiquote.wikiscan.org ru], [http://eswikiquote.wikiscan.org es], [http://dewikiquote.wikiscan.org de], [http://frwikiquote.wikiscan.org fr], [http://ptwikiquote.wikiscan.org pt], [http://trwikiquote.wikiscan.org tr], [http://hewikiquote.wikiscan.org he].

Wikivoyage (9) : [http://enwikivoyage.wikiscan.org en], [http://dewikivoyage.wikiscan.org de], [http://itwikivoyage.wikiscan.org it], [http://frwikivoyage.wikiscan.org fr], [http://ruwikivoyage.wikiscan.org ru], [http://eswikivoyage.wikiscan.org es], [http://nlwikivoyage.wikiscan.org nl], [http://hewikivoyage.wikiscan.org he], [http://ptwikivoyage.wikiscan.org pt].

Wikiversity (5) : [http://enwikiversity.wikiscan.org en], [http://dewikiversity.wikiscan.org de], [http://frwikiversity.wikiscan.org fr], [http://ruwikiversity.wikiscan.org ru], [http://itwikiversity.wikiscan.org it].

Others (9) : [http://commons.wikiscan.org commons], [http://wikidata.wikiscan.org wikidata], [http://meta.wikiscan.org meta], [http://mediawiki.wikiscan.org mediawiki], [http://species.wikiscan.org species], [http://incubator.wikiscan.org incubator], [http://test.wikiscan.org test], [http://test2.wikiscan.org test2], [http://outreach.wikiscan.org outreach].
</div>
'''Visitor goal''' : reach at least 300 unique users per month on all new subdomains (thus excluding fr.wikiscan.org and future portal wikiscan.org). The measurement tool is Google Analytics.

==Get involved==
<!--Leave this section blank -->
===Participants===

* [[User: Akeron | Akeron]]: software developer, I have all the required technical knowledge to fully realize this project, I know Wikiscan very well  because I fully designed it, and I've been a regular Wikimedia editor since 2006, mainly on French Wikipedia.

===Community Notification===

This project was presented on some Francophone projects and on English Wikipedia: [//fr.wiktionary.org/w/index.php?title=Wiktionnaire:Wikid%C3%A9mie/f%C3%A9vrier_2016&oldid=21093608#Projet_d.27extension_de_Wikiscan_.C3.A0_d.27autres_projets Wiktionary], [//fr.wikisource.org/w/index.php?title=Wikisource:Scriptorium/F%C3%A9vrier_2016&oldid=5619792#Projet_d.27extension_de_Wikiscan_.C3.A0_d.27autres_projets Wikisource], [//fr.wikiversity.org/w/index.php?title=Wikiversit%C3%A9:La_salle_caf%C3%A9/f%C3%A9vrier_2016&oldid=532575#Projet_d.27extension_de_Wikiscan_.C3.A0_d.27autres_projets Wikiversity], [//en.wikipedia.org/w/index.php?title=Wikipedia:Village_pump_%28idea_lab%29&oldid=711114162#Statistics_Website English Wikipedia (idea lab)].

===Endorsements===
<!--This section is for community members to describe why they think a project idea is of value.-->
Do you think this project should be selected for an Individual Engagement Grant? Please add your name and rationale for ''endorsing'' this project in the list below. (Other constructive feedback is welcome on the [[Grants talk:{{PAGENAME}}|talk page]] of this proposal).

* ''Community member: add your name and rationale here.''
# {{Endorsement}} Wikiscan is a great tool and I really wish it supports more wikis. [[User:Thibaut120094|Thibaut120094]] ([[User talk:Thibaut120094|talk]]) 17:25, 8 April 2016 (UTC)
# {{Endorsement}} [[User:JackPotte|JackPotte]] ([[User talk:JackPotte|talk]]) 17:41, 8 April 2016 (UTC)
# {{Endorsement}} [[s:fr:User:Ernest-Mtl|Ernest-Mtl]] ([[s:fr:User talk:Ernest-Mtl|talk]]) 18:10, 8 April 2016 (EDT)
# {{Endorsement}} Wikiscan was designed for the French Wikipedia, where I edit the most, and it worked well. I used Wikiscan once a week from 2010 to 2014, because it provided lots of useful infos on users, on articles and so on. I highly regard the product and the software developer. [[User:Cantons-de-l&#39;Est|Cantons-de-l&#39;Est]] ([[User talk:Cantons-de-l&#39;Est|talk]]) 16:23, 9 April 2016 (UTC)
# {{Endorsement}} [[User:Unsui|Unsui]] ([[User talk:Unsui|talk]]) 20:18, 9 April 2016 (UTC)
# {{Endorsement}} --[[User:Yann|Yann]] ([[User talk:Yann|talk]]) 15:06, 11 April 2016 (UTC)
# {{Endorsement}} [[User:Lyokoï|Lyokoï]] ([[User talk:Lyokoï|talk]]) 15:15, 11 April 2016 (UTC)
# {{Endorsement}} [[User:Tomthepsg|Tomthepsg]] ([[User talk:Tomthepsg|talk]]) 17:26, 11 April 2016 (UTC)
# {{Endorsement}} [[User:Pamputt|Pamputt]] ([[User talk:Pamputt|talk]]) 15:36, 11 April 2016 (UTC)
# {{Endorsement}} [[User:Otourly|Otourly]] ([[User talk:Otourly|talk]]) 15:36, 11 April 2016 (UTC)
# {{Endorsement}} essential & expected (so). --[[User:Benoît Prieur|Benoît Prieur]] ([[User talk:Benoît Prieur|talk]]) 15:37, 11 April 2016 (UTC)
# {{Endorsement}} [[User:Wuyouyuan|Wuyouyuan]] ([[User talk:Wuyouyuan|talk]]) 13:14, 12 April 2016 (UTC)
# Soutient [[User:Lionel Scheepmans|Lionel Scheepmans]] <sup><big>✉ </big> [[User talk:Lionel Scheepmans|Contact]]</sup> <sub>French native speaker, désolé pour ma [[w:dysorthographie|dysorthographie]]</sub> 15:42, 11 April 2016 (UTC)
#{{Endorsement}} Highly interesting and useful to deal with. Many thanks [[fr:User:Kaviraf|Kaviraf ]] [[User:Kaviraf|Kaviraf]] ([[User talk:Kaviraf|talk]]) 16:18, 11 April 2016 (UTC)
# {{Endorsement}} [[User:Thierry613|Thierry613]] ([[User talk:Thierry613|talk]]) 16:52, 11 April 2016 (UTC)
# {{Endorsement}} I had the opportunity to see pleasantly run this tool on the french wikipedia and the use that could be done. Other projects can profitably use this tool. Request worth studying with the utmost seriousness [[User:Crochet.david|Crochet.david]] ([[User talk:Crochet.david|talk]]) 17:05, 11 April 2016 (UTC)
# {{Endorsement}} --[[User:Zyephyrus|Zyephyrus]] ([[User talk:Zyephyrus|talk]]) 22:14, 11 April 2016 (UTC)
# {{Endorsement}} [[User:Viticulum|Viticulum]] ([[User talk:Viticulum|talk]]) 00:24, 12 April 2016 (UTC)
# {{Endorsement}} A very good tool that could help a lot of communities of the Wikimedia movement. [[User:Pyb|Pyb]] ([[User talk:Pyb|talk]]) 11:49, 12 April 2016 (UTC)
# {{Endorsement}} Looks to fill an important gap in information about Wikipedias. [[User:Fences and windows|Fences and windows]] ([[User talk:Fences and windows|talk]]) 16:43, 12 April 2016 (UTC)
# {{Endorsement}} Very helpful tool! [[User:Dyolf77|Dyolf77]] ([[User talk:Dyolf77|talk]]) 23:04, 12 April 2016 (UTC)
# {{Endorsement}} To understand the community and the readers, nothing is more important than clear data, which is provided by this tool. Also, the pageviews stats are the best quantitative data we have to measure the effective sharing of knowledge. We need more of this. [[User:Plyd|Plyd]] ([[User talk:Plyd|talk]]) 09:28, 13 April 2016 (UTC)
# {{Endorsement}} Because this project need acknowledgement of progression. [[User:Noé|Noé]] ([[User talk:Noé|talk]]) 21:26, 13 April 2016 (UTC)
# {{Endorsement}} high time to recognise this effort properly. extremly valuable to judge new groups editing. a license fitting to mediawiki would be key, e.g. GPL? --[[User:ThurnerRupert|ThurnerRupert]] 10:13, 14 April 2016 (UTC)
# {{Endorsement}} Akeron, I am Dan Andreescu (milimetric) of the WMF Analytics team.  I think what you're doing here is very valuable for the wiki movement, and I support your proposal.  We are in the process of overhauling the kind of data that's available to query, so that hopefully building projects like Wikiscan in the future will become much easier.  We do not have yet all the technical details, but we're calling the project Wikistats 2.0 and you can come to #wikimedia-analytics and talk to me (milimetric) or Joseph Allemandou (joal) about it anytime.  We would love your input on what would make a more useful infrastructure for you.  Also I am curious what kind of technology you're using, it seems like it would help us with our efforts. [[User:Milimetric|Milimetric]] ([[User talk:Milimetric|talk]]) 11:24, 27 May 2016 (UTC)
#:Hi, thank you for your support. The technology is basic : PHP/Mysql and some Memcached, this is working fine actually and will work with multi-wikis but in the future I would like to switch to a NoSql solution. Wikiscan needs are mainly low level data from revision and logging tables, something that would be useful is a single table (or some kind of data flow) which contains all this data (ideally with deleted edits too). Like the recentchanges table but with whole project history, other things interesting in this table are old_len/new_len to calculate diff and new page flag. There is also some needs for pageviews data, like retrieving all data for one project for 1 hour or 24 hours. [[User:Akeron|Akeron]] ([[User talk:Akeron|talk]]) 15:17, 27 May 2016 (UTC)
# {{Endorsement}} I'm suitably impressed by the coverage and depth of Wikiscan, as well as its mission to provide near-instantaneous stats. I also like the flexibility of the interface, where a user can easily zoom-in on one year, or month, or even date, where also most tables are sortable, even by a combination of criteria. Well done! I added a [https://stats.wikimedia.org/cgi-bin/search_portal.pl?search=Akeron section] on the project in Wikistats portal. It seems to me euro 6690 is a modest amount, given the scope of the project. Even more when the inevitable rise in complexity (coordination-wise) for a multi-lingual site with many more stakeholders is taken into account. Having said that, the quote is so modest that I wouldn't feel bad if only 70%-80% of the targets were reached. One comment on the scope of Wikiscan: the threshold of 100,000 edits seems rather high to me. For people who care about emerging communities a much lower threshold would be useful. I assume that once Wikiscan can handle 336 wikis, inclusion of even more languages will be a minor effort configuration-wise. [[User:Erik Zachte|Erik Zachte]] ([[User talk:Erik Zachte|talk]]) 08:55, 29 May 2016 (UTC)
#:Hi, thank you very much. Wikiscan was initially built and optimized for the size of French Wikipedia, supporting the 100k+ wikis (which represents 99.5% of total edits) is the first step, it shouldn't be hard to support all the small wikis with some dedicated optimization so the current server can handle it. [[User:Akeron|Akeron]] ([[User talk:Akeron|talk]]) 15:26, 30 May 2016 (UTC)