User:Guillom/UploadWizard: Difference between revisions
breakdown by year, alpha comment |
expand |
||
| Line 36: | Line 36: | ||
Let's see if we can count the number of files uploaded with UploadWizard over time. It would be nice to make a chart, and we might even be able to see spikes for large contribution campaigns like [[c:Commons:Wiki Loves Monuments|Wiki Loves Monuments]] if we break down by month. We'll have to combine counts for the 3 different upload comment patterns identified above. |
Let's see if we can count the number of files uploaded with UploadWizard over time. It would be nice to make a chart, and we might even be able to see spikes for large contribution campaigns like [[c:Commons:Wiki Loves Monuments|Wiki Loves Monuments]] if we break down by month. We'll have to combine counts for the 3 different upload comment patterns identified above. |
||
=== Breakdown by year === |
|||
Let's start with yearly counts. [[quarry:query/92164|query/92164]] provides the breakdown by year for the original upload comment until October 2020 (<code>User created page with UploadWizard</code>), [[quarry:query/92165|query/92165]] the breakdown for own works from 2020 (<code>Uploaded own work with UploadWizard</code>), and [[quarry:query/92166|query/92166]] the breakdown for own works from 2020 (regex <code>'^Uploaded a work by .+ from .+ with UploadWizard$'</code>). The results are tabulated below. |
Let's start with yearly counts. [[quarry:query/92164|query/92164]] provides the breakdown by year for the original upload comment until October 2020 (<code>User created page with UploadWizard</code>), [[quarry:query/92165|query/92165]] the breakdown for own works from 2020 (<code>Uploaded own work with UploadWizard</code>), and [[quarry:query/92166|query/92166]] the breakdown for own works from 2020 (regex <code>'^Uploaded a work by .+ from .+ with UploadWizard$'</code>). The results are tabulated below. |
||
| Line 130: | Line 131: | ||
|} |
|} |
||
The breakdown by year starts in 2012, which is concerning. UploadWizard was deployed to Commons in beta in 2010, and as the main upload interface in 2011. It's possible that the upload comment message wasn't added until 2012, which would mean we're missing over a year's worth of data. It's also curious that we have files with the old message in the years after 2020 and up to 2025. |
The breakdown by year starts in 2012, which is concerning. UploadWizard was deployed to Commons in beta in 2010, and as the main upload interface in 2011. It's possible that the upload comment message wasn't added until 2012, which would mean we're missing over a year's worth of data. It's also curious that we have files with the old message in the years after 2020 and up to 2025. This requires further investigation, and we still haven't counted the uploads with the third message pattern. But let's first take a look at the breakdown by month. |
||
=== Breakdown by month === |
|||
Let's break it down by month: |
Let's break it down by month: |
||
| Line 1,053: | Line 1,055: | ||
|288377 |
|288377 |
||
|} |
|} |
||
Charting these results shows nice, very visible spikes every September, which is the month of Wiki Loves Monuments! |
|||
| ⚫ | |||
== Uploads between 2010 and 2012 == |
|||
[[quarry:query/92174|query/92174]] identifies <code>comment_text = "User created page with UploadWizard (alpha)"</code> as <code>comment_id =</code>, and from there we can count the number of files uploaded with that comment. [[quarry:query/92177|query/92177]] yields a count of . |
|||
| ⚫ | Further research indicates that the message used from 2012 to 2020 was changed in [[gerrit:9714]]. Prior to August 2012, the message read <code>User created page with UploadWizard (alpha)</code> , which might have given us a fourth message pattern to find files uploaded with UploadWizard between 2010 and 2012. |
||
Unfortunately, the same code commit shows that the early "alpha" message wasn't recorded as a comment during the upload. This is confirmed by [[quarry:query/92174|query/92174]] , which yields no <code>comment_id</code> for <code>comment_text = "User created page with UploadWizard (alpha)"</code> |
|||
Checking some of my own UploadWizard uploads from that period ([[:File:Fortaleza do Pico Funchal Madeira - 076 - Feb 2008.jpg|December 2010]], [[:File:Jonah Hill and Channing Tatum - 21 Jump Street 027 - WonderCon 2012.jpg|March 2012]]), I can confirm that they have no log comment or initial edit summary. Their history confirms that I uploaded them with UploadWizard, since the category <code>Uploaded with UploadWizard</code> was [[c:Special:Permalink/222387780|removed by a bot in 2016]], when the Commons community decided to get rid of the category (the unfortunate decision that started me on this whole wikiarchaeology expedition in the first place). |
|||
The category removal might give us another way to identify early files uploaded with UploadWizard between 2010 and 2012: they will contain an entry in their edit history with the mention "Category:Uploaded with UploadWizard removed per community decision". We can count those files, making sure to exclude those we have already counted using the log comment. |
|||
Using [[:File:Fortaleza do Pico Funchal Madeira - 076 - Feb 2008.jpg|the image I uploaded in December 2010]] as a starting point, we can get the comment ID from the history entry where [[c:Special:Permalink/222387780|the bot removed the category]], which is faster than browsing the comment table to match the edit's summary text. [[Quarry:query/92177|query/92177]] gets the <code>rev_comment_id</code> for that history entry in the [[mw:Manual:revision_table|revision table]], and [[Quarry:query/92193|query/92193]] verifies that it corresponds to the <code>comment_text = Category:Uploaded with UploadWizard removed per <nowiki>[[Commons:Categories for discussion/2016/08/Category:Uploaded with UploadWizard|community decision]]</nowiki>"</code>. Now let's count them, minus the files uploaded once there was a log comment. |
|||
== Number of contributors == |
== Number of contributors == |
||
Revision as of 21:56, 27 March 2025
I'm trying to estimate the number of files uploaded using UploadWizard. To my knowledge there is no easy way to measure this, so this page describes my attempt to find a way.
Previous attempts
There used to be a hidden category, Category:Uploaded with UploadWizard, automatically added by UploadWizard during upload. This was an easy way to track the main metric (number of files uploaded) but the Commons community decided to delete it because it didn't serve a purpose, as "UploadWizard is the default mode of upload" and "This category is like having an 'Articles created using the Edit Button' category on Wikipedia." A comment in this discussion notes that as of August 2016, the category contained around 9 million files.
Another way was to query the Commons database (or rather a copy, using Quarry) and look for log actions for uploads with the content "User created page with UploadWizard", which was also automatically set by UploadWizard during upload (example). This approach was used by User:Steinsplitter on October 16, 2016 and yielded a count of 10,228,537.
Current problem
The method used by Steinsplitter is no longer reliable for several reasons:
log_commentis deprecated, andlog_comment_id(in theloggingtable) along withcomment_idandcomment_text(in thecommenttable) should be used instead.- Changes made by the Multimedia team to UploadWizard in 2020 resulted in two possible patterns of upload comments, multiplied by their translations:
Uploaded own work with UploadWizard(example)Uploaded a work by $1 from $2 with UploadWizard, where $1 is the copyright holder, and $2 the source, per the message documentation.
.jpg){kind=link}
All the language variations of the "own work" comment could reasonably be listed and queried. However, the customization of the "third party" comment with two variables makes identifying all files upload that way non-trivial. First, the comment table would need to be queried to get all comment_ids for all the comments using the pattern of the message (and all its translations), and then the logging table would need to be queried to count the number of uploads that match this (presumably large) number of comments.
Combined approach
If we ignore the 2020 changes for now, let's try to adapt Steinsplitter's method to use the logging and comment tables. query/41469 identifies comment_text = "User created page with UploadWizard" as comment_id = 44. Although it takes a while to count (11 minutes), as of March 23, 2025, query/42024 yields a count of 20,231,572. The same query run by User:Reedy on October 5, 2020 yielded a count of 20,231,414, confirming that the count has barely changed, due to the comment message changing in the software around that date.
If we now explore the new comment messages, query/91991 identifies comment_text = "Uploaded own work with UploadWizard" as comment_id = 144143204. As of March 23, 2025, query/42025 runs for ~10 minutes and yields a count of 14,118,636 files uploaded with that comment. This brings the total of files uploaded with UploadWizard to a minimum of 20,231,572 + 14,118,636 = 34,350,208, so around 30% of all 116,784,255 files on Commons.
Let's see if the comment translations are actually used. In my experience, Spanish-language contributors often prefer to set their interface to Spanish, so Spanish is a good language to test if the comment is actually ever translated. In query/91992 we look for the comment_id corresponding to comment_text = "Trabajo propio subido con el Asistente de subidas" (per translatewiki:MediaWiki:Mwe-upwiz-upload-comment-own-work/es). The query returns no result, meaning that the translated comment can't be found. query/91993 shows a similar result for the French translation. On further inspection, the original code commit for the message change confirms that "These messages are localized server-side in order to use
the wiki's language, not the user's language" (gerrit:625864)
If we don't have to take into account translated messages, this dramatically simplifies the count for own-work uploads, and brings counting third-party uploads back into the realm of the possible. Let's see if we could count third-party works.
Looking at the second post-2020 comment message, query/91995 attempts to get all comment_ids for comment_text matching the regex '^Uploaded a work by .+ from .+ with UploadWizard$', and yields 3,136,590 entries after running for 51 minutes.
Until we can count how many files were uploaded for each comment_id, we know that at least one file was uploaded for each comment. This brings the total of files uploaded with UploadWizard to a minimum of 34,350,208 + 3,136,590 = 37,486,798, so around 32% of all files on Commons.
Evolution over time
Let's see if we can count the number of files uploaded with UploadWizard over time. It would be nice to make a chart, and we might even be able to see spikes for large contribution campaigns like Wiki Loves Monuments if we break down by month. We'll have to combine counts for the 3 different upload comment patterns identified above.
Breakdown by year
Let's start with yearly counts. query/92164 provides the breakdown by year for the original upload comment until October 2020 (User created page with UploadWizard), query/92165 the breakdown for own works from 2020 (Uploaded own work with UploadWizard), and query/92166 the breakdown for own works from 2020 (regex '^Uploaded a work by .+ from .+ with UploadWizard$'). The results are tabulated below.
| upload_year | upload_count (User created page with UploadWizard)
|
upload_count (Uploaded own work with UploadWizard)
|
upload_count ('^Uploaded a work by .+ from .+ with UploadWizard$')
|
upload_count (total) |
|---|---|---|---|---|
| 2012 | 840273 | 840273 | ||
| 2013 | 2118570 | 2118570 | ||
| 2014 | 2507717 | 2507717 | ||
| 2015 | 2647913 | 2647913 | ||
| 2016 | 2608031 | 2608031 | ||
| 2017 | 2895540 | 2895540 | ||
| 2018 | 3024205 | 3024205 | ||
| 2019 | 3075870 | 3075870 | ||
| 2020 | 513263 | 2087611 | 2600874 | |
| 2021 | 111 | 2520956 | 2521067 | |
| 2022 | 25 | 2690304 | 2690329 | |
| 2023 | 50 | 2986794 | 2986844 | |
| 2024 | 1 | 3103117 | 3103118 | |
| 2025 | 4 | 779291 | 779295 |
The breakdown by year starts in 2012, which is concerning. UploadWizard was deployed to Commons in beta in 2010, and as the main upload interface in 2011. It's possible that the upload comment message wasn't added until 2012, which would mean we're missing over a year's worth of data. It's also curious that we have files with the old message in the years after 2020 and up to 2025. This requires further investigation, and we still haven't counted the uploads with the third message pattern. But let's first take a look at the breakdown by month.
Breakdown by month
Let's break it down by month:
| upload_month | upload_count (User created page with UploadWizard) (query/92168)
|
upload_count (Uploaded own work with UploadWizard)
|
upload_count ('^Uploaded a work by .+ from .+ with UploadWizard$')
|
upload_count (total) |
|---|---|---|---|---|
| 2012-08-31 | 31154 | 31154 | ||
| 2012-09-30 | 423680 | 423680 | ||
| 2012-10-31 | 148209 | 148209 | ||
| 2012-11-30 | 122685 | 122685 | ||
| 2012-12-31 | 114545 | 114545 | ||
| 2013-01-31 | 125211 | 125211 | ||
| 2013-02-28 | 120502 | 120502 | ||
| 2013-03-31 | 131330 | 131330 | ||
| 2013-04-30 | 145044 | 145044 | ||
| 2013-05-31 | 170437 | 170437 | ||
| 2013-06-30 | 152843 | 152843 | ||
| 2013-07-31 | 164438 | 164438 | ||
| 2013-08-31 | 173549 | 173549 | ||
| 2013-09-30 | 468730 | 468730 | ||
| 2013-10-31 | 166780 | 166780 | ||
| 2013-11-30 | 148827 | 148827 | ||
| 2013-12-31 | 150879 | 150879 | ||
| 2014-01-31 | 162282 | 162282 | ||
| 2014-02-28 | 156435 | 156435 | ||
| 2014-03-31 | 183458 | 183458 | ||
| 2014-04-30 | 177654 | 177654 | ||
| 2014-05-31 | 240287 | 240287 | ||
| 2014-06-30 | 179651 | 179651 | ||
| 2014-07-31 | 175011 | 175011 | ||
| 2014-08-31 | 196834 | 196834 | ||
| 2014-09-30 | 414755 | 414755 | ||
| 2014-10-31 | 270995 | 270995 | ||
| 2014-11-30 | 184402 | 184402 | ||
| 2014-12-31 | 165953 | 165953 | ||
| 2015-01-31 | 168781 | 168781 | ||
| 2015-02-28 | 174220 | 174220 | ||
| 2015-03-31 | 205279 | 205279 | ||
| 2015-04-30 | 189857 | 189857 | ||
| 2015-05-31 | 306529 | 306529 | ||
| 2015-06-30 | 218902 | 218902 | ||
| 2015-07-31 | 206162 | 206162 | ||
| 2015-08-31 | 218577 | 218577 | ||
| 2015-09-30 | 394290 | 394290 | ||
| 2015-10-31 | 220380 | 220380 | ||
| 2015-11-30 | 180760 | 180760 | ||
| 2015-12-31 | 164176 | 164176 | ||
| 2016-01-31 | 162122 | 162122 | ||
| 2016-02-29 | 154465 | 154465 | ||
| 2016-03-31 | 178055 | 178055 | ||
| 2016-04-30 | 192188 | 192188 | ||
| 2016-05-31 | 264177 | 264177 | ||
| 2016-06-30 | 231859 | 231859 | ||
| 2016-07-31 | 196906 | 196906 | ||
| 2016-08-31 | 219677 | 219677 | ||
| 2016-09-30 | 441282 | 441282 | ||
| 2016-10-31 | 210406 | 210406 | ||
| 2016-11-30 | 176574 | 176574 | ||
| 2016-12-31 | 180320 | 180320 | ||
| 2017-01-31 | 189767 | 189767 | ||
| 2017-02-28 | 175423 | 175423 | ||
| 2017-03-31 | 201818 | 201818 | ||
| 2017-04-30 | 219977 | 219977 | ||
| 2017-05-31 | 305834 | 305834 | ||
| 2017-06-30 | 261426 | 261426 | ||
| 2017-07-31 | 211190 | 211190 | ||
| 2017-08-31 | 239974 | 239974 | ||
| 2017-09-30 | 437621 | 437621 | ||
| 2017-10-31 | 234242 | 234242 | ||
| 2017-11-30 | 222166 | 222166 | ||
| 2017-12-31 | 196102 | 196102 | ||
| 2018-01-31 | 202578 | 202578 | ||
| 2018-02-28 | 211861 | 211861 | ||
| 2018-03-31 | 219672 | 219672 | ||
| 2018-04-30 | 227838 | 227838 | ||
| 2018-05-31 | 313746 | 313746 | ||
| 2018-06-30 | 244249 | 244249 | ||
| 2018-07-31 | 234726 | 234726 | ||
| 2018-08-31 | 238016 | 238016 | ||
| 2018-09-30 | 454189 | 454189 | ||
| 2018-10-31 | 266254 | 266254 | ||
| 2018-11-30 | 204673 | 204673 | ||
| 2018-12-31 | 206403 | 206403 | ||
| 2019-01-31 | 214757 | 214757 | ||
| 2019-02-28 | 236267 | 236267 | ||
| 2019-03-31 | 247818 | 247818 | ||
| 2019-04-30 | 225782 | 225782 | ||
| 2019-05-31 | 311503 | 311503 | ||
| 2019-06-30 | 252651 | 252651 | ||
| 2019-07-31 | 240456 | 240456 | ||
| 2019-08-31 | 236220 | 236220 | ||
| 2019-09-30 | 428250 | 428250 | ||
| 2019-10-31 | 236568 | 236568 | ||
| 2019-11-30 | 222327 | 222327 | ||
| 2019-12-31 | 223271 | 223271 | ||
| 2020-01-31 | 235188 | 235188 | ||
| 2020-02-29 | 247295 | 247295 | ||
| 2020-03-31 | 30716 | 149853 | 180569 | |
| 2020-04-30 | 1 | 175078 | 175079 | |
| 2020-05-31 | 34 | 207037 | 207071 | |
| 2020-06-30 | 8 | 201967 | 201975 | |
| 2020-07-31 | 221848 | 221848 | ||
| 2020-08-31 | 3 | 190197 | 190200 | |
| 2020-09-30 | 9 | 346647 | 346656 | |
| 2020-10-31 | 1 | 242662 | 242663 | |
| 2020-11-30 | 3 | 180792 | 180795 | |
| 2020-12-31 | 5 | 171530 | 171535 | |
| 2021-01-31 | 1 | 202500 | 202501 | |
| 2021-02-28 | 167393 | 167393 | ||
| 2021-03-31 | 1 | 204365 | 204366 | |
| 2021-04-30 | 215591 | 215591 | ||
| 2021-05-31 | 2 | 222428 | 222430 | |
| 2021-06-30 | 3 | 213424 | 213427 | |
| 2021-07-31 | 9 | 192787 | 192796 | |
| 2021-08-31 | 9 | 217760 | 217769 | |
| 2021-09-30 | 301713 | 301713 | ||
| 2021-10-31 | 220434 | 220434 | ||
| 2021-11-30 | 185093 | 185093 | ||
| 2021-12-31 | 86 | 177468 | 177554 | |
| 2022-01-31 | 199614 | 199614 | ||
| 2022-02-28 | 167138 | 167138 | ||
| 2022-03-31 | 211900 | 211900 | ||
| 2022-04-30 | 203873 | 203873 | ||
| 2022-05-31 | 237919 | 237919 | ||
| 2022-06-30 | 1 | 239714 | 239715 | |
| 2022-07-31 | 1 | 248807 | 248808 | |
| 2022-08-31 | 18 | 235854 | 235872 | |
| 2022-09-30 | 2 | 304853 | 304855 | |
| 2022-10-31 | 1 | 235687 | 235688 | |
| 2022-11-30 | 2 | 201640 | 201642 | |
| 2022-12-31 | 203305 | 203305 | ||
| 2023-01-31 | 8 | 197215 | 197223 | |
| 2023-02-28 | 3 | 201681 | 201684 | |
| 2023-03-31 | 21 | 223063 | 223084 | |
| 2023-04-30 | 234738 | 234738 | ||
| 2023-05-31 | 1 | 249348 | 249349 | |
| 2023-06-30 | 11 | 245198 | 245209 | |
| 2023-07-31 | 1 | 281165 | 281166 | |
| 2023-08-31 | 1 | 265888 | 265889 | |
| 2023-09-30 | 2 | 346333 | 346335 | |
| 2023-10-31 | 290008 | 290008 | ||
| 2023-11-30 | 227178 | 227178 | ||
| 2023-12-31 | 2 | 224979 | 224981 | |
| 2024-01-31 | 209808 | 209808 | ||
| 2024-02-29 | 216116 | 216116 | ||
| 2024-03-31 | 1 | 257500 | 257501 | |
| 2024-04-30 | 241950 | 241950 | ||
| 2024-05-31 | 251723 | 251723 | ||
| 2024-06-30 | 259635 | 259635 | ||
| 2024-07-31 | 301447 | 301447 | ||
| 2024-08-31 | 249734 | 249734 | ||
| 2024-09-30 | 358738 | 358738 | ||
| 2024-10-31 | 310661 | 310661 | ||
| 2024-11-30 | 222455 | 222455 | ||
| 2024-12-31 | 223350 | 223350 | ||
| 2025-01-31 | 225721 | 225721 | ||
| 2025-02-28 | 265324 | 265324 | ||
| 2025-03-31 | 4 | 288373 | 288377 |
Charting these results shows nice, very visible spikes every September, which is the month of Wiki Loves Monuments!
Uploads between 2010 and 2012
Further research indicates that the message used from 2012 to 2020 was changed in gerrit:9714. Prior to August 2012, the message read User created page with UploadWizard (alpha) , which might have given us a fourth message pattern to find files uploaded with UploadWizard between 2010 and 2012.
Unfortunately, the same code commit shows that the early "alpha" message wasn't recorded as a comment during the upload. This is confirmed by query/92174 , which yields no comment_id for comment_text = "User created page with UploadWizard (alpha)"
Checking some of my own UploadWizard uploads from that period (December 2010, March 2012), I can confirm that they have no log comment or initial edit summary. Their history confirms that I uploaded them with UploadWizard, since the category Uploaded with UploadWizard was removed by a bot in 2016, when the Commons community decided to get rid of the category (the unfortunate decision that started me on this whole wikiarchaeology expedition in the first place).
The category removal might give us another way to identify early files uploaded with UploadWizard between 2010 and 2012: they will contain an entry in their edit history with the mention "Category:Uploaded with UploadWizard removed per community decision". We can count those files, making sure to exclude those we have already counted using the log comment.
Using the image I uploaded in December 2010 as a starting point, we can get the comment ID from the history entry where the bot removed the category, which is faster than browsing the comment table to match the edit's summary text. query/92177 gets the rev_comment_id for that history entry in the revision table, and query/92193 verifies that it corresponds to the comment_text = Category:Uploaded with UploadWizard removed per [[Commons:Categories for discussion/2016/08/Category:Uploaded with UploadWizard|community decision]]". Now let's count them, minus the files uploaded once there was a log comment.