Wikisource talk:Former language domain requests

Language requests: Page names and archiving

May I suggest that this page be moved to "Language Domains Requests", and that the current "requests" page (which I started) be made an archive of it? (Some people have even continued to edit the old requests page during the new vote, which means that some do find it confusing.)

Furthermore, the old "language domains proposal" could be moved to an archive of the current vote, and the link removed from Sriptorium.

If no one has any objections the moves can be made (I won't be able to get back online for this for a while though, so someone else can do it if they want in the meantime). Dovi 09:02, 14 Apr 2005 (UTC)

sure, go ahead. don't forget to update all the links. ThomasV 09:07, 14 Apr 2005 (UTC)

Links between languages and projects

I don't understand. Wouldn't a link like wikisource:fr:French Text be sufficient? Ambush Commander 19:59, 10 May 2005 (UTC)

No, that sort of link which should work, doesn't actually work on at least some of the sister projects. w:fr may give you an "fr" title in English wikipedia; "en" or "fr" will give you English or French Wikipedias, with no way to link to the other projects except externally (with single square brackets). Dovi 20:43, 10 May 2005 (UTC)

Wait... do you mean link redundancy, as in w:en:Wikipedia (invalid) w:Wikipedia (valid, but may not always go to en?) I'm not sure I exactly understand, can you give some examples? Ambush Commander 21:03, 10 May 2005 (UTC)

Nope, not just redundancies. Check it out in some languages to see. Dovi 07:27, 11 May 2005 (UTC)

Moving pages to language domains

User:ThomasV has investigated this problem, and it would be great if he could summarize what he has learned so far here!

Since this is such a critical job, we may want to start a separate technical help page for it soon, but for now we might as well start right here.

Is there any possibility to make use of the language categories for this (i.e. a bot that might recognize them and transfer them to the correct domain)? Or a bot that might recognize certain language-specific characters in the title or contents of a page? Or might it be possible for a human to compose a list of page titles, and somehow "feed" it to a bot? Dovi 20:53, 10 May 2005 (UTC)

We already do have listings that are organized by language... the only problem we might have is preserving page history (which a bot cannot do). Ambush Commander 21:07, 10 May 2005 (UTC)

A proposal for moving pages

In order to keep the history of each page, we should not move pages to subdomains. It makes more sense to create a subdomain that contains a raw copy of the whole database, and then to delete what should does not belong to it. This is, at least for the french subdomain, how I want to proceed.

So, I suggest that we list all the subdomains for which this creation method is requested. In order to reduce the load of work, it is better not to create all subdomains simultaneously, but to wait until enough pages have been deleted from the main subdomain. Given that the majority of wikisource articles here are in English, I think we should start with this language. Here is how I propose to proceed: (ongoing sections don't necessarily have to be completed in order for the next step to continue, but are eventually going to be finished).

1. create en.wikisource.org as a copy of wikisource.org.
2. delete all articles in English from wikisource.org. This phase would last a few weeks.
3. (ongoing) delete all non-English articles from en.wikisource.org
4. create fr.wikisource.org as a copy of wikisource.org
5. delete all articles in French from wikisource.org. hopefully this phase will be shorter!
6. (ongoing) delete all non-French articles from fr.wikisource.org
7. create de.wikisource.org as a copy of wikisource.org
8. delete all articles in German from wikisource.org.
9. (ongoing) delete all non-German articles from de.wikisource.org

and so on...

Of course the order of creation might be different from what I suggest here. But it reduces the load to start with languages that have a lot of pages. I'd like to know who is interested in using this method. It implies that people are willing to wait a few extra weeks before their subdomain is created, and to take part in the deletion process (oh come on, we've been waiting for 8 months, what are a few weeks?)

ThomasV 22:48, 10 May 2005 (UTC)

I'm interested in this method. I really can't think of any other ways to go about moving pages to new subdomains while still preserving the page history. This method, while very tedious, would at least allow us to do that. Zhaladshar 23:46, 10 May 2005 (UTC)

I'm interested too. I will help deleting articles. --LadyInGrey 00:11, 11 May 2005 (UTC)

I do have a couple questions, though, about this. First, this will entail a LOT of deleting, won't it? I mean, if we copy ALL of Wikisource into en.wikisource.org, we will then have to delete all English articles from wikisource.org, but won't we also have to delete all non-English documents from en.wikisource.org? Since it IS the English version, all French, German, Spanish, etc. documents will have to go. Likewise, this will happen for every other subdomain as well, correct? Zhaladshar 00:22, 11 May 2005 (UTC)

The main problem with this method is it's not scalable: we'll need admins like Zhaladshar to go into overtime deleting articles like crazy, cuz we normal users can't delete articles. However, personally, I think this is the best way to go about seperating the domains without nuking the page history. I also added an extra step, which states that the subdomains have to be cleaned too. Ambush Commander 01:28, 11 May 2005 (UTC)

I think what Thomas writes makes a lot of sense, though it will involve a huge amount of work, work that will fall entirely on the shoulders of administrators. We should try to think of ways to lessen that load.

One suggestion: When we speak about deleting all English articles from wikisource.org (after a raw copy is made at en:), followed by French, etc. - this should probably apply only to the main namespace.

But the project namespace ("Wikisource:"), with its policy and technical help pages, should probably not be deleted, even though most of their content is English. Those pages belong to the entire project, both because of their historical value, and so that the still relevant ones can be copied and adapted/translated for each language domain. Dovi 05:47, 11 May 2005 (UTC)

Sure, the namespace should not be deleted. concerning scalability, I guess we can create a few more admins for this particular task. if some people have problems with this, we can even make the statement that those creations are temporary. Concerning the amount of work, there's about 20,000 articles now, and about 45% of them are in English. For statistics, check here : Wikisource:Scriptorium#distribution_of_languages_in_wikisource. Okay they are a bit old and unreliable, but I guess it's ok to say that we'll have to delete about 10,000 articles in English during phase 2. I guess I'm able to delete more than 100 pages in one hour. ThomasV 06:48, 11 May 2005 (UTC)

This definitely sounds like an interesting idea, and a good way to keep page histories. I am willing to give a hand in the deletion process, and I think that creating new admins to assist in this process is a very good idea (serious users only, of course, not totally at random). We should start, as Thomas V suggests, with the main languages (English, French, German, Chinese...) and delete all texts in those languages at the main site before copying the remaining contents to domains for smaller languages. I would definitely prefer to wait a month or two for a major part of the contents to be deleted before copying the remaining contents to the Danish subdomain, rather than having to delete some 19.000+ non-Danish pages at the subdomain :)

I also have a question: Can we start to add new content to the subdomains before the bulk of the main site is copied, or will that process delete all contents at the subdomain? --Christian S 13:21, 11 May 2005 (UTC)

In reply to Christian S's question - it seems to me the answer is definitely yes! There is no reason not to. Plus, not to do so would be to create even more extra work... :-)

This also brings up another issue: We need to decide on some sort of criterion for when language wikis should be set up. Any thoughts? Dovi 13:27, 11 May 2005 (UTC)

hum, I am not sure if I really understood the question, but it seems to me that the answer should be no, because I do not know if it is possible to merge two databases easily. anyway, this is a technical question, we should ask developers. concerning the schedule, I believe we should first request a creation of en.wikisource, that will be a copy of the current wikisource. then we'll see how long it takes to complete phase 2. phase 2 will last until some people decide that it is time to create their subdomain. ThomasV 13:37, 11 May 2005 (UTC)

You understood the question correctly, it was if it is possible to merge two databases. If they can't be merged, then a subdomain should't be created untill it is ready to recieve a copy of the current (reduced) wikisource database. I see two ways to do this:

1: Start with the 6 largest languages one at a time, that is English, then French, Romanian, Spanish, German and Polish. According to the stats at Wikisource:Scriptorium#distribution_of_languages_in_wikisource this adds up to about 86% of the texts. When the texts in these languages have been deleted we can create the rest of language domains by copying the remaining part of the database, then reduced to about 3.000 pages, a size that most languages should be able to clean up.

2: Start with the English domain, and then let the users who requested other language domains decide for themself when they feel that the database is sufficiently reduced to be copied to their domain.

--Christian S 13:59, 11 May 2005 (UTC)

This project is going to take quite some time, obviously. We are toying around with the idea of adding new Admins to Wikisource for this, which is a great idea. But do we really have enough serious users who would be willing to do that? Most users register, add something, and never visit the site again. I can name just a handful of serious users--and most of them are admins!

Another question I have is that of sysop status among subdomains. Over at Wikipedia, sysop status on one subdomain does not entail status on another. You have to apply on every subdomain to become an admin. I'm sure the same will happen here, too, once language domains are created. But this seems like it will drastically reduce the number of eligable admins for any language domain to go about cleaning up each additionally created one (like English and French, which are immense). How will we get past this? Will all 19 admins on Wikisource.org be given sysop status on new subdomains as well to help with the clean-up? Zhaladshar 14:39, 11 May 2005 (UTC)

You are all right. I didn't think of the problem that it may only be possible to "dump" the entire contents of a database to a new location, and not to add all pages to an extant site. The only people who can answer all these questions are the developers. Dovi 15:10, 11 May 2005 (UTC)

Nevermind, I think I answered my own questions. Sysop status will be given to those who are requesting the new subdomain. That way, they understand the language and they are serious users. Zhaladshar 15:24, 11 May 2005 (UTC)

I agree that those who request a new subdomain are likely candidates for sysop status at the subdomains. Getting the nessesary manpower to do the clean-up at the subdomain may be an issue, and maybe a database dump isn't the best solution for very small languages. The cleaning up is going to be quite a lot of work to save the history of, say, 10 or 20 pages in some small language, and it may be better (or at least a lot easier) for such very small languages just to sacrifice the histories of a small number of texts by using the transwiki method instead of a database dump. This will minimise the workload at least for the smallest languages, which will probably also have the lowest number of sysop candidates to do the job.--Christian S 15:52, 11 May 2005 (UTC)

It's by no means an ideal situation, but it might have to be done just out of necessity. Of course, for the smallest languages, a good number of articles (those in English, French, German, etc.) will already have been removed, so the process will be much lighter for smaller subdomains, and the three or four users who requested it will have a lot less work to do than those working on en.-/fr.-/de.wikisource.org. Zhaladshar 16:05, 11 May 2005 (UTC)

Yes, I agree that the transwiki method is only meant as an "emergency option", the general procedure suggested above (the database reduction and dumping) is definitely to be preferred if at all possible.--Christian S 16:33, 11 May 2005 (UTC)

Concerning the fact that only admins can do this, I bet we could greatly optimize this process by using a combination of bots and non-admin help. It could go something like this:

Bot Specs

• Must hook into an admin account (I know, we're all leery about giving bots admin perms, but wait...)
• Must be manually operated (that is, it speeds up the Admin in deleting pages, but isn't supposed to work by itself)

1. Harvest links from a special page that non-admins compiled of pages that need to be deleted
2. Present links, one by one, to admin. They mark it delete, or not-delete, and the bot takes appropriate action
3. Bot automatically removes processed links from page and adds them to it's own "ignore" cache
4. After ~80% of the pages are done, switch to "All pages" view and finish by scanning all of that.

The main idea is to reduce the amount of interpage jumping admins have to do, so when they're deleting, it may only take, say 2 seconds for them to issue a verdict and then go to the next page. We could easily get a 1000 pages done in one hour.

Even if no one gets around to coding the bot (I might take a look into doing that), we need a listing of things that should be deleted that's easy for admin's to traverse, and then remove links as pages are deleted or verified as 'keeps'. Perhaps we could get a database dump of all pages and put them into a wiki page (like Special:Allpages). With the bot, perhaps we could set up a web interface (kay, maybe it's not a bot anymore) that presents a list of pages, with little excerpts, and then you just check or uncheck them off. Does mediawiki let you do that? What do you think? Ambush Commander 19:34, 11 May 2005 (UTC)

I guess it's very dangerous to let a robot delete pages, but I must say that I too have been investigating the question of using a bot. My idea was to have the robot decide whether a page is in a given language, and then to perform deletion manually. but if you want to semi-automate deletions, one practical issue is that existing robots (at least those in pymediawiki) do not have a deletion capability. ThomasV 20:20, 11 May 2005 (UTC)

I could try coding one myself, I'd just need to do a bit of investigation with MediaWiki to see exactly how the HTTP interface works. Ambush Commander 20:47, 11 May 2005 (UTC)

First, I apologie for my poor english.

Second, I don't know exactly what bot can do and can't do. Perhaps a method to selecting page (by bot) according to langage is a combination of : specific langage characters (ç, é, è, à ê in French for example) AND users register current language. In fact, I think a lot of us are contributing in only one language (except in discussion page like here).

The steps can be :

1. copy of the entire database in each subdomain. The main database, wikisource, is freezed and can be used only as a backup copy, in case of bad operation during next steps. After copy, contributions can be done only on new subdomains.
2. in each subdomains, creation of specific page, one for each language.
3. each language has its own bot which can calculate, for each page, the probability this page has the "good language". calculation can be done as explain above.
4. admins must flag items they want to delete in its specific "language page" (for example french admin do that in the "french language page", and a bot can delete them
5. admins must also flag items which are considered as "correct language"
6. a croos comparison of admins'flags of each language can be done by bot to verify / validate admins'selections, and to re-compute for pages which aren't flag as "deleted" or "correct".
7. process ends when the "specific langage page" is empty.

I think this method has some advantages :

1. It minimises error or bad selection
2. Time to do this is not important. The process can be done a long time after the creation of subdomain without blocking new contributions.
3. Subdomain is not dependant of the others.

Perhaps a combination of this method with the ThomasV's method could be a good solution ?

Nota Bene : some pages must be in more than one subdomains. For example, the latin version of Bible can be used by all of subdomains. François Rey 23:53, 12 May 2005 (UTC)

Don't apologize for your English, it is not that poor.

In general I don't like the idea of a bot being able to delete pages. It may be quite a programming job to make a robot that can correctly distinguish specific languages from each other, the special character recognition is not nessesarily enough, as many special characters are used, more or less frequently, in more than one language. If users have to register the language of each text anyway, then it would be a lot easier to just make a robot that recognises those language tags.

You suggest that admins should flag texts in their own language and create pages with lists of texts in a given language at all the subdomains. This seems like a cumbersome way to do it, as it takes just as long, or even longer, to flag a page and update a language list as it does to an admin to delete it manually, and if this tagging process must be done at each subdomain, then we have about 20.000 pages multiplied by the number of language subdomains (=MANY pages) to tag.

I don't agree with you that your suggestion makes the time it takes unimportant. When a subdomain is created I would like it to get cleaned up as soon as possible, so to me the time it takes and minimizing the workload is important.

I very much prefer the procedure suggested by ThomasV and let the creation of subdomains for small languages wait until the major part of the texts, that is, those in the largest languages, have been deleted from the main site (I consider my own language to be one of the smaller ones).

We shouldn't freeze the main database either, as it would make it impossible to use this domain for discussions about and coordination of the implementation of the subdomains. Instead, as the subdomains are being created, it should be strongly advertised that texts in the languages of created subdomains should be uploaded there and not at the main site (is it possible to use the MediaWiki:Sitenotice for this?). --Christian S 06:50, 13 May 2005 (UTC)

yeah, we can use Sitenotice for this. perhaps I should restate what I said above more clearly: I do not like the idea of having a bot delete pages automatically. it's just too dangerous. I'm sure a delete-bot would never be allowed in wikipedia. (and semi-automatically sounds kind of dangerous too).

in my opinion the latin version of the Bible should go to a latin subdomain if it is created. if no such domain is created, it should stay on wikisource.org. if a page contains a bilingual version of a text (for example, this ancient greek translation), then it should go to the subdomain of the language it is translated in.

concerning the creation of en.wikisource.org, I asked TimStarling about it. He said we'll have to wait a few more days, because he's too busy now. ThomasV 07:34, 13 May 2005 (UTC)

I agree that manual deletion is to be preferred over deletion bots, and yes, such a bot is definitely a no-go in most wiki projects.

The Bible in Latin should go to a latin subdomain (if created). There is no need for it in other language domains, as it will be as easily accessible by interwiki links as it would be if it were on any other language domain.

Thanks for the update on the creation of en wikisource, I really look forward to starting the process.--Christian S 07:56, 13 May 2005 (UTC)

On Latin, I agree with Thomas and Christian. Latin texts belong on la.wikisource when that is set up, and until then they belong here, linked to from the current Latin Main Page. Special editions, as Thomas pointed out (bilingual translations, or versions of Latin text with notes in modern languages) belong on the website of that language. Dovi 14:24, 13 May 2005 (UTC)

Furthermore, we are likely to get lots more questions and comments like this, so we need a place for people to discuss them and summarize the conclusions. Plus, we never formally decided our criterion for setting up language wikis (though the general trend is pretty clear, and nice, on the language requests page).

Suggestion: Let's open up pages on Language Policy (for general discussions of what pages go where, and how the subdomains relate to the "main pages" here) plus Start a new language edition (on the technical criteria for setting up a new wiki). These should be listed as links at the top of "Language domain requests" and at "Scriptorium." Dovi 14:37, 13 May 2005 (UTC)

I'am sorry, I don't explain my idea clearly.

For more easily understanding, I will describe how to do for french language, and we could imagine the same for the others.

First, a full copy of wikisource database is done on the fr subdomain. After this moment, all french contributions must be done on this subdomain. I agree we must have a common part with other wikisource subdomain to continue this discussion or to exchange somme data.

After the full copy, a bot will automatically create N pages, N equals the number of differents languages plus a page named "undefined langage". Each page, for example the german page, contains wiki links of all supposed german articles. On the side of each link, a specific button can be explicitally pressed by the admin (the french admin) if he is considerating this article as effectively a german article. It can verify this with only the title of the article, if it is an evidence, or by reading the content of the article.

The button can be a "suppress" button if the action can be done rapidly without re-reading the entire page, or it can be a "flag" button which indicates the article we must delete.

Then admin can use a second bot which only delete articles which are flagged.

These two bots don't decide which article must be delete, they only help admin to, first, sort articles ; second to delete flagged articles

Collaborative actions could be done between all wikisource subprojects, french admin can verify french articles list, etc ... In this case, the button will be a "confirm" button rather than a "delete" button.François Rey 11:19, 13 May 2005 (UTC)

the method I proposed does not require to program new tools, so it can be started immediately. I believe it is also more efficient, although I am not quite sure if I really understood what you propose. it seems to me that your method would take O(kN) deletions, where N is the number of articles and k the number of subdomains (not to mention the time spent programming bots). The method I proposed takes less than that. ThomasV 15:22, 13 May 2005 (UTC)

I understand the method I proposed will put a high load of work on admins. In order to nominate more admins, I requested bureaucrat rights on wikisource.org. ThomasV 16:22, 13 May 2005 (UTC)

I must say that I agree with ThomasV on this, ThomasV's method is both faster and simpler, and by using his method we can also avoid the controversial deletion bots. Also, I don't see the need for all this sorting into languages. Lets take your example with a full copy at the fr domain. Then, at the fr domain, it would only be nessesary to sort texts into two categories, "French" and "not French". Why confirm that a German text is in German, when it has to be deleted anyway as it is not French? And why not just delete the page at once, when it is being evaluated whether it should be kept or not? As an admin I find that just as easy. I support that we use the plan of ThomasV! That way the workload will be much less, especially for the smaller languages which are also the largest in number, than it would be if a full copy is made to every language subdomain, even with the aid of deletion bots.

I support that ThomasV gets bureaucrat rights. Have you made the request at meta:Requests for permissions? You probably should, as I'm not sure Ec will be around to grand you that status. --Christian S 16:43, 13 May 2005 (UTC)

Agreed. We need a bureaucrat around here. If deletion bots are not that preferable, then we don't have to use them (might take too long to program them anyway). So far, we've proposed:

• Making a full copy of the database to language subdomains and then deleting from them in order to preserve page history
• Make a global notification about the ongoing work? (personally, I think this is a must)
• Creation of new pages to coordinate the language task (such as lists of English works, French works, etc.) Ambush Commander 21:27, 13 May 2005 (UTC)

I also prefer ThomasV's method. It should be kept simple. There are nearly 32,000 pages on Wikisource. With this method there are perhaps 60,000, 80,000 or so pages to be deleted. So you need admins and you need to organise it. So I suggest to announce the start of language domains and to invite all admins at all Wikimedia projects to help deleting the pages. I think most of them can be trusted and it's easier and faster to give them temporarly admin status than to elect new admins for Wikisource. Then there should be a plan who deletes which pages. You could create a list with entries like "Deleting all non-English pages starting with an A at en.wikisource" and a person who want to do this task can add his name. So you would make sure that there are not 10 people searching for non-English articles starting with "A" and the articles with "Z" are forgotten. If you find 50 admins, each working 2 hours a day (about 250 deletions, depending very much on the servers), the work could be done in less than a week. --Jofi 21:44, 13 May 2005 (UTC)

I would prefer to promote regular users on Wikisource to permanent admin status than give some Wikipedia admins only temporary status. Besides, with the new language domains, each language is going to need a few sysops to do maintenance on those pages, so why not just give the users who are requesting a subdomain adminstrator status? That way, they'll probably be more inclined to stay with the project (which the Wikipedians will probably not be) and will kill two birds with one stone. Zhaladshar 21:49, 13 May 2005 (UTC)

Normally there are only very rare situation in which an admin has to delete a page. So you don't need many admins. But this is a different situation. If you wait until each sub-Wikisource has elected it's admins and they have time and are willing to do the deletion jobs, than it will take many months until the work is done. So why not using "special" admins for a special situation? It's not important if they stay, it's important first to start the projects where they maybe stay in the end. --Jofi 22:22, 13 May 2005 (UTC)

Hmm... we could do the elections while the cleanup is proceeding on the main domain! Ambush Commander 00:36, 14 May 2005 (UTC)

Ok, I suppose I should speak in English, so excuse me if mine' is REALLY horrible. I was reading a lot about all the stuff, and I started speaking on it.wiki and on the italian irc channel, and now i have an opinion. First, a question:

• Is it possible, for developers, to move pages from wikisource.org to xx.wikisource preserving histories?

If so, should not be faster and smarter to begin moving articles in a language that is less used? I don't know if my opinion is intellegible, so i make examples:

• Let's suggest we have language xx. On wikisource there are a dozen articles written in xx. Developers should only create xx.wikisource and move a dozen articles. Then xx.wikisource should be able to start working

This is a way specular from ThomasV. Why should we make in this way? Simple: moving and deleting work is similar, BUT:

• In the same time, we'll have more local wikisources operative. With ThomasV's way, we'll have en.wikisource in 2 weeks (i hope), with mine we'll have xx.wikisource, yy.wikisource, zz.wikisource
• Minor wikisources should not be forced to wait a lot of time, until majors end with their work.

Ooooookay. I don't know if I was clear, so... in case, just ask. --Gatto Nero 16:13, 14 May 2005 (UTC)

Well - it sounds reasonable, but there is one MAJOR problem in what you suggest: There is a limited number of developers, and they are already overloaded with development tasks. Moving pages from one domain to another will probably be of very low priority to the developers. Deleting as ThomasV suggests can be done by ordinary admins, and new admins to assist in the process can be created relatively easy, while developer access is harder to get (with developer access you can really mess things up if you are not sure what you are doing). I don't think anybody would be granted developer status just to move articles. Not that your suggestion is bad, I just don't think it is workable, as minor (or major) wikisources probably will have to wait a long time for developers to move the relevant pages due to low priority compared to development tasks and bug fixes.--Christian S 16:50, 14 May 2005 (UTC)

Uh, you're right too. Guess should we ask to developers before, maybe. I think one or two developers should be enough to make the work, leaving at the end (to admins) all the "deletion" work. It depends on them, so... --Gatto Nero 17:00, 14 May 2005 (UTC)

If it is tecnically possible (whether it is or not, I have no idea) you may be able to persuade a developer to move a small number of pages to a small language domain. But don't expect them to be willing to move hundreds of pages. And there is no guarantee that subdomains for the smallest languages will be requested in any near future. Unless subdomains for one or more of the smallest languages are requested it is probably not relevant to ask the developers.--Christian S 18:56, 14 May 2005 (UTC)

List of admins who want to take part in the deletion work

To use ThomasV's method there are admins of this (the main) Wikisource needed who want to do the deletions here. So if you want to, please make a note here. --Jofi 22:56, 13 May 2005 (UTC)

• Ambivalenthysteria ?
• ArnoLagrange ?
• Brion VIBBER ? (inactive since Dec 2004)
• Caton ? (= Marc)
• Christian S ?
• Dr Absentius ? (inactive since Apr 2004)
• Eclecticology ?
• Kalki ? (inactive since Dec 2004)
• LadyInGrey ?
• Marc ?
• Maveric149 ?
• Menchi ? (inactive since Sep 2004)
• Mxn ?
• Samuel ? (inactive since May 2004)
• Shin-改 ?
• Shizhao ?
• ThomasV ?
• Yann ?
• Zhaladshar ?

I am not sure if this list is useful; many of these admins are inactive. in addition, I believe we will create new admins for this deletion work. ThomasV 08:56, 14 May 2005 (UTC)

The list is useful if there won't be additional admins. Folowing your suggestion, for example the Italians have to wait until the English, French, German, ... articles are deleted before they can start their Wikisource. If there are too few admins doing the work and the deletions take a long time, the Italians (or others) may decide not to wait until the deletions are done, but to start with a complete copy of Wikisource at once. --Jofi 09:32, 14 May 2005 (UTC)

If, say, the italians or others don't want to wait, and the community requesting the language subdomain agree on it, then i see no problem in letting them start with a full copy. The suggested method will decrease the workload at the subdomains, but if the individual language community prefer the greater workload to waiting, then why not? I think waiting for some of the major languages to be deleted will be beneficial to many language communities, and that most of them can see the point in waiting. Meanwhile they could allways help deleting pages here to speed up the process and hence reduce the waiting time - what they delete here they won't have to delete at their own domain.

About the list of admins, I think it would be better to start two new lists: one for current admins who want to help and one for non-admin users who want to become admins in order to help.

--Christian S 11:24, 14 May 2005 (UTC)

Certainly it's better to wait until the deletions are made, than to start with a full copy. So there have to be many people able to delete pages to decrease the waiting time. --Jofi 11:44, 14 May 2005 (UTC)

List of non-admin users who want to take part in the deletion work

Enter your name here if you want to help deleting the pages. If you aren't known at Wikisource add a short description. --Jofi 11:44, 14 May 2005 (UTC)

• Jofi (admin at de.wikipedia)
• François Rey (at wikisource since April 9, 2005 ; at wikipedia since november 2004 ; at wikisource project, I am working on french bible (improvement of presentation, cutting up texts in chapters) and in adding Les Misérables. Probably I could not be active at the end of june and the beginning of july, cause a moving out.
• Ambush Commander (wikisource since May 8th, Wikipedia since August 2004 with 500+ edits) Currently, on wikisource, I'm working on The Economic Consequences of the Peace, the tables are really killing me (I so need to use regexps), and I know it's been quite a short time since I've joined wikisource, so I don't mind if I don't get a chance to help.

What about google?

It is annoying to search for a term, find a link, click on it and find an empty page. It would be nice to preserve the links, at least for a few months until the search engines update their links. So, http://wikisource/wiki/stuff should have a http redirect to http://fr.wikisource.org/wiki/stuff. Bogdan 14:55, 14 May 2005 (UTC)

First of all, Great Idea! Second of all, Requires MediaWiki Changes. Third of all, Requires more work for Admins. Fourth of all, can we compromise? Instead, put a global notice on all empty pages that if what you're looking for here is missing, *it might not*, and then give a list of links to all the new subdomains and their respective entries (telling them to go to the language of this text), and maybe they'll find it. Ambush Commander 15:54, 14 May 2005 (UTC)

Yes, you only need to edit MediaWiki:Nogomatch and give the users links to articles with the same name in the subdomains. They will know what language they want and it's nearly no work for the admins. But the articles shouldn't be renamed in the subdomains. --Jofi 17:36, 14 May 2005 (UTC)

However, setting up Search Engine friendly HTTP redirects is going to be difficult. Let's just let google reindex the whole thing. Ambush Commander 19:46, 14 May 2005 (UTC)

Creating a list

For the languages with a smaller number of articles, it would be easier to move the pages by creating a list with all the articles in that language, then:

1. copy the main database to the new subdomain database
2. a bot in the language's wikisource deletes all the articles, but the ones in the list;
3. a bot in the main wikisource deletes only the articles in the list.

For the Romanian language, for example, I could easily create one, as most of pages were contributed by me and I kept a pretty consistent standard.

Should I start making the list ? :-) Bogdan 19:07, 14 May 2005 (UTC)

hum... I am not in favor of using a delete-bot, at least not in the main wikisource. it's just too dangerous.

for a subdomain, I guess a delete-bot could be used; however it has yet to be written, tested, debugged, etc...

concerning Romanian, I thought it was one of the main languages in wikisource. but I do not speak Romanian, so I am not really able to tell if a page is in Romanian, or some other language that ressembles it. this means that the statistics I made are probably false, at least wrt romanian (see those stats in the Scriptorium). since you know that lanuage, maybe you could help fix those stats. ThomasV 19:21, 14 May 2005 (UTC)

Going through all articles to create a list of articles in a specific language should be almost the same work as going through all articles to delete articles in a specific language. I think the slowest part of the work won't be human beings but the servers and I prefer solutions without deletion bots. --Jofi 21:02, 14 May 2005 (UTC)

No, it's not the same work. Take this author Autor:Ion Luca Caragiale for example. To making a list of the 350 pages from that page you need only one click and a copy and paste. To delete those 350 pages manually, you need 350 x 3 clicks = 1050 clicks. Bogdan 21:49, 14 May 2005 (UTC)

It's less work if there is such a list. But many texts are not listed in lists or the lists are not divided into languages. And what you mentioned is only the list of one author, not the list of pages in one language. But if you want, you can create such a bot. If it's really safe and a copy of the database is made, one could try it out. --Jofi 22:44, 14 May 2005 (UTC)

::aside - Dang, that's a lot of pages we're going to be deleted. We haven't put it in perspective have we? - endaside:: Does MediaWiki have "mass deletion" tools? Ambush Commander 22:29, 14 May 2005 (UTC) Nope, I took a look at my own installation of Mediawiki and it didn't seem to have anything like that.

A "delete instantly without question if i'm sure" button would be very useful. But if there are enough people helping it should be possible without extra tools. --Jofi 22:49, 14 May 2005 (UTC)

::laments:: The whole problem is that Wikimedia wasn't built for transwiki tranfers. Meh. Ambush Commander 23:19, 14 May 2005 (UTC)

I swung over to bugzilla and found this: bugzilla:606, they've set it for the 1.5 release. I don't know how long that's going to take though. Ambush Commander 02:53, 15 May 2005 (UTC)

I don't know if it would work well for so many pages, if the revision history shall be kept, because it uses the xml-export without compression or anything else. Any minor edit in a 100kb page results in another 100kb to extract, move and import. --Jofi 09:39, 15 May 2005 (UTC)

To Wait? 1.5 MediaWiki promises Full History exports/imports of Articles

I sort of started a discussion in the list article, but I think it deserves it's own topic. Bugzilla:606 has the bug report for Special:Import, an automated transwiki system. It is set for the 1.5 MediaWiki release.

If we wait for this system to be implemented, we may be able to stop our admins from having to delete loads of articles: instead, we will have them man the Transwiki tools and start moving pages.

Jofi brought up one point that the Special:Import system may not be well adapted for mass moves of pages, as marking it up in XML will inflate the page size. However, we must also contrast this with the amount of deleting the other method will take. We are, effectively, deleting the entire Wiki for just ONE copied database: a percentage consisting of the English entries from the main, and the rest from the English copy. Should we wait? Ambush Commander 00:18, 16 May 2005 (UTC)

Does anybody know when the 1.5 MediaWiki is expected to be released? If it is to be released soon, then it can be an interesting option to wait for it, but I'm not sure we want to wait many months for the release. --Christian S 10:51, 16 May 2005 (UTC)

I do not know, but this feature is probably already in cvs, so we do not need to wait the release in order to test it. it would be great if somebody could test it. ThomasV 11:30, 16 May 2005 (UTC)

As far as I can see, only the export feature is up and running, but it is still not possible to import pages into a wiki. So, we can export pages with histories to a simple XML-file, but we have to wait for 1.5 in order to import the XML-files into the new language-wikis. Pity, it would be a really great feature to get up and running right now! --Christian S 12:10, 16 May 2005 (UTC)

It's here. It needs WikiError.php (will come in 1.5 but is not important for SpecialImport) and doesn't seem to work yet. It simply uses Special:Export from the source and then (should) import the XML file. But whe haven't to wait until 1.5. There are no dependencies. Special:Import just has to be completed. --Jofi 12:18, 16 May 2005 (UTC)

I have to correct myself: There ARE dependencies. It can't be used until 1.5 is ready. --Jofi 12:57, 16 May 2005 (UTC)

I guess if this feature is available soon, we should wait for it. however, I believe there will be no big difference between current cvs and 1.5. if it dos not work cvs yet, I do not believe it will be in 1.5. So what should I do? I already asked Tim to create en.wikisource.org (which he seems to have forgotten); maybe we should ask on meta what the best strategy is. ThomasV 12:28, 16 May 2005 (UTC)

Brion Vibber has called it a 1.5 release blocker. So it should be in 1.5.. Then we have to make lists of articles in a specific language, import them (only admins of the subdomain), mark them as deleteable in main wikisource and delete them (only admins of main). --Jofi 12:36, 16 May 2005 (UTC)

Are you sure that this feature would really save us some work? what you describe is about 4 operations per article. I do not know exactly how many deletions my method will involve, but if we make the assumption that languages have a geometric distribution (one language amounts 50% of articles, the next language 25%, the next one 12.5%, and so on...) then it is possible to show that my method involves 2 deletions per article. the actual distribution of languages is different from this simplified case, but not very different. ThomasV 12:44, 16 May 2005 (UTC)

I'm not sure what's better. Your suggestion: Go through "Allpages", click on article to delete, click on delete, select "I'm sure", click on delete. Do this about 60,000 times. Using SpecialImport: Go through "Allpages", add each article to a language list (copy&paste), add those lists partially to SpecialImport, make sure the articles are really imported, go to the articles at main wikisource, mark them as deletable, delete them. Do this about 30,000 times. --Jofi 13:06, 16 May 2005 (UTC)

MediaWiki 1.5 is due to go live around June. I've just asked Brion Vibber about this and he says Special:Import will definitely be part of 1.5, and that this will be the best way of splitting of the various language versions from here. Angela 13:14, 16 May 2005 (UTC)

Jofi: from what you describe, I believe my method is faster. Note that yours also involves a deletion. Another point is that my method can be performed by many users in parallel, it does not require coordination; otoh if people start building lists, they are likely to have conflicts.

Angela: did Brion say this specifically for wikisource?

ThomasV 13:36, 16 May 2005 (UTC)

Yes, he said this today specifically about Wikisource. Also, there is a deletion bot program available. See w:User:AngBot for example. Angela 14:40, 16 May 2005 (UTC)

After speaking to Tim Starling about Special:Import, I strongly suggest this method is used. The operation would need to be done once in total, not once per article. You just feed the list of articles to be moved to the French Wikisource into it, and it moves them all at once in one simple operation. Then you feed that same list to the deletion bot and it removes them from the original Wikisource. All that is needed is a list of which articles need to move where. Angela 14:49, 16 May 2005 (UTC)

Has the bot been tested? I don't see many "user" contributions. If it's the best method, we should start creating lists with articles by language. --Jofi 15:10, 16 May 2005 (UTC)

There is nothing in its contribution list because deletions are not recorded in a user's contributions (this might change in 1.5). It was run by a number of people to clear up page creation vandalism on the Chinese Wikipedia last year. It worked fine at the time, but I will check with the people maintaining the Pywikipedia framework that it will still work under MediaWiki 1.5. Angela 15:34, 16 May 2005 (UTC)

Following the irc discussion with Tim and Angela, I believe the Special:Import method has drawbacks: the operation of creating the list will take a lot of work (about as much as deleting pages, and maybe more if coordination between users is required). However, if we could have a robot categorize the pages, it would be much easier. There are programs around, that can tell if a text is written in French, with pretty good accuracy. I previously did not want to use such a program (see above) because I was considering to use it for the deletion process, where a mistake would be dramatic. However, I believe it makes sense to use such a bot with the Special:Import feature. I'm willing to give it a try. ThomasV 15:22, 16 May 2005 (UTC)

Here is what needs to happen if the Import method is used:

1. Make a list of which pages belong to which language:

   There are two ways to do this.

   1. Add a category tag to every page stating which language it is in, then derive the list for Special:Import automatically from that. No admins are needed for this. This could be done with a bot (or many bots).
   2. Create a manual list like Wikisource:List of French articles and add the relevant articles there. No admins are needed for this.

2. When the list is finished, one person pastes the list in Special:Export, saves the result, then pastes the same list into Special:Import. An admin needs to do the Import. This is one operation per language, not per article.
3. Run a deletion bot to remove the articles that were just imported. The deletion bot already exists and can be run by multiple people if necessary.

Angela 15:34, 16 May 2005 (UTC)