Experiments: Difference between revisions
new page |
updated page |
||
| Line 10: | Line 10: | ||
== Principles == |
== Principles == |
||
=== Customers v. colleagues === |
=== Customers v. colleagues === |
||
Anyone working with Wikimedia editors has to treat them as colleagues, not as customers. Experiments are often an attempt to optimize human behavior and workflow. There's nothing inherently wrong with such goals, but Wikimedians are not customers in the same way that users of Facebook are customers of its site. They instead should be viewed as colleagues. Would you go into the office of someone you work with and start fucking with them to optimize their behavior? Surely not. But this is exactly the type of behavior the Wikimedia Foundation is now engaging in. Disrupting the work of long-time editors in the name of questionable experimentation. |
|||
Anyone working Wikimedia editrs |
|||
== Implementation == |
== Implementation == |
||
| Line 68: | Line 68: | ||
> As I said, new editors are generally ignorant of past behavior. |
> As I said, new editors are generally ignorant of past behavior. |
||
> So you're not disrupting their workflow. |
> So you're not disrupting their workflow. |
||
> You're trying to optimize behavior in a lot of ways. |
|||
> Which is fine. |
|||
> But only when looking at the users as cusotmers. |
|||
> customers |
|||
> Really you have to look at them as colleagues. |
|||
> Would you go into someone's office and start fucking with them to optimize their behavior? |
|||
> Someone you work with? |
|||
> I think this is a crucial point. |
|||
> Yeah, that's a fair point. I have obscene amounts of respect and admiration for active contributors, but I can see how what I wrote didn't reflect that |
|||
> Well, heh. we haven't really discussed what you wrote and the stupidity of it. |
|||
> Just in terms of it coming back to bite you in the ass, if nothing else. |
|||
> Brooke: it did? |
|||
> In the future, I mean. |
|||
> oh. yeah, maybe. |
|||
> It looks like you joined Wikipedia in 2005, but you weren't around during the Usability team, were you? |
> It looks like you joined Wikipedia in 2005, but you weren't around during the Usability team, were you? |
||
> They had a similar incident with one of Trevor Parscal's reverts. |
> They had a similar incident with one of Trevor Parscal's reverts. |
||
Revision as of 19:09, 19 August 2012
 |
Just brainstorming. More on the talk page. |
This is a guiding document on the use of experiments on Wikimedia wikis.
Background
Experiments on Wikimedia wikis, generally conducted by the Wikimedia Foundation, have become increasingly common in recent years. In 2009, the Usability team conducted a number of experiments. In 2012, there is now a dedicated experiments team (E3).
These experiments have varying goals and motivations, but we've reached a point where some clearer guidelines need to be established about what is and is not appropriate.
Principles
Customers v. colleagues
Anyone working with Wikimedia editors has to treat them as colleagues, not as customers. Experiments are often an attempt to optimize human behavior and workflow. There's nothing inherently wrong with such goals, but Wikimedians are not customers in the same way that users of Facebook are customers of its site. They instead should be viewed as colleagues. Would you go into the office of someone you work with and start fucking with them to optimize their behavior? Surely not. But this is exactly the type of behavior the Wikimedia Foundation is now engaging in. Disrupting the work of long-time editors in the name of questionable experimentation.
Implementation
Mitigation
Experimentation often comes with high costs and high risks. Adding extra weight by including people in a test group who ought not be tested on adds considerable cost and risk without any benefit.
Smarter code should ensure that only editors who meet specified criteria load extra code (JavaScript). A number of factors can be taken into account when determining whether to load extra JavaScript for a particular user, including:
- user's logged-in status;
- user's edit count;
- user's registration date; and
- whether the user is using the default skin.
Looking at and using smarter metrics is important. If a user is using a non-default skin, it's fairly safe to assume that they probably don't want to be fucked with. Experiments on these users should undergo the most scrutiny and require the most consideration.
Opt-out
Any and all experiments should have an opt-out feature. However, an opt-out feature is not a license to be more obnoxious simply because people can opt out of your experiment.
See also