User talk:WikiInquirer

Requesting to place online survey link

Original question posted at the Village Pump (assistance):
Hi, I wish to make a request to the administrators here. I am doing some research on Wikipedia and I wish to survey fellow Wikipedians on what motivates them to contribute their time, effort and knowledge to this great resource. I have prepared an online survey form hosted on my school server and I wish to contact Wikipedians to help me fill out this survey form, by email or by posting the link on the user's discussion page. Should the user not reply or delete my post , I would not pester them. Is this acceptable behaviour on WP? I don't wish to unwittingly flout the rules here. And also, any data collected would be kept private and confidential. I would only be asking questions that are related to my research and probably the most sensitive questions I would ask for are the Wikipedian's username and simple demographics (no income and such). I would require the Wikipedian's username because I am going to engage in a lucky draw for gift certificates as a reward for respondents who complete my survey. Is this OK? --WikiInquirer 15:21, 27 January 2007 (UTC)talk to me

You could post the link here and see who bites; that's what's usually done and ensures more of a random sample than putting it on people's pages. ~ ONUnicorn^{(Talk|Contribs)}problem solving 18:02, 30 January 2007 (UTC)

Or post at Wikipedia:Village pump (miscellaneous). But, to answer your question, as long as you don't post to individual user pages requesting partipation (that's considered internal spam), you're pretty much okay. Other places you might want to post to would be the talk pages of Wikipedia:Who writes Wikipedia, Wikipedia:Meetup, and Wikipedia:Wikipedians. Try to keep the posting down to four sentences or less - interested editors can follow a link to your site to read about privacy, for example, or any specifics on the type of questions; the shorter your posting, the less likely someone will object to it. -- John Broughton (☎☎) 21:59, 1 February 2007 (UTC)

But I have drawn up a sample of Wikipedians to be surveyed (to satisfy some research criteria) and the only options open to me would be either to contact these people via email or post to the user's talk page. Can I post on the user's talk page and limit my request to just 4 lines like you said? I would KISS. In addition, I would state clearly that if the user deletes my post, then it is understood as a sign of objection and I would not pester them again. I would also send the soliciting message block to you for approval before circulation. --WikiInquirer 06:49, 3 February 2007 (UTC) talk to me

I'm just an editor here, like you, so I'm in no position to approve anything. And, in fact, there probably isn't anyone that can approve this (yes, frustrating); the good news is that there isn't anyone who has to approve it. But I can offer advice, for which I need one key piece of information: How large is your sample? -- John Broughton (☎☎) 06:55, 3 February 2007 (UTC)

Okay, 200 to 300. That's way more than I hoped you'd say. The good news is that the policies I checked, regarding spam, canvassing, and multi-posting, don't actually address this. Which doesn't make it okay - this is an encyclopedia project, and there certainly is some policy out there that can be read as "no surveys". But it does mean that if you do this very courteously, you may be okay.

First, I strongly suggest that you post only the briefest of messages on user talk pages - the four line limit was really for project talk pages. Something perhaps like this (in its entirety):

If you'd be interested in participating in a small survey, please see User:WikiInquirer/February 2007 survey. Thanks! (signature)

Next, on that subpage, you need to carefully explain how you selected your sample, and why you'd really appreciate as high as possible a participation rate, and what the editor will get from it (e.g., you'll send them a link to the on-line report once you've finished it). The subpage also needs to explain a bit about the project, and give the editor a sense of how much time is being requested (e.g., number of questions). And it needs to say that there will be no further followup on the initial posting on your talk page. (Whether an editor leaves your posting up or deletes it, if he/she fails to act, do not, repeat, do not ever post again on that page.)

Finally, you need to very carefully think through how many questions you're going to ask each editor, and how much personal information you ask. The more the questions, and the more the personal information, the more likely that someone is going to complain (and/or just quit in mid-survey).

If you'd like help on the subpage or on the survey questions, I'd be happy to offer some advice. Again, whatever you do, you need to totally respect the fact that you're asking editors for a bit of their time, and absolutely to minimize how much of that you're asking for. -- John Broughton (☎☎) 07:40, 3 February 2007 (UTC)

Missing page hit count in the database dump?

Original question posted at the Village Pump (technical).
Hi, I noticed that the page hit count in table page is missing. Is this normal or a faulty dump? Specifically, I'm looking at stub-meta-current.xml.gz from the dump on 20061130. Alternatively, would stub-meta-history.xml.gz have the page hit count instead? Thank you! --WikiInquirer 11:58, 16 January 2007 (UTC) ^talk

The hit counters were turned off for performance reasons, so I wouldn't expect their data to be in the database dumps. --ais523 12:12, 16 January 2007 (UTC)

The official announcement is in the Very Frequently Asked Questions. --WikiInquirer 11:29, 17 January 2007 (UTC)

Wikicharts offers a machine-readable version of its data, which will give you approximate hit counts for the most viewed pages recently. Tra (Talk) 16:07, 16 January 2007 (UTC)

I checked Wikicharts but it can show the hit counts for the top 1000 (or so, but less than 1100) most viewed pages. Are there any other internal or external tools out there that has captured the hit count of each page in the English Wikipedia? --WikiInquirer 01:23, 18 January 2007 (UTC)^TALK
And I suppose the Hitcounter table is also empty in enwiki? Sigh --WikiInquirer 01:33, 18 January 2007 (UTC)^{talk to me}

Pagecounts are certainly disabled on enwp - we'd bring the servers to their knees in a snap otherwise. Wikicharts is the only statistical data I am aware of, unfortunately - and it's really only a rough sampling, since it takes one in every few hundred visits. Sorry there isn't more - we'd like it too... Shimgray | talk | 01:37, 18 January 2007 (UTC)

Thank you, Tra and Shimray, for your replies. It's understandable and hey looking at the silver lining, this is a good sign -- Wikipedia is growing and growing! --WikiInquirer 05:05, 18 January 2007 (UTC)^{talk to me}

When would the next database dump be out?

Hi, may I know when the next enwiki db dump would be released? As for previous dumps, can someone point me to a historical timeline, if available? Thank you --WikiInquirer 11:16, 16 January 2007 (UTC)

The dumps are supposed to be generated once a week but recently, they seem to be generated less often so there's no guarantees on when the next one will come out. If you go to http://download.wikimedia.org/enwiki/ you'll see a list of dumps generated in the past few months for the English Wikipedia, but if you need dumps from before then, you might be able to get one of the existing dumps and run an SQL query on it to remove all revisions and log entries for pages that are dated after a certain date in order to 'simulate' how the wiki looked historically. Tra (Talk) 15:57, 16 January 2007 (UTC)

Where "recently" is the last few years.... Rich Farmbrough, 22:20 3 March 2007 (GMT).

The difference between current and history in the database dump

Hi there. I have downloaded a portion of the English Wikipedia database dump on 30th Nov 2006 and I have imported stub-meta-current.xml.gz (364.7 MB) on my machine. Curiously, I found that each of the following three tables: page, revision and text has exactly 6,635,199 records. As the label 'current' might suggest, is any of the data in the three tables truncated? I have a hunch that the tables revision and text are truncated at whatever the number of records that table page has. Is this correct? What does 'current' actually mean?
Furthermore, I wish to understand what are the differences among the following three files in the db dump:

• stub-meta-history.xml.gz 3.1 GB
• stub-meta-current.xml.gz 364.7 MB -- This is the one that I'm staring at now.
• stub-articles.xml.gz 238.9 MB

Thank you --WikiInquirer 11:36, 10 January 2007 (UTC)(Talk)

Meta current is all pages - including user pages etc.

Meta history has all the history of the pages, so you can analyse changes, authorship or do roll backs.

Articles is just article pages plus possibly a few bits and pieces...

That's my best take on it. Rich Farmbrough, 11:55 10 January 2007 (GMT).

Thanks Rich for the above reply. Does that mean that stub-articles.xml.gz contain only pages and no revisions/text? In stub-meta-current.xml.gz, why would the tables page/revision/text have exactly the same number of records? Any missing data here? --WikiInquirer 04:41, 11 January 2007 (UTC) (Talk)

"Current" means no histories, as opposed to complete or whatever (the much bigger one). Thus for each page, you have one entry in revision (the current one) and one entry in text (that of the current revision). —Simetrical (talk • contribs) 05:02, 11 January 2007 (UTC)

Thank you very much, Simetrical. You answered my question on the spot. --WikiInquirer 07:19, 12 January 2007 (UTC) (Talk)

Contacting other researchers on WP

Posted the question on Helpdesk on 4th Jan item 5.8 --WikiInquirer 05:27, 6 January 2007 (UTC)

Private tables in the db dump

Hi there,
I am an IS undergrad doing some social research on the Wikipedian community -- What motivates people to contribute their knowledge to Wikipedia, in the absence of pecuniary compensation.
I am currently swimming through the archival data from the English Wikipedia database dump and I wish to find out how I can possibly get my hands on isolated fields in the private tables that should not violate Wikipedia's privacy policy?
Example being: In the user table (which is private), can I take a harmless peek at only the following three fields: user_id, user_name and user_registration? --with the intention of mapping the user's registration timestamp. Or are there any points of contact whom I can approach to make research-related inquiries?
Your help in this matter is very much appreciated. --WikiInquirer 09:03, 4 January 2007 (UTC)

You could probably download the log dump to get the dates of registration for the newer users. For the older users this data won't be available but you could make a reasonable estimate on when they signed up by looking at the date of their first edit. For the very oldest users, i.e. as old as Jimbo Wales not even this data would be available but you could probably find out their dates of signup elsewhere, e.g. in the archives of Wikipedia:List of Wikipedians in order of arrival. You won't be able to get the users' id this way but the user names are all unique (although bear in mind that usernames have changed for some people, you can find details of these name changes in the user rename log). Tra (Talk) 12:37, 4 January 2007 (UTC)

User renaming

Thank you for the advice on Private tables in the db dump. I shall go and comb the archives. Are user renaming actions captured in the logging table?
WikiInquirer 05:41, 6 January 2007 (UTC)

Yes, and you can view them at Special:Log/renameuser so I would presume that they would also be available in the database dump, although I havn't looked at that particular download myself. If they aren't available, you could always just scrape the website. Be aware, however, that if a user performs an action before they are renamed then the logs aren't always consistent on whether the old or new username is used. You might want to look through the logs for a few recently renamed users to familiarize yourself on how the actions are recorded. Tra (Talk) 13:14, 6 January 2007 (UTC)

Tra, thanks for the above reply. Just to be sure, a user cannot rename herself as another user who already has that username right? Meaning to say, there can't be more than one Jimbo Wales. I checked the db schema and it says that user_name is an unique index and user_id is the primary key to the user table. --WikiInquirer 05:06, 10 January 2007 (UTC)

It is not possible to rename a user to a username that is already taken, and it is not possible to have two users with the same username at a given point in time, but it is possible to rename a user to another username, which frees up their old username for someone to rename a user to, e.g. the following renames could be performed:

1. JoeBloggs -> JoeBloggs2
2. JohnSmith -> JoeBloggs (because JoeBloggs is now available)

This happens very rarely, however (since only Beaurocrats can rename users, and most renames are normal and straightforward), so you would probably be able to manually fix your statistics when this happens, if it causes a problem. Tra (Talk) 17:55, 10 January 2007 (UTC)

Thank you very much, Tra. You have been a great help =) --WikiInquirer 04:28, 11 January 2007 (UTC)

User groups

Hi, I'm a relatively new user who is conducting some research on the English Wikipedia. I downloaded the database dump on 20061130 and I have a few questions about the user_groups table: Are normal registered users listed in this table because it has only 1,319 records? Only 7 distinct user groups are shown here: boardvote, bot, bureaucrat, checkuser, developer, oversight, steward, sysop? Does this mean that normal registered users do not belong to any of the above 7 user groups? Thanks --WikiInquirer 05:42, 31 December 2006 (UTC)

Yes, I believe only users with extra rights are shown, though I haven't looked at the table in a while so I can't be sure.