SourceWatch talk:RefConverterBot

Referencing systems

Hi again AI,

Sheldon mentioned to me recently that you're not very happy with the new Cite.php referencing system. So I thought I'd drop you a line and get your take. I actually disliked it at first, too, but I've come round to it now. Here are the things I like about it:

• References travel with the text. If someone splits an article in half, in most cases the references will still be fine in both new articles (although one has to be a little careful with citations that are referenced more than once in the same article).
• The links to/from the citations actually work properly now.
• Putting the citation text in the main body of the article is less ugly to edit than I expected, in fact I'm beginning to like it.
• And having to do that forces people to provide citations along with their references (because the citation actually generates the ref link).
• It's become the standard over at wikipedia (after a long period of controversy!)
• And it means that references are now properly integrated into the mediawiki software - ie. the software "knows" they are refs.

There's still a place for the old-style citations in "Other external links". In the articles I've converted so far, this is where I put external links that don't directly serve to justify the article's content, but are good background reading.

So, what's your take on this? (Since you delete your talk page messages once you've replied, I'll cc this onto my talk page too...)

Regards, --Neoconned 07:56, 18 May 2007 (EDT)

Neoconned - I've used it a few times and it's okay, I guess. However, I tend to input quickly at times and it's just a major annoyance so have not been using it much. Will have to try to use it more .. if I don't forget! Artificial Intelligence 08:01, 18 May 2007 (EDT)

OK... but what's the annoyance, from your point of view? After all, in principle it involves less typing than the old system, as you only need to put the citation link in one place, not two! --Neoconned 08:07, 18 May 2007 (EDT)

It's the forgetting part ... practice, practice, practice, as they say. I just don't think of it ! Artificial Intelligence 08:09, 18 May 2007 (EDT)

Ha ha true... and the proliferation of referencing systems over at wikipedia over the last few years hasn't exactly inspired confidence in users that it's worth learning yet another one! But it does seem this is the end of that evolutionary process. Er, I hope.

I was planning to write a manual converter in JavaScript, (at User:Neoconned/SourceWatchRefConverter), to allow people to convert individual articles to the new system.

But Sheldon has suggested writing a bot (robot user) to automatically convert all the articles on SW to the new system. And I think it does make sense to get it done automatically. I see this as being governed by the following principles:

• Getting it right is more important than getting it running soon. We won't run it until it's very thoroughly tested.
• It can run slowly. If it converts one article every 3 minutes, it'll take about a month to do the whole wiki. And that's enough time for human users to keep an eye on what it's doing.
• There'll be a panic button. I envisage a "control page" on the wiki where any SW admin (including you, Bob, etc) can pause the bot simply by putting some suitable text (eg. "OFF").

How does that sound to you? --Neoconned 08:21, 18 May 2007 (EDT)

---

I like the automatic part ... but here I see a couple future (and past) MAJOR PROBLEMS for references. These problems, for the most part, originate with the source, not SW or CP, although users contribute to the problems. I speak here from personal experience, as I used Yahoo! back in the "good ole' days" of Disinfopedia.

• The old and (unfortunately) current use of Yahoo! links. Although I've personally beat this drum nearly to death all over SW and CP, the Yahoo! news links keep appearing and expire almost overnight. I realize that those email and instant message notifications are handy, but provide a "lazy man's" access to the news. An automatic conversion system will not be able to fix that. Sometimes having the article's title helps to replace a broken link, but not always, as Yahoo! like many others using wire services, renames or rewords the article's title. Also, Yahoo! links are not archived in the Wayback Machine (nor are many other expired newspaper links).

• Without an active Yahoo! (or other) news link, it is nearly impossible to find an accurate article title or the original source, such as Agence France Presse or other such sources, which Yahoo! aggregates.

• Another problem is the use of the Associated Press as a source. The AP is a wire service, not a publication (this has been a tough one to explain to folks), and AP subscribers can rename and pick and choose from the wire report to suit their own needs, plus add local applicable information. Additionally, the reporter's name on the article can vary, depending on whether an article is posted "as is" or is edited or embellished by a local reporter. An expired AP link will often, like Yahoo! links, be impossible to replace.

• On the subject of the AP-related links, it also matters which news sources are used as to whether an AP link will survive. For example, ABC News and CBS News, and even Fox News, are pretty enduring. Forbes usually is, as well. Others are not. The most reliable of enduring links are those posted by Free Republic, but then those articles are not always whole and usually come with numerous lengthy comments not pertinent to the article itself.

• For that matter, these expired links will be embedded within the article as well as listed in the References section, doubling the lack of documentation.

My two cents ... Artificial Intelligence 08:52, 18 May 2007 (EDT)

Good points... Actually, the bot will give us a chance to at least flag up the Yahoo links. It will work by matching up links within the page. This is because a plain numbered ref link [1] should point to the same address as the link in its corresponding citation. So it'll be easy to make the conversion bot add a warning message to those citations that use Yahoo links. I think the best thing would be for it to add a template call, to eg. Template:YahooLink. That template would add a warning message to the link within the citation. And then one would be able to get a list of all the pages in the wiki which contain Yahoo links, simply by looking at which pages invoke that template.

Another (and technically unrelated) measure could be for me to tweak the Javascript in MediaWiki:Monobook.js so that users who attempt to add a link to Yahoo get a warning message, dissuading them... (or possibly even blocking them altogether).

So the reference converter bot can tell us which articles in the wiki already contain Yahoo news links. And a little Javascript can discourage users from adding new links to Yahoo. :) --Neoconned 09:57, 18 May 2007 (EDT)

That sounds like a marvelous solution. Just blocking future Yahoo news links will be a help. Replacing all the dead Yahoo links will be, unfortunately, nearly impossible ... too, too many of them with too, too little info to follow. Artificial Intelligence 10:38, 18 May 2007 (EDT)