User talk:SMcCandlish/Replacement of Template:Rp

Some comments on the draft
You have made a great start, so not a huge lot to say.

Lead
What I'm really saying is that we need to make a solid case that has passed its 'use-by' date and it is time to retire it. There are going to be many editors who don't like change and will need to be persuaded that this change is really needed. 𝕁𝕄𝔽 (talk) 16:06, 21 December 2023 (UTC)
 * An example or two of the horrors enabled by rp might be good.[1]:123 or [2]:321, footnote 27 to demonstrate what we are talking about
 * this necessarily includes parenthetical partial citations like those created by. Is that indisputable and self-evident? I think it needs more work. I'd have written it as this arguably includes parenthetical partial citations like those created by. if only to pre-empt those who read no further on the first assertion that they disagree with.
 * Mandy Rice Davies Applies but I would want to assert that adds visual clutter and is poor typography. (But we need to be careful to avoid the "hard cases make bad law" response and make clear that although this[2]:321, footnote 27 is the extreme end, simple cases like[1]:123 are ugly too.)
 * Is there an MOS:ACCESS challenge?
 * Two tweaks:
 * other than the simple superscript indicators
 * the most common me n thod of shortened footnotes, the harvp/sfnp template set, which work directly with  citation markup.
 * Taking these in order:
 * I added such examples.
 * A general rule for implementing change (in any aspect of life) is to assert that the change is needed and give the reasons; if someone else has doubts, they can raise them on their own. Don't seed them with doubt and provide them ammunition to resist change, especially since people tend to resist change simply because it is change, without an actual substantive objection. For more on this, see this user-talk comment.
 * Done.
 * I thought of one and added it, though it's just about eyesight not a strong technical objection (not WCAG/ARIA sort of matter).
 * I think the intro makes the case strongly now.
 * Tweaks:
 * Done.
 * Done.
 * I also made other replies below.  — SMcCandlish ☏ ¢ 😼  09:29, 30 December 2023 (UTC)

Summary of the process
𝕁𝕄𝔽 (talk) 16:06, 21 December 2023 (UTC)
 * 1) If it's only being cited one time at all True but getting off-topic: omit.
 * 2) Many articles especially about writers already have a section called ==Bibliography== for the list of works. I think that ==Sources== is more common (or at least not as distracting or potentially confusing).
 * 3) I don't understand what you have written here. You know that you can use any amount of sfnps citing different pages of the same source in an article?  ...  ...  freely in the same article. You can also have multiple instances of  ...  ...  (citing the same page fronm three different points in the text) and "the system" will give them all the same number (as it does for  ) Have I misunderstood your point?
 * 4) I think you are referring here to repeated instances  (as in  ...  ...  but I'm afraid that it is not really clear, sorry.
 * 5) I'm getting lost now, you need to explain what is being fixed.
 * 6) Repeating what I said at #1. These are good points but are clutter where you have put them. I would put then in a new section called something like "and while you are here" (though in my experience, it is always best to concentrate on one aspect at a time, that makes it easier to identify where the error crept in.
 * Do we need to mention /
 * Overhauled this entire section to account for not creating duplicates, so no need for a "defining" instance of.
 * Moved it to "Concurrent cleanup".
 * Fixed.
 * Fixed. I was unaware at the time that had been made so sophisticated. For a long time now I've been doing   at the first named occurrence of a reused citation, when this wasn't actually necessary. I have a lot of stuff to eventually clean up!
 * This material has been replaced.
 * Ditto.
 * Ditto. There are only four steps now.
 * Done.
 * — SMcCandlish ☏ ¢ 😼  09:29, 30 December 2023 (UTC)

The process in more detail, for specific kinds of cases

 * Why would you ever advise
 * I had an example just today! Something like  but that's the only use-case I can think of. --𝕁𝕄𝔽 (talk) 19:30, 22 December 2023 (UTC)
 * I seem to remember struggling with how to use quote= with sfnp but I think I resolved it.
 * You haven't mentioned the loc= option. I found that very useful at Calendar (New Style) Act 1750 with some very old sources. Though I'm not sure that it would ever have been in competition with

That's it for now. 𝕁𝕄𝔽 (talk) 16:06, 21 December 2023 (UTC)

I'm off to a holiday party for now, and will get to the rest of this later. Quick answer to a dual point: The reason I was using  with successive cases of , instead of using  and then more instances of  later, was an expectation that the latter would create duplicate short citations (which it did do, back in the day). This seems to have been fixed without me ever noticing (since I would do the harvp markup habitually to avoid it), so I need to rewrite the material to just use for such cases, and that will certainly simplify things!

Blah blah blah.

Yak yak yak.



I was expecting (based on "olden days" behavior) for the "Yokohama (1999), pp. 22–30." short cite in the demo to show up twice.

That said, if  (with no ref name) appeared in the tutorial material so far, that was probably an error on my part, though if the article is consistently templated this way already, WP:CITEVAR would have us comply with it when doing  cleanup. — SMcCandlish ☏ ¢ 😼  01:41, 23 December 2023 (UTC)
 * Haven't forgotten about all the above; I've been banging all day long with ChatGPT on improving the ref cleanup regex. Will try to get to the rest of this stuff tomorrow.  — SMcCandlish ☏ ¢ 😼  07:37, 25 December 2023 (UTC)
 * I'm ultra intrigued to know how you used chatgpt for this but I guess I'll have to wait until you have nothing better to do. 𝕁𝕄𝔽 (talk) 08:52, 25 December 2023 (UTC)
 * Turned out to largely be a waste of several days' time. ChatGPT cannot reliably parse really long and complex regexes. It not only gets confused about various of the operator-overloaded functions of  in a regex, it will just forget (even after being reminded) of the necessity to not change one part of the regex that is working for particular cases when it is trying to address an issue with another part.  In the end, I switched to using regex101.com, which provides precise syntax highlighing of the regex, and a breakdown of what it is doing step by step, with live-updated test cases. See here for one of the ones I've been banging on: https://regex101.com/r/xubdCt/14  As for ChatGPT, I still use it for debugging, but I'll isolate the problem part of the regex and only have the "AI" work on that part alone, so it does not mess with any of the other parts that are already doing what they are supposed to do. If you're just curious, you can see some of my more "painful" ChatGPT sessions here and here.  — SMcCandlish ☏ ¢ 😼  09:29, 30 December 2023 (UTC)
 * The rp-replacement-guide section addressed above is the next one I will overhaul, but I need a TV break and a snack first. I already addressed the first two sections above, item by item.  — SMcCandlish ☏ ¢ 😼  09:29, 30 December 2023 (UTC)
 * Got caught up in working on the ref cleanup scripting. The "meat" of it is already working. Need to add a test for nested quotation marks, and fix an issue with line-breaks not being addressed when the format begin with . Then see about working in some other stuff. The hardest part seems to be done already. Other than reworking what works for horizontal citations into a version for vertical ones, often used in WP:LDR, and which people will defend (at least in the LDR block, not in mid-articles) as a WP:CITEVAR "citation style".  — SMcCandlish ☏ ¢ 😼  14:28, 2 January 2024 (UTC)
 * That ref cleanup script is now usable, at User:SMcCandlish/TidyRefs. Still needs a few features added, but it mostly will already do what is needed to normalize  and   markup (especially when used alongside User:SMcCandlish/TidyCitations to normalize the citation templates inside the p) to make search-replace operations on such things reliable and predictable across an article.  — SMcCandlish ☏ ¢ 😼  17:26, 22 January 2024 (UTC)

Template:R, etc.
Stuff to deal with: This is basically just a note-to-self about some things to keep track of and deal with at some point, and to account for in the replace-rp documentation. — SMcCandlish ☏ ¢ 😼  07:35, 25 December 2023 (UTC); updated: 01:05, 5 February 2024 (UTC)
 * , a wrapper for, needs to be replaced (perhaps by a bot) when using  features; then its code changed to no longer use . I marked it  on 2024-01-20 , because of the next item below. EEng reverted me on 2014-01-22; took it to his user talk.
 * Centralized discussion/Citation discussion: Overwhelming consensus to "suspend" implementation and use of, unless/until: "all relevant policies and guidelines have approved the use of ; all tools / bots for maintaining citations had be[en] proved to work as well with as with ref; use of  in an article has gained active consensus, not just apathy." Zero of that ever happened, so this template is already deprecated since 2010, and should be removed from various information pages, etc., citing that RfC.
 * I've made a Template:R/sandbox copy of that template (same with its subtemplates, and changed the code in the sandbox versions to refer to the sandbox versions of the code); asked for help at Help talk:Substitution on making the template cleanly subst'able so we can get at the underlying p markup it outputs.
 * Instructions to use it at Help:Reftags need to be replaced with and  summary. I marked this section  on 2024-01-20, though it needs to be replaced with a summary of usage of , etc., not left historical indefinitely.
 * Same with Help:Citation merging. Needs to be replaced with instructions on use of . I marked this section on 2024-01-20.
 * Same with Nesting footnotes. Both of these pages need to be overhauled to account for all methods, including, etc. I marked it on 2024-01-20
 * Yet another one: Help:Converting between references formats. I marked it on 2024-01-20.
 * Citing sources also needs revision, but after the above, as a summary.
 * Recommendation of was also found in WP:Citing sources. I removed it. We'll see if that sticks.
 * , another wrapper (also found recommended at WP:CITE and removed from there). As with, it needs to be replaced (perhaps by a bot) when using  features; then its code changed to no longer use . It should not be entirely deleted/deprecated, since it provides other features. This citation system is only used in 237 articles (plus a few user pages, etc.)
 * The Template:Harvard citation documentation shared by, etc., needs to be updated to say how to use sfnref instead of recommending the old 2023a crap.
 * May also need updating: Citation templates and reference anchors.
 * See also Wikipedia talk:Citing sources/Archive 50 and note in particular the disputation between Guy_Macon, Izno, and Moxy vs. SarahSV and David Eppstein (Whatamidoing kind of in the middle), and a few other parties; more pertinent to quotation marks around attributes in the particular, but the "lines in the sand" are clear that various editors are in favor of incremental minor improvements, while others have a walled-garden approach to "citation styles" at "their" articles).
 * Article heavily using a mixture of and  to use as a cleanup testbed: Ballistic Missile Early Warning System.
 * Documentation at Template:Sfnp, etc., include examples of the deprecated Harvard inline-parenthetical templates, and these need to be replaced; may have to split the shared documentation between current and deprecated versions.

Rp deprecation
As with rp itself, the first and very significant hurdle we have to get over is to secure consensus to have it declared obsolete and deprecated. We mustn't underestimate how much of a challenge that is going to be. There will be many editors for whom the quick'n'dirty kludge is just too convenient. 𝕁𝕄𝔽 (talk) 08:59, 25 December 2023 (UTC)
 * Just going to do a bit of a brain-dump here. As noted in the first section above, I'm not going to "seed" people with the idea that there is a controversy here, a challenge to them, a cause to take up. I'm just going to provide tools to do what needs to be done, clear rationales for doing it, and then just start doing it, and encouraging others to do it. If someone wants to play butt-hurt victim about it, they can do that on their own time without any help from me. I'm not going to provide them any "well, maybe ..." rationales they can try to use, any excuses they can hide behind, any wedges they can drive, any wiggle room to try to "creatively interpret" their way out of deprecation of inline parenthetical referencing. The RfC about it didn't make an exception for rp, so there is no such exception unless and until the community retroactively decides to make one. I'm sticking with that, or all of this is just a waste of time. As for "the quick'n'dirty kludge is just too convenient": Even with WP:CITESTYLE demanding a consistent citation style throughout an article, plus explicit deprecation of inline parenthetical referencing, no one is ever going to be punished for using it, or even for just dumping in bare URLs as "citations", or otherwise not following CITESTYLE in the course of being otherwise productive. The guidelines don't work that way, since they aren't policies much less core content policies. While people can't be forced to comply with CITESTYLE's consistency requirements while adding new content and citations, what  legitimately happen under that guideline (or its WP:CITEVAR subsection), however, is stonewalling to prevent citation improvement to comply with the guideline, nor going around changing good citations to crap ones against the guideline. (MoS and other guidelines work on the same principle: no one's sanctionable for being a poor speller or unclear about when to use capital letters or italics or dashes; but if someone stubbornly goes around changing correct spelling or formatting to erroneous style, or reverting people correcting their demonstrable errors, then that  disruptive and sanctionable.) A permissible "citation style" that can be defended under CITEVAR doesn't include  of how citations are formatted. (Some of the WT:CITE regulars tried really hard to get such a rule – down to spacing within citation templates, I kid you not – with various RfCs years ago, and dismally failed.) Lots of cite cleanup  permissible, without it constituting a "change in citation style" that someone can legitimately object to per CITEVAR. More on that below.  When it really comes down to it, we don't need to care if some particular editor is playing "don't you dare touch anything about citations at  article" WP:OWN games at a particular page. They'll give it a rest eventually, especially after  (and the rare ) are marked as deprecated at some point, and there's no hurry to do that. It'll be easier after a robust method of replacement is in play and being used. I'll probably remove the  recommendations from the above-listed pages sooner than later, though, since it conflicts with the community consensus to deprecate inline parenthetical referencing.  — SMcCandlish ☏ ¢ 😼  09:29, 30 December 2023 (UTC)

CITEVAR "Generally considered helpful" list and its implications
WP:CITEVAR's "Generally considered helpful" list is key here: The main points are 3 and 6 in that list, as they specifically enable replacement of parenthetical referencing, and are grounded in human utility not technical requirements.
 * 1) Rp cleanup arguably qualifies as "improving existing citations by adding missing information", since the page numbers are missing from the citation and are instead jammed into the article body. That one's kind of a stretch, but still an argument worth making.
 * 2) "Replacing some or all general references with inline citations" isn't relevant.
 * 3) "Imposing one style on an article with inconsistent citation styles (e.g., some of the citations in footnotes and others as parenthetical references): an improvement because it makes the citations easier to understand and edit" is definitely salient, since it permits replacement of parenthetical referencing (see also below), it is grounded in an "easier to understand and edit" rationale that is generalizable and has nothing to do with technicalities but with human users.
 * 4) A secondary, weak argument can be made with regard to "fixing errors in citation coding, including incorrectly used template parameters, and ref markup problems: an improvement because it helps the citations to be parsed correctly": Now that we have templates with parameters for this information, kept within the citations, using obsolete alternatives that put this information inline in the article text arguably amounts to a template-parameter misusage error today, as well as a markup problem (while the server and the end-user device can parse an rp-formatted citation without a rendering error, the user (the actually important figure in this equation) is impeded in parsing the citation by having to jump up and down in the article to get all of the salient information, instead of it all being together at the bottom of the page).
 * 5) "Combining duplicate citations": some of this rp cleanup will do that directly, and in spirit is doing that from top to bottom, since the point of this provision is getting rid of redundant citation code in the article; but, yeah, it's also one of the weaker arguments.
 * 6) "Converting parenthetical referencing to an acceptable referencing style": spot-on, word-for-word what we're doing here.
 * 7) Finally, 'replacing opaque named-reference names with conventional ones, such as "Einstein-1905" instead of ":27".': Cleanup of rp will also entail some of that, though again this is not one of the main arguments.

That's an off-the-cuff rundown on the sorts of arguments I'm prepared with already, and can probably think of more. Doing this kind of cleanup is something I'll go to the mat for. Another way of looking at this is the "It is better to ask for forgiveness than permission" principle of getting stuff done.

By way of direct analogy: If Trappist_the_monk had sought all kinds of buy-in and agreement on details before developing and deploying CS1, what would have happened is endless miring in demands to account for this-and-that nitpick of thus-and-such external citation style, and if anything functional had come out of it, it would be an essentially unusable mess in which everyone would have to go down a checklist of parameter nit-picks for a particular article's citation style. Instead, he just invented a "Wikipedia citation style" (with no bone thrown to other styles, other than grudgingly adding vauthors later, which was arguably a mistake), and it is now used in about 95% of our content (with most of the hold-outs being old content no one's bothered updating, aside from unformatted and incomplete "citations" injected by noobs, usually as URL copy-pastes). In ye olde tymes, it was common for editors who were professionals in a particular field to write articles here in a particular all-manual, no-template citation style that closely mirrored what they used at work (AMA, APA, MHRA, MLA, Turabian, IEEE, Chicago, Vancouver, whatever), and which was an ass-pain for nearly everyone else on the system. This practice has almost completely stopped now because of the CS1 templates and their widespread use. This happened because Trappist built and deployed and used the tool and encouraged others to use it, and just left CITEVAR pundits to go about their not-so-merry way defying consistency and ease-of-use until they were simply overwhelmed by more practical-minded users. "Build it and they will come." Etc. This is how most actually practical improvement at WP and other WMF projects happens, not through endless discussion that tries to appease everyone. (That tedious process is better for working on policy language, since buy-in about what rules should apply has to happen for people to respect the rules.)  — SMcCandlish ☏ ¢ 😼  09:29, 30 December 2023 (UTC)
 * is used to such an extent that I think it will be most effective to secure a formal community consensus that it is "scheduled to be deprecated" and that CITEVAR cannot be used a reason to obstruct its being replaced with sfnp, as happened at Eric Gill. People adopted CS1 because it was easier all round and provided a good checklist for what should be in a citation. I don't think we can really expect to achieve this change by stealth. --𝕁𝕄𝔽 (talk) 11:42, 14 January 2024 (UTC)
 * Yes, I agree that we shouldn't go looking for trouble. So how about we simply propose an update to WP:Citing sources that explicitly says that rp falls within its scope, that it is being phased out and it should be replaced as the opportunity arises using this here tool what you have developed. Would that work? --𝕁𝕄𝔽 (talk) 19:04, 14 January 2024 (UTC)
 * That's basically the plan, though developing the tools is taking longer than expected (especially after some drama distracted me). I still have a bunch of regex and js work to do.  — SMcCandlish ☏ ¢ 😼  21:00, 17 January 2024 (UTC)

More templates to account for
Some other stuff to detect and work around: — SMcCandlish ☏ ¢ 😼  19:50, 13 January 2024 (UTC)
 * – has a "maximal" format of, but would most often be

Citevar strikes back?
Just leaving this sea-mine outside your harbour to think about because I can't think of a way round it.&dagger; It kind of relates to our discussion at talk:John Dee. If an article uses the ref/ref style very heavily, then I can see an objection to replacing the refs that use rp to sfnp because of the underlying harv ref style. "So we have to keep using rp, obvs".

&dagger; short of a bot that changes every use of ref/ref to harvard. aka a nuclear minefield. 𝕁𝕄𝔽 (talk) 19:48, 28 January 2024 (UTC)
 * Well, "Harvard" is a misnomer here. is called that because it was originally devised as part of a system for doing Harvard referencing (inline parenthetical citations). Now it's relegated to use within , while most of the  templates are simply deprecated (though they're not all marked with  yet).  is the answer to the hypothetical issue here. If someone is weirdly and stubbornly resistant to using  in place of their   because, they insist, their "established citation style" is to always use  tags even when unnecessary, then the replacement is  .  But they should be convinced to go along with  as functionally identical but more concise. Anyway, I'm not going to spend any time trying to come up with resistance scenarios, per WP:BEANS. Just going to build the tools and use them and "advertise" them. It's not necessary that every page change over night and that  die instantly. It's fine for it to just fade away, exactly as it faded in.  For the moment, I'm a bit bollixed. The Windows machine I use most of the time (my "gaming PC" I haven't gamed on in 2 years, because I have more productive stuff to do), with its ultrawide monitor, and most of my stuff on it, is temporarily kaput due to a Windows Update screwup. I have to repair various permissions on the original boot drive, and reinstall the OS and all apps.  Been putting it off and using my old Mac for a while, but the regex development has stopped in the interim.  User:SMcCandlish/TidyRefs is, nevertheless, working remarkably well so far, though I need to build in a few extra kinds of error-condition handling, e.g. for mistakes like  . That will ultimately be the ref-code normalization tool to run first, that makes some kinds of semi-automated replacement of  actually feasible.  — SMcCandlish ☏ ¢ 😼  08:55, 3 February 2024 (UTC)
 * re ref/harv/ref, yes, I should have thought of that. Obvious really.
 * re your PC in resus, you have my deepest sympathy. I have a shadow drive and offline USB clone drives but any fool can make backups, actually restoring back to a working system takes a little longer. I have nightmares that I will ever have to do it for real. 𝕁𝕄𝔽 (talk) 16:45, 3 February 2024 (UTC)
 * Haven't lost any user data, it's just that once Windows gets ucked fup it's just ruined and you have to start over with it. Stupid an annoying. PS: As for "obvoius", I didn't think of it myself untl about 2 months ago. There are way too many referencing-related templates doing too many inconsistent things. It's all very hard to figure out and keep track of. Part of why I wrote up User:SMcCandlish/How to use the sfnp family of templates recently.  — SMcCandlish ☏ ¢ 😼  10:52, 4 February 2024 (UTC)