Template talk:NUMBEROF

Feedback
Hi. Here are the things I noticed about this template: Overall, I would see this as a really useful feature if it was part of the toolserver or a built-in extension to the MediaWiki software so that it could tap into the database or via an API call. Until then, as a static approximation (or worse, an outdated number that is not updated very often), then its usefulness is very limited. It would also be much more useful if it included all current projects and all current languages. This seems like a good starting point from which to expand and make it even more useful. I do a lot of cross-wiki templating, and something like this might come in very handy. -- Will scrlt ( →“¡¿Talk?!” ) 20:55, 19 April 2009 (UTC)
 * Where is this data from? How does it get updated? How often does it get updated? It appears to be static information that would have to be updated manually or by bot. In that case, the article numbers probably should be rounded to the nearest 250 or 500, and other numbers rounded appropriately, too. The documentation should indicate that these are approximations current as of YYYY-MM-DD HH:MM.
 * I only see Wikipedia projects, but some of the Wikibooks, Wikinews, Wiktionary, and Wikinews projects are larger than these. *The order of the entries appears jumbled. It would seem that alphabetical would be best to make it easy to maintain and for humans to understand more easily.
 * I would suggest making the default option (i.e., no attributes) return the live information from the current project (i.e., display the actual variable values for en.wiki in this case). #default: values should be used as catch-alls for each of the options, probably returning the article count (so remove NUMBEROFARTICLES from the check, and just use it as the #default: value for each language).
 * The "Raw" format of numbers should be the default, because your use of decimals for commas is confusing (How can you have 308 and 619/1000th of an administrator in 308.619? In American English, that would be 308,619. To you, that probably looks as confusing as the other way looks to me. To avoid that ambiguity, make Raw be the default.
 * The name is a little ambiguous, too. The name "Variables" should probably be used for something more to do with manipulating variables or displaying variables within other templates or something. Perhaps counts or numof) would be a more precise name for describing this template. Then the order of the attributes could be changed to make it more human readable. would be equivalent to your current  and  would be equivalent to your current  except returning "6,290" instead of "6.290".
 * I agree with most of what you said. However, I'd stick with the "R" parameter to get a numeric value, and no parameter to get a formatted value (with a comma as thousands separator, there's no need for a period here on en-wiki). Defaulting to raw and a formatting option for thousands separator is inconsistent with mw:Help:Magic words. Also, I'd actually rename it NUMBEROF so that it is used as or  .  Amalthea  21:24, 19 April 2009 (UTC)

Where is this data from? How does it get updated? How often does it get updated?
Where is this data from? How does it get updated? How often does it get updated? I know it says "The template is updated by a pywikipedia bot using this script", but it isn't clear how often the script runs

Also for NUMBEROFACTIVEUSERS, what is the definition of "Active User"? Logged on? Made an edit in the last year? Month? Someone who gets a lot of exercise? Guy Macon (talk) 15:45, 7 June 2011 (UTC)


 * The data is taken from pages like en:Special:Statistics, it is updated once per day. Definition of "Active user" can be found at the Statistics page: "Users who have performed an action in the last 30 days". Sorry for delay in answering, I seldom look at this talk page. — Ace111 (talk) 01:18, 4 December 2011 (UTC)

Largest Wikipedia
It would be great if the Wikipedia's rank in terms of article count can be added. For example, English being the largest, should have a rank of 1. This will allow us to do something like: The Dutch Wikipedia is the th largest Wikipedia project. Best regards. Naudefj (talk) 07:27, 13 September 2015 (UTC)
 * It is already possible:
 * The Dutch Wikipedia is the -largest Wikipedia.
 * The Dutch Wikipedia is the -largest Wikipedia. —Iketsi (talk) 19:33, 14 September 2015 (UTC)

Bot no longer updating data
Acebot appears to have stopped updating the data for this template. Could we get it working again? Sdkb (talk) 08:14, 10 March 2020 (UTC)

Requested move 3 May 2020

 * The following is a closed discussion of a requested move. Please do not modify it. Subsequent comments should be made in a new section on the talk page. Editors desiring to contest the closing decision should consider a move review after discussing it on the closer's talk page. No further edits should be made to this discussion. 

The result of the move request was: Moved, see below. (non-admin closure) * Pppery * it has begun... 03:49, 6 May 2020 (UTC)

– Match name with template * Pppery * it has begun... 03:38, 3 May 2020 (UTC)
 * Module:Wikipedia stats → Module:NUMBEROF
 * Module:NUMBEROF → Module:NUMBEROFSECTIONS
 * Can we please discuss what to do with a bigger problem first (see what I will post soon, below). Let's at least get GreenC's thoughts before jumping into a formal procedure. Johnuniq (talk) 03:58, 3 May 2020 (UTC)


 * Support This is logical and a good idea to keep the module and template names in sync, particularly the plan is to copy the module to many other language wikis and it would be great if they were all the same name everywhere. I believe had the idea that other "number of" functions could be added so they kept the module name generic, but it wouldn't make sense in this case (I don't think due to discussion below). Also Module:NUMBEROF is only used on one other wiki, so it wouldn't create much confusion elsewhere. --  Green  C  14:58, 3 May 2020 (UTC)
 * That's what I meant about discussing the issue with the module creator first. What is the point of holding a central discussion if the creator agrees with the move? It should just happen, particularly now that a new submodule will be created (see section below). Johnuniq (talk) 00:24, 4 May 2020 (UTC)
 * Makes sense to me :)  20:10, 3 May 2020 (UTC)


 * The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Overhead
@GreenC: There is a problem at List of Wikipedias which is showing several "The time allocated for running scripts has expired." In the old scheme, apparently a bot updated Template:NUMBEROF/data with current values, and all NUMBEROF had to do was work through some switches. I haven't had time to examine Module:Wikipedia stats but a quick look makes me think it cannot be made more efficient. Any thoughts on what to do? Johnuniq (talk) 04:02, 3 May 2020 (UTC)


 * I assumed with mw.ext.data.get "The loaded module is evaluated only once per page" similar to mw.loadData. This is not a time consuming operation particularly with cache for a 17KB object. However if it loads it for each instance of the template then it could time out on network I/O, when there are many instances like with List of Wikipedias. (Lua has a 10 second limit). If so, that is a bummer, it would make the Lua module impractical. -- Green  C  04:35, 3 May 2020 (UTC)
 * ... unless you wrap the mw.ext.data.get in a mw.loadData'd submodule. * Pppery * it has begun... 04:39, 3 May 2020 (UTC)
 * That sounds intriguing but I don't understand. -- Green  C  04:47, 3 May 2020 (UTC)
 * There would need to be a data module in addition to the main module. The main module would use mw.loadData on the data module. The data module would extract all data from mw.ext.data.get and store in a table (a table of pure data, no code). The data module would finish by returning that table. The main module would look up what it needed in the table. The point of that is that the calculation in the data module would occur once per page. However, it would always occur, even if there was only one call on the page. Johnuniq (talk) 05:09, 3 May 2020 (UTC)
 * Ok cool makes sense. I'll try it. -- Green  C  05:34, 3 May 2020 (UTC)
 * I wondered if all data could be extracted from the call so I put a test of dumping the result of the call at Module talk:Sandbox/Johnuniq/temp. Viewing that table might help see what a module would need to do. Johnuniq (talk) 07:08, 3 May 2020 (UTC)
 * I've had a look at the module now. If you wanted to do something else I would be happy to create Module:Wikipedia stats/data and update Module:Wikipedia stats. I'll wait to see what you want to do. Johnuniq (talk) 07:26, 3 May 2020 (UTC)
 * Johnuniq, Lua is not something I do very often, though I could work my way through this. Since the idea is to roll the Module out to dozens or 100s of wikis, it would be good to have it in top condition from the start, and you are more experienced then I am. If you want to develop it that would be wonderful. --  Green  C  14:38, 3 May 2020 (UTC)
 * FYI, I have nearly finished but I have to do a bunch of testing and upload. I'm thinking of putting the data in Module:NUMBEROF/data and we could either move that later or contemplate using Module:NUMBEROF for the main module, as well as what it currently does. Johnuniq (talk) 08:03, 4 May 2020 (UTC)

I have created Module:NUMBEROF/data and merged Module:Wikipedia stats into Module:NUMBEROF. I edited the merged code to use the data module. It needs more testing and a final check but I suspect it will work. Previewing suggests it can handle List of Wikipedias fairly quickly. The new module code can be tested with NUMBEROF/sandbox as is done at Template:NUMBEROF/testcases. Example: Problem: The current NUMBEROF (which does not use a module) has a special case for en.wikipedia, namely it uses the magic variables such as  to get the number of edits. Therefore its results are more up to date than the module which uses data that is updated once per day. We can think about that. I don't care where the code goes or even if it is used. However, it seemed reasonable to merge the two "number of" functions into Module:NUMBEROF which has documentation saying "may be expanded to include any other NUMBEROFthings". Any thoughts? Johnuniq (talk) 11:27, 4 May 2020 (UTC)
 * Update: The module now uses magic variables for the local site. Johnuniq (talk) 07:58, 5 May 2020 (UTC)
 * Merge works. The code is precise, interesting technique with map and data. Nice work, this is a cool module, and really a model for similar ideas to host template data on Commons. Feature request: some languages use "." instead of "," for number breaks (for example Turkish is 1.000.000 for 1 million). It can be hard-coded since it will always be the same, but would be good to have it easily visible for users to toggle, maybe a variable at the top of the module, or an in-line comment? -- Green  C  13:08, 4 May 2020 (UTC)
 * I wonder, if the module resides on enwiki and the language argument is "en.wikipedia" instead of downloading the file from Commons, run frame:preprocess(" ") - that would speed it up and resolve the accuracy lag. Commons updates 2x a day, I could increase but wanted to balance against too many page diffs over many years. --  Green  C  13:20, 4 May 2020 (UTC)
 * Good idea, I'll look at that, despite the consequent code elegance degradation. :( There is a language thingy for formatting numbers. ISTR it had a quirk but I don't recall the details. I'll think about that. Johnuniq (talk) 23:06, 4 May 2020 (UTC)
 * Maybe pass the data module which stat is requested, and the data module seeds the Lua array with only the required value via frame:preprocess, that way there wouldn't much change to the main module code. -- Green  C  00:09, 5 May 2020 (UTC)
 * But, for example, if a page had a request for enwiki stats first, mw.loadData would get an array with the information only for enwiki. Then if a second call on the page was for some other wiki, the array would not hold that data. The problem is that there is no way for an individual call to know if more calls will be needed on the page. The original code you wrote showed that on a crazy page like List of Wikipedias there is too much overhead to get the data individually. There could be two templates: a simple one that people should use for a page that only has a couple of calls, and a special one for long lists. The special one would use mw.loadData to get the single massive array. That might be unpleasant. Johnuniq (talk) 03:32, 5 May 2020 (UTC)
 * Oh right, it would defeat the purpose of using mw.loadData oh well .. maybe the solution is a doc sub-section named "Template vs. magic numbers" the pros and cons when to use each. In that case the current template has support for "en" and "en2" so requires backwards compatibility, "en2" as an alias of "en" -- Green  C  03:59, 5 May 2020 (UTC)
 * What is (or was) en2? I'm just looking at mw.site.stats which generates a nice table of numbers that exactly matches the magic words. For example, and mw.site.stats.edits give the same result, both here at enwiki and also at bnwiki. By contrast, at bnwiki,  gives a result (with commas) in Bangla digits, and their equivalent list of Wikipedias uses Bangla digits. I think it would be fairly straightforward for the module to use mw.site.stats for the local wiki, and the large table otherwise. I'll be thinking about that for a day or two. What about the move request in the section above? Any thoughts on whether Module:NUMBEROF should combine both functions, or should counting sections be separate? I don't care but I'd prefer to get that sorted out quickly. Johnuniq (talk) 05:10, 5 May 2020 (UTC)
 * The module now uses magic variables if the local site is wanted, and it uses local digits and formatting if want commas. I'll try it on bnwiki as a test in due course. Johnuniq (talk) 07:58, 5 May 2020 (UTC)
 * That is great. You can see what en and en2 are in the "/data" file. I have no idea why Acerbot added it. It only appears in the "/data" file on enwiki. Theoretically we could drop it and deal with the fallout since it's no longer needed. It might not even be used. -- Green  C  14:58, 5 May 2020 (UTC)

Other Wikipedias
After a look at bnwiki, I'm wondering if this development is worthwhile. They use bn:Template:NUMBEROF which gets its data from bn:Template:NUMBEROF/data. The latter is updated daily by Jembot, which is operated by -jem- from the Spanish Wikipedia. As well as what works here, they have a DEPTH field which is some kind of indication of how frequently the average page is edited (a number around zero means articles are hardly edited at all). For example, previewing the following at bnwiki gives the values shown: Presumably DEPTH is calculated by the bot. Other Wikipedias may have other systems. The enwiki Template:NUMBEROF/data has been updated manually each week by copying from it:Template:NUMBEROF/data where it is updated daily by ItwikiBot but with no DEPTH. Perhaps the development is worthwhile even if only for enwiki? Johnuniq (talk) 09:48, 5 May 2020 (UTC)
 * → 1000
 * → 342
 * User:Acebot was maintaining local "/data" pages for dozens of wikis. It went MIA in December. Since then, most of those sites have been broken either entirely, or sporadic manual updates like enwiki. A few sites have operating local solutions, like itwiki and bnwiki. To replicate Acebot will require bot perms across 100s of wikis, which is laborious and if anything happens to GreenC bot (which would theoretically replace Acerbot's function) then someone else will have to do the same in the future. This solution requires no bot perms, can be easily rolled out to 100s of wikis by anyone (unless they have an operational local solution like itwiki), and if GreenC bot (which maintains the data on Commons) ever went MIA then it can easily be replaced by another single bot updating a single page which works across all participating wikis. --  Green  C  14:41, 5 May 2020 (UTC)
 * OK, now I see that en2 means to use the bot values for en rather than the current values from the magic variables. I will support that if it's simple. I also see your point that the module will be worthwhile even if it's not used everywhere. One day (not now, I can't handle any more info atm!), I'll get you to explain how GreenC bot creates c:Data:Wikipedia statistics/data.tab and how that can be fetched with . Clever stuff.I have to resolve what's going to happen to the module with respect to number-of-sections. I originally thought that merging them would be desirable, but they are different concepts and if NUMBEROF is going to be used on other Wikipedias, it might be better to split the modules again. I'm now inclined to move the NUMBEROF material from Module:NUMBEROF, putting it back into Module:Wikipedia stats. Then implement the move request above. Can we thrash that out please—I can't make up my mind on whether to merge or split but I'm inclining to split on the basis that the simpler NUMBEROF is, the better, if it's going to be used elsewhere. That is, we would end up with Module:NUMBEROF containing the project we're working on, and Module:NUMBEROFSECTIONS with the previous code to count the number of sections on a page. Thoughts? Johnuniq (talk) 02:09, 6 May 2020 (UTC)
 * Johnuniq Honestly I don't care either way I'll probably copy the module without the section code, so it wouldn't matter. I'd just like to firm up the code so I can start copying there is a lot of mundane work to do there yet. --  Green  C  02:33, 6 May 2020 (UTC)
 * I think we are in agreement then. Please change your vote in the move request above and I will proceed with splitting them. Johnuniq (talk) 03:09, 6 May 2020 (UTC)
 * I'm moving to wrap this up so please change your vote above. Johnuniq (talk) 03:25, 6 May 2020 (UTC)
 * Done. -- Green  C  03:38, 6 May 2020 (UTC)

I have investigated the depth field displayed at bnwiki. It turns out that depth is defined at Wikipedia article depth. Translating the formula from that page into Lua syntax, using the statistics names gives:

I checked that bnwiki implements that formula with a strange twist: after calculating the above with, the value is rounded to two decimal places. Pretty odd. The bot data they have at bnwiki includes a depth field, but it is an integer. At any rate, the module could easily calculate depth if wanted. Shall I implement that?I will soon upload a new version of the module that implements en2. I'm thinking it should also implement aliases for the parameter names. For example, the number of articles can be fetched with ARTICLES or NUMBEROFARTICLES. That is, any "NUMBEROF" at the start of a parameter should be removed. I assume it's ok to do that? Johnuniq (talk) 05:19, 6 May 2020 (UTC)
 * More questions. Q1. The "not found" error return is  whereas the current template uses  . For example, fetching EXITS (typo) instead of EDITS would give -1. Is -1 wanted? Q2. Now that Module:NUMBEROF contains only one function, and if there are no plans for other functions, normal procedure would be to rename   to  . That would change the template from   to  . Johnuniq (talk) 06:47, 6 May 2020 (UTC)
 * I couldn't leave it alone and have implemented depth. I checked that the calculations agree with the data at bnwiki.
 * With answers for Q1 and Q2, this can be finished. Then I need 24 hours break, then I can do a final code review. Johnuniq (talk) 10:56, 6 May 2020 (UTC)
 * With answers for Q1 and Q2, this can be finished. Then I need 24 hours break, then I can do a final code review. Johnuniq (talk) 10:56, 6 May 2020 (UTC)
 * With answers for Q1 and Q2, this can be finished. Then I need 24 hours break, then I can do a final code review. Johnuniq (talk) 10:56, 6 May 2020 (UTC)


 * I'd never heard of depth before, but since it is a standard statistic it is awesome to include. For Q1: -1 to indicate an error because "0" is a legitimate value in the data. I suppose it could return a string but there would be translations which I was hoping to avoid. Q2 sounds good didn't know that. -- Green  C  13:32, 6 May 2020 (UTC)
 * Noticed the error message in English in trimArg oh well I probably won't translate it. Also it uses Module:Math which is not standard library.. what happens if the module does not exist at a site, does the module not run at all, or only if depth is requested? It is pretty commonly copied so this would be an uncommon scenario. -- Green  C  13:57, 6 May 2020 (UTC)
 * OK, we'll keep -1 as the error return. I changed the entry function to . In due course I'll do some more serious testing and checking. There are only 326 transclusions of NUMBEROF here so we could swap that template to use the module soon, but I'll wait at least 24 hours in case I want to see current articles with the old template. Re Module:Math, are you referring to math.floor? That does not use a module—math.xxx is built into Lua and is available everywhere. There is only one English message in the module and it's simplest for people to translate it directly rather than fiddle about with a more general i18n procedure. Johnuniq (talk) 01:38, 7 May 2020 (UTC)

Q83161953
Any ideas about what should happen with ? That was created when ckb:Module:NUMBEROF was copied from what is now at Module:NUMBEROFSECTIONS. It doesn't matter to me, but if the new Module:NUMBEROF gets copied to other Wikipedias, someone will want to create a Wikidata item for the module and confusion between Module:NUMBEROF and Module:NUMBEROFSECTIONS will occur. Johnuniq (talk) 04:01, 6 May 2020 (UTC)
 * I renamed the item on Wikidata to "Module:NUMBEROFSECTIONS". I don't see much potential for confusion, as long as the CKBwiki module is renamed before the new module spreads to other wikis, which will presumably happen now that you've pinged Aram. * Pppery * it has begun... 04:06, 6 May 2020 (UTC)
 * Thanks, I thought that would be more involved but I guess that's the whole point of a Q id, namely that the name can be changed without fuss. Johnuniq (talk) 04:33, 6 May 2020 (UTC)
 * Hello By these discussions, I updated the module, template and there contents. Thanks! ⇒  Aram  Talk  15:10, 6 May 2020 (UTC)

Statistics problem
I have been wondering why there is a large discrepancy when using Module:NUMBEROF on List of Wikipedias. To see that, edit NUMBEROF and replace the contents with  but do not save. Paste "List of Wikipedias" into the "Preview page with this template" box and click Show preview. The most noticeable difference is the pie chart at the top where, for example, English shrinks from 11.4% to 5.3%. I exported the data from bnwiki and compared it with the data from commons:Data:Wikipedia statistics/data.tab. There are minor differences due to the different times at which the two sets of data were prepared. However there is the following major difference. site                articles

---commons data--- commons.wikimedia   60,020,863 en.wikibooks            86,030 en.wikiquote            36,791 en.wikivoyage           29,758 meta.wikimedia          92,568 species.wikimedia      722,714 total              114,234,009

---bnwiki data--- [only has .wikipedia data] total               53,237,984 The discrepancy above is because the commons data has a total that is twice that of bnwiki, because commons includes the non-wikipedia sites shown. I checked the data at bnwiki and itwiki and eswiki (created by two different bots). They have essentially identical values which are very similar to enwiki except that enwiki's data is a few days old.

One way to handle this would be for GreenC's bot that generates the commons data to calculate the total data from only the .wikipedia sites. It could perhaps also include new grandtotal data equivalent to what is now total. Johnuniq (talk) 02:32, 8 May 2020 (UTC)

Ah yes. Each project with language sub-domains would have its own total. I believe there are 7 plus a grand: I can seed the stats with one from each "en.wiki " and add some GreenC code for project totals -- Green  C  03:02, 8 May 2020 (UTC)
 * total.wikipedia
 * total.wikibooks
 * total.wikiquote
 * total.wikivoyage
 * total.wikisource
 * total.wikinews
 * total.wiktionary
 * total.all
 * Good, that would cover everything. By the way, I have done a superficial check of the module at bnwiki and its use of local digits can be seen at bn:User talk:Johnuniq. Johnuniq (talk) 03:15, 8 May 2020 (UTC)
 * The grand totals are now accurate, at least for *.wikipedia tested with the above preview method .. the other projects are not complete since not every lang.project is in the config.tab (eg. ru.wikisource); which is possible, but it adds to the file size and so far no one has seemed to want it. Also config.tab is manually maintained so will not reflect new lang.project until someone adds them.. I thought about automating the list creation at least for *.wikipedia so there would be no need to maintain config.tab except for optional non-wikipedia projects, but have not figured out how to determine the list of all wikipedia langs. -- Green  C  21:30, 8 May 2020 (UTC)
 * That's great. I checked the totals for xxx.wikipedia and they look good. The other projects can wait for a time when they are requested. I'm thinking of trying this at nowiki (that's no.wikipedia.org) where they use non-breaking spaces for thousands separators. They don't have much activity and their list article (no:Liste over Wikipediaer) uses manually-entered numbers and is very incomplete. Johnuniq (talk) 00:03, 9 May 2020 (UTC)
 * I had previously installed the original version of the module on the Afrikaanse wiki and never rolled it back since it was basically working (albeit poor performance). Tonight I installed the new version it looks like commas are not working? Example: Afrikaanse Wikipedia -- Green  C  03:01, 9 May 2020 (UTC)
 * The module uses formatNum to produce localized numbers (when N is used as the third parameter) and that should give the same result as the parser function. Editing af:Afrikaanse Wikipedia and replacing its content with  then previewing shows "1 234 567 890" (with spaces). Doing the same thing with   also shows a number with spaces, so I assume it's working. Johnuniq (talk) 04:02, 9 May 2020 (UTC)
 * heh it didn't occur to me spaces are a style. -- Green  C  13:43, 9 May 2020 (UTC)

User:Johnuniq, things are looking pretty good. How about we go live and see if any problems show up, enwiki is a good stress test. -- Green  C  23:00, 11 May 2020 (UTC)
 * Yes, I'm ready. Please switch it over and deal with any complaints! Johnuniq (talk) 23:09, 11 May 2020 (UTC)
 * Alright! It's active and docs are updated. -- Green  C  01:05, 12 May 2020 (UTC)
 * It's looking good. At the time of my last message, I kept a file with the HTML source of List of Wikipedias, and I just purged that page and got the source again. Comparing the two shows expected minor variations in numbers, and the pie chart is unchanged. The Lua time usage rose from 0.76 seconds using the old template to 3.87 seconds using Module:NUMBEROF because it's doing a bunch of work, but the total CPU time usage fell from 7.47 to 6.89 seconds. Other performance numbers also show a minor improvement. Johnuniq (talk) 03:34, 12 May 2020 (UTC)
 * I set up a test page User:GreenC/numberof with 16 templates and Lua time usage is 0.021 (it was 0.015 with only 1 invocation) so we know retrieving the file from Commons is not inherently slow. List of Wikipedias has 422 invocations. Adding those to the test page and Lua time is 0.273 .. ok but the page does some fancy arithmetic so I added those formulas and Lua time is 0.323 .. I can't think of how to make it higher. The jump from 0.76 to 3.87 is odd.  --  Green  C  14:47, 12 May 2020 (UTC)
 * I just put all 438 from List of Wikipedias in User:Johnuniq/sandbox2. It shows Lua time usage 0.340 seconds, Lua memory usage 1.94 MB. However, there are many more calls to NUMBEROF, for example in WP7 and its variations. It seems unlikely that there would be sufficiently many more calls to get the Lua time to 3.87 seconds but I don't know. Johnuniq (talk) 00:00, 13 May 2020 (UTC)
 * Hmm I added the WP7's to the test page and it jumped to 3 seconds, so that is it. I'm thinking the file is loaded from Commons with each call to the Wp7 template. -- Green  C  00:16, 13 May 2020 (UTC)
 * Actually there are 309 instances of Wp7? and if each have a dozen or so instances of NUMBEROF which is around 3,708 .. and if 422 = 0.273 seconds then "((3708/422)+1)*0.273" = 2.67 seconds which is not far from 3 seconds. --  Green  C  00:28, 13 May 2020 (UTC)
 * The file from Commons should only be loaded once per page, regardless of what WP7 is doing. Johnuniq (talk) 04:20, 13 May 2020 (UTC)

en2.wikipedia
There is no en2.wikipedia.org in use on enwiki (and I doubt anywhere else). I'd like to remove this part of the code it no longer needs to be supported. -- Green  C  14:26, 13 May 2020 (UTC)
 * Sure, if you want. However, it can be handy to have a way to display what the data file contains for enwiki at enwiki. If en2 is removed, there is no easy way of doing that. I was thinking of incorporating the little dump program (output currently here) into the main module so the whole data file could be shown in a sandbox, in preview if wanted. That sort of debugging is sometimes useful. If you still want to remove en2, either do it or confirm here and I'll do it. Johnuniq (talk) 00:47, 14 May 2020 (UTC)
 * No problem if there is a use for it. I have not been copying it to other languages. -- Green  C  01:23, 14 May 2020 (UTC)
 * I looked at an example of what you've been doing: bg:Module:NUMBEROF. Hey, better not edit war with bg:User:PSS 9 because I see that it is an admin bot run from meta. It reverts and escalates to blocking opponents! Very interesting. In the module you use af instead of en for the comments. From my work at hiwiki and bnwiki I understand the reason for that. I'm happy to put that along with your other changes in my personal repo so everything is synced. I'll remove en2 as you've done—it can be added if ever needed. You think the comment at the top of Module:NUMBEROF should be removed? Do you really want that "Load data..." comment? Right before code with  and the name of data module? That's great work at keeping the other projects up to date. Johnuniq (talk) 02:45, 14 May 2020 (UTC)
 * Yeah I didn't realize bg:User:PSS 9 was a bot so left it alone after the second time. The inline .af .en etc.. comments I got tired of localizing them, some I did, some didn't, doesn't seem critical. It's a lot of work, like 6 to 7 pages and customizations, no two seem to go alike, kind of a brain burner to get all the details right, the doc pages are the hardest. The comment at the top doesn't hurt but is probably redundant with the /doc information. Yeah the extra load data comment is redundant and can be removed. Currently using af:Module:NUMBEROF as the template for copying. -- Green  C  14:53, 14 May 2020 (UTC)
 * Please don't localize the comments. I've transwikied convert to several places and the more sites are the same, the better (apart from necessary localization of messages). An edit summary like  would be ideal. I updated Module:NUMBEROF here so it is the same as afwiki except that I kept the two-line introductory comment at the top. That can be omitted elsewhere. Johnuniq (talk) 01:47, 15 May 2020 (UTC)

@GreenC: By the way, I noticed a couple of other Wikipedias where you changed  by replacing   with the local name. Are you aware that is not needed? In the interests of keeping the modules at different sites as similar as possible for easier updates if needed in the future, I leave terms like Module unchanged for convert. Johnuniq (talk) 03:13, 17 May 2020 (UTC)
 * I think I only did that when the template name itself changed. I don't recall exactly but generally don't change it. -- Green  C  03:28, 17 May 2020 (UTC)

Conversion notes
This is public record/notes concerning roll-out of the template to other wikis


 * At cawiki, EstaBot maintaining the data file. A message was sent concerning the existence of this template and offer of help install.
 * At eswiki, JemBot is updating data once-daily. Not notified about new template.
 * At eowiki, template is pending review.
 * At hywiki, ԱշբոտՏՆՂ (local bot) is running. Not yet notified or installed.
 * At itwiki, ItwikiBot is running. Not installed. Notified about new template on talk page.
 * At krwiki, unable to install
 * At ruwiki, Lua Module:NumberOf - Data is stored in Module:NumberOf/today updated daily by MBHbot
 * At tgwiki, Lua Module:NumberOf - Модул:NumberOf/today not kept up to date. Template has 1 transclusion.
 * At trwiki, YBot is running. Not installed. Not notified.
 * At tkwiki, template is edit protected, requested change.
 * At ukwiki, Lua Module:NumberOf - Data is stored in Module:NumberOf/today updated daily by MBHBot.
 * At urwiki, ShumariyatBot updating once a day. Not notified or installed.

All wikis processed successfully with special cases noted above. -- Green  C  19:57, 17 May 2020 (UTC)

Some wikis have dedicated bots updating statistics internally in articles: example, example. -- Green  C  23:54, 18 May 2020 (UTC)

Bot on Github
Bot is on Github, template docs updated with link. -- Green  C  16:47, 9 June 2020 (UTC)

Out of memory
ckb:Template:Largest Wikipedias/graph is generating red errors from Lua memory exceeding 50MB. Some relevant looking debug info from the HTML is below when the new versions of NUMBEROF and 'Wikipedia rank by size' were in place. I restored the non-Lua version of ckb:Template:Wikipedia rank by size but that didn't solve it. I then restored ckb:Template:NUMBEROF to its non-Lua version and same problem. Whatever it is not directly related to the new Lua code, but it would be nice to be able to determine the cause. I wonder if it's related to ckb:Template:Wikipedia rank by size/WP somehow. There is an opened discussion ckb:Template talk:Largest Wikipedias/graph. As of this post the non-Lua versions are in place but they can be easily reverted back. -- Green  C  13:55, 11 June 2020 (UTC)


 * Lovely, a puzzle on a right-to-left wiki, my favorite:) It looks like you're right that there was some pre-existing problem but I'll examine it although unless I get very lucky it won't be for a day or two. Johnuniq (talk) 23:21, 11 June 2020 (UTC)
 * Experimenting with parts of that page shows that the following is sufficient to generate the "not enough memory" error:
 * Previewing that shows that only the following are used: ckb:Module:Arguments + ckb:Module:Language/name + ckb:Module:Language/name/data. I'll pursue it more later. Johnuniq (talk) 00:49, 12 June 2020 (UTC)
 * You found the problem. ckb:Module:Language/name/data is fubar. I copied the version from enwiki and that seems to have solved it. ckb:Module:Language and ckb:Module:Language/name are already in sync with enwiki. (BTW I wonder how they are kept in sync? I could use it for NUMBEROF. ) -- Green  C  01:54, 12 June 2020 (UTC)
 * The original /data file contained a call to mw.loadData to itself creating a recursive loop, may have been the problem. --  Green  C  03:49, 12 June 2020 (UTC)
 * Good work, recursion would definitely cause that problem. Johnuniq (talk) 06:13, 12 June 2020 (UTC)
 * Hello and ! I REALLY THANK YOU BOTH! You both helped us more and we will appreciate it! This also fixed the pie chart errors (See ckb:ویکیپیدیا). Thank you both again! ⇒  Aram  Talk  11:39, 12 June 2020 (UTC)
 * , you are very welcome. Glad it worked out. -- Green  C  02:18, 13 June 2020 (UTC)
 * , you are very welcome. Glad it worked out. -- Green  C  02:18, 13 June 2020 (UTC)

Version 2 completed
Version 2 of NUMBEROF has been installed on ~60 language wikis, even if they don't immediately need the new ranking feature. This will ensure old versions are not confusing things. About 24 wikis had the Lua version of Wikipedia rank by size installed. Full documentation updated on enwiki and metawiki, with briefer versions elsewhere. -- Green  C  23:47, 12 June 2020 (UTC)

Site site total
User:Johnuniq, looking at this sentence "Wikipedia articles have been created in 313 languages, with 303 active and 10 closed": The numbers could be derived from data.tab  What do you think? -- Green  C  16:49, 4 September 2020 (UTC)
 * Groan, I suppose the module could handle another calculation. I think 313 is counted by the number of rows in c:Data:Wikipedia statistics/data.tab where the site contains ".wikipedia", excluding all "total." rows? How are 303 or 10 counted? Johnuniq (talk) 00:12, 5 September 2020 (UTC)
 * Ah shoot I linked the wrong tab it would be c:Data:Wikipedia statistics/config.tab .. but this is no longer updated as the bot now generates the list of sites internally. It could be updated agaon, in which case it would be another tab for Lua to load, which is probably not worthwhile for a singular use case. Unless Lua doesn't load the tab unless needed, I forget how it works. --  Green  C  02:45, 5 September 2020 (UTC)
 * OK, I'll think about it soon. We have NUMBEROF and Wikipedia rank by size which call different functions in Module:NUMBEROF. There could be a third template calling a different function. I suppose you want counts for each project so "wikipedia" or "wikibooks" etc. would be needed as a parameter, defaulting to "wikipedia". We could modify NUMBEROF so it accepts additional identifiers for parameter 1: languages, active, closed. If one of those was used, parameter 2 would be, for example, "wikipedia" not "en.wikipedia". That would be easier for maintenance (no new template) but the documentation would be a little more complex. Thoughts? Johnuniq (talk) 04:02, 5 September 2020 (UTC)

I decided it was a good idea so: Now we just need c:Data:Wikipedia statistics/config.tab updated. Johnuniq (talk) 11:19, 5 September 2020 (UTC)
 * → (error)
 * → (error)
 * → (error)
 * Perhaps I shouldn't overthink this but now I'm wondering whether is a bit cryptic. Perhaps the "wikipedia" parameter should be required (no default).  reads better and the template is never used frequently. Thoughts? Johnuniq (talk) 03:39, 6 September 2020 (UTC)

That could also make documentation a little easier. And updated config.tab it has 303 "active" wikipedia entries. tools.botwikiawk@tools-sgebastion-07:~/numberof$ grep active datac.tab | grep wikipedia | wc   303     303    9581 ..confirmed the template says 303. Excellent. -- Green  C  03:56, 6 September 2020 (UTC)
 * Added new docs Template:NUMBEROF/doc and updated The List . -- Green  C  04:13, 6 September 2020 (UTC)

John, there is a naming profusion. The tab is called config.tab (the copy on Commons is not actually used as a config file). The Module is called /count. And the docs call it "Meta statistics". I'm partial to meta stats since that differentiates it from other stats. Suggest meta.tab and Module /meta .. what do you think? -- Green  C  05:06, 6 September 2020 (UTC)
 * OK, I'm happy with any short preferably one-word name. Can you create meta.tab and leave config.tab in place? Then reply here and I will update the module and make parameter 2 required with no "wikipedia" default. Johnuniq (talk) 06:49, 6 September 2020 (UTC)
 * meta.tab is ready. -- Green  C  13:41, 6 September 2020 (UTC)
 * OK, it's all done, and parameter 2 is required. Johnuniq (talk) 03:12, 7 September 2020 (UTC)

I sometimes need to view what structured data looks like in Lua and have enhanced Module:Dump so a preview of something like  works. See demo in User:Johnuniq/sandbox (permalink). Johnuniq (talk) 09:58, 7 September 2020 (UTC)
 * Nice! -- Green  C  14:21, 7 September 2020 (UTC)

Wikiquote [article]
Is now a frequent invoker of +. Turning out to be useful in unexpected ways like the list of top 10 sites. Plan to do similar for other project articles. -- Green  C  15:35, 6 September 2020 (UTC)

Wikiversity and meta.tab
User:Johnuniq: Wikiversity does not work with meta.tab

The rest work:

I've looked at the code and JSON carefully and can not figure out why it would different. Only thought might be a 0 vs 1 starting offset error, but since it's associative arrays it wouldn't matter. Added meta.tab to User:Johnuniq/sandbox and it looks fine. -- Green  C  00:18, 11 September 2020 (UTC)
 * It's some bug related to the fact that wikiversity has no closed projects. I'll fix it very soon. Johnuniq (talk) 00:35, 11 September 2020 (UTC)
 * Done. For the record, wikiversity was showing -1 for active because it had no closed entries and that led the module to believe the project was not valid. Johnuniq (talk) 00:58, 11 September 2020 (UTC)
 * Great. Thanks! -- Green  C  01:50, 11 September 2020 (UTC)

Status
User:Johnuniq: I tried to add a new feature that returns the site status: But it doesn't work. Would be "active" or "closed". The code looks pretty simple for this not sure where it went wrong. This is for the last column in the table at Wikivoyage. -- Green  C  03:28, 11 September 2020 (UTC)
 * I'll think about it soon but it's a completely different operation. Johnuniq (talk) 03:37, 11 September 2020 (UTC)
 * Oh yeah I see, it needs to parse the json by language and project.   --  Green  C  03:53, 11 September 2020 (UTC)
 * I've been distracted here so "soon" will be another 24 hours. Johnuniq (talk) 11:42, 11 September 2020 (UTC)

Johnuniq, was experimenting how to generate the table automatically using and found a solution which is visible at Wikivoyage. Really awesome to create the table full auto sorted by rank, never done before and with wide application. However because it uses, it looks like /other is only including active sites ie. the 21 active entries. For example, if the for loop is increased from 21 to 22, it gives a "-1" error on the last closed site pswiki (Pashtun). Example:

Produces:

It says count ie. and  is the for-loop counter (1..22). It works fine by changing  to   which is 21.

So, I had to create a second table for closed sites, and manually add a record for pswiki (Pashtun). Thus, the need for a status argument is immediately no longer required. However, it would be great if there was a way to designate active, closed or active+closed when doing rankings, to get all in one table. At which point the status column and feature would be needed again. -- Green  C  15:21, 11 September 2020 (UTC)
 * I'm still caught up elsewhere and will wait before looking at this. Johnuniq (talk) 10:11, 12 September 2020 (UTC)

Wikidata
Brought up by User:Yair rand we are missing Wikidata (API:Siteinfo). When I get a chance I'll add it to the bot and post here when done. -- Green  C  14:57, 23 February 2022 (UTC)



It now works. It would be nice to have aliases for these one-off sites:
 * www.wikidata
 * meta.wikimedia
 * commons.wikimedia
 * foundation.wikimedia
 * wikimania.wikimedia
 * wikitech.wikimedia

Thus one could say  or   is more natural language. Wikidata is a bit different as the alias would be the right part of the dot while the others on left, static definition required. What do you think User:Johnuniq? -- Green  C  15:51, 23 February 2022 (UTC)
 * Would this involve a tweak to the module to add aliases? I'm happy to do that and certainly "wikidata" rather than "www.wikidata" should be supported. I think a magic table of aliases built-in to the module would be ok. Johnuniq (talk) 00:01, 24 February 2022 (UTC)
 * Sure that sounds good! -- Green  C  03:22, 24 February 2022 (UTC)

Please test the sandbox template as below. Normal Using alias If that is all that is needed, I will update the main module. Johnuniq (talk) 03:23, 25 February 2022 (UTC)
 * Did you see this? Johnuniq (talk) 04:04, 26 February 2022 (UTC)
 * Hi. Yes that's great. Much better that way. -- Green  C  04:16, 26 February 2022 (UTC)
 * Docs now updated. --  Green  C  04:22, 26 February 2022 (UTC)
 * And Module:NUMBEROF is updated, so I removed "/sandbox" from the examples above. Johnuniq (talk) 04:31, 26 February 2022 (UTC)

Module:NUMBEROF on Meta
https://meta.wikimedia.org/wiki/Module:NUMBEROF

Someone made some major changes on Meta (Jan 2021). No idea if these are worthwhile, but, it is concerning as we now have a significant fork in the code, which makes keeping the code updated across wikis nearly impossible. For example, the above new alias feature. Meta is a common location for other wikis to retrieve code from so it's likely to get propagated. Such is the problem where every wiki is essentially its own fork and there is not git-like mechanism to keep things in sync. -- Green  C  04:49, 26 February 2022 (UTC)
 * Not much we can do about that. I had a quick look at the difference and don't see anything useful. Some of the claimed optimizations are not helpful because there is only a single call. You could try overwriting the code with the current version from here but that would be reverted. Might be worthwhile anyway. Johnuniq (talk) 06:23, 26 February 2022 (UTC)
 * OK. Most of it is optimizations. I'll try to set up some tests for the biggest page. The results of that will determine what to do next. There are pros and cons to those changes and need to see if the pros outweigh the cons. -- Green  C  16:01, 26 February 2022 (UTC)
 * No, there are no pros. Johnuniq (talk) 02:25, 27 February 2022 (UTC)

The code at meta returns 0 for an unknown project (see "xy" below). That is a defensible choice but it's new behavior. I don't know why, but the depth calculation also gives 0. In the following, the meta results use the current modules which were last edited in 2021.

I have examined the differences between enwiki and meta and the changes there are not needed. There is no reason to cache a result for performance when that result is used once. The changes obfuscate the code. I will make a trivial edit at Module:NUMBEROF/other and Module:NUMBEROF/rank to match one change at meta. I will then copy the enwiki modules to meta which will restore them to how they were with the new alias feature, including returning -1 for an unknown project. Johnuniq (talk) 02:25, 27 February 2022 (UTC)


 * OK. Let me know if you need help.
 * Re: -1 we discussed it 18 months ago, above, to use "-1 to indicate an error because 0 is a legitimate value in the data". And not a string ("error") to avoid localization complexity. Verdy said in this edit "return 0, not -1, for missing data (otherwise gives some bad results in sums)". Even still, -1 symbolically means error, and errors should be checked for prior to doing sums. Otherwise how do you know if a site is missing vs. one that exists but has 0 edits. --   Green  C  03:09, 27 February 2022 (UTC)
 * I have updated the meta modules and left an explanation at m:Module talk:NUMBEROF. Please let me know if you notice a response and I'll join in. The depth calculation shown above now returns 250 at meta. Johnuniq (talk) 03:33, 27 February 2022 (UTC)
 * Great. Now monitoring m:Module:NUMBEROF for changes will get an alert. The talk page they'll need to ping. -- Green  C  05:42, 27 February 2022 (UTC)
 * FYI, a post at m:Module talk:NUMBEROF explains that multilanguage wikis should use a more elaborate (with higher overhead) method of determining the language code. The point being that it appears to be unfortunately inevitable that meta will have a slightly different version of the module, and maybe one or two other places such as Commons if anyone sets it up there. Johnuniq (talk) 23:31, 1 March 2022 (UTC)

Synchronizer
The Module and sub-pages can be maintained across many wikis with Synchronizer. -- Green  C  16:27, 9 September 2023 (UTC)


 * It turns out certain sub-page names are reserved: /doc, /XML, /meta, /sandbox, /testcases and /TemplateData. These reserved names are disallowed on WikiData from having items created since they are not considered notable pages. Without an item, the Syncronicity program won't work [for that page]. We accidentally created Module:NUMBEROF/meta not realizing it is a reserved name on Wikidata - if it was /feta it would be OK. I'm pleading with an admin on Wikidata to create the item anyway since this is not a normal /meta sub-page but a Lua module part of the template, the sort of thing we normally have items for. But so far no luck. That leaves renaming in the lua modules. I am not ready to update everything, since all it gains is the ability to run Syncronicity on /meta and that page is stable cross wiki. Maybe next time we need to do a major change cross wiki this can be included. --  Green  C  01:09, 10 September 2023 (UTC)

numbers with gaps
how can one get the number with gaps instead of commas? ThurnerRupert (talk) 08:05, 10 December 2023 (UTC)
 * The template uses the MediaWiki API which follows the convention for the project where it is used. Using it here will show commas but using it, for example, at no: would show gaps (just spaces, I think). Johnuniq (talk) 08:25, 10 December 2023 (UTC)

active admins
how could one get the number of active admins, i.e. an admin which is active user at the same time. ThurnerRupert (talk) 08:05, 10 December 2023 (UTC)
 * I'm pretty sure that cannot be done by a template. See List of administrators where, I think, the number of active administrators is updated by a bot. Am I right in thinking this is not handled by your Commons data? I don't think it would be useful to add it because the information is available at the link in this comment. Johnuniq (talk) 08:30, 10 December 2023 (UTC)

This tool doesn't track individual user names, only counts of users. Thus not possible to compare the list of active users with the list of admins to see where they intersect. It would require a separate tool that retrieves the lists of names, then does a list intersection.

You could contact User:JamesR who maintains WP:ADMINSTATS. There is already which has sub-pages like Template:Adminstats/Adam_Bishop, which is kept updated by User:Cyberbot I ie. User:Cyberpower678. An additional field for "last edit" might be easy to generate. In any case, as noted by Johnuniq, there is already List of administrators generated by a bot written by User:Rick Block. What would be needed is way to count those lists, post the counts somewhere probably as JSON, and a template to display the counts. It would probably make most sense for Rock Block to do the counting as part of the 'List of administrators' bot process, rather than somebody else counting the lists with another bot. -- Green  C  18:42, 10 December 2023 (UTC)