Talk:Information theory

Sidebar template for information theory articles
I propose creating a sidebar template similar to Template:Probability_fundamentals to better connect the information theory related articles. What do you think? Fvultier (talk) 22:34, 7 October 2018 (UTC)
 * This would work Limit-theorem (talk) 01:10, 8 October 2018 (UTC)

Good grief, have mercy on the Information Age
There are two classes of information theories: Statistical and Algorithmic. Scientific induction may start with statistical information but induction of *causal* models generally entails *algorithmic* information for the same reason the statistical model of the first billion bits of Pi are worthless for predicting the next bit of Pi compared to the shortest algorithm that calculates the first billion bits of Pi. This misunderstanding of the very concept of "information" has been inexcusable ever since Solomonoff published his papers on data-driven induction in the mid 1960s at the dawn of the information age. This article goes to an incredible length to obscure this first-order distinction. Jim Bowery (talk) 21:04, 23 August 2022 (UTC)

bits and binary digits
This article makes the classic mistake of confusing bits with binary digits. Clarification can be found here (see final 3 paragraphs): https://plus.maths.org/content/information-bits-and-20-questions Jvstone (talk) 10:21, 9 April 2023 (UTC)
 * That article's concluding paragraph admits that "Sadly, in modern usage, the terms bit and binary digit have become synonymous." That is simply a fact, not necessarily a mistake – bit is a term formed as a contraction of "binary digit". While some people may try to draw a distinction between what those two terms mean, most do not –  even in academic literature (as I suspect you know, since it seems you wrote that). Context or additional explanation is needed (or alternative terminology, such as using the term binary symbol instead of binary digit to draw the distinction, or talking about bits of data versus bits of entropy or bits of information). Many words have multiple meanings and depend on context for their exact interpretation. —⁠ ⁠BarrelProof (talk) 02:16, 10 April 2023 (UTC)