Talk:GPT-2

GA criteria—focus
Right now, half(!) the body text is devoted to the background section. The article tries to give an overview of developments in AI over 70 years, which understandably takes a lot of space. But the article shouldn't try to do that. A short background section maybe 3 paragraphs could give relevant background on the problem that researchers are trying to solve, but the current version contains a great deal of extraneous detail. (t &#183; c)  buidhe  18:17, 24 April 2021 (UTC)


 * @JPxG I was around to start the GA Review but right now I would agree on @Buidhe opinion - as soon as you addressed this ping me please. Thanks. CommanderWaterford (talk) 09:38, 27 April 2021 (UTC)
 * Will do. jp×g 16:59, 29 April 2021 (UTC)
 * , Nudge as this issue doesn't seem to have been addressed yet. Maybe consider yanking the GAN template and putting it back when this is ready for a full review. (t &#183; c)  buidhe  06:59, 22 May 2021 (UTC)

OpenAI https://chat.openai.com/chat was released more than a week ago
OpenAI released a new version of GPT to everyone. Kazkaskazkasako (talk) 14:21, 8 December 2022 (UTC) ChatGPT

Background section
hey, ! Why do you think the long historical background should be there? It's not about gpt, it's not about llm, it's about history of ai and that belongs to different articles (as template on top of the section already says). We don't have long background sections in gpt-3, gpt-4 or PaLM article, but by this logic EVERY article should have a long history section. I agree that part about Transformers can be useful, but everything else is just redundant. Artem.G (talk) 10:22, 23 March 2023 (UTC)
 * The answer here consists of a few parts. Firstly, when I wrote this, it was the only place for this stuff to go; an article did not exist at Generative Pre-trained Transformer (GPT-3 and 4 didn't exist period). Now that a set index article exists for the whole series of models, it would probably make more sense for this to be there, which I will probably get around to doing, but at the time there was no such thing so this was the only place it could to.
 * Otherwise, the situation with GPT-3 and 4 is a bit strange and there is some history. OpenAI was founded as a non-profit to perform research that was, well, open (on the basis that humanity would benefit from neural network research being publicly accessible). This was true in 2019, which is why you can download source code and weights for GPT-2. In the last couple of years, they started having to weigh these noble principles against the prospect of making a bunch of money, which is why you cannot find much in the way of documentation for GPT-3 and 4; the sole technical publication for the latter is a 10-page-long "technical report" containing zero information on architecture, training or implementation. Because of this, the articles for those models (or "models" -- there's no actual evidence of what GPT-4 is, it could be its own model or a collection of models or an older model augmented by... etc etc) are necessarily scant and focus on surface-level details like what op-ed writers thought about them. Suffice it to say that the lengthy background section here is a historical artifact that I can deal with. jp×g 06:11, 24 March 2023 (UTC)
 * Thanks for detailed comment, I agree that a more general article like Generative Pre-trained Transformer would be a better place for intro and historical sections, though I still think that its place is in History of artificial intelligence (already slightly outdated) or maybe in Neural_network. Artem.G (talk) 07:10, 24 March 2023 (UTC)

Wiki Education assignment: Linguistics in the Digital Age
— Assignment last updated by Tell19 (talk) 08:28, 14 March 2024 (UTC)