Once you’ve trained your large language model on the entire written output of humanity, where do you go? Here’s Ilya Sutskever, ex-OpenAI, admitting to Reuters that they’ve plateaued: [Reuters] Th…
LLMs are quite impressive as chatbots all things considered. The conversations with them are way more realistic and almost as funny as the ones with the IRC markov chain my friend made as a freshman CS student.
Of course, out bot’s training data only included the IRC channel’s logs of a few years and the Finnish Bible we later threw in for shits and giggles. A training set of approximately zero terabytes in total.
LLMs are less a marvel of machine learning algorithms (though I admit they might play a part) and more one of data scraping. Based on their claims, they have already dug through the vast majority of publicly accessible world wide web, so where do you go from there? Sure, there are a lot of books that are not on the web, but feeding them in the machine is about as hard as getting them on the web to begin with.
Senior year of college, I took an elective seminar on interactive fiction. For the final project, one of my classmates wrote a program that scraped a LiveJournal and converted it into a text adventure game.
LLMs are quite impressive as chatbots all things considered. The conversations with them are way more realistic and almost as funny as the ones with the IRC markov chain my friend made as a freshman CS student.
Of course, out bot’s training data only included the IRC channel’s logs of a few years and the Finnish Bible we later threw in for shits and giggles. A training set of approximately zero terabytes in total.
LLMs are less a marvel of machine learning algorithms (though I admit they might play a part) and more one of data scraping. Based on their claims, they have already dug through the vast majority of publicly accessible world wide web, so where do you go from there? Sure, there are a lot of books that are not on the web, but feeding them in the machine is about as hard as getting them on the web to begin with.
Senior year of college, I took an elective seminar on interactive fiction. For the final project, one of my classmates wrote a program that scraped a LiveJournal and converted it into a text adventure game.