Big README
- Saw a "stupid" post saying internal FAQ at company will disapper given that we'll interact with internal knowledge mostly through LLM Assistant. Stupid because those assistant will still have to learn from something and this something is internal FAQ and documentations. It will have be be i) searchable so that assistant can query it (Retrieval augmented or with API) and ii) parsable so that it can be used from training/fine-tuning. Now, i) has already been the goal for long long time and ii) is pretty straighforward given that LLM can process a lot with limited pre-processing. A different question is if the Assistant will be able to generate such FAQ and it's likely that it'll help of lot to do so. Interesting view on dynamic FAQ/documentation as much of the knowledge will be generated by interactions with this assistant. Thus, one need to be able to query/summarize such interactions and progressively improve the FAQ from learnings with minimal system owner intervention if possible.
- Will likely start a project soon on democratic assembly of AI: given several LLM interacting with a given task in mind, what consensus system is optimal to reach best decision ? Dictature, democracy, fluid democracy ? Classical aggregation for boolean decisions (eg vote, stacking, bagging...) are a bit outdated for text data.
- ChatGPT+Book: Use babyAGI ideas to make GPT write a book from scratch on a given selected topic.
- Summarize ideas from Eliezer Yudkowsky podcast.
- AI, Innovation and Economics: I recall reading an article arguing China "weakened" software companies to prevent pool of talents to divert from real tech innovation (hardware, healthcare ...) to so-called "fake" tech innovation (ads, logistics, HR... ) as spearheaded by Meta, Amazon and co. Interestingly, LLM were possible thanks to funding from such companies to i) develop hardware and tools, ii) educate talents and motivate new ones to join the filed iii) provide useful market to AI in "early" days.
- I bet we'll all have our own trained Assistant in the next 5/10 years. Personalization is finally getting some tools to become interesting, after years where only multi-armed bandits were available. Tons of questions: lifelong learning, plugin, personnal training, feedback process.... One way I see this happening is through a LORA adaption of a shared LLM weight. Incoming
- List of Newsletter I read [30min/day]: Noahpinion [Econ, Society], DataMachina [AI/ML], TDLR [AI/Tech news], TDLR AI [AI/Science version of previous], The Economist [News/Society], Astral Codex Ten [Lot], Hackernewsletter [AI/Tech/Eng], Not Boring, Platformers, The Generalist, The Diff, Strange Loop Canon, Eric Newcomer, Dynomight, PunchCard Investor, EA Newsletter, Alex Kantrowitz from Big Tech, Dkb, Divisare Journal, Archilovers News, Holden Luntz Gallery Simple Pleasures
- What if GPT-4 is actually not that better ? More generally, does it matter or most of the innovation will be around connecting the core block to products ? Or specialization ?
- Could we switch from an API-connected world to a Chatbot-connected one ? Jere a generalist chatbot asks to experts chatbot or to search engine answers to question and aggregates results.
- Is the data generation bottleneck going to be solved by Chatbot generating artificial discussions and content. Derive economics using this data.
- Write a bit more on what this unlocks for personalization, finaly moving beyond old and boring recommander systems.