AI Blog · AI 博客 · MAE

AI Blog

Essays · Writers
Lil'Log · Lilian Weng Harness Engineering for Self-Improvement The concept of recursive self-improvement (RSI) dates back to I. J. Good (1965), where he defined an… 138 天前 Lil'Log · Lilian Weng Scaling Laws, Carefully Scaling laws are one of the most critical empirical findings in deep learning. The observation is si… 148 天前 Lil'Log · Lilian Weng Why We Think Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post. … 567 天前 Lil'Log · Lilian Weng Reward Hacking in Reinforcement Learning Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the … 721 天前 Lil'Log · Lilian Weng Extrinsic Hallucinations in LLMs Hallucination in large language models usually refers to the model generating unfaithful, fabricated… 865 天前 Lil'Log · Lilian Weng Diffusion Models for Video Generation Diffusion models have demonstrated strong results on image synthesis in past years. Now the research… 951 天前 Lil'Log · Lilian Weng Thinking about High-Quality Human Data [Special thank you to Ian Kivlichan for many useful pointers (E.g. the 100+ year old Nature paper “V… 1018 天前 Lil'Log · Lilian Weng Adversarial Attacks on LLMs The use of large language models in the real world has strongly accelerated by the launch of ChatGPT… 1121 天前 Lil'Log · Lilian Weng LLM Powered Autonomous Agents Building agents with LLM (large language model) as its core controller is a cool concept. Several pr… 1245 天前 Lil'Log · Lilian Weng Prompt Engineering Prompt Engineering, also known as In-Context Prompting, refers to methods for how to communicate wit… 1345 天前 Lil'Log · Lilian Weng The Transformer Family Version 2.0 Many new Transformer architecture improvements have been proposed since my last post on “The Transfo… 1392 天前 Lil'Log · Lilian Weng Large Transformer Model Inference Optimization [Updated on 2023-01-24: add a small section on Distillation.] Large transformer models are mainstrea… 1408 天前 Lil'Log · Lilian Weng Some Math behind Neural Tangent Kernel Neural networks are well known to be over-parameterized and can often easily fit data with near-zero… 1532 天前 Lil'Log · Lilian Weng Generalized Visual Language Models Processing images to generate text, such as image captioning and visual question-answering, has been… 1623 天前 Lil'Log · Lilian Weng Learning with not Enough Data Part 3: Data Generation Here comes the Part 3 on learning with not enough data (Previous: Part 1 and Part 2). Let’s consider… 1678 天前 Lil'Log · Lilian Weng Learning with not Enough Data Part 2: Active Learning This is part 2 of what to do when facing a limited amount of labeled data for supervised learning ta… 1733 天前 Lil'Log · Lilian Weng Learning with not Enough Data Part 1: Semi-Supervised Learning When facing a limited amount of labeled data for supervised learning tasks, four approaches are comm… 1810 天前 Lil'Log · Lilian Weng How to Train Really Large Models on Many GPUs? [Updated on 2022-03-13: add expert choice routing.] [Updated on 2022-06-10]: Greg and I wrote a shor… 1882 天前 Lil'Log · Lilian Weng What are Diffusion Models? [Updated on 2021-09-19: Highly recommend this blog post on score-based generative modeling by Yang S… 1957 天前 Lil'Log · Lilian Weng Contrastive Representation Learning The goal of contrastive representation learning is to learn such an embedding space in which similar… 1998 天前 Lil'Log · Lilian Weng Reducing Toxicity in Language Models Large pretrained language models are trained over a sizable collection of online data. They unavoida… 2069 天前 Lil'Log · Lilian Weng Controllable Neural Text Generation [Updated on 2021-02-01: Updated to version 2.0 with several work added and many typos fixed.] [Updat… 2147 天前 Lil'Log · Lilian Weng How to Build an Open-Domain Question Answering System? [Updated on 2020-11-12: add an example on closed-book factual QA using OpenAI API (beta). A model th… 2212 天前 Lil'Log · Lilian Weng Neural Architecture Search Although most popular and successful model architectures are designed by human experts, it doesn’t m… 2296 天前