文件夹: llm/2021

2021年12月01日

A General Language Assistant as a Laboratory for Alignment

2021年12月01日

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

2021年12月01日

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

2021年12月01日

Improving language models by retrieving from trillions of tokens (RETRO)

2021年12月01日

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

2021年12月01日

WebGPT: Browser-assisted question-answering with human feedback

2021年10月01日

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

2021年10月01日

Introducing Pathways: A next-generation AI architecture

2021年10月01日

Multitask Prompted Training Enables Zero-Shot Task Generalization (T0)

2021年10月01日

Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning

2021年9月01日

Finetuned Language Models Are Zero-Shot Learners (FLAN)

2021年9月01日

What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers

2021年8月01日

Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (ALiBi)

2021年8月01日

Jurassic-1: Technical Details and Evaluation (White Paper)

2021年7月01日

Evaluating Large Language Models Trained on Code (Codex)

2021年7月01日

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

2021年6月01日

CPM-2: Large-scale Cost-effective Pre-trained Language Models

2021年5月01日

Anthropic raises $124 million to build more reliable, general AI systems

2021年5月01日

CogView: Mastering Text-to-Image Generation via Transformers

2021年5月01日

LaMDA: our breakthrough conversation technology

2021年4月01日

Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM

2021年4月01日

PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation

2021年4月01日

RoFormer: Enhanced Transformer with Rotary Position Embedding

2021年4月01日

ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning

2021年3月01日

FastMoE: A Fast Mixture-of-Expert Training System

2021年3月01日

GLM: General Language Model Pretraining with Autoregressive Blank Infilling

2021年3月01日

M6: A Chinese Multimodal Pretrainer

2021年1月01日

AI Research 技术调研

探索

文件夹: llm/2021

A General Language Assistant as a Laboratory for Alignment

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

Improving language models by retrieving from trillions of tokens (RETRO)

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

WebGPT: Browser-assisted question-answering with human feedback

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Introducing Pathways: A next-generation AI architecture

Multitask Prompted Training Enables Zero-Shot Task Generalization (T0)

Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning

Finetuned Language Models Are Zero-Shot Learners (FLAN)

What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers

Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (ALiBi)

Jurassic-1: Technical Details and Evaluation (White Paper)

Evaluating Large Language Models Trained on Code (Codex)

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

CPM-2: Large-scale Cost-effective Pre-trained Language Models

Anthropic raises $124 million to build more reliable, general AI systems

CogView: Mastering Text-to-Image Generation via Transformers

LaMDA: our breakthrough conversation technology

Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM

PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation

RoFormer: Enhanced Transformer with Rotary Position Embedding

ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning

FastMoE: A Fast Mixture-of-Expert Training System

GLM: General Language Model Pretraining with Autoregressive Blank Infilling

M6: A Chinese Multimodal Pretrainer

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity