To comply with applicable regulations, starting on September 1st, 2025, our services will no longer be accessible from your location.

Learn More

Learn more

For more information or if you need help retrieving your data, please contact Weights & Biases Customer Support at support@wandb.com

Skip to content

Platform
Weave: AIOps

Traces
Explore and debug AI applications

Evaluations
Rigorous evaluations of AI applications

Playground
Explore prompts and models

Agents
Observability tools for agentic systems

Guardrails
Block prompt attacks and harmful outputs

Monitors
Continuously improve in production
Inference: Hosting

OpenAI OSS
GPT OSS 20B, GPT OSS 120B

Alibaba Qwen3
235B A22B, 235B Thinking, Coder 480B

Meta Llama
Llama 4 Scout, Llama 3.3 70B, Llama 3.1 8B

MoonshotAI Kimi
Kimi K2

Microsoft Phi
Phi 4 Mini 3.8B

Hangzhou DeepSeek
DeepSeek V3-0324, DeepSeek R1-0528
Models: MLOps

Experiments
Track and visualize your ML experiments

Sweeps
Optimize your hyperparameters

Tables
Visualize and explore your ML data
Core

Registry
Publish and share your ML models and datasets

Artifacts
Version and manage your ML pipelines

Reports
Visualize and explore your ML data

SDK
Log ML experiments and artifacts at scale

Automations
Trigger workflows automatically
Solutions
Use Cases

Train LLMs

Fine-tune LLMs

Computer Vision

Time Series

Recommender Systems

Classification & Regression

Industries

Autonomous Vehicles

Communications

Financial Services

Healthcare & Life Sciences

Public Sector

Scientific Research

Case Studies

Canva
Learn how Canva leverages W&B to deploy models

Microsoft
Learn how Microsoft uses W&B for their ML projects

Toyota
Learn how Toyota uses W&B for autonomous driving

OpenAI
Learn how OpenAI Robotics uses W&B for large scale ML
Enterprise
Security

Deployment

Performance

Partners

Support
Resources
AI Courses

AI Advisory Services

Blog

Articles

Podcast

Whitepapers

Events & Webinars

Press
Docs
Pricing

ko
ja
en
de

Current best practices for training LLMs from scratch

Introduction
The scaling laws
Hardware
Dataset collection
Dataset pre-processing
Pre-training steps
Model evaluation
Bias and toxicity
Instruction tuning
RLHF
Conclusion
References
Appendix

Introduction
The scaling laws
Hardware
Dataset collection
Dataset pre-processing
Pre-training steps
Model evaluation
Bias and toxicity
Instruction tuning
RLHF
Conclusion
References
Appendix

Current best practices for training LLMs from scratch

References

What Language Model Architecture and Pre-training Objective Work Best for Zero-Shot Generalization?
GPT-3 Paper – Language Models are Few-Shot Learners
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
OPT: Open Pre-trained Transformer Language Models
PaLM: Scaling Language Modeling with Pathways
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM
How To Build an Efficient NLP Model
Emergent Abilities of Large Language Models
Beyond the Imitation Game Benchmark (BIG-bench)
Talking About Large Language Models
Galactica: A Large Language Model for Science
State of AI Report 2022
Finetuned Language Models are Zero-Shot Learners
Scaling Instruction-Fine Tuned Language Models
Training Language Models to Follow Instructions with Human Feedback
Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques, and Tools
New Scaling Laws for Large Language Models by DeepMind
New Scaling Laws for Large Language Models
Understanding the Difficulty of Training Transformers

X-twitter Linkedin-in Youtube

The Platform

Weave
Traces
Evaluations
Playground
Agents
Guardrails
Online Evaluations
Models
Experiments
Sweeps
Tables
Inference
Registry
Artifacts
Reports
SDK
Automations

Weave
Traces
Evaluations
Playground
Agents
Guardrails
Online Evaluations
Models
Experiments
Sweeps
Tables
Inference
Registry
Artifacts
Reports
SDK
Automations

Article

What are AI agents?
What are AI guardrails?
LLM evaluations
What is RAG?
What is MLOps?
ML model registry
What is LLMOps?
What is reinforcement learning?

What are AI agents?
What are AI guardrails?
LLM evaluations
What is RAG?
What is MLOps?
ML model registry
What is LLMOps?
What is reinforcement learning?

Resources

Documentation
AI Courses
Articles
Blog and tutorials
Podcast
Events
Support

Documentation
AI Courses
Articles
Blog and tutorials
Podcast
Events
Support

Company

About us
Partners
Trust and security
Legal
Careers
Contact
Brand guide
Press

About us
Partners
Trust and security
Legal
Careers
Contact
Brand guide
Press

Copyright © Weights & Biases. All rights reserved.

Website terms of use
Privacy policy
English

Website terms of use
Privacy policy
English

Platform
Experiments
Lightweight experiment tracking

Reports
Collaborative dashboards

Artifacts
Dataset and model versioning

Tables
Interactive data visualization

Sweeps
Hyperparameter optimization

Models
Centralized model registry

Launch
Automated ML workflows

Prompts
LLMOps and prompt engineering

Monitoring
Observability for production ML

Weave
Interactive ML app builder
Solutions
Use cases

What are AI agents?

What are AI guardrails?

LLM evaluations

What is RAG?

What is MLOps?

ML model registry

What is LLMOps?

What is reinforcement learning?

Industries

What are AI agents?

What are AI guardrails?

LLM evaluations

What is RAG?

What is MLOps?

ML model registry

What is LLMOps?

What is reinforcement learning?
Ecosystem
What are AI agents?

What are AI guardrails?

LLM evaluations

What is RAG?

What is MLOps?

ML model registry

What is LLMOps?

What is reinforcement learning?

What are AI agents?

What are AI guardrails?

LLM evaluations

What is RAG?

What is MLOps?

ML model registry

What is LLMOps?

What is reinforcement learning?
Docs
Pricing
Enterprise
What are AI agents?

What are AI guardrails?

LLM evaluations

What is RAG?

What is MLOps?

ML model registry

What is LLMOps?

What is reinforcement learning?
Company
What are AI agents?

What are AI guardrails?

LLM evaluations

What is RAG?

What is MLOps?

ML model registry

What is LLMOps?

What is reinforcement learning?