Meta Llama 3.1 8B on W&B Inference
Price per 1M tokens
$0.22 (input)
$0.22 (output)
Parameters
8B
Context window
128K
Release date
July 2024
Meta Llama 3.1 8B inference details
Llama 3.1 8B provides efficient multilingual conversational support ideal for applications where responsiveness and computational efficiency are critical. Effective for building chatbots, automated customer interactions, and applications needing fast yet reliable language understanding.
Created by: Meta
License: llama3.1
🤗 model card: Llama-3.1-8B-Instruct
import openai
import weave
# Weave autopatches OpenAI to log LLM calls to W&B
weave.init("<team>/<project>")
client = openai.OpenAI(
# The custom base URL points to W&B Inference
base_url='https://api.inference.wandb.ai/v1',
# Get your API key from https://wandb.ai/authorize
# Consider setting it in the environment as OPENAI_API_KEY instead for safety
api_key="<your-apikey>",
# Team and project are required for usage tracking
project="<team>/<project>",
)
response = client.chat.completions.create(
model="meta-llama/Llama-3.1-8B-Instruct",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me a joke."}
],
)
print(response.choices[0].message.content)
Llama 3.1 resources
W&B Inference powered by CoreWeave guide
AI engineering course: Agents
