Add AI Model Fallback Mechanism for Improved Reliability

[jelveh]

Well, we just got banned by Anthropic without any warning, which highlights the importance of being resilient when it comes to using 3rd party services.

Currently, when an AI model request fails (e.g., due to provider errors), the entire request fails. We should implement an automatic fallback mechanism to improve reliability and ensure requests succeed whenever possible.

Current Behavior

• When a model provider (e.g., OpenAI) returns an error, the request fails completely
• No automatic fallback to alternative models
• Users experience complete failure rather than degraded service

Proposed Solution
Implement an automatic fallback chain for AI model requests:

1. Try requested model first (e.g., claude-3-5-sonnet)
2. If that fails, fall back to gpt-4o mini
3. If gpt-4o mini fails, fall back to llama
4. Only fail completely if all fallback options are exhausted

Implementation Details

1. Response Enhancement:

   • Add a new response attribute (e.g., usedModel) indicating which model actually handled the request
   • This allows clients to know if a fallback was used

2. Configuration Option:

   • Add a new option to disable fallback behavior: disableFallback (default: false)
   • Example usage:

   puter.ai.chat("Hello!", {
     model: "claude-3-5-sonnet",
     disableFallback: true  // Disable automatic fallback
   })

3. Fallback Chain:

   • Define clear fallback hierarchy
   • Consider timeout/retry logic before moving to next fallback

Expected Behavior

// Example 1: With fallback enabled (default)
const response = await puter.ai.chat("Hello!", { model: "claude-3-5-sonnet" });
console.log(response.text);         // The response text
console.log(response.usedModel);    // e.g., "gpt-4o-mini" if Claude failed

// Example 2: With fallback disabled
try {
  const response = await puter.ai.chat("Hello!", {
    model: "claude-3-5-sonnet",
    disableFallback: true
  });
} catch (error) {
  // Original error from Claude is thrown
}