Open Source OpenAI+Azure Fail Safe

OpenAI + Azure load balancer and fail safe for GPT requests

2 votes·2 comments·Nov 22, 2023

About

The OpenAI Load Balancer is a Python library designed to distribute API requests across multiple endpoints (supports both OpenAI and Azure). It implements a round-robin mechanism for load balancing and includes exponential backoff for retrying failed requests.