Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
Kiro
on July 10, 2024
|
parent
|
context
|
favorite
| on:
RouteLLM: A framework for serving and evaluating L...
I don't think that's a use case since you don't get rate limited when using the API.
Onawa
on July 10, 2024
|
next
[–]
We get rate limited when using Azure's OpenAI API. As a gov contractor working with AI, I have limited means for getting access to frontier LLMs. So routing tools that can fail over to another model can be useful.
fkyoureadthedoc
on July 10, 2024
|
parent
|
next
[–]
Same. Initially we just load balanced between various regions, ultimately bought some PTUs.
kordlessagain
on July 10, 2024
|
prev
[–]
Anthropic Build Tier 4: 4,000 RPM, 400,000 TPM, 50,000,000 TPD for Claude 3.5 Sonnet
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
