I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp).
It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency
Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test
The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including the growing number of third-party “proxy” or “cheap LLM API” services.
It supports: • OpenAI-compatible APIs (official + proxies) • Claude (via Anthropic) • Local endpoints (custom/self-hosted)
You can also self-host it with docker-compose. Config is clean, adding a new provider only requires a simple plugin-style addition.
Would love feedback, PRs, or even test reports from APIs you’re using. Especially interested in how some lesser-known services compare.
Comments URL: https://news.ycombinator.com/item?id=44413921
Points: 4
# Comments: 2
Article URL: https://www.economist.com/science-and-technology/2025/06/27/is-being-bilingual-good-for-your-brain
Comments URL: https://news.ycombinator.com/item?id=44406140
Points: 24
# Comments: 13
Comments URL: https://news.ycombinator.com/item?id=44390072
Points: 7
# Comments: 0
Article URL: https://phys.org/news/2025-06-tomatoes-galpagos-quietly-de-evolving.html
Comments URL: https://news.ycombinator.com/item?id=44380872
Points: 9
# Comments: 0