Inferock Bench

💬 نظرات

• کاربر ProductHunt 1auditing your own inference bill is such an obvious gap 🔍 nice one. seeing much overbilling in the wild yet?
• کاربر ProductHunt 2Congrats on the launch of Inferock Bench. Love the idea of having an independent receipt for every LLM API call. Transparent usage, billing, and overpayment tracking can be a huge win for teams working with multiple LLM providers.
• کاربر ProductHunt 3Retries are the part I’d put in giant font. ‘$0.04/call’ means nothing if a successful job secretly takes 6 calls. I want cost per accepted outcome.
• کاربر ProductHunt 4The failed-calls and retries breakdown is the part I'd use first. One case I keep hitting might not show up there: a tool call that returns 200 with a silently corrupted value. I measured this on Anthropic models, 40 calls, none flagged the value was wrong, so it bills as a clean success and the retry logic never fires. Can the receipt catch a call that looked fine but wasn't? Or is that out of scope by design?