Testing if "bash is all you need"

Vercel and Braintrust compared three agent architectures for querying structured data: SQL databases, bash+filesystem, and a hybrid approach. SQL achieved 100% accuracy with minimal tokens; bash lagged at 53% accuracy but could verify results; the hybrid approach matched SQL while adding self-verification. The real lesson was that evals themselves require detailed iteration to surface ground-truth issues.