sks-website/src/app/codeindex/benchmarks/ at main · russ.fugl.dev/sks-website · Tangled

russ.fugl.dev / sks-website

0

build smart knowledge-sharing systems that combine the best of human expertise with smart technology

0

sks-website / src / app / codeindex / benchmarks /

at main 2 files

Russ T. Fugal fix(codeindex): fall back to placeholder Convex URL during build 2mo ago

opengraph-image.tsx

style(codeindex): apply prettier formatting to OG image components Pure reformat — line-wraps long string literals and JSX prop objects across the four codeindex opengraph-image.tsx files. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2 months ago

feat(codeindex): update benchmarks with 19-repo multi-language eval results Replace self-eval test fixture benchmarks with real public repo results: - 330 queries across 19 public repos (17 GitHub, 2 tangled.org), 16 languages - Per-language data now from representative repos (e.g. Kong/kong for Lua, kubernetes for Go, elasticsearch for Java) - Lua jumps from "broken/17%" to "excellent/82%" on real-world Kong codebase - Update multipliers from 3.1x (naive grep) to 1.4x (vs LLM-crafted expert grep patterns) — more rigorous baseline - Update cost from $0.17 to $0.70, file count to 100k+ - Add services section anchor, language tags on repo cards - Qualify commit-boost signal contribution (25.8% on self-eval, varies on broader corpus — investigating) - Update all marketing copy, metadata, and OG images to match Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

4 months ago