RTK context reduction
felan-rtk-optimizer vs felan-no-rtk-optimizer
Cost · minimize · Ratio of reduced sums · lower is better
+26.6%primary outcome · Ratio of reduced sums·+15.4% to +32.8%Case range (macro mean)
Costprimary · minimize · Ratio of reduced sums+26.6%Aggregate values: baseline $6.2598; candidate $4.5969 · case mean +24.1%; Case range (macro mean) +15.4% to +32.8%
Prompt tokenssecondary · minimize · Ratio of reduced sums+40.6%Aggregate values: baseline 7,240,443 tokens; candidate 4,302,107 tokens · case mean +34.4%; Case range (macro mean) +21.3% to +47.5%
Metrics
Arm values remain macro means across cases. The objective headline uses the ratio of reduced sums after per-case median trial reduction.
| Metric | felan-no-rtk-optimizerbaseline · 6/6 runs · 6/6 passed | felan-rtk-optimizercandidate · 6/6 runs · 5/6 passed; 1 failed · timeout | Delta |
|---|---|---|---|
| Cost | $3.1299 | $2.2985 | -$0.8315 |
| Prompt tokens | 3,620,222 tokens | 2,151,054 tokens | -1,469,168 tokens |
| quality.passRate | 1 | 1 | 0 |
| Duration | 583942.5000 | 587463 | +3520.5000 |
| Uncached input tokens | 120,318 tokens | 109,032 tokens | -11,286 tokens |
| Cache-read input tokens | 3,499,904 tokens | 2,004,288 tokens | -1,495,616 tokens |
| Cache-write input tokens | 0 tokens | 0 tokens | 0 tokens |
| Output tokens | 25,813 tokens | 23,673 tokens | -2,140 tokens |
| Requests | 43.5000 | 38 | -5.5000 |
Tests
TestBaseline (Median trials; Ratio of reduced sums)Candidate (Median trials; Ratio of reduced sums)vs baseline
prewalk-checkout$2.25373/3 passed$1.90672/3 passed; 1 failed · timeout+15.4%
Attempt 1$2.6695Passed$2.4292Timed out+9.0%
Attempt 2$2.0541Passed$1.9067Passed+7.2%
Attempt 3$2.2537Passed$1.8581Passed+17.6%
rtk-felan-project-instructions$4.00613/3 passed$2.69023/3 passed+32.8%
Attempt 1$4.0061Passed$3.2122Passed+19.8%
Attempt 2$4.6089Passed$2.6902Passed+41.6%
Attempt 3$3.9922Passed$2.2193Passed+44.4%