MarkItDown conversion
felan-markitdown vs felan-no-markitdown
Cost · minimize · Ratio of reduced sums · lower is better
+31.0%primary outcome · Ratio of reduced sums·+18.5% to +47.8%Case range (macro mean)
Costprimary · minimize · Ratio of reduced sums+31.0%Aggregate values: baseline $0.0555; candidate $0.0383 · case mean +31.8%; Case range (macro mean) +18.5% to +47.8%
Prompt tokenssecondary · minimize · Ratio of reduced sums+13.8%Aggregate values: baseline 21,569 tokens; candidate 18,603 tokens · case mean +14.9%; Case range (macro mean) -6.4% to +32.7%
Metrics
Arm values remain macro means across cases. The objective headline uses the ratio of reduced sums after per-case median trial reduction.
| Metric | felan-no-markitdownbaseline · 12/12 runs · 12/12 passed | felan-markitdowncandidate · 12/12 runs · 12/12 passed | Delta |
|---|---|---|---|
| Cost | $0.0139 | $0.0096 | -$0.0043 |
| Prompt tokens | 5,392 tokens | 4,651 tokens | -741 tokens |
| quality.passRate | 1 | 1 | 0 |
| Duration | 22795.5000 | 16107 | -6688.5000 |
| Uncached input tokens | 5,008 tokens | 3,849 tokens | -1,159 tokens |
| Cache-read input tokens | 768 tokens | 768 tokens | 0 tokens |
| Cache-write input tokens | 0 tokens | 0 tokens | 0 tokens |
| Output tokens | 371 tokens | 151 tokens | -219 tokens |
| Requests | 3.2500 | 2.7500 | -0.5000 |
Tests
TestBaseline (Median trials; Ratio of reduced sums)Candidate (Median trials; Ratio of reduced sums)vs baseline
markitdown-document-configuration$0.02083/3 passed$0.01693/3 passed+18.5%
Attempt 1$0.0208Passed$0.0169Passed+18.5%
Attempt 2$0.0803Passed$0.0234Passed+70.9%
Attempt 3$0.0201Passed$0.0148Passed+26.2%
markitdown-docx-extraction$0.00863/3 passed$0.00703/3 passed+18.8%
Attempt 1$0.0085Passed$0.0070Passed+17.6%
Attempt 2$0.0117Passed$0.0070Passed+40.3%
Attempt 3$0.0086Passed$0.0070Passed+18.8%
markitdown-pptx-extraction$0.01383/3 passed$0.00723/3 passed+47.8%
Attempt 1$0.0129Passed$0.0075Passed+42.1%
Attempt 2$0.0147Passed$0.0072Passed+51.2%
Attempt 3$0.0138Passed$0.0072Passed+47.8%
markitdown-xlsx-extraction$0.01243/3 passed$0.00723/3 passed+42.0%
Attempt 1$0.0086Passed$0.0072Passed+16.7%
Attempt 2$0.0144Passed$0.0072Passed+50.1%
Attempt 3$0.0124Passed$0.0072Passed+42.1%