# Script guide and workflow

## End-to-end flow (build → disasm → pseudo → func_map → evaluation)
1) Build binaries at all O levels  
   - `scripts/build-host-opt-levels.sh` runs `TARGET=host run-tests` with `-O0…-O3`, renaming outputs to `*.host.O<opt>`.
2) Emit objdump listings (`*.s`)  
   - `OBJDUMP=/path/to/objdump JOBS=<N> scripts/disasm-all-objdump.sh` walks all `*.O0`~`*.O3` binaries and writes sibling `*.s`.
3) Dump IDA pseudocode (`*.pseudo`)  
   - `IDA_BIN=/path/to/idat scripts/decompile-all-pseudo.sh` calls `idat -A -S dump_pseudo.py` for each binary.
4) Build func maps (`*.func_map.jsonl`)  
   - `python3 scripts/build-func-maps.py` (or `--binary <path>`). Requires the `.pseudo` + `.s` files and `clang-format`.
5) Batch evaluation  
   - Example: `python3 scripts/eval_infer_out.py reports/SK2Decompile/merged.O2.func_map.infer.jsonl --target host --jobs 8`. The script currently replaces functions with `case["pseudo_normalize"]` (IDA-style content). To evaluate SK2 outputs instead, change both occurrences of that field in `scripts/eval_infer_out.py` (around lines 250 and 375) to `case["infer-out-model2"]` and rerun. Outputs land in `reports/infer_out_eval/`.

### Report locations
- `reports/SK2Decompile/`: evaluation results for SK2-generated pseudocode at each O level.
- `reports/IDA-Pro/`: evaluation results for IDA/Hex-Rays pseudocode at each O level.

### Current evaluation snapshot (host, merged.func_map.infer, 2025-11-21)
Counts are shown as `success / rate`.

**SK2Decompile** (`reports/SK2Decompile/merged.O*.func_map.infer-host-*.{json,md}`)  
| O level | Buildable | Runnable |
| --- | --- | --- |
| O0 | 181 / 44.36% | 140 / 34.31% |
| O1 | 173 / 44.59% | 113 / 29.12% |
| O2 | 159 / 42.18% | 94 / 24.93% |
| O3 | 140 / 38.36% | 72 / 19.73% |
| Avg | — / 42.37% | — / 27.02% |

**IDA-Pro** (`reports/IDA-Pro/merged.O*.func_map.infer-host-*.{json,md}`)  
| O level | Buildable | Runnable |
| --- | --- | --- |
| O0 | 116 / 28.43% | 104 / 25.49% |
| O1 | 90 / 23.20% | 84 / 21.65% |
| O2 | 88 / 23.34% | 81 / 21.49% |
| O3 | 71 / 19.45% | 67 / 18.36% |
| Avg | — / 23.61% | — / 21.75% |

**SK2 vs. IDA (relative lift based on averages)**  
- Build rate: SK2 42.37% vs IDA 23.61% → +79.5% lift  
- Run rate: SK2 27.02% vs IDA 21.75% → +24.3% lift

## Key scripts (what they do)
- **build-host-opt-levels.sh**: Host builds at O0–O3, renames to keep all variants.
- **disasm-all-objdump.sh**: Batch `objdump -d` over O0–O3 binaries, writes `*.s` siblings.
- **decompile-all-pseudo.sh** + **dump_pseudo.py**: Batch IDA headless decompilation to `*.pseudo`.
- **decompile-with-mctoll.py**: Optional `llvm-mctoll -d` over host binaries, with bundled headers and macro defines.
- **build-func-maps.py**: Cross-links source, pseudo, and assembly per function into `*.func_map.jsonl`.
- **pseudo_token_stats.py**: Token stats/plots over `*.func_map.jsonl` pseudo content.
- **eval_infer_out.py**: Workspace-based build/test after function replacement. Default replacement body is `case["pseudo_normalize"]`; switch both occurrences to `case["infer-out-model2"]` (around lines 250 and 375) when evaluating SK2 outputs.