6 min
Marketing AI benchmark: three public ones, what each measures, and how to pick
A marketing AI benchmark only helps if it tests your task. AD-Bench, the Creativity Benchmark and marketing-bench compared, with five checks to run.
LLM benchmark
Benchmarks and evaluation
Most marketing AI is benchmarked on an eval nobody outside the company can see. These posts are about measuring it properly, including the times our own measurements turned out to be wrong.
The measured evidence for this topic lives in marketing-bench.
3 posts
6 min
Benchmark contamination, and five other ways an AI score lies: what we found in our own
Benchmark contamination is one of six ways an AI score misleads. Each failure, a real example from our own benchmark, and the check that would have caught it.
7 min
LLM as a judge: the biases that skew AI evaluation, and the protocol we use against them
Using an LLM as a judge is fast and cheap, but judges favour certain positions, formats, lengths and their own writing. The research, and a protocol to copy.
Other topics
- AI ad platforms
What Meta, Google and TikTok are automating in advertising, what the announcements actually ship, and what that leaves for an independent model to check.
- Marketing LLMs and open models
Open-weights and small language models for marketing work: what exists, what they cost to run, and how they compare on a public benchmark.
- Ad creative and copy
Hooks, angles, ad copy and creative analysis: how to tag what an ad is doing, and what a model can and cannot learn about why it works.
- Fine-tuning and building models
When fine-tuning beats prompting and when it does not, with measured results: LoRA adapters, scaling curves, quantization and label quality.
- Platform specs and compliance
Safe zones, character limits, aspect ratios and ad policy: the rules that decide whether an ad renders correctly before anyone judges whether it is good.