Pulpie: A 20x Cheaper HTML Content Extractor That Matches SOTA

Feyn Labs has released Pulpie, a family of Pareto-optimal models for extracting main content from HTML pages. The smallest model, pulpie-orange-small (210M parameters), scores 0.862 ROUGE-5 F1 on WebMainBench — nearly identical to Dripper's 0.864 — but costs one twentieth as much to run. On an NVIDIA L4 GPU, it processes 13.7 pages per second versus Dripper's 0.68. Cleaning 1 billion pages would cost $7,900 with Pulpie versus $159,000 with Dripper.

The key architectural insight: Pulpie uses an encoder that labels every HTML block as content or boilerplate in a single forward pass, whereas Dripper is a decoder that emits labels one token at a time, tying speed to memory bandwidth. This makes Pulpie smaller, faster, and cheaper while maintaining quality.

Pulpie's pipeline runs in four stages: simplify HTML (remove scripts, styles), chunk blocks into 8,192-token segments, classify each block via a forward pass, and return the kept blocks as HTML or Markdown. About 80% of pages fit in a single chunk.

Training required building a labeled dataset from scratch. The team sampled 16,670 English pages from Common Crawl, labeled blocks using DeepSeek V3.2, then cross-checked with Dripper 0.6B, keeping 14,959 pages with ≥70% inter-labeler agreement. A teacher model (EuroBERT-2.1B) was fine-tuned on this data, scoring 0.873 ROUGE-5 F1. That teacher was then distilled into two smaller encoders: Pulpie Orange Base (610M) and Pulpie Orange Small (210M), using KL-divergence loss weighted 0.7 and hard-label cross-entropy at 0.3.

The distilled models retain nearly all the teacher's quality. The 210M model is within 1.1 F1 points of the 2.1B teacher, and it ties Dripper at a third the size. Dripper also fails on 135 pages in the benchmark (mostly due to context window overflow), while Pulpie handles arbitrarily long pages by chunking.

Data quality matters. The post cites AICC research showing that cleaner extraction from Common Crawl improved benchmark accuracy by 1.08 percentage points, beating heavily filtered corpora like FineWeb and RefinedWeb. At inference, a single irrelevant passage can derail a model's answer — so clean context is critical for both training and production.

Pulpie is open source and available on Hugging Face. For anyone doing large-scale web extraction — whether for pre-training data pipelines or RAG context management — this is a significant step forward in cost and throughput.