← All open vacancies

Backend developer · Lead / principal

Staff Software Engineer (Agentic Search) - Crawler

  • Amsterdam area
  • On-site
  • Permanent job
  • Salary not listed
  • Sponsors work permits
  • Posted 16-10-2026
  • English is enough

Technology asked for

  • Go
  • C++
  • Linux
••• is building an agent-native search platform that gives AI agents low-latency, observable, programmatic search APIs grounded in fresh real-world data. This Staff Engineer role sits in the content acquisition team, which owns the crawling and ingestion infrastructure feeding the search index. Main tasks: design, build and operate web-scale distributed crawlers that handle billions of URLs; create ingestion workflows for crawlers, structured feeds and partner sources; develop crawl scheduling, prioritisation, recrawl and freshness strategies; build URL discovery, deduplication and content extraction; keep the systems reliable under high throughput; define metrics for coverage, freshness and quality; track bandwidth and infrastructure cost; and work with indexing and ML teams. Must-haves: 8+ years building backend or distributed systems, strong Go or C++, experience with large-scale systems (10k+ RPS, billions of URLs), solid knowledge of HTTP, DNS and TLS, crawling and scraping, and experience running and debugging production distributed systems. Streaming pipelines, Kafka-like messaging and Spark or Flink are a plus. The process includes coding interviews.

See how well this fits you

A free profile shows your match with this vacancy and your permit route. Showing interest in it and being put forward by Kinit is part of Career Plus.