Anti-Bot Engineer (Remote, Full-Time)

About the role

Smart Working is looking for a senior Anti-Bot Engineer to design, build, and operate large‑scale web scraping systems that can reliably extract high‑quality data from heavily protected, constantly changing web environments. This remote, full‑time position focuses on creating production‑grade pipelines that survive adversarial anti‑bot measures and dynamic page rendering.

Key responsibilities

• Design, develop and maintain Python‑based scraping infrastructure targeting highly protected sites, including Google‑like environments.
• Implement resilient extraction solutions for JavaScript‑heavy pages using browser automation and hybrid techniques.
• Continuously adapt systems to frequent changes in page structure, request flows, and anti‑bot mechanisms.
• Build robust data pipelines with strong emphasis on accuracy, observability, fault tolerance, and cost efficiency.
• Deploy advanced strategies such as proxy rotation, fingerprinting, session management, and request routing to ensure stability at scale.
• Monitor system health, detect anomalies proactively, and debug complex production failures across distributed cloud environments.
• Collaborate with data engineering and product teams to validate and structure scraped data.
• Document architecture, scraping logic, and operational procedures for long‑term maintainability.

Required profile

• Several years of experience building and operating large‑scale distributed systems in production.
• Proven ability to own end‑to‑end mission‑critical data pipelines.
• Strong systems thinking and comfort working independently in ambiguous, adversarial environments.

Required skills

• Python programming
• JavaScript knowledge
• Browser automation tools (e.g., Selenium, Playwright)
• Proxy rotation and fingerprinting techniques
• Session management and request routing
• Experience with cloud environments and distributed systems