Software Engineer III

Babel Street is the trusted technology partner for the worlds most advanced identity intelligence and risk operations. We deliver advanced AI and data analytics solutions providing unmatched analysis-ready data regardless of language proactive risk identification 360-degree insights high-speed automation and seamless integration into existing systems. Babel Street empowers government and commercial organizations to transform high-stakes identity and risk operations into a strategic advantage. The actionable insights we deliver safeguard lives and protect critical assets around the world. Babel Street is headquartered in Reston Virginia with regional offices in Boston MA and Cleveland OH and international offices in Australia Canada Israel Japan and the U.K. For more information visit .

ROLE SUMMARY:

The primary purpose of this Software Engineer III position is to architect develop andmaintainadvanced automated data extraction systems (spiders) that harvest critical business intelligence from complex web environments. This role ensures the continuous reliable flow of high-quality data into our internal databases by engineering solutions to overcome sophisticated technical barriers and anti-bot security measures. Candidates near our Somerville MA or Reston VA offices are preferred but remote work it potential for the ideal candidate.

ROLE FOCUS;

As a Software Engineer III specializing in data extraction you willbe responsible forthe end-to-end lifecycle of web-based data collection. This includes designing scalable crawling architectures reverse-engineering web applications toidentifydata points and implementing evasion techniques to bypass IP rate-limiting and bot detection. You will also manage the storage and integrity of this data using advanced SQL and relational database management.

We are looking for a Senior Python Engineer with a hacker mindset to join our team as a Software Engineer III. This role is dedicated to large-scale web scraping and data harvesting. If you have deep experience with Scrapy or Playwright know how to defeat Cloudflare orDataDome and can write high-performance SQL to manage millions of records we want to hear from you. This is a specialized role for an engineer who enjoysreverse-engineeringthe web to unlock data.

The primary purpose of this Software Engineer III position is to architect develop and maintainadvanced automated data extraction systems (spiders) that harvest critical business intelligence from complex web environments. This role ensures the continuous reliable flow of high-quality data into our internal databases by engineering solutions to overcome sophisticated technical barriers and anti-bot security measures.

Responsibilities:

• Spider Development: Design and deploy robust distributed spiders and crawlers to extract data from a variety of web architectures (SPAs SSR etc.).
• Bot Evasion Engineering: Research and implement strategies to bypass anti-scraping technologies including proxy rotation browser fingerprinting and CAPTCHA solving.
• Database Management: Create andoptimizeSQL schemas for large-scale data storage and perform complex data transformations and validation.
• System Maintenance: Proactively monitor the health of extraction agents and refactor code quickly in response to target website updates or layout changes.
• Performance Optimization: Utilize asynchronous Python programming to maximize the throughput and efficiency of data collection pipelines.

Requirements:

• Advanced Python: Mastery of Python 3.x with deep experience in extraction frameworks (Scrapy Playwright Selenium or Puppeteer).
• Technical Resilience: Proven ability to bypass high-level bot detection (e.g. Cloudflare Akamai orPerimeterX).
• Database Mastery: Expert-level SQL skills and experience managing relational databases like PostgreSQL or MySQL.
• Network Proficiency: Expert understanding of HTTP/S TCP/IP TLS fingerprinting and browser-header manipulation.
• Problem Solving: A specialized ability to reverse-engineer JavaScript-heavy websites and hidden API endpoints.
• Able to write debug and deploy complex Python code in a distributed environment.
• Must be able to analyze and interpret complex web structures and network traffic using browser developer tools.
• Ability to design and maintain relational database tables containing millions of rows.
• Able to pivot and respond quickly to technical break-fixes to ensure data continuity for the business.
• Collaboration with data analysts to define and validate data requirements and output formats.

Education & Experience:

• Bachelors degree in Computer Science Information Systems or a related field (or equivalent professional experience).
• Minimum of 5 years of experience in Software Engineering with at least 23 years focused specifically on large-scale web scraping or data extraction.

Benefits at Babel Street (just to name a few...)

• Health Benefits: Babel Street covers 85-100% monthly premium costs for Medical Dental Vision Life & Disability insurances for you and your family!
• Retirement Plans:Babel Street offers both a Traditional and Roth 401(K) with a very competitive match.
• Unlimited Flexible Leave: We trust our employees to manage their own time and balance their personal and work lives.
• Holidays: Babel Street provides employees with 12 paid Federal Holidays
• Tuition Reimbursement: We are committed to investing in our employees. One way we do that is with our Tuition Reimbursement Program for continuing education.

Babel Street is anequalopportunity/affirmative actionemployer. All qualified applicants will receive consideration foremploymentwithout regard to sex gender identity sexual orientation race color religion national origin disability protected Veteran status age or any other characteristic protected by law. FurtherBabel Street will not discriminate against applicants for inquiring about discussing or disclosing their pay or in certain circumstances the pay of their coworkerPay Transparency Nondiscrimination.In addition Babel Streets policy is to provide reasonable accommodation to qualified employees who have protected disabilities to the extent required by applicable laws regulations and ordinances where a particular employee works. Upon request we will provide you with more information about such accommodations.