Software Development Engineer - Cloud Infrastructure
Software Development Engineer - Cloud Infrastructure
Posted on :
08-05-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
This is a remote position.
We are seeking a Software Development Engineer Cloud Infrastructure to join our team with expertise in cloud infrastructure Big Data and web crawling technologies. This role bridges system reliability engineering with scalable data extraction solutions ensuring our infrastructure remains robust and capable of handling highvolume data collection. You will design resilient systems optimize automation pipelines and tackle challenges posed by advanced botdetection mechanisms.
Responsibilities:
- Architect deploy and manage scalable cloud environments (AWS/GCP/DO) to support distributed data processing solutions to handle terabytescale datasets and billions of records efficiently.
- Automate infrastructure provisioning monitoring and disaster recovery using tools like Terraform Kubernetes and Prometheus.
- Optimize CI/CD pipelines to ensure seamless deployment of web scraping workflows and infrastructure updates.
- Develop and maintain stealthy web scrapers using Puppeteer Playwright and headless chromium browsers.
- Reverseengineer botdetection mechanisms (e.g. TLS fingerprinting CAPTCHA solving) and implement evasion strategies.
- Monitor system health troubleshoot bottlenecks and ensure 99.99% uptime for data collection and processing pipelines.
- Implement security best practices for cloud infrastructure including intrusion detection data encryption and compliance audits.
- Partner with data collection ML and SaaS teams to align infrastructure scalability with evolving data needs.
- Research emerging technologies to stay ahead of antibot trends including technologies like Kasada PerimeterX Akamai Cloudflare and more.
Requirements
- 4 6 years of experience in site reliability engineering and cloud infrastructure management.
- Proficiency in Python JavaScript for scripting and automation.
- Handson experience with Puppeteer/Playwright headless browsers and antibot evasion techniques.
- Knowledge of networking protocols TLS fingerprinting and CAPTCHAsolving frameworks.
- Experience with monitoring and observability tools such as Grafana Prometheus Elasticsearch and familiarity with monitoring and optimizing resource utilization in distributed systems.
- Experience with data lake architectures and optimizing storage using formats such as Parquet Avro or ORC.
- Strong proficiency in cloud platforms (AWS GCP or Azure) and containerization/orchestration (Docker Kubernetes).
- Deep understanding of infrastructureascode tools (Terraform Ansible) .
- Deep experience in designing resilient data systems with a focus on fault tolerance data replication and disaster recovery strategies in distributed environments.
- Experience implementing observability frameworks distributed tracing and realtime monitoring tools.
- Excellent problemsolving abilities with a collaborative mindset and strong communication skills.
Benefits
- Work Location: Remote
- 5 days working
4 6 years of experience in site reliability engineering and cloud infrastructure management. Proficiency in Python, JavaScript for scripting and automation. Hands-on experience with Puppeteer/Playwright, headless browsers, and anti-bot evasion techniques. Knowledge of networking protocols, TLS fingerprinting, and CAPTCHA-solving frameworks. Experience with monitoring and observability tools such as Grafana, Prometheus, Elasticsearch, and familiarity with monitoring and optimizing resource utilization in distributed systems. Experience with data lake architectures and optimizing storage using formats such as Parquet, Avro, or ORC. Strong proficiency in cloud platforms (AWS, GCP, or Azure) and containerization/orchestration (Docker, Kubernetes). Deep understanding of infrastructure-as-code tools (Terraform, Ansible) . Deep experience in designing resilient data systems with a focus on fault tolerance, data replication, and disaster recovery strategies in distributed environments. Experience implementing observability frameworks, distributed tracing, and real-time monitoring tools. Excellent problem-solving abilities, with a collaborative mindset and strong communication skills.
Employment Type
Full Time
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
