Lead Analyst/Senior Software Engineer - Data Engineer with Python, Apache Spark, HDFS
Lead Analyst/Senior Software Engineer - Data Engineer with Python, Apache Spark, HDFS
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Position Description:
At CGI were a team of builders. We call our employees members because all who join CGI are building their own company - one that has grown to 72000 professionals located in 40 countries. Founded in 1976 CGI is a leading IT and business process services firm committed to helping clients succeed. We have the global resources expertise stability and dedicated professionals needed to achieve. At CGI were a team of builders. We call our employees members because all who join CGI are building their own company - one that has grown to 72000 professionals located in 40 countries. Founded in 1976 CGI is a leading IT and business process services firm committed to helping clients succeed. We have the global resources expertise stability and dedicated professionals needed to achieve results for our clients - and for our members. Come grow with us. Learn more at .
This is a great opportunity to join a winning team. CGI offers a competitive compensation package with opportunities for growth and professional development. Benefits for full-time permanent members start on the first day of employment and include a paid time-off program and profit participation and stock purchase plans. We wish to thank all applicants for their interest and effort in applying for this position however only candidates selected for interviews will be contacted. No unsolicited agency referrals please.
Job Title: Lead Analyst/Senior Software Engineer - Data Engineer with Python Apache Spark HDFS
Position Title: Lead Analyst/Senior Software Engineer - Data Engineer with Python Apache Spark HDFS
Experience: 6 to 8 Years
Category: Software Development/ Engineering
Main location: Hyderabad
Shift Timings: General Shift
Employment Type: Full Time-Permanent
Position ID: J0325-2041
Your future duties and responsibilities:
Job Overview:
CGI is looking for a talented and motivated Data Engineer with strong expertise in Python Apache Spark HDFS and MongoDB to build and manage scalable efficient and reliable data pipelines and infrastructure
Youll play a key role in transforming raw data into actionable insights working closely with data scientists analysts and business teams.
Key Responsibilities:
Design develop and maintain scalable data pipelines using Python and Spark.
Ingest process and transform large datasets from various sources into usable formats.
Manage and optimize data storage using HDFS and MongoDB.
Ensure high availability and performance of data infrastructure.
Implement data quality checks validations and monitoring processes.
Collaborate with cross-functional teams to understand data needs and deliver solutions.
Write reusable and maintainable code with strong documentation practices.
Optimize performance of data workflows and troubleshoot bottlenecks.
Maintain data governance privacy and security best practices.
Required qualifications to be successful in this role:
Minimum 6 years of experience as a Data Engineer or similar role.
Strong proficiency in Python for data manipulation and pipeline development.
Hands-on experience with Apache Spark for large-scale data processing.
Experience with HDFS and distributed data storage systems.
Strong understanding of data architecture data modeling and performance tuning.
Familiarity with version control tools like Git.
Experience with workflow orchestration tools (e.g. Airflow Luigi) is a plus.
Knowledge of cloud services (AWS GCP or Azure) is preferred.
Bachelors or Masters degree in Computer Science Information Systems or a related field.
Preferred Skills:
Experience with containerization (Docker Kubernetes).
Knowledge of real-time data streaming tools like Kafka.
Familiarity with data visualization tools (e.g. Power BI Tableau).
Exposure to Agile/Scrum methodologies.
Skills:
- Hadoop Hive
- Python
- SQL
- English
What you can expect from us:
Together as owners lets turn meaningful insights into action.
Life at CGI is rooted in ownership teamwork respect and belonging. Here youll reach your full potential because
You are invited to be an owner from day 1 as we work together to bring our Dream to life. Thats why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our companys strategy and direction.
Your work creates value. Youll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas embrace new opportunities and benefit from expansive industry and technology expertise.
Youll shape your career by joining a company built to grow and last. Youll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.
Come join our teamone of the largest IT and business consulting services firms in the world.
Required Experience:
Senior IC
Employment Type
Full-Time
