Data Engineer
Share
Job Location:
Dallas, IA - USA
Monthly Salary:
$ 96250 - 125000
Posted on:
8 hours ago
Vacancies:
1 Vacancy
Job Summary
Data Engineer
About the Role
As a Data Engineer youll play an integral role in building the roadmap of the companys technology strategy.
Responsibilities
- Engineer robust AZURE data factory pipelines using Linked Services/Datasets to ingest high volume diverse data from disparate source systems into Azure BLOB storage
- Design and implementation of scalable data pipelines using Azure Databricks and Azure Data factory adhering to Medallion architecture principles to ensure data integrity consistency and performance
- Develop complex & advanced transformation logic using PySpark (Python)user-defined functions (UDFs) within Azure Databricks to design and implement data processing and ETL workflow enabling efficient transformation of heterogeneous source data into an integrated advanced Benefit and Eligibility system
- Creating modular and reusable notebooks in Azure Databricks utilizing functions parameters widgets for dynamic data processing and various python libraries such Panda numpyPyspark SQL SQL alchemy etc.
- Developing highly optimized Databrick notebooks using PySpark (Python & SQL) for advanced data joining data filtering pre-aggregation and processing of large data sets.
- Working with SQL Server and Oracle writing optimized queries with CTEs ANSI SQL window functions Views materialized Views stored procedures and Oracle-specific syntax to perform data profiling validation balancing reconciliation to ensure data accuracy (data quality) and data completeness.
- Solving complex data engineering challenges including Delta merge for hundreds of millions of records deduplication performance tuning and improving ETL run time efficiency using partitioning strategies Z-Ordering etc.
- Closely working with Client and Business analysts to define and assess complex data conversion and data integration requirements translating business needs into technical solutions.
Requirements
Bachelors Degree in Computer Science or related field
Total experience 10 as data analyst/engineer
Federal government or State level experience
2 years experience in Health and Huma Services programs (SNAP and TANF)
Flexible Working
At Conduent we want you to be yourself. We recognize that everyone is different and that how people want to work and deliver at their best is different for everyone too.
In this role you can expect the following working conditions:
Remote work: Enjoy the convenience of working from home and maximize your time by unplugging at the end of your workday.
Working For You
Perks and rewards designed for you:
Career Growth Opportunities: We help you thrive so together we can grow. We provide opportunities to advance your career with a vast portfolio of businesses and a global footprint.
Great Work Environment: We are proud of our award-winning culture and the recognition weve received for our diversity efforts.
Join Us
At Conduent we are one team one mission. We understand that our success is directly related to the success of our associates. We strive to create a culture where you can:
Bring your authentic self to work
Grow and thrive both personally and professionally
Make a difference with our clients in our communities and with the millions of people we support
When you join Conduent you are engaged in creating the future - both our companys and your own. With more than 60000 associates across 24 countries we will provide you the opportunity to grow with a team of people who will challenge and inspire you to be the best!
Pay Transparency Laws in some locations require disclosure of compensation-related information. For this position actual salaries will vary and may be above or below the range based on various factors including but not limited to location experience and addition to base pay this position based on business need may be eligible for a bonus or addition Conduent provides a variety of benefits to employees including health insurance coverage voluntary dental and vision programs life and disability insurance a retirement savings plan paid holidays and paid time off (PTO) or vacation or sick time. The estimated salary range for this role is $96250 - $125000
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
COVID-19 presents an unprecedented and challenging situation for all of us. At Conduent, our top priority is to protect our associate base and the communities where we live and work, while continuing to serve and help our clients at a time when they need us most. We’d like to thank ev ... View more
