Solution IT Inc. is looking for an Databricks Data Engineer for one of its clients in Houston TX (Hybrid)
US Citizens & Locals Only
Job Title: Databricks Data Engineer
Responsibilities:
- Design develop and implement robust data solutions using modern data platforms like Databricks.
- Ensure scalable reliable and secure data environments that support advanced analytics and meet business needs.
- Lead the migration of data from traditional RDBMS systems to Databricks environments.
- Design scalable data pipelines and infrastructure to meet evolving data demands.
- Develop and manage ETL processes using Databricks for efficient data extraction transformation and loading.
- Optimize ETL workflows to improve performance and ensure data integrity.
- Ensure seamless data transitions with minimal disruption to ongoing operations.
- Monitor and optimize data system performance reliability scalability and cost-effectiveness within Databricks environments.
- Collaborate with data engineers scientists analysts and product managers to define and deliver data solutions.
- Establish and enforce best practices and standards for data engineering and data warehouse processes.
- Evaluate and implement tools and technologies to improve data pipeline efficiency and effectiveness.
- Mentor junior data engineers and provide technical leadership.
- Design and operationalize end-to-end machine learning pipelines on Databricks including data wrangling feature engineering training and tuning.
- Work with DevOps and infrastructure teams to deploy and manage production data systems.
- Ensure compliance with organizational industry and regulatory data standards.
- Collaborate with enterprise architects and IT leaders to align data architecture with overall IT strategy.
Required Skills:
- BS or MS in Computer Science or a related field.
- Extensive experience with Databricks including ETL and data migration.
- Databricks certifications are a strong plus.
- Experience with cloud platforms like AWS Azure or GCP.
- Familiarity with big data technologies and frameworks.
- Experience implementing and optimizing ML workflows using Databricks ML MLflow scikit-learn Spark MLlib etc.
- Strong knowledge of data warehousing modeling and SQL.
- Experience transitioning from RDBMS systems to cloud-native platforms.
- Proficiency in Python SQL and scripting languages.
- Understanding of data governance data quality and data security best practices.
- Expertise in database technologies particularly SQL Server.
- Familiarity with Docker and orchestration tools like Kubernetes.
- Excellent analytical and problem-solving skills.
- Strong communication and collaboration skills especially in cross-functional environments
Work Site: Houston TX (Hybrid)
Duration: 6 Months
Expected Start Date: 2 Weeks
Number of Positions: 01
Please send your responses back to
About Solution IT
Solution IT is a national IT consulting company specializing in: Technology Staffing and Oracle E-Business Solutions based in Boston Massachusetts.