Data Scientist (Big Data Engineer) 2

California Creative Solutions Inc.

Job Location:

Austin, TX - USA

Monthly Salary: Not Disclosed

Posted on: 1 hour ago

Vacancies: 1 Vacancy

Job Summary

Roles/Responsibilities:

The Worker is responsible for developing maintaining and optimizing big data solutions using the Databricks Unified Analytics Platform.

This role supports data engineering machine learning and analytics initiatives within this organization that relies on large-scale data processing.

Duties include:

Designing and developing scalable data pipelines
Implementing ETL/ELT workflows
Optimizing Spark jobs
Integrating with Azure Data Factory
Automating deployments
Collaborating with cross-functional teams
Ensuring data quality governance and security.

Mandatory Skills:

Implement ETL/ELT workflows for both structured and unstructured data- 4years
Automate deployments using CI/CD tools- 4years
Collaborate with cross-functional teams including data scientists analysts and stakeholders- 4years
Design and maintain data models schemas and database structures to support analytical and operational use cases- 4years
Evaluate and implement appropriate data storage solutions including data lakes (Azure Data Lake Storage) and data warehouses- 4years
Implement data validation and quality checks to ensure accuracy and consistency- 4years
Contribute to data governance initiatives including metadata management data lineage and data cataloguing- 4years
Implement data security measures including encryption access controls and auditing; ensure compliance with regulations and best practices- 4years
Proficiency in Python and R programming languages- 4years
Strong SQL querying and data manipulation skills- 4years
Experience with Azure cloud platform- 4years
Experience with DevOps CI/CD pipelines and version control systems- 4years
Working in agile multicultural environments- 4years
Strong troubleshooting and debugging capabilities- 4years
Design and develop scalable data pipelines using Apache Spark on Databricks- 3 years
Optimize Spark jobs for performance and cost-efficiency- 3 years
Integrate Databricks solutions with cloud services (Azure Data Factory) - 3 years
Ensure data quality governance and security using Unity Catalog or Delta Lake- 3 years
Deep understanding of Apache Spark architecture RDDs DataFrames and Spark SQL- 3 years
Hands-on experience with Databricks notebooks clusters jobs and Delta Lake- 3 years

Desirable Skills:

Knowledge of ML libraries (MLflow Scikit-learn TensorFlow)- 1years
Databricks Certified Associate Developer for Apache Spark- 1years
Azure Data Engineer Associate- 1 years

Roles/Responsibilities: The Worker is responsible for developing maintaining and optimizing big data solutions using the Databricks Unified Analytics Platform. This role supports data engineering machine learning and analytics initiatives within this organization that relies on large-scale data pro...

Roles/Responsibilities:

The Worker is responsible for developing maintaining and optimizing big data solutions using the Databricks Unified Analytics Platform.

This role supports data engineering machine learning and analytics initiatives within this organization that relies on large-scale data processing.

Duties include:

Designing and developing scalable data pipelines
Implementing ETL/ELT workflows
Optimizing Spark jobs
Integrating with Azure Data Factory
Automating deployments
Collaborating with cross-functional teams
Ensuring data quality governance and security.

Mandatory Skills:

Implement ETL/ELT workflows for both structured and unstructured data- 4years
Automate deployments using CI/CD tools- 4years
Collaborate with cross-functional teams including data scientists analysts and stakeholders- 4years
Design and maintain data models schemas and database structures to support analytical and operational use cases- 4years
Evaluate and implement appropriate data storage solutions including data lakes (Azure Data Lake Storage) and data warehouses- 4years
Implement data validation and quality checks to ensure accuracy and consistency- 4years
Contribute to data governance initiatives including metadata management data lineage and data cataloguing- 4years
Implement data security measures including encryption access controls and auditing; ensure compliance with regulations and best practices- 4years
Proficiency in Python and R programming languages- 4years
Strong SQL querying and data manipulation skills- 4years
Experience with Azure cloud platform- 4years
Experience with DevOps CI/CD pipelines and version control systems- 4years
Working in agile multicultural environments- 4years
Strong troubleshooting and debugging capabilities- 4years
Design and develop scalable data pipelines using Apache Spark on Databricks- 3 years
Optimize Spark jobs for performance and cost-efficiency- 3 years
Integrate Databricks solutions with cloud services (Azure Data Factory) - 3 years
Ensure data quality governance and security using Unity Catalog or Delta Lake- 3 years
Deep understanding of Apache Spark architecture RDDs DataFrames and Spark SQL- 3 years
Hands-on experience with Databricks notebooks clusters jobs and Delta Lake- 3 years