Data Engineer
Data Engineer
Posted on :
27-06-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Location: Remote
Exp: 5 to 8 years
BDatabricks Data Engineer-
We are seeking a skilled and experienced SQL to Databricks Migration Specialist to join our team. The primary responsibility of this role is to migrate SQL-based Crystal Reports to Databricks ensuring seamless integration and optimal performance. The ideal candidate should have a strong background in SQL Databricks and Crystal Reports along with excellent problem-solving skills and attention to detail.
Responsibilities: As a SQL to Databricks Migration Specialist for Crystal Reports your key responsibilities will include:
1. Analyzing Existing Crystal Reports: - Reviewing existing Crystal Reports that utilize SQL queries. - Identifying the underlying data sources and dependencies. - Assessing the complexity of each report and its compatibility with Databricks.
2. Designing Data Models on Databricks: - Collaborating with stakeholders (business analysts developers) to understand reporting requirements. - Translating existing SQL queries into optimized SparkSQL or Pyspark code for on Databricks. - Designing efficient data models using Delta Lake or other appropriate storage formats.
3. Migrating Crystal Reports: - Developing migration strategies based on the analysis of existing reports. - Rewriting or modifying existing reports using Apache Spark APIs within the Databricks environment. - Ensuring data integrity during the migration process.
4. Performance Optimization: - Profiling and optimizing migrated reports for improved performance on Databricks clusters. - Implementing best practices for query optimization caching mechanisms partitioning techniques etc.
5. Documentation and Training: - Documenting the migration process including steps taken challenges faced and solutions implemented. Providing training sessions or documentation for end-users on how to use migrated reports in Databricks.
Qualifications: -
1. Bachelors degree in Computer Science or related field (or equivalent work experience).
2. Strong proficiency in SQL programming language. - In-depth knowledge of Apache Spark ecosystem including SparkSQL Pyspark and Python programming languages.
3. Familiarity with cloud-based platforms such as Azure Databricks would be advantageous
4. Familiarity with Crystal Reports or any reporting tools would be advantageous
5. Excellent problem-solving skills with an ability to analyze complex systems efficiently
6. Strong communication skills with an ability to collaborate effectively across teams
delta lake,apache spark,communication,problem-solving,sql,databricks,sparksql,cloud-based platforms (azure databricks),crystal reports,python,apache,pyspark
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
