TE-1409 Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Department:
Job Summary
Job Title: Data Engineer
Location: Hyderabad
Experience: 8 12 Years
Employment Type: Full-Time
Shift: Overlap with US Team (evening IST preferred)
Staff Data Engineer
Data Platform & Analytics Engineering Hyderabad (On-site Softobiz office)
About the Role
We are looking for a Staff Data Engineer to own our end-to-end data platform from raw event ingestion through the warehouse to the analytics marts that the business product and marketing teams rely on every day. You will be the person who makes sure the numbers we make decisions on are accurate fresh reproducible and trusted.
This is a hands-on individual-contributor role. You will partner daily with engineering analytics product growth and external data partners. Success looks like: one canonical patient or customer key across every system a well-modelled warehouse that answers business questions in SQL rather than spreadsheets and a data platform that product marketing and leadership genuinely trust.
Scope note: this role owns warehouse architecture pipelines and data modelling. The sister MarTech Engineering role owns event capture at the edge and the delivery of conversion signals to ad platforms. Both roles share one source of truth (the warehouse) and collaborate on the hand-off between event streams and warehouse tables.
What Youll Own
- Warehouse architecture and modelling BigQuery (or equivalent) as the single source of truth designed to serve both analytics and activation use cases.
- dbt project ownership model layering (staging intermediate marts) naming conventions tests documentation exposures and the promotion workflow from dev to production.
- Ingestion pipelines event streams from the CDP (Segment or equivalent) landing cleanly in the warehouse plus structured loads from the product database billing / payments ad platforms and third-party SaaS systems.
- Identity resolution in the warehouse stitching anonymous edge-level and identified user records into one canonical key and publishing that key to downstream consumers.
- Core marts revenue orders subscriptions cohorts LTV funnel conversion channel performance clinical or operational metrics as the business requires.
- Reverse-ETL / activation models warehouse tables purpose-built to feed ad-platform conversions APIs lifecycle marketing audiences and experimentation tools.
- Data quality and observability dbt tests freshness monitors row-count alerts reconciliation checks between systems (for example: warehouse orders vs payment processor warehouse conversions vs ad-platform reports).
- Data governance PII / PHI handling column-level masking row-level security where required access policies and the documentation that makes those policies auditable.
- Orchestration and CI/CD scheduled runs retries incremental strategies pull-request-based dbt reviews automated tests in CI.
- Partnership with external data partners and the sister MarTech Engineering function clear ownership lines shared conventions joint on-call for pipeline incidents.
Required Experience
- 812 years in data engineering or analytics engineering at consumer-facing digital businesses ideally including at least one direct-to-consumer e-commerce subscription or marketplace product.
- Deep hands-on experience designing and maintaining a warehouse on BigQuery Snowflake or Redshift. You can explain partitioning clustering materialised views and cost / performance trade-offs from memory.
- Production experience with dbt Core or Cloud including model layering incremental strategies snapshots tests documentation exposures and managing a multi-contributor repo through pull requests.
- Fluent SQL advanced window functions CTEs pivoting session stitching slowly-changing-dimension patterns. You can read a 200-line query and tell us what is wrong with it.
- Experience designing identity resolution in the warehouse merging anonymous cookie-level logged-in and server-generated identifiers into one canonical customer key and publishing that key for downstream systems to join on.
- Hands-on experience building marts for revenue orders subscriptions cohorts LTV and funnel analysis and reconciling those marts against authoritative systems (payment processors billing ad platforms).
- Python for data work transformations orchestration DAGs small internal tools and scripting around the warehouse. You do not need to be a backend engineer.
- Working knowledge of at least one workflow orchestrator Airflow Prefect Dagster or equivalent and the operational discipline that goes with it (idempotency backfills retries alerting).
- Comfort integrating warehouse output into reverse-ETL tooling (Polytomic Hightouch or Census) so that audiences conversion events and customer traits can be activated in ad platforms lifecycle tools and experimentation systems.
- A bias for data contracts and tests over heroics you treat a broken dashboard as a platform failure to be prevented not a one-off to be patched.
- Experience operating under ambiguity a platform migration re-platforming or a major tracking overhaul where the right schema had to be invented mid-flight.
Tools and Technologies
The following stack describes what you will work with day-to-day. You do not need hands-on experience with every single tool depth in 6070% of this list is what we are looking for along with the pattern-matching to pick up the rest quickly.
| Category | Tools |
| Data Warehouse | BigQuery (required) Snowflake or Redshift. Strong grasp of partitioning clustering cost controls. |
| Transformation | dbt Core or dbt Cloud (required) dbt tests snapshots exposures packages. |
| Orchestration | Airflow Prefect Dagster or an equivalent DAG runner. dbt Cloud scheduling for dbt-only pipelines. |
| Ingestion | Segment (CDP warehouse) Fivetran or Airbyte (SaaS warehouse) custom Python ingestion where needed. |
| Reverse-ETL / Activation | Polytomic (preferred) Hightouch Census mapping marts to ad-platform lifecycle and experimentation destinations. |
| Product Analytics (consumer) | Mixpanel Amplitude or Heap you will not own these tools but you will land their events and feed their user properties from the warehouse. |
| BI / Visualisation | Omni Looker Mode Metabase Tableau warehouse-native semantic modelling preferred. |
| Languages | SQL (advanced) Python (fluent) Jinja (dbt); shell scripting as needed. |
| Storage & Cloud | Google Cloud Platform primary (Cloud Storage IAM Pub/Sub Cloud Run / Cloud Functions). |
| Governance & PII | Column-level security masking policies row-level access data-classification tagging audit logging. |
| Dev workflow | Git GitHub / GitLab pull-request reviews dbt CI unit and schema tests dbt docs. |
| Collaboration | Linear Slack Google Docs & Sheets Confluence or Notion. |
Nice to Have
- Healthcare fintech or other privacy-sensitive domain experience designing a warehouse that respects PII / PHI boundaries and survives audit.
- Experience operating a dbt repo shared between an internal team and an external data partner (or vendor) conventions code review ownership boundaries.
- Experience with Segment Unify identity-graph services or custom identity-resolution frameworks.
- Hands-on with experimentation data building the exposures assignment and outcome tables that power A/B test readouts and feeding them to an experimentation platform.
- Hands-on with LTV or propensity modelling (regression survival or tree-based) and with getting those predictions into the warehouse as first-class tables.
- Familiarity with streaming patterns Pub/Sub Kafka Kinesis and when to prefer them over batch.
- Prior early-hire data platform experience at a growth-stage product company where you set the conventions rather than inherited them.
How You Work
- You treat the warehouse as a product versioned documented tested with clear ownership of every model and every downstream exposure.
- You are as comfortable debugging a BigQuery cost regression as you are explaining an attribution or LTV model to senior leadership.
- You write dbt tests before you write dashboards. You would rather catch the problem in CI than in a Monday standup.
- You draw scope lines honestly you know where your layer ends and the MarTech product or ML layer begins and you document the hand-off.
- You reconcile aggressively warehouse vs payment processor warehouse vs ad platforms warehouse vs the product UI. Anything that does not tie out is a ticket not a footnote.
- You write things down. The team should not need to re-learn a modelling decision you have already made.
Compensation and Logistics
- Location: On-site at our Hyderabad (Softobiz) office. This is not a remote role.
- Reporting: Reports to the Engineering Owner with regular working interactions across product analytics growth and external data partners.
- Compensation: Competitive base salary plus performance bonus and standard benefits.
- Schedule: Full-time with some overlap expected with partner teams in other time zones.
How to Apply
Send your resume and a short note (510 sentences) describing the most complex data platform or warehouse migration you have personally led what you inherited what you shipped and what you would do differently. Links to open-source contributions dbt packages public writing or talks are welcome.
About Softobiz:
Innovation begins with like-minded people aiming to transform the world together. At Softobiz we invite you to become a part of an organization that has been helping clients transform their business by fusing insights creativity and technology. With a team of 300 technology enthusiasts we have been trusted by leading enterprises around the globe for over 12 years.
At Softobiz we foster a culture of equality learning collaboration and creative freedom empowering our employees to grow and excel in their careers. Our technical craftsmen are pioneers in the latest technologies like AI machine learning and product development.
Why Should You Join Softobiz
- Work with technical craftsmen who are pioneers in the latest technologies.
- Access training sessions and skill-enhancement courses for personal and professional growth.
- Be rewarded for exceptional performance and celebrate success through engaging parties.
- Experience a culture that embraces diversity and creates an inclusive environment for all employees.
Softobiz is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will be afforded equal employment opportunities without discrimination based on race creed color national origin sex age disability or marital status.
Follow us
Required Experience:
IC
Key Skills
About Company
Softobiz prepares businesses for transformative success by embracing change and engineering innovative digital products.
