Key Responsibilities
- Design and maintain ELT pipelines using Dataform, SQLX, and BigQuery.
- Implement Bronze, Silver, and Gold data layers and dimensional data models.
- Develop incremental models, reusable JavaScript functions, assertions, and workflow configurations in Dataform.
- Optimize BigQuery SQL, partitioning, clustering, query performance, and compute and storage costs.
- Build and maintain CDC pipelines using Datastream, including handling updates, deletes, and schema evolution.
- Manage GCP infrastructure using Terraform, including BigQuery datasets, Datastream, IAM, and service accounts.
- Implement GitHub-based CI/CD, pull requests, code reviews, and automated Dataform validation.
- Ensure data quality, security, governance, and pipeline monitoring.
- Troubleshoot production failures, perform root-cause analysis, and implement preventive measures.
- Collaborate with BI teams and business stakeholders to deliver reliable, analytics-ready data.
Required Skills
- 6–10 years of experience in Data Engineering.
- Strong hands-on experience with GCP and BigQuery.
- Production experience with Dataform, SQLX, incremental processing, assertions, and dependency management.
- Advanced SQL skills, including window functions, MERGE, ARRAY, STRUCT, UNNEST, and deduplication.
- Experience with BigQuery architecture, dimensional modeling, and cost optimization.
- Working knowledge of Datastream and CDC concepts.
- Experience with Terraform, GitHub, and CI/CD practices.
- Strong troubleshooting, problem-solving, and communication skills.
#NemetschekIndia #LI-Hybrid #LI-SR2
