The Data Engineer is responsible for designing, building, and maintaining the data infrastructure that powers reporting, analytics, and decision-making across the organization. This role works extensively with the Microsoft Azure data ecosystem — including Azure Data Factory (ADF), Azure SQL Database/Synapse, and Azure Data Lake — to ingest, transform, and deliver high-quality, trusted datasets.
In addition to building pipelines, the Data Engineer must have the ability to analyze and correlate multiple datasets from ERP, MES, CRM, and other sources, designing star and snowflake schema data models that support performance and scalability. This role ensures that data is accurate, reliable, and available to stakeholders in a timely manner.
Essential Duties and Responsibilities
- Pipeline Development: Design, build, and monitor ADF pipelines, dataflows, and triggers to automate ingestion and transformation of data from multiple systems.
- SQL Development: Write and maintain stored procedures, views, and reporting tables; tune queries for optimal performance.
- Data Modeling: Design and implement star/snowflake schemas to enable efficient reporting and analytics.
- Data Correlation: Analyze cross-system data relationships to create unified reporting views and metrics.
- Automation: Develop Python scripts for validation, reconciliation, and process automation where needed.
- Data Quality: Implement validation checks, reconciliation processes, and alert to maintain data accuracy and reliability.
- Documentation: Create detailed technical documentation for pipelines, data models, and schedules.
- Stakeholder Collaboration: Work closely with finance, supply chain, and operations teams to translate requirements into scalable solutions that deliver measurable business value.
Qualifications
- Bachelor’s degree in Computer Science, Data Engineering, or related field.
- 5+ years of experience in data engineering, ETL, or database development.
- Proven hands-on experience with Azure Data Factory, Azure SQL Database/Synapse, and Azure Data Lake.
- Advanced SQL development skills, including stored procedures, functions, indexing, and performance tuning.
- Expertise in star and snowflake schema design for analytical data models.
- Familiarity with data governance, security, and best practices in Azure environments.
- Experience using Python scripting for data transformation and workflow automation.
- Strong understanding of ETL/ELT design patterns and incremental loading strategies.
- Excellent problem-solving, critical-thinking, and communication skills.
