Senior QA Engineer – Data (Databricks)
About the Role
The Senior QA Engineer for Data will test and validate the data platform behind our internal back-office systems, including Databricks pipelines and the Power BI reports that business users rely on. This is a hands-on senior role: you'll make sure our data is accurate, complete, and timely, build automated data tests, and mentor onshore and nearshore QA testers. You'll partner closely with data engineers and business stakeholders and work within an agile environment.
Responsibilities
- Design and carry out test strategies for data pipelines built in Databricks, covering ingestion, transformation, and output.
- Validate data accuracy, completeness, and consistency between sources and targets, including reconciliation and data comparison processes.
- Build and maintain automated data tests in Python and Pytest, using SQL and PySpark to validate tables and transformations.
- Test and validate Power BI reports against source data.
- Define and implement data quality checks, and help set QA standards for the data platform.
- Integrate data tests into CI/CD pipelines.
- Investigate data defects and pipeline performance issues, and coordinate fixes with data engineering.
- Work with business users to understand how they use data and reports, and propose technical and non-technical solutions to business problems.
- Mentor onshore and nearshore QA testers, and review their test code against coding standards.
- Track and report on data quality metrics, test coverage, and release readiness.
- Stay current on QA tools and practices, including AI-assisted testing.
Required Qualifications
- 6+ years of QA experience, including 3+ years testing data pipelines, data warehouses, or analytics platforms.
- Hands-on experience testing in Databricks.
- Strong SQL skills for data validation and reconciliation.
- Proficiency in Python and Pytest.
- Experience with PySpark or Spark SQL.
- Solid understanding of ETL/ELT processes and data modeling concepts.
- Experience testing in cloud environments (Azure preferred).
- Experience with a test management tool (Qase preferred).
- Experience working in an agile environment.
- Strong process discipline and attention to detail.
- Excellent verbal and written communication skills, with a customer-service mindset toward business users.
Preferred Qualifications
- Experience testing or validating Power BI reports.
- Experience with Delta Lake and Unity Catalog.
- Experience with data quality frameworks (e.g., Great Expectations or Databricks pipeline expectations).
- Experience with CI/CD pipelines (e.g., Azure DevOps).
- Experience mentoring QA testers or working with distributed or nearshore teams.
- Familiarity with AI-assisted development tools such as Claude Code.
Current Tech Stack
Python, Pytest, SQL, PySpark, Databricks, Power BI, Qase, Azure, Claude Code
