Overview
As the Senior Data Engineer, you will own the pipelines and platform that move Nerdy’s data from source systems into a governed lakehouse. You’ll build streaming and batch ingestion, run the Iceberg lakehouse on Starburst Galaxy, and ship reliable, well-tested dbt layers that analytics engineers and analysts build on—leveraging AI to accelerate development, QA, and operations.
Qualifications
- Bachelor’s degree in Computer Science, Engineering, Math/Stats, or equivalent experience.
- 3+ years as a Data Engineer (or similar) in high-growth environments.
- Expert SQL and strong Python; experience building production pipelines that run unattended and recover on their own.
- Hands-on with open table formats and lakehouse engines: Iceberg on Trino/Starburst Galaxy, or similar (Delta Lake, Hudi, Spark, Databricks, Presto/Athena).
- Streaming experience with Kafka/Confluent or similar (Kinesis, Redpanda), including schema registry, CDC (e.g., Debezium), and idempotent loading into lakehouse tables.
- Proficiency with dbt and dbt Cloud (incremental models, macros, tests, docs) and orchestration tools such as Airflow or Dagster.
- Solid AWS experience (S3, IAM, Glue/REST catalogs) and infrastructure as code (Terraform or similar).
- Solid grasp of data modeling, partitioning, incremental processing, and backfill strategy on large datasets.
- Git-based development and CI/CD familiarity; disciplined approach to reviews, releases, and on-call.
- Experienced with AI-native tools that enhance productivity and speed (e.g., Cursor, Make, Supabase, Netlify, Claude Code, n8n, Firecrawl, ChatGPT, Grok, Bolt, Vercel, etc).
- Alignment with Nerdy’s apolitical, mission-focused culture.
Responsibilities
- Ingestion & streaming: Build and run streaming and batch ingestion from Kafka and third-party APIs into Iceberg tables; own CDC, schema evolution, and late-arriving data.
- Lakehouse platform: Operate the Iceberg lakehouse on Starburst Galaxy (Trino)—table layout, partitioning, compaction, retention, and cluster sizing.
- Layered modeling with dbt: Build and maintain the bronze, silver, and gold layers with incremental models, idempotent merges, and safe backfills; push shared logic upstream so downstream consumers stay simple.
- Orchestration & reliability: Design dbt Cloud job schedules, dependencies, retries, and alerting; keep production jobs within their freshness SLAs and drive incidents to root cause.
- Data quality & governance: Implement tests, contracts, freshness and volume monitoring, lineage, and access controls; handle PII within privacy and security guardrails.
- Performance & cost: Tune queries, file layout, and cluster usage to improve speed and manage compute and storage spend.
- Infrastructure as code: Manage clusters, storage, and access through Terraform and CI, with changes that are reviewed, reversible, and documented.
- Collaboration: Partner with Analytics Engineering, Product Engineering, and business teams to turn source changes and new requirements into dependable data products.
- Documentation & change control: Maintain versioned code (Git), review PRs, run CI/CD for pipelines and dbt, and document sources, models, and runbooks.
- AI-enabled workflows: Use AI for code generation/review, test scaffolding, pipeline debugging, data profiling, and anomaly detection—operating within privacy and security guardrails.
Unleash Your Potential at Nerdy:
At Nerdy, you'll earn your success through meritocracy and exceptional outcomes. Our benefits reward performance and initiative:
- Competitive USD-Benchmarked Compensation: Earn a market-leading rate set in U.S. dollars, paid to you in local currency.
- 100% Remote (Home Country Only): Work from anywhere in your home country—no relocation required, no borders crossed.
- Flexible Time Off: Our flexible PTO lets you recharge on your own terms and when you need it the most.
- Local Holiday Pay: We honor your nation’s official holidays with paid time off—celebrate what matters to you.
- Continuous Learning: Get a free, all-inclusive learning membership for you and your household—including 1-on-1 tutoring hours, unlimited on-demand classes, and access to our full suite of learning products and services.
- Supercharge with AI: Gain exclusive access to cutting-edge AI tools that boost your productivity, making you feel almost super-human (cape not included).
- Feedback-Rich, Collaborative Culture: Tap into regular training, peer reviews, and a team that treats every team member as a vital collaborator and owner in our success.
- Make a Global Impact: Your expertise fuels an innovative platform used by learners around the world—be part of something transformative.
The Bottom Line:
If you're driven by impact, energized by ownership, and excited to help shape what’s next, you’ll thrive here. We move fast, think big, and reward those who deliver. This isn’t a traditional corporate environment - it’s a place to do the most meaningful work of your career.
