Ability to work independently and take initiative to solve technical problems.
Excellent communication skills to effectively collaborate with technical and non-technical teams.
You will be part of a team of 5 developers and will report to the Data domain manager.
- Analyze and refactor existing code primarily developed by data engineers.
- Establish and maintain modular and clean development standards.
- Experience with code reviews and provide feedback to improve code quality and development practices.
- Write and execute unit tests, integration tests, performance tests, and regression tests to ensure code robustness and quality.
- Ensure migration and compatibility with recent versions of Python 3 while managing the existing code.
- Apply the SOLID principles of object-oriented design to ensure code quality and maintainability.
- Work with tools and environments such as Cloudera to manage and optimize data.
- Understand and apply distributed computing principles to optimize data processing and computation.
DevOps responsibilities:
- Experience with GitLab CI/CD for automation of build and deployment processes.
- Collaborate with data engineering teams to understand and improve existing data pipelines.
- Implement Continuous Integration (CI) using Jenkins and Jenkins files/groovy, SonarQube
- Familiarity with SonarQube for code quality analysis.
- Develop, deploy, and manage containerized applications using Docker.
- Understanding of deployment strategies using Virtuozzo Containers and Docker.
- Knowledge of Cloudera platform and its integration with Python applications.
- Experience with Jira API for task and project management.
- Familiarity with Nexus Repository for artifact management.
- Strong troubleshooting skills and ability to optimize performance.
- Knowledge of package management with PyPI and pip.
- Auto feed DEV, SIM, UAT envs
Methodology:
Work in an Agile methodology, participating in daily stand-ups, sprint planning, and retrospectives.
Proven experience (3+ years) in Python development, with a strong understanding of object-oriented and modular programming principles.
- Good knowledge of Python 3 and the challenges associated with migrating from older versions.
- Deep understanding of the SOLID principles of object-oriented design.
- Expertise in code refactoring and improving code quality.
- Proficiency in unit testing, integration testing, performance testing, and regression testing.
- Experience with distributed computing principles and optimizing data processing (for example: Spark).
- Understanding of data modelling and creating data structures on Hadoop.
- Experience with big data environments, particularly Cloudera, is a significant plus.
DevOps skills:
- CI/CD Pipeline Automation
- Version Control & Branching
- Deployment Automation
- Artifact Management
- Containerization & Orchestration
- Automation & Monitoring
