Role :-
- Design, implementation, and management of data platform, data pipelines to help fulfill data access and aggregation functionality for users in cloud environment.
Responsibilities :-
- Designing, constructing, developing, installing, testing and maintaining architectures of large-scale data processing systems.
- Experiment with various data models to optimize the search queries and come up with the proposals.
- Build data pipelines that capture, process and store organized data into large databases which would be ready for consumption.
- Expertly use Query programming languages such as SQL and functional programming languages such as Python, for mining and storing data into databases.
- Perform ETL tasks on the Big Data, using highly-scalable algorithms written in Scala / Java / PySpark so that in can utilize the power distributed computing.
- Have end- to- end responsibility for leading projects focused on extracting, merging, analyzing and managing large sets of data across multiple, disparate databases
- Be able to transform unstructured raw data in to formats suitable for statistical modeling, visualization and machine learning environments.
- Establish methodologies for quickly rolling out new data analysis capabilities for standalone data- driven products and service to support our associates.
- Be able to understand varies data sources and may be able to implement automated data quality and auditing through standard tools and custom code.
- Demonstrate a deep knowledge of and ability to operationalize, leading data technologies and best practices
- Be responsible for maintaining project plans, clean code, and well- written documentation
- Be able to work in teams and collaborate with stakeholders todefine requirements
- Make decisions independently on analytical problems & methods.
- Be able to identify and suggest novel areas of future work for themselves or the team
- Be able to work in a globally distributed team in an Agile/ Scrum approach
Competency Requirements :-
- Hands-on data engineering or other data- intensive development experience
- Experience processing large amounts of structured and unstructured data
- Advanced knowledge of programming languages such as Python /Java
- Experience building scalable data models and performing complex relational databases queries using SQL (Oracle, MySQL), etc.
- Experience with Big Data tools like Spark, Hadoop, Amazon EMR.
- Experience working on Cloud platforms like AWS,IBM,Azure including EMR, RDS, EC2, Vault, ECS, S3, EFS, etc.
Desired but not essential :-
- Experience in IoT time series data domain
- Exposure to automotive data
Educational Qualification :-
