Analyst, Data Engineer, Consumer Banking Group Technology

Business Function

Group Technology and Operations (T&O) enables and empowers the bank with an efficient, nimble and resilient infrastructure through a strategic focus on productivity, quality & control, technology, people capability and innovation. In Group T&O, we manage the majority of the Bank's operational processes and inspire to delight our business partners through our multiple banking delivery channels.

Roles & Responsibilities

• Data Engineer should be able to understand the Business requirements, Functional and Technical requirements and translate them effectively using the Spark/SCALA Framework

• Should be able to extract the data from source systems (Open/Mainframes) to Data Lake (Hive/Parquet)

• Understand the complex transformation logic and translate them to Hive/Spark-SQL queries to load them onto Enterprise Data Domain tables and Data Marts

• Hands on experience on dealing with live streaming datasets using Kafka/Spark Streaming

• Excellent oral and written communication skills, On-time delivery and good team player

Requirements

• 3 to 4.5 years hands on experience on Spark Core, Spark-SQL, Scala-Programming and Streaming datasets in Big Data platform

• Should have extensive working experience in Hive and other components of Hadoop eco system(HBase, Zookeeper, Kafka and Flume)

• Should be able to understand the complex transformation logic and translate them to Spark-SQL queries

• Unix Shell Scripting and setting up CRON jobs

• Should have worked on Cloudera distribution framework, Oozie Workflow (or any Scheduler), Jenkins (or any version controller)

• Prior experience in Consumer Banking Domain is an advantage

• Prior experience in agile delivery method is an advantage

• Excellent understanding of technology life cycles and the concepts and practices required to build big data solutions

• Familiar with Data Warehouse concepts and Change Data Capture (CDC and SCD Types)

• Core Java skillset, Micro services understanding is an added advantage. Airflow is plus to have

• Good Knowledge and experience in any database (Teradata or Data Lake or Oracle or SQL Server) is a plus

• Bachelor of Engineering, Bachelor of Technology, MCA (or equivalent master’s degree)

Schedule: Full-time

Travel: