Data Engineer II (azure Databricks / Azure Data Factory) (US)

Data Engineer

The Data Engineer (Azure Databricks/ Azure Data Factory) II provides a broad range of data engineering functions including data modeling, data quality, data profiling, data acquisition and ingestion, extract transform load (ETL), metadata enrichment and management, data provenance and lineage, and other specialized data management functions.

Depth & Scope:

• Advanced knowledge of data engineering frameworks, technologies, tools, processes, patterns, and procedures, including how they impact other technology areas such as Architecture or Infrastructure
• Performs complex technical tasks independently
• Advanced knowledge of TD applications, systems, networks, innovation, design activities, business, organization, best practices, and standards
• Designs and develops to meet business and technical requirements; analyzes, adapts, integrates, codes, tests, debugs, and executes
• Uses and evolves established patterns to solve complex problems; leads the development of new patterns where necessary
• May be recognized as a subject matter expert in areas directly related to key accountabilities

Education & Experience:

• Degree, Postgraduate Degree, or Technical Certificate in Data Management or related discipline (e.g. Computer Science, Engineering), or equivalent practical experience
• 3+ years of relevant experience

Customer Accountabilities:

• Performs data analysis and assesses data management requirements for a specific Platform or Journey, including complex analysis involving multiple pods or products
• Maintains expert knowledge of upstream data, including knowledge provided through data profiling, data quality reporting, and via the production of metadata
• Supports the acquisition and ingestion of data
• Articulates complex, large scale, and high impact technical design and development details to non-technical business partners
• Elicits, analyzes, and understands business and data requirements to develop complete business solutions, including data models (entity relationship diagrams, dimensional data models), ETL and business rules, data life-cycle management, governance, lineage, and metadata
• Ensures data is maintained in compliance with enterprise data standards, policies, and guidelines
• Develops and maintains complex data models using industry standard modeling tools
• Develops and maintains complex ETL jobs and frameworks using the Bank's standard tools
• Provides support to the development and testing teams to resolve data issues, including escalation support on complex issues
• Supports partners and stakeholders in interpreting and analyzing data
• Builds effective working relationships within own pod and across partner teams to encourage collaboration on all pod deliverables

Shareholder Accountabilities:

• Coordinates with technology work teams such as ITS, ARE, Architecture, Enterprise Protect etc. to ensure overall delivery success
• Supports the QA team with data analysis of complex issues/ test cases as part of SIT/UAT/PAT testing
• Provides oversight on post implementation activities during the warranty period
• Executes & approves code check-in/ check-out into source code repository as part of source code management
• Works closely with ITS/ ARE teams to support code packaging & deployment (CI & CD) into higher environments
• Is the lead participant in the design & architecture reviews or the application
• Raises service-now requests and works with the change management team to support release management activities
• Leads data engineering initiatives and capabilities, data governance principles and how they apply across the organization
• Ensures metadata and data lineage is captured and compatible with enterprise metadata and data management tools and processes
• Adheres & contributes towards standard security coding practices to ensure application is free of most common coding vulnerabilities
• Ensures technical decisions, technical risks and lessons learned are identified, clearly documented and enhancements are accordingly implemented
• Protects the interests of the organization – identifies and manages risks, and escalates non-standard, high-risk activities as necessary
• Adheres to internal policies/procedures and applicable regulatory guidelines
• Keeps current on emerging trends/ developments and grow knowledge of the business, related tools, and techniques
• Enables team members by sharing knowledge and leveraging engineering best practices

Employee/Team Accountabilities:

• Participates fully as a member of the team, supports a positive work environment that promotes service to the business, quality, innovation and teamwork and ensures timely communication of issues/ points of interest
• Provides thought leadership and/or industry knowledge for Data engineering best practices and participates in knowledge transfer within the team and business unit
• Keeps current on emerging trends/ developments and grows knowledge of the business, related tools and techniques
• Participates in personal performance management and development activities, including cross training within own team
• Keeps others informed and up-to-date about the status / progress of projects and/or all relevant or useful information related to day-to-day activities
• Actively mentors and enables team members by sharing knowledge and leveraging engineering best practices
• Supports the team by providing guidance and proactively identifying and resolving issues
• Leads, motivates and develops relationships with internal and external business partners / stakeholders to develop productive working relationships
• Contributes to a fair, positive and equitable environment that supports a diverse workforce
• Acts as a brand ambassador for your business area/function and the bank, both internally and/or externally

Preferred Qualifications:

• Databricks
• Azure
• Python
• PySpark
• ADF
• Data Products
• Banking domain experience

Physical Requirements:

Never: 0%; Occasional: 1-33%; Frequent: 34-66%; Continuous: 67-100%

• Domestic Travel – Occasional
• International Travel – Never
• Performing sedentary work – Continuous
• Performing multiple tasks – Continuous
• Operating standard office equipment - Continuous
• Responding quickly to sounds – Occasional
• Sitting – Continuous
• Standing – Occasional
• Walking – Occasional
• Moving safely in confined spaces – Occasional
• Lifting/Carrying (under 25 lbs.) – Occasional
• Lifting/Carrying (over 25 lbs.) – Never
• Squatting – Occasional
• Bending – Occasional
• Kneeling – Never
• Crawling – Never
• Climbing – Never
• Reaching overhead – Never
• Reaching forward – Occasional
• Pushing – Never
• Pulling – Never
• Twisting – Never
• Concentrating for long periods of time – Continuous
• Applying common sense to deal with problems involving standardized situations – Continuous
• Reading, writing and comprehending instructions – Continuous
• Adding, subtracting, multiplying and dividing – Continuous

The above statements are intended to describe the general nature and level of work being performed by people assigned to this job. They are not intended to be an exhaustive list of all responsibilities, duties and skills required. The listed or specified responsibilities & duties are considered essential functions for ADA purposes.