Mandatory Skills: Azure Data Engineering, Strong programming skills with Azure Data Bricks & Pyspark
Requirements
· Degree in computer science or equivalent preferred
· Demonstrable experience in architecture, design, implementation, and/or support of highly distributed applications
· 12+ Years of Hands-on experience with data modelling, database design, data mining, and segmentation techniques.
· Working knowledge and experience with “Cloud Architectures” (e.g., SaaS, PaaS, IaaS) and the ability to address the unique security considerations of secure Cloud computing
· Designing and building distributed systems capable of processing massive data volumes
· Should have architected solutions for Cloud environments such as Microsoft Azure and/or GCP
· Experience with debugging and performance tuning in distributed environments
· Strong analytical skills with the ability to collect, organize, analyse, and broadcast significant amounts of information with attention to detail and accuracy
· Experience dealing with structured, unstructured data.
· Must have Python, PySpark experience.
· EDW experience required – Azure Databricks, RedShift, Azure Synapse, etc
· Experience in ML or/and graph analysis is a plus
Responsibilities
· Act as subject matter expert providing best-practice guidance on data lake and ETL architecture frameworks suitable for handling big data for unstructured and structured information
· Drive business and Service Layer development with the customer by finding new opportunities based on expanding existing solutions creating new ones
· Provide hands-on subject matter expertise to build and implement Azure-based Big Data solution
· Research, evaluate, architect, and deploy new tools, frameworks, and patterns to build sustainable Big Data platforms for our clients
· Facilitate and/or conduct requirements workshops
· Responsible for collaborating on the prioritization of technical requirement
· Collaborates with peer teams and vendors on the solution and delivery
· Has overall accountability for project delivery
· Works collaboratively with the Product Management, Data Management, and other Architects to deliver for the cloud data platform, Data as a service
· Consults with clients to assess current problem states, define desired future states, define solution architecture and make solutions recommendations