Azure Senior Data Ops/ Gen AI Engineer(NCS/Job/ 1052)

Co-develop and help manage the unstructured data pipeline lifecycle from dev through prod
•Enable NLP, entity extraction, permissibility tagging, and redaction modules for unstructured content
•Advise and manage automation/orchestration workflows
•Help onboard new data ops engineers
•Establish reuseable data engineering patterns for consuming from document stores
•Data modeling and architecture throughout Lakehouse zones
•Advise on engineering standards, best practices, and frameworks
•Enable and maintain CI/CD DataOpscapabilities


• Over 10 years of IT experience
• 4+ recent years in Azure stack
• 3+ recent years in ADF creating pipelines with several activities
• 3+ recent years of hardcore experience developing PySpark notebooks in Azure databricks cluster
• 2 years in SDLC agile and using CI/ CD pipeline using Github actions.
• 3 years working with offshore and onsite teams for coordinated delivery in SDLC Agile methodology
• 3+ years of Scrum, StandUps, JIRA board maintenance, estimation and staff augmentation at a project level.
• 3+ years in deployment and post deployment activities in production.
• Exposure to RAG and PyTorch or python using API Key to code against vector databases.
• Exposure to vectorization using OpenAI’s ( or any other LLM’s) embedding scheme and storing data in vector database.