Data Engineer - Databricks & PySpark Specialist

Job Description & Summary: Manager_Data Scientist Gen AI_Data & Analytics_Advisory_Kolkata 

Responsibilities: 

 We are seeking a highly skilled and motivated Manager with expertise in artificial intelligence and machine learning. This is a full-time, hybrid position that offers an opportunity to work with cutting-edge technologies and contribute to innovative projects in the AI/ML domain. 

 ## Key Responsibilities  

- Lead and manage projects centered around AI and ML applications. 

-Work on large language models (LLMs) such as OpenAI, Gemini, and Anthropic. 

- Implement retrieval-augmented generation (RAG) strategies, including embedding and chunking techniques. 

- Develop and optimize chatbots and conversational agents. 

- Leverage frameworks like LangFlow and LangChain for AI applications. 

- Utilize coding tools (e.g., Cursor, v0, Claude Code, VS Code + Copilot, Codex) to enhance development processes. 

- Perform specialized or small language model training and fine-tuning. 

- Collaborate with cross-functional teams to deliver high-quality solutions. 

Technical Skills required (must have)  

· HandsOn Exposure to Generative AI (Design, development of GenAI application in production)  

· Strong understanding of RAG, Vector Database, Lang Chain and multimodal AI applications.  

· Strong understanding of deploying and optimizing AI application in production.  

· Strong knowledge of statistical and data mining techniques like Linear & Logistic Regression analysis, Decision trees, Bagging, Boosting, Time Series and Non-parametric analysis.  

· Strong knowledge of DL & Neural Network Architectures (CNN, RNN, LSTM, Transformers etc.)  

· Strong knowledge of SQL and R/Python and experience with distributing data/computing tools/IDEs. 

 · Experience in advanced Text Analytics (NLP, NLU, NLG). · Understanding of LLM Ops, ML Ops for scalable ML development.  

· Working knowledge on agents like PyTorch, TensorFlow  

· Proven capability in using Generative AI services like OpenAI, Google (Gemini) 

 · Understanding of Agentic AI Framework (Lang Graph, Auto gen etc.)  

· Understanding of fine-tuning for pre-trained models like GPT, LLaMA, Claude etc. using LoRA, QLoRA and PEFT technique 

Mandatory skill sets: 

AI chatbots, Data structures, GenAI object-oriented programming, IDE, API, LLM Prompts, PyTorch, TensorFlow 

Preferred skill sets: 

AI chatbots, Data structures, GenAI object-oriented programming, IDE, API, LLM Prompts, PyTorch, TensorFlow 

Years of experience required: 

8-13 Years 

Education qualification: 

BE, B. Tech, M. Tech