Debasis Bhattacharjee

An AI agent uses an LLM as a reasoning engine to autonomously plan use tools and complete multi-step tasks. Unlike a single LLM call that maps input to output an agent operates in a loop: observe think act observe again — until the task is complete.

Deep Dive: The ReAct pattern (Reason + Act) describes the core agent loop: the LLM receives a task and available tools generates a thought (reasoning about what to do) selects an action (a tool call) receives the observation (tool output) and repeats until producing a final answer. Tools are functions the LLM can invoke: web search code execution database queries API calls file operations. Agent architectures range from simple (single LLM with tools) to complex (multi-agent systems where specialized agents collaborate with a planner/orchestrator agent routing tasks). Key engineering challenges: tool design (tools must have clear descriptions for the LLM to select them correctly) error handling (agents can get stuck in loops or make wrong tool calls) context management (the agent's action history grows and fills the context window) and cost control (multi-step agents can make many API calls).

Real-World: A customer onboarding agent at a SaaS company replaces a 12-step manual process: it receives a new customer email calls the CRM API to create a contact queries the provisioning API to set up an account generates and sends a personalized welcome email creates a Jira ticket for account review and posts a Slack notification to the account manager — all autonomously from a single trigger.

⚠ Common Mistakes: Building agents without observability — impossible to debug why an agent made wrong decisions without logging the full thought-action-observation trace. Not implementing maximum step limits — agents can loop indefinitely on ambiguous tasks. Giving agents too many tools — LLMs struggle to select from large tool sets. Not handling tool failures gracefully in the agent loop.

🏭 Production Scenario: A document processing agent for an insurance company was processing claims autonomously. Without a step limit it entered an infinite loop trying to resolve a document parsing error making 10000 API calls in 8 minutes and generating a $400 API bill before being detected. Implementing a 20-step maximum and exponential backoff on tool errors fixed the runaway behavior.

Follow-up questions: What is the difference between ReAct Plan-and-Execute and Reflexion agent patterns? How do you implement agent memory (short-term vs long-term)? What is LangGraph and how does it implement agent state machines?

// ID: AI-ADV-002  ·  DIFFICULTY: 8/10  ·  ★★★★★★★★☆☆

RAG retrieves relevant documents from a vector database using semantic similarity search injects them into the LLM context and generates a response grounded in the retrieved content. Main failure modes are retrieval failures context window overflow and hallucinations about retrieved content.

Deep Dive: RAG has three main components: indexing (documents are chunked embedded using an embedding model and stored in a vector database like Pinecone Weaviate or pgvector) retrieval (the user query is embedded and semantically similar chunks are retrieved using approximate nearest neighbor search) and generation (retrieved chunks are inserted into the LLM prompt as context and the model generates a response). Key design decisions: chunk size (too small loses context too large wastes context window and dilutes relevance) embedding model choice number of retrieved chunks (k) whether to use reranking to improve retrieved chunk ordering and metadata filtering to constrain retrieval. Advanced patterns include hybrid search (semantic + keyword/BM25) HyDE (hypothetical document embeddings) and multi-hop retrieval for complex questions.

Real-World: A legal research assistant RAG system at a law firm used chunk sizes of 512 tokens for case documents. Attorneys complained answers lacked context. Investigation showed important legal reasoning spanned across chunk boundaries. Implementing larger overlapping chunks (1024 tokens with 200 token overlap) and a reranker (Cohere Rerank) improved answer quality significantly.

⚠ Common Mistakes: Chunking documents arbitrarily without considering semantic boundaries (splitting mid-paragraph). Using cosine similarity retrieval without reranking causing less relevant chunks to appear in context and confuse the model. Not handling the case where no relevant documents are retrieved — the model hallucinates instead of saying it does not know. Embedding the entire document instead of chunking exceeding context limits.

🏭 Production Scenario: A production customer support RAG system was giving confidently wrong answers about product return policies. Investigation revealed the retrieval was returning chunks from old policy documents because they had higher semantic similarity scores than newer updates. Implementing date-based metadata filtering to prefer recent documents and adding a retrieval confidence threshold solved the problem.

Follow-up questions: What is the difference between RAG and fine-tuning — when do you use each? What is a vector database and how does HNSW indexing work? What is RAGAS and how do you evaluate a RAG system?

// ID: AI-ADV-001  ·  DIFFICULTY: 8/10  ·  ★★★★★★★★☆☆

Model drift is the degradation of model performance over time as the real-world data distribution changes after deployment. Detect with monitoring (input distribution prediction distribution and ground truth metrics). Handle with automated retraining triggers shadow deployments and champion-challenger frameworks.

Deep Dive: There are two types of drift: data drift (input feature distributions change — customer demographics shift new product categories appear) and concept drift (the relationship between inputs and outputs changes — what predicts churn changes as customer behavior evolves). Detecting data drift: monitor statistical properties of input features using tests like KS test Population Stability Index (PSI) or Jensen-Shannon divergence. Detecting concept drift: monitor prediction distribution shifts and when labels are available track accuracy/AUC over time. PSI > 0.2 typically signals significant drift requiring investigation. Handling drift: trigger model retraining when drift metrics exceed thresholds use sliding window retraining on recent data implement champion-challenger deployment to safely test retrained models and maintain feature stores that can be queried at training and serving time to ensure consistency.

Real-World: A credit scoring model deployed in January showed 0.81 AUC. By September AUC had dropped to 0.71. PSI analysis of input features revealed significant drift in employment status and income features — COVID-19 had fundamentally changed the distribution. Emergency retraining on recent data restored AUC to 0.79.

⚠ Common Mistakes: Not monitoring model performance after deployment — treating deployment as the end of the ML lifecycle. Retraining on all historical data including outdated periods instead of using a recent sliding window. Not having rollback capability when a retrained model performs worse than the current champion. Ignoring the feedback loop where model predictions affect future training data.

🏭 Production Scenario: A fraud detection model at a payment processor declined from 89% recall to 74% recall over 6 months as fraudsters adapted their behavior patterns. Monthly retraining on recent fraud cases and implementing a fast-response challenger model that retrained weekly restored recall to 86% while reducing false positives.

Follow-up questions: What is Population Stability Index and how is it calculated? What is a feature store and why does it matter for training-serving skew? How do you implement a champion-challenger deployment?

// ID: ML-MLO-001  ·  DIFFICULTY: 8/10  ·  ★★★★★★★★☆☆

A feature store is a centralized repository for ML features that solves the training-serving skew problem by ensuring features computed at training time are computed identically at serving time. It also enables feature reuse across teams and models.

Deep Dive: Training-serving skew is one of the most common and damaging production ML problems: features computed during training using the full historical dataset are computed differently at serving time using real-time data leading to performance degradation. A feature store has two components: offline store (historical feature values for training — typically a data warehouse like BigQuery or Redshift) and online store (latest feature values for low-latency serving — typically Redis or DynamoDB). Feature pipelines write to both stores ensuring identical computation logic. Feature engineering logic is defined once and shared — a 'user_30_day_purchase_total' feature computed for a recommendation model can be reused by a fraud model without re-implementation. Modern feature stores (Feast Tecton Hopsworks) also handle: feature versioning (audit trail) feature sharing across teams and point-in-time correct feature lookup (critical for preventing temporal data leakage in training).

Real-World: At a major e-commerce company the customer lifetime value model the recommendation model and the fraud model all needed 'user_purchase_frequency_last_30_days'. Before the feature store each team computed it differently with subtle differences (timezone handling business day vs calendar day) producing inconsistent results. The feature store defined one authoritative computation shared across all three models.

⚠ Common Mistakes: Implementing offline-only features (fast to build but creates training-serving skew when serving). Computing features in the model serving code itself (no reuse performance overhead). Not handling point-in-time correctness in the offline store (using features from after the label timestamp in training data — a form of feature leakage). Building a feature store before having more than 2-3 models (premature optimization).

🏭 Production Scenario: A churn prediction model performing at 0.84 AUC in offline evaluation dropped to 0.71 AUC in production. Investigation revealed that customer engagement features were computed using UTC timestamps in training but local time in the serving API — a seemingly minor difference that caused dramatic feature value shifts for users in non-UTC timezones. Centralizing feature computation in a feature store with explicit timezone handling fixed the skew.

Follow-up questions: What is point-in-time correct feature lookup and why does it prevent data leakage? What is the difference between Feast Tecton and Hopsworks? How do you handle feature freshness requirements for real-time models?

// ID: ML-ADV-003  ·  DIFFICULTY: 9/10  ·  ★★★★★★★★★☆