Debasis Bhattacharjee

pytest discovers and runs test functions automatically providing rich assertion introspection fixtures for dependency injection and parametrize for data-driven tests. A good unit test is fast isolated deterministic and tests one specific behavior.

Deep Dive: pytest looks for files named test_*.py functions named test_* and classes named Test*. When an assert fails pytest shows you exactly what the actual and expected values were — no need for assertEqual(). Fixtures (@pytest.fixture) provide setup/teardown and dependency injection for tests — database connections temporary files mock objects. Parametrize (@pytest.mark.parametrize) runs the same test with multiple input/output combinations eliminating test duplication. Mocking with unittest.mock.patch replaces real dependencies with controlled fakes making tests fast and isolated. Good unit tests: test one behavior run in milliseconds do not hit databases/networks/file systems (mock these) are deterministic (same result every run) and fail with clear messages.

Real-World: A FastAPI endpoint test: the test uses a pytest fixture providing a TestClient (mock HTTP client) patches the database dependency with an in-memory mock uses parametrize to test valid/invalid/edge case inputs and has clear test names like test_create_user_returns_201_for_valid_input. Each test runs in under 5ms with no external dependencies.

⚠ Common Mistakes: Writing tests that test implementation details instead of behavior — tests should not break when you refactor internals. Not mocking external dependencies making tests slow and flaky. Using a single large test function that tests multiple behaviors (impossible to tell which behavior failed). Asserting too broadly (assert response is not None) or too narrowly (asserting on exact internal state).

🏭 Production Scenario: A Django e-commerce platform's test suite took 45 minutes to run because 800 tests were hitting the actual test database. Refactoring to use pytest fixtures with database mocking and factory_boy for test data generation reduced the suite to 3 minutes enabling CI to run on every commit.

Follow-up questions: What is the difference between mocking and stubbing? How do you test async functions with pytest? What is property-based testing with Hypothesis?

// ID: PY-INT-006  ·  DIFFICULTY: 4/10  ·  ★★★★☆☆☆☆☆☆

Use generators file iteration (files are iterators in Python) or chunk-based reading. Never use read() or readlines() on large files — they load the entire file into memory.

Deep Dive: Python file objects are iterators — you can iterate over them line by line without loading the entire file. For binary files or files where line iteration is not appropriate use file.read(chunk_size) to read fixed-size chunks in a loop. For CSV files use csv.DictReader (which iterates lazily) or pandas with chunksize parameter (pd.read_csv('file.csv' chunksize=10000) returns an iterator of DataFrames). For JSON use ijson for streaming JSON parsing. The with statement ensures the file is properly closed. For very large files (100GB+) memory-mapped files (mmap module) allow treating file content as if it were in memory while the OS handles paging.

Real-World: A log analysis system needed to process 50GB daily log files to extract error counts. Using open(file).read() caused OOM crashes. Refactoring to iterate line by line (for line in file) reduced memory usage from 50GB to under 10MB while processing the same file.

⚠ Common Mistakes: Using file.readlines() which builds a complete list of all lines in memory. Using pd.read_csv() without chunksize on multi-GB files. Not closing files (always use with statement). Forgetting to handle encoding explicitly — defaulting to system encoding causes silent corruption on non-ASCII data.

🏭 Production Scenario: A production data pipeline at a logistics company was crashing nightly when processing a 30GB shipment data CSV. The fix used pandas chunked reading: processing 50000 rows at a time aggregating results and writing summaries — reducing peak memory from 45GB (crashing the server) to 2GB.

Follow-up questions: What is the mmap module and when would you use it? How does ijson enable streaming JSON parsing? How do you process a large file in parallel in Python?

// ID: PY-INT-005  ·  DIFFICULTY: 4/10  ·  ★★★★☆☆☆☆☆☆

Context length is the maximum number of tokens an LLM can process in a single call (input + output combined). It determines how much text you can send and receive. Exceeding it causes errors or truncation and longer contexts increase cost and latency.

Deep Dive: Tokens are the fundamental units LLMs process — roughly 3-4 characters or 0.75 words per token in English. Context length limits how much text fits in one API call: GPT-4's 128k context allows roughly 96000 words while smaller models might allow only 4096 tokens. The entire prompt (system prompt + conversation history + retrieved documents + user message) plus the response must fit within this limit. Context length matters for: conversation history management (older messages must be truncated or summarized) RAG systems (limiting how many retrieved chunks can be included) document processing (whether you process entire documents or must chunk them) and cost (most APIs charge per token — 128k context calls cost much more than 4k calls even for short responses).

Real-World: A legal contract analysis system tried to process 200-page contracts as a single API call. For contracts over the context limit the API truncated silently (depending on the implementation) causing the model to analyze only part of the contract and miss critical clauses. The fix required a map-reduce approach: analyze sections independently then synthesize.

⚠ Common Mistakes: Assuming context length = input length (output tokens count against the limit too). Sending entire conversation history without truncation strategy causing errors as conversations grow. Not monitoring token usage in production getting surprised by cost and latency. Thinking larger context is always better — models have attention degradation in very long contexts (the 'lost in the middle' problem).

🏭 Production Scenario: A customer service chatbot was working correctly in testing (short conversations) but failing in production for customers with long support history. Investigation revealed conversations exceeding the context limit caused the API to throw errors. Fix required implementing a sliding window that kept the system prompt + last 10 messages + current message within limits.

Follow-up questions: What is the 'lost in the middle' attention problem in long contexts? How do you implement conversation history management? What is the difference between context window and KV cache?

// ID: AI-INT-003  ·  DIFFICULTY: 5/10  ·  ★★★★★☆☆☆☆☆

Temperature controls the randomness of token selection by scaling the probability distribution. Top_p (nucleus sampling) limits selection to the smallest set of tokens whose cumulative probability exceeds p. Both control output diversity but differently.

Deep Dive: Language models output a probability distribution over the vocabulary for the next token. Temperature scales this distribution before sampling. Temperature=1 is the raw distribution. Temperature1 flattens it (more random more creative more likely to produce unusual tokens). Temperature=0 is greedy — always picks the highest probability token. Top_p=0.9 means: sort tokens by probability keep the top tokens until their cumulative probability reaches 90% sample only from those. This dynamically adjusts the candidate set size based on the distribution shape. Use temperature for general creativity control. Use top_p for better diversity control when the distribution is very peaked. Most APIs recommend using one or the other not both simultaneously.

Real-World: A customer support chatbot needs low temperature (0.1-0.3) for consistent accurate responses to FAQs. A creative writing assistant needs higher temperature (0.7-0.9) for varied imaginative output. A code generation tool typically uses temperature=0 or very low values because there is usually one correct answer and creativity increases bugs.

⚠ Common Mistakes: Using temperature=0 for tasks requiring diversity (the model gets stuck in repetitive loops). Using high temperature for factual tasks (increases hallucination significantly). Setting both temperature and top_p to non-default values — they interact in complex ways and most practitioners use one or the other. Not understanding that temperature=0 does not mean truly deterministic — floating point variations can still cause different outputs.

🏭 Production Scenario: A legal document summarization API was producing inconsistent outputs that caused compliance issues. The temperature was set to 0.7 (appropriate for creative tasks) by a developer who copied settings from a creative writing example. Setting temperature to 0.1 made outputs consistent and predictable for the compliance use case.

Follow-up questions: What is top_k sampling? How does beam search differ from sampling? What is the repetition penalty parameter?

// ID: AI-INT-001  ·  DIFFICULTY: 5/10  ·  ★★★★★☆☆☆☆☆

A decorator is a function that wraps another function to add behavior. Without functools.wraps the wrapper loses the original function's metadata like __name__ and __doc__.

Deep Dive: Decorators work by taking a function as input and returning a new function that adds behavior before or after the original call. The syntax @decorator is syntactic sugar for function = decorator(function). The core problem is that the returned wrapper function has its own identity — its __name__ is 'wrapper' not the original function's name. This breaks logging debugging and documentation tools. functools.wraps(original_func) applied to the wrapper copies the original function's metadata to the wrapper. This is especially critical in Flask and FastAPI where the routing system uses function names to identify view functions — without wraps all decorated routes have the same name and only one will be registered.

Real-World: In a Flask application a custom authentication decorator without functools.wraps caused all protected routes to map to the same endpoint name 'wrapper' making url_for() return wrong URLs and breaking the entire navigation system. Adding @functools.wraps(f) to the inner wrapper function fixed it immediately.

⚠ Common Mistakes: Forgetting @functools.wraps on the inner wrapper function. Decorators that do not preserve the function signature breaking tools that inspect function parameters. Applying decorators in the wrong order when stacking multiple decorators.

🏭 Production Scenario: A production Flask API broke its authentication after a refactor added a logging decorator without functools.wraps. The route registration system saw multiple routes all named 'wrapper' and silently dropped all but one making several API endpoints return 404 despite the code being correct.

Follow-up questions: How do class-based decorators work? How do you write a decorator that accepts its own arguments? How does decorator stacking (applying multiple decorators) work in Python?

// ID: PY-INT-002  ·  DIFFICULTY: 5/10  ·  ★★★★★☆☆☆☆☆

Chain-of-thought (CoT) prompting asks the LLM to show its reasoning step by step before giving a final answer. It significantly improves performance on multi-step reasoning tasks: math logic code debugging and complex analysis. It does not help (and can hurt) simple classification or recall tasks.

Deep Dive: Standard prompting asks for the answer directly. CoT prompting adds 'Let's think step by step' or provides examples where the reasoning is shown before the answer. The improvement comes from the model using its output tokens to work through intermediate reasoning steps — effectively using the context window as a scratchpad. Zero-shot CoT adds 'think step by step'. Few-shot CoT provides worked examples. Auto-CoT automatically generates reasoning chains. CoT helps when: the task requires multiple steps errors in early steps compound (math logic) or when the model needs to 'check its work'. CoT does NOT help for: simple fact retrieval single-step tasks or tasks where the reasoning process cannot be decomposed into steps.

Real-World: A financial analysis assistant was making errors on complex revenue calculations with multiple steps. Adding 'Calculate step by step showing each calculation:' to the prompt reduced calculation errors by 65% because the model would catch its own arithmetic mistakes when the intermediate steps were visible.

⚠ Common Mistakes: Using CoT for every task regardless of complexity — it increases token usage and cost with no benefit for simple tasks. Not providing few-shot CoT examples for novel reasoning patterns — zero-shot CoT underperforms when the reasoning pattern is unfamiliar. Trusting CoT reasoning as ground truth — the model can reason confidently but incorrectly.

🏭 Production Scenario: A legal contract analysis tool was misclassifying contract risk levels. The system prompt was updated to require: 'First identify all risk factors present. Then assess the severity of each. Then determine the aggregate risk level. Finally state your conclusion.' This structured CoT approach improved classification accuracy from 71% to 88%.

Follow-up questions: What is self-consistency prompting? What is tree-of-thought prompting? How do you measure whether CoT is actually helping for a specific task?

// ID: AI-INT-004  ·  DIFFICULTY: 5/10  ·  ★★★★★☆☆☆☆☆

Context managers use __enter__ and __exit__ methods to manage setup and teardown of resources. The 'with' statement calls these automatically ensuring cleanup even if an exception occurs.

Deep Dive: When you use 'with open(file) as f' Python calls f.__enter__() to set up and f.__exit__() to clean up. You can create custom context managers two ways: implement __enter__ and __exit__ in a class or use the @contextmanager decorator from contextlib with a generator function that yields once. The __exit__ method receives exception information and can suppress exceptions by returning True. Context managers are the Pythonic way to handle any resource that needs guaranteed cleanup: database connections locks temporary directories timers and transaction management.

Real-World: A database transaction context manager in a Django-like ORM: __enter__ begins the transaction __exit__ commits if no exception occurred or rolls back if one did. This pattern ensures no transaction is ever left open regardless of what happens inside the with block.

⚠ Common Mistakes: Not handling exceptions in __exit__ letting them propagate when they should be caught. Creating context managers with @contextmanager and forgetting to wrap the yield in try-finally skipping cleanup on exceptions. Using try-finally everywhere instead of the cleaner with statement.

🏭 Production Scenario: A production PostgreSQL service had intermittent connection failures traced to database transactions being left open. The root cause was exception handling that bypassed the connection cleanup code. Refactoring to use a context manager with proper __exit__ eliminated the issue permanently.

Follow-up questions: What is the contextlib module? How do nested context managers work? What is contextlib.ExitStack used for?

// ID: PY-INT-004  ·  DIFFICULTY: 5/10  ·  ★★★★★☆☆☆☆☆

Python dictionaries are hash tables. Lookup insertion and deletion are O(1) average case. Hash collisions can degrade this to O(n) worst case but Python's implementation makes this extremely rare. Python 3.7+ guarantees insertion-order preservation.

Deep Dive: Dictionaries store key-value pairs in a hash table. When you set d[key] = value Python computes hash(key) maps it to a bucket and stores the value. When you access d[key] Python recomputes the hash and looks up the bucket directly — O(1). Hash collisions (two different keys mapping to the same bucket) are resolved via open addressing in CPython. Python 3.6 introduced a compact dictionary representation that stores insertion order as a side effect. Python 3.7 made insertion order preservation official. Only hashable objects can be dictionary keys (immutable types: strings integers tuples — but not lists or other dicts). dict.get(key default) avoids KeyError for missing keys. collections.defaultdict automatically creates default values. collections.Counter counts hashable objects.

Real-World: In a word frequency counter processing millions of log lines dict-based counting with Counter outperforms sorting-based approaches by orders of magnitude — O(n) with hash table vs O(n log n) for sort-then-count. In a URL routing system a dict of {path: handler} enables O(1) route lookup regardless of how many routes exist.

⚠ Common Mistakes: Using a list to check membership (if item in list is O(n) — use a set or dict instead). Modifying a dictionary while iterating over it (raises RuntimeError — iterate over list(d.items()) instead). Using mutable objects as dictionary keys (unhashable type TypeError). Not using setdefault() or defaultdict() and writing verbose if-key-in-dict patterns instead.

🏭 Production Scenario: A production request deduplication service was checking if a request ID had been seen using a list (if request_id in seen_list). At 10000 requests per second the O(n) membership check was consuming 60% of CPU time. Replacing with a set (O(1) lookup) reduced CPU usage to 2% with identical functionality.

Follow-up questions: How does Python set differ from dict internally? What is the difference between dict and OrderedDict after Python 3.7? What is dict comprehension and when should you use defaultdict instead?

// ID: PY-INT-008  ·  DIFFICULTY: 5/10  ·  ★★★★★☆☆☆☆☆

Precision is the fraction of positive predictions that are actually positive. Recall is the fraction of actual positives that were correctly identified. F1 is their harmonic mean. Which matters depends on the cost of each type of error.

Deep Dive: Precision = TP / (TP + FP). High precision means when you predict positive you are usually right (few false alarms). Recall = TP / (TP + FN). High recall means you catch most actual positives (few misses). There is a precision-recall tradeoff — increasing the classification threshold raises precision but lowers recall. F1 score = 2 * (precision * recall) / (precision + recall) balances both. Choose based on business cost: in spam detection low precision (legitimate emails marked spam) is worse than low recall (some spam gets through) — optimize precision. In cancer screening low recall (missing cancers) is catastrophic — optimize recall. In fraud detection both matter differently depending on churn cost vs fraud loss.

Real-World: A medical imaging AI for tumor detection: recall is paramount — missing a tumor (false negative) is far worse than a false alarm (false positive) that leads to an additional test. The model was tuned to 98% recall at 60% precision flagging many non-tumors for human review rather than risking misses.

⚠ Common Mistakes: Using accuracy as the primary metric for imbalanced datasets — 99% accuracy on a dataset where 99% of examples are negative tells you nothing useful. Not understanding that F1 is undefined when both precision and recall are zero. Optimizing the wrong metric because the business cost of each error type was not clearly defined.

🏭 Production Scenario: A production spam filter optimized for F1 score was generating too many false positives (legitimate business emails marked as spam). The client measured success by user complaints about missed emails not by spam caught. Reframing as a precision optimization problem and raising the threshold resolved the operational issue.

Follow-up questions: What is the ROC curve and AUC? When do you use macro vs micro vs weighted averaging for multiclass metrics? What is the Matthews Correlation Coefficient?

// ID: ML-INT-002  ·  DIFFICULTY: 5/10  ·  ★★★★★☆☆☆☆☆

A Random Forest builds many decision trees on random subsets of data and features then aggregates their predictions. It is better than a single tree because averaging many uncorrelated trees reduces variance without increasing bias.

Deep Dive: A single decision tree is prone to overfitting — it can grow arbitrarily complex and memorize training data. Random Forest addresses this with two randomness sources: bagging (each tree trains on a bootstrap sample — random sample with replacement of the training data) and feature randomness (at each split only a random subset of features is considered). These two mechanisms ensure the trees are decorrelated. Aggregating many decorrelated slightly overfit trees through voting (classification) or averaging (regression) dramatically reduces variance. Random Forests also provide feature importance scores by measuring how much each feature reduces impurity across all trees.

Real-World: At a financial institution a Random Forest model for loan default prediction consistently outperformed single decision trees by 8-12% AUC across quarterly retraining cycles. The interpretability of feature importance scores also helped explain decisions to regulators making it preferable to black-box alternatives.

⚠ Common Mistakes: Assuming more trees always help — there is a point of diminishing returns (typically 100-500 trees). Not tuning max_depth and min_samples_split allowing trees to overfit. Ignoring class imbalance when using Random Forest for classification. Using Random Forest for very high-dimensional sparse data where gradient boosting typically performs better.

🏭 Production Scenario: A production fraud detection model using a single deep decision tree had to be retrained daily due to instability — small changes in training data caused large swings in predictions. Switching to a Random Forest made predictions stable across daily retraining reducing manual monitoring overhead significantly.

Follow-up questions: What is the difference between Random Forest and Gradient Boosting? How does feature importance work in Random Forest? What is out-of-bag (OOB) error?

// ID: ML-INT-001  ·  DIFFICULTY: 5/10  ·  ★★★★★☆☆☆☆☆