Generative AI and LLM questions — RAG, prompts, hallucinations, and production basics.
An LLM is a neural network trained on massive text to predict the next token, enabling it to generate human-like text, code, and summaries. Examples: GPT-4 class models, Claude, Gemini, Llama. They power chatbots, copilots, and RAG assistants in product teams.
Discriminative models classify or predict labels (spam vs not spam). Generative models create new content (text, images, code). LLMs are generative; you still use discriminative models for fraud detection or ranking alongside GenAI pipelines.
Crafting instructions, examples, and constraints so the model produces reliable outputs. Techniques: clear role ('You are a Java tutor'), few-shot examples, output format (JSON only), chain-of-thought for reasoning tasks. Critical for interns building internal tools without fine-tuning.
Tokens are subword pieces the model reads/writes; billing and context limits are token-based. Long prompts + long answers cost more and fill the context window. Optimization: summarize history, retrieve only relevant chunks (RAG), and cache repeated system prompts.
Maximum tokens the model can attend to in one request (prompt + completion). Exceeding it truncates or fails. Large windows help long documents but increase latency and cost. Design apps to chunk documents and retrieve selectively.
Sampling randomness: low temperature (0–0.
Model confidently states false facts. Mitigations: RAG with verified sources, ask model to cite passages, lower temperature, structured outputs, human review for high-stakes answers, and evaluation sets measuring factual accuracy on your domain.
Retrieve relevant documents from a vector database using embedding similarity, inject them into the prompt, then generate an answer grounded in those chunks. Standard pattern for company knowledge bases, college notes bots, and support assistants.
Dense vector representations of text where semantic similarity ≈ cosine similarity in vector space. Used for search, clustering, and RAG retrieval. Produced by embedding models (OpenAI, Cohere, open-source E5/BGE). Store in vector DB (Pinecone, pgvector, Weaviate).
Prompt + RAG: fastest for changing knowledge without retraining. Fine-tuning: adapt style, format, or domain when you have quality labeled data and stable requirements. Full training from scratch is rare for product teams. Most campus projects combine API + RAG.
LLM loop that plans steps, calls tools (APIs, SQL, search), observes results, and continues until task completes. Needs guardrails: allowed tools, max steps, human approval for destructive actions. Different from single-shot Q&A chatbots.
Input/output filters (PII redaction, toxicity), topic boundaries, rate limits, auth, logging, and refusal policies for unsafe requests. Enterprise Indian IT clients demand compliance — mention DPDP Act awareness and not sending secrets to public APIs.
Offline: golden Q&A sets, human rubrics, semantic similarity, LLM-as-judge with caution. Online: thumbs up/down, task success rate, latency, cost per session. A/B test prompt versions. Avoid evaluating only with vibe checks.
Ask the model to reason step by step before the final answer, improving multi-step math and logic tasks. Can be hidden from end users (internal reasoning). Trade-off: more tokens and latency.
Passwords, API keys, Aadhaar/PAN, unreleased financial data, customer PII without consent, proprietary source code under strict NDAs. Use enterprise agreements, private endpoints, or on-prem models when required.
Proprietary APIs: best quality, managed scaling, higher ongoing cost, data policy depends on vendor. Open models (Llama, Mistral): self-host for control and cost at scale but need GPU ops expertise. Many startups hybridize.
Model returns structured JSON specifying which function to invoke (getWeather, queryDB) with arguments; your app executes and feeds result back. Enables reliable integrations vs parsing free-form text. Core to copilots and agents.
Assists code review, test generation, docs, and boilerplate — but humans own architecture, security, and correctness. Teams expect you to use copilots responsibly, verify outputs, and write tests. Interview angle: AI augments, not replaces, fundamentals.
User behavior or upstream model updates change output quality over time. Monitor eval metrics, log failures, version prompts and models, rollback when regressions detected. Similar to ML ops but faster iteration cycle.
Example: 'Campus placement FAQ bot — PDF notes chunked, embedded into pgvector, user question retrieves top-k chunks, GPT answers with citations, fallback if confidence low.' Mention stack, evaluation, and privacy — shows practical, hands-on thinking.
