∇GradientSchool
Intermediate
15 min read
#generative ai#Guide

Generative AI for Code

Comprehensive guide on Generative AI for Code.

Generative AI for Code

1. Learning Objectives#

By the end of this notebook, you should be able to:

  1. Explain how generative AI models represent and generate source code.
  2. Understand code language models, code completion, code generation, and code transformation.
  3. Distinguish code generation from repository-level software engineering.
  4. Design effective prompts and structured workflows for coding assistants.
  5. Understand repository context, code RAG, embeddings, and dependency-aware retrieval.
  6. Build AI-assisted workflows for implementation, debugging, refactoring, testing, and documentation.
  7. Design coding agents that use files, terminals, test runners, linters, and version control safely.
  8. Understand code execution sandboxes and security boundaries.
  9. Evaluate generated code for correctness, security, maintainability, and style.
  10. Understand unit-test generation, test-driven generation, and verification loops.
  11. Design repository-aware and multi-file coding systems.
  12. Apply human-in-the-loop controls to high-impact code changes.
  13. Understand software-engineering agents, planning, state, tool use, and recovery.
  14. Build production-grade coding assistants and practical portfolio projects.

2. Why Generative AI for Code Matters

Code is a particularly interesting target for generative AI because software is both:

  • natural-language-like
  • highly structured

A coding model can generate:

Architecture & Data Flow
Natural language intent
 |
 v
 Code
 |
 v
 Compiler / Runtime
 |
 v
 Observable result

This creates something special.

For ordinary text, correctness can be subjective.

For code, we can often execute the generated artifact and obtain measurable feedback.

That enables a powerful loop:

Architecture & Data Flow
Generate
 |
 v
Execute
 |
 v
Test
 |
 v
Observe failure
 |
 v
Repair
 |
 v
Test again

This verification loop is one of the most important ideas in AI-assisted software engineering.

3. Code Language Models

A code language model learns patterns from programming-related data such as:

  • source code
  • documentation
  • configuration
  • tests
  • examples
  • repository structure
  • issue descriptions
  • natural-language instructions

At a simplified level:

Architecture & Data Flow
Input tokens
 |
 v
Transformer
 |
 v
Probability distribution
 |
 v
Next token

For code generation:

Architecture & Data Flow
"Create a Python function that..."
 |
 v
 Code model
 |
 v
 Python tokens
 |
 v
 Source code

The model is not a compiler.

It generates probable code.

Compilation, execution, tests, and static analysis provide additional verification.

4. Code Is More Than Text

Consider:

🐍 Python
def calculate_total(items):
 return sum(item.price for item in items)

A model must understand relationships such as:

Architecture & Data Flow
calculate_total
 |
 +--> items
 |
 +--> item.price
 |
 +--> sum()

At repository scale, relationships become much larger:

Architecture & Data Flow
API endpoint
 |
 v
Service
 |
 v
Repository
 |
 v
Database

Therefore, advanced coding systems need more than a single prompt.

They need context about the software system.

5. Code Completion

Code completion predicts what should come next.

Example:

🐍 Python
def calculate_discount(price, rate):
 return

The model might complete:

🐍 Python
price * (1 - rate)

Completion is generally a narrow task.

The model has immediate context:

Architecture & Data Flow
Current file
 |
 v
Local context
 |
 v
Prediction

This can be efficient and highly useful.

6. Code Generation

Code generation starts from an explicit task.

Example:

›"Create a FastAPI endpoint that returns
employee information by employee ID."

The model may generate:

🐍 Python
@app.get("/employees/{employee_id}")
def get_employee(employee_id: str):
 ...

But generated code should not automatically be considered production-ready.

It needs:

  • validation
  • tests
  • security review
  • integration checks
  • style checks
  • dependency checks

7. Code Transformation

Generative AI can transform existing code.

Examples:

Architecture & Data Flow
Python -> TypeScript
REST -> GraphQL
Sync -> Async
Old API -> New API
Monolith -> Service

Another common transformation:

Architecture & Data Flow
Unreadable code
 |
 v
AI refactoring
 |
 v
Readable code

The transformation should preserve behavior unless behavior change is explicitly requested.

8. Code Explanation

A coding assistant can explain:

  • functions
  • classes
  • modules
  • algorithms
  • dependencies
  • errors
  • architecture

Example:

Architecture & Data Flow
Repository
 |
 v
Select function
 |
 v
AI explanation
 |
 +--> Purpose
 +--> Inputs
 +--> Outputs
 +--> Dependencies
 +--> Side effects

Explanation quality improves when the model has the relevant repository context.

9. Code Debugging

Debugging is naturally iterative.

Architecture & Data Flow
Bug report
 |
 v
Inspect code
 |
 v
Generate hypothesis
 |
 v
Modify code
 |
 v
Run test
 |
 +--> Pass --> Finish
 |
 +--> Fail --> Inspect
 |
 v
 Repair

A strong coding agent should use actual execution results instead of relying only on reasoning.

10. Why Execution Feedback Matters

Suppose the model generates:

🐍 Python
result = total / count

The model may not know whether:

Mathematical Formulation
count == 0

is possible.

A test can reveal:

›ZeroDivisionError

Now the agent has concrete evidence.

This creates:

Architecture & Data Flow
Model hypothesis
 |
 v
Executable experiment
 |
 v
Observed evidence
 |
 v
Improved implementation

This is one of the biggest advantages of code generation over unconstrained text generation.

11. Code as a Verifiable Artifact

A useful mental model:

Architecture & Data Flow
Natural language
 |
 v
Generated code
 |
 +--> Compiler
 +--> Type checker
 +--> Linter
 +--> Unit tests
 +--> Integration tests
 +--> Security scanner
 |
 v
Evidence

The AI should be one component of a verification pipeline.

12. Prompting for Code

A weak prompt:

›"Write an API."

A stronger prompt:

text
Build a FastAPI endpoint for employee lookup.

Requirements:
- GET /employees/{employee_id}
- validate employee_id
- return structured JSON
- return 404 when not found
- separate route and service logic
- include unit tests
- do not expose sensitive fields

Useful prompt components:

  • objective
  • constraints
  • existing architecture
  • interfaces
  • expected inputs
  • expected outputs
  • error behavior
  • testing requirements

13. Structured Coding Prompts

A practical structure:

text
Goal
Context
Existing interfaces
Constraints
Requirements
Non-requirements
Expected output
Tests
Acceptance criteria

Example:

text
Goal:
Add employee lookup.

Context:
FastAPI application with service/repository layers.

Constraints:
Do not modify database schema.

Requirements:
GET /employees/{employee_id}

Acceptance:
404 when employee does not exist.
200 with approved fields when found.

This reduces ambiguity.

14. Repository Context

A real project may contain:

text
project/
├── app/
│ ├── api/
│ ├── services/
│ ├── repositories/
│ └── models/
├── tests/
├── config/
├── migrations/
├── requirements.txt
└── README.md

Sending the entire repository to the model is often inefficient.

Instead:

Architecture & Data Flow
User request
 |
 v
Repository retrieval
 |
 v
Relevant files
 |
 v
Context construction
 |
 v
Coding model

This is code-focused RAG.

15. Code RAG

Code RAG retrieves relevant repository information.

Potential retrieval units:

  • files
  • functions
  • classes
  • symbols
  • documentation
  • tests
  • configuration
  • dependency metadata

Architecture:

Architecture & Data Flow
Repository
 |
 v
Parser / Indexer
 |
 +--> Symbols
 +--> Imports
 +--> Functions
 +--> Classes
 +--> Docs
 |
 v
Search / Retrieval
 |
 v
Relevant context
 |
 v
Coding model

16. Why Naive Code Chunking Fails

Generic text chunking may split:

🐍 Python
class UserService:
 ...

from:

🐍 Python
def create_user(...):
 ...

This can destroy useful structure.

Better code retrieval can preserve:

  • file boundaries
  • symbol boundaries
  • class relationships
  • imports
  • call relationships
  • tests

17. Symbol-Aware Retrieval

Instead of retrieving arbitrary character ranges:

›characters 1-1000

retrieve semantic units:

Architecture & Data Flow
Class: UserService
 |
 +--> create_user()
 +--> update_user()
 +--> delete_user()

This gives the model more coherent context.

18. Dependency-Aware Retrieval

Suppose the requested function is:

›create_order()

The relevant context may include:

Architecture & Data Flow
create_order()
 |
 +--> validate_order()
 |
 +--> OrderRepository
 |
 +--> PaymentService
 |
 +--> Order model
 |
 +--> tests

A good coding assistant should retrieve important dependencies, not just the target function.

19. Repository Graphs

A repository can be represented as a graph:

Architecture & Data Flow
File A
 |
 +--> imports File B
 |
 +--> calls Function C
 |
 +--> uses Class D

Graph-based retrieval can combine:

text
Semantic similarity
+
Symbol relationships
+
Dependency relationships

This is often more useful than vector search alone.

20. Code Embeddings

Code can be embedded into vectors.

Architecture & Data Flow
Function
 |
 v
Code embedding
 |
 v
Vector index

A query:

›"Where is employee authentication handled?"

can retrieve semantically relevant code.

Hybrid retrieval can combine:

text
Keyword search
+
Symbol search
+
Vector search
+
Dependency graph

21. Hybrid Code Search

A robust system may use:

Architecture & Data Flow
Query
 |
 +--> Keyword search
 |
 +--> Symbol search
 |
 +--> Vector search
 |
 +--> Dependency traversal
 |
 v
Candidate files
 |
 v
Reranking
 |
 v
Final context

This reduces the risk of missing exact symbol names while preserving semantic search.

22. Repository Summaries

Large repositories can maintain summaries:

Architecture & Data Flow
Repository summary
 |
 +--> Architecture
 +--> Services
 +--> APIs
 +--> Data layer
 +--> Important conventions
 +--> Security rules

Then retrieve detailed code only when needed.

This creates hierarchical context:

Architecture & Data Flow
Repository
 |
 v
Architecture summary
 |
 v
Service summary
 |
 v
Relevant files
 |
 v
Relevant symbols

23. Codebase Understanding

An advanced coding agent should understand:

  • architecture
  • conventions
  • dependency structure
  • build system
  • tests
  • deployment
  • configuration
  • API contracts

Example:

Architecture & Data Flow
User request
 |
 v
Understand repository
 |
 +--> Architecture
 +--> Relevant module
 +--> Existing patterns
 +--> Tests
 |
 v
Plan change

This is repository-level reasoning.

24. Coding Agent Architecture

Architecture & Data Flow
+------------------------------------------------------+
| Coding Agent |
| |
| Goal |
| | |
| v |
| Planner |
| | |
| v |
| Repository Retriever |
| | |
| v |
| Context Builder |
| | |
| v |
| Coding Model |
| | |
| v |
| Proposed Change |
| | |
| v |
| Policy / Diff Validator |
| | |
| v |
| Sandbox |
| | |
| +--> Tests |
| +--> Linter |
| +--> Type checker |
| +--> Build |
| | |
| v |
| Results |
| | |
| v |
| Repair / Verify |
+------------------------------------------------------+

25. Tool Use for Coding Agents

Useful tools include:

text
read_file()
search_code()
list_directory()
write_file()
apply_patch()
run_tests()
run_linter()
run_type_checker()
run_build()
git_diff()
git_status()

Each tool should have clear boundaries.

Avoid:

›execute_any_shell_command()

when a narrower tool can perform the task.

26. Diff-Based Editing

Instead of rewriting entire files, an agent can produce a patch.

Architecture & Data Flow
Original file
 |
 v
Patch
 |
 v
Validated result

Example:

diff
- return user.name
+ return {"name": user.name}

Advantages:

  • smaller changes
  • easier review
  • easier rollback
  • easier auditing

27. Human Code Review

A coding agent should not automatically merge every change.

A safer workflow:

Architecture & Data Flow
Agent
 |
 v
Generate patch
 |
 v
Run tests
 |
 v
Run security checks
 |
 v
Human review
 |
 +--> Approve --> Merge
 |
 +--> Reject --> Revise

For critical repositories, human review should remain mandatory.

28. Test-Driven Code Generation

A powerful workflow:

Architecture & Data Flow
Requirement
 |
 v
Generate tests
 |
 v
Generate implementation
 |
 v
Run tests
 |
 +--> Fail --> Repair
 |
 +--> Pass --> Review

Tests provide an executable specification.

However, generated tests can themselves be incorrect.

Test quality must also be reviewed.

29. Unit-Test Generation

Suppose:

🐍 Python
def add(a, b):
 return a + b

The assistant might generate:

🐍 Python
def test_add():
 assert add(2, 3) == 5

More complete testing should consider:

  • normal cases
  • boundary values
  • invalid input
  • exceptions
  • empty input
  • large input

30. Property-Based Testing

Instead of testing only examples, test properties.

For example:

Mathematical Formulation
Sorting property:

sort(sort(x)) == sort(x)

An AI coding system can propose properties and tests.

This can reveal bugs that example-based tests miss.

31. Test Execution Loop

Architecture & Data Flow
Generated code
 |
 v
Unit tests
 |
 +--> Pass
 |
 +--> Fail
 |
 v
 Error analysis
 |
 v
 Patch
 |
 v
 Tests again

Limit the loop.

Otherwise the agent may endlessly modify code.

32. Static Analysis

Generated code should be checked with:

  • linters
  • formatters
  • type checkers
  • dependency scanners
  • static analyzers

Pipeline:

Architecture & Data Flow
Generated code
 |
 +--> Formatter
 |
 +--> Linter
 |
 +--> Type checker
 |
 +--> Security scanner
 |
 v
Quality gate

Static analysis is a valuable deterministic complement to model reasoning.

33. Security of Generated Code

AI-generated code can introduce:

  • injection vulnerabilities
  • insecure authentication
  • broken access control
  • unsafe deserialization
  • secrets exposure
  • weak cryptography
  • path traversal
  • command injection
  • SQL injection

Never assume generated code is secure because it looks professional.

34. Secure Code Generation

A security-aware prompt might include:

text
Requirements:
- validate all external input
- use parameterized database queries
- do not log secrets
- enforce authorization
- avoid shell execution
- include negative security tests

But prompts are not a substitute for security tooling and review.

35. Generated SQL

A model may generate:

sql
SELECT * FROM employees WHERE id = '...';

The application should use parameterized queries.

For example:

🐍 Python
cursor.execute(
 "SELECT name FROM employees WHERE id = ?",
 (employee_id,)
)

Do not concatenate untrusted input into SQL.

36. Generated Shell Commands

A dangerous pattern:

🐍 Python
os.system(user_generated_command)

A safer architecture uses narrow tools.

Instead of:

›Agent -> arbitrary shell

prefer:

Architecture & Data Flow
Agent -> run_tests()
Agent -> build_project()
Agent -> inspect_git_diff()

Each tool can enforce its own constraints.

37. Sandboxed Code Execution

Some coding agents need to execute generated code.

Use isolation:

Architecture & Data Flow
Coding Agent
 |
 v
Execution Sandbox
 |
 +--> CPU limit
 +--> Memory limit
 +--> Time limit
 +--> Filesystem isolation
 +--> Network policy
 |
 v
Test result

Never execute untrusted generated code directly on a production host.

38. Repository Permissions

A coding agent should receive limited access.

Example:

Architecture & Data Flow
Agent workspace
 |
 +--> Read source
 +--> Modify selected files
 +--> Run tests
 +--> Generate patch
 |
 X--> Production secrets
 X--> Production credentials
 X--> Unrelated repositories
 X--> Deployment credentials

Separate development and deployment authority.

39. Git as an Agent Safety Boundary

Git provides useful controls:

Architecture & Data Flow
Working tree
 |
 v
Diff
 |
 v
Review
 |
 v
Commit
 |
 v
Pull request
 |
 v
CI
 |
 v
Merge

An agent should generally work through reviewable changes rather than directly rewriting production state.

40. Agentic Software Engineering Loop

A mature coding agent may follow:

Architecture & Data Flow
Understand issue
 |
 v
Inspect repository
 |
 v
Form plan
 |
 v
Implement
 |
 v
Run tests
 |
 v
Inspect failures
 |
 v
Repair
 |
 v
Run tests again
 |
 v
Review diff
 |
 v
Generate summary

This is much closer to software engineering than simple code completion.

41. Planning Repository Changes

A plan might look like:

text
Task:
Add employee search endpoint.

Plan:
1. Inspect existing employee routes.
2. Inspect service/repository pattern.
3. Add search method.
4. Add route.
5. Add validation.
6. Add tests.
7. Run full test suite.
8. Review diff.

Planning reduces accidental architectural violations.

42. Minimal-Change Principle

Agents should avoid unnecessary modifications.

Bad:

text
Requested:
Add one endpoint.

Agent:
Refactors 25 files.

Better:

text
Requested:
Add one endpoint.

Agent:
Changes only required files.

Minimal changes improve:

  • reviewability
  • reliability
  • rollback
  • debugging

43. Codebase Conventions

The agent should follow existing conventions.

If a repository uses:

text
services/
repositories/
schemas/
routers/

do not introduce a completely different architecture unless explicitly requested.

The repository itself is an important source of truth.

44. Dependency Management

AI agents may suggest new libraries.

Before adding one, evaluate:

  • license
  • maintenance
  • security
  • package size
  • compatibility
  • transitive dependencies
  • project standards

A coding agent should not add dependencies casually.

45. Documentation Generation

AI can generate:

  • docstrings
  • README sections
  • API documentation
  • changelogs
  • architecture summaries
  • code comments

But comments should explain:

›Why

rather than merely:

›What

Avoid:

🐍 Python
# Increment i by one
i += 1

Prefer explaining non-obvious business logic.

46. Code Review Assistance

An AI reviewer can inspect a diff:

Architecture & Data Flow
Git diff
 |
 v
Review model
 |
 +--> Correctness
 +--> Security
 +--> Performance
 +--> Maintainability
 +--> Tests
 |
 v
Review findings

AI review should complement—not replace—human review for important systems.

47. Bug Triage

An AI system can classify issues:

Architecture & Data Flow
Issue
 |
 v
Classifier
 |
 +--> Bug
 +--> Feature
 +--> Documentation
 +--> Security
 +--> Performance
 |
 v
Routing

Then assign:

  • priority
  • likely component
  • reproduction requirements
  • suggested owner

Automation should remain reviewable.

48. Code Search Assistant

A repository assistant can answer:

"Where is employee authentication handled?"

Pipeline:

Architecture & Data Flow
Question
 |
 v
Hybrid code search
 |
 v
Relevant symbols
 |
 v
Dependency traversal
 |
 v
Context
 |
 v
Answer

The answer should cite:

  • file
  • symbol
  • relevant lines
  • related dependencies

This makes repository exploration much easier.

49. Repository-Level RAG Architecture

Architecture & Data Flow
+-----------------------------------------------------+
| Code Intelligence |
| |
| Repository |
| | |
| v |
| Parser / AST / Symbol extractor |
| | |
| +--> File index |
| +--> Symbol index |
| +--> Vector index |
| +--> Dependency graph |
| +--> Documentation index |
| |
| Query |
| | |
| v |
| Hybrid Retrieval |
| | |
| v |
| Reranking |
| | |
| v |
| Context Builder |
| | |
| v |
| Coding / Reasoning Model |
+-----------------------------------------------------+

50. Code Agent State

A useful state object:

🐍 Python
from dataclasses import dataclass, field


@dataclass
class CodingTaskState:
 goal: str
 status: str = "PLANNING"
 changed_files: list[str] = field(default_factory=list)
 test_results: list[str] = field(default_factory=list)
 errors: list[str] = field(default_factory=list)
 step_count: int = 0

State makes long-running coding tasks resumable.

51. Python: Simple Tool Registry

🐍 Python
TOOLS = {
 "search_code": search_code,
 "read_file": read_file,
 "apply_patch": apply_patch,
 "run_tests": run_tests,
 "git_diff": git_diff,
}


def get_tool(name: str):
 if name not in TOOLS:
 raise PermissionError(f"Tool not allowed: {name}")

 return TOOLS[name]

In production, add:

  • user authorization
  • repository authorization
  • argument validation
  • audit logging
  • timeouts
  • resource limits

52. Python: Test-Gated Change

🐍 Python
def validate_change(run_tests, get_diff):
 test_result = run_tests()

 if not test_result.success:
 return {
 "approved": False,
 "reason": "Tests failed",
 "test_output": test_result.output,
 }

 diff = get_diff()

 if not diff:
 return {
 "approved": False,
 "reason": "No changes detected",
 }

 return {
 "approved": True,
 "diff": diff,
 }

This illustrates a deterministic gate around agent-generated changes.

53. Coding Agent Model Routing

Not every task needs the largest model.

Architecture & Data Flow
Simple completion
 |
 v
Small code model

Code explanation
 |
 v
Medium model

Repository planning
 |
 v
Reasoning model

Complex debugging
 |
 v
Strong reasoning model

Visual UI debugging
 |
 v
Multimodal model

Routing can reduce cost and latency.

54. Context Window Management

A repository task can require:

text
Issue
+
Architecture
+
Relevant files
+
Tests
+
Error output
+
Diff

But including everything may overwhelm the context.

Use staged retrieval:

Architecture & Data Flow
Issue
 |
 v
Repository summary
 |
 v
Relevant module
 |
 v
Relevant symbols
 |
 v
Tests
 |
 v
Error output

Retrieve more only when necessary.

55. Error-Driven Retrieval

Suppose a test reports:

›AttributeError:
UserService has no attribute search_users

The agent can retrieve:

text
UserService
+
search_users references
+
related tests
+
callers

This is more efficient than rereading the entire repository.

56. Verification Hierarchy

A coding agent can verify at increasing levels:

Architecture & Data Flow
Syntax
 |
 v
Type checking
 |
 v
Unit tests
 |
 v
Integration tests
 |
 v
End-to-end tests
 |
 v
Security checks
 |
 v
Human review

Not every change requires every level.

Risk should determine the verification depth.

57. Risk-Based Code Review

Example:

Architecture & Data Flow
Documentation change
 -> lightweight checks

Internal utility
 -> tests + review

Authentication code
 -> tests + security review

Payment logic
 -> extensive tests + security + human approval

Agent autonomy should decrease as impact increases.

58. Software Engineering Agents

An advanced coding agent can support:

Architecture & Data Flow
Issue
 |
 v
Planning
 |
 +--> Repository exploration
 |
 +--> Implementation
 |
 +--> Testing
 |
 +--> Debugging
 |
 +--> Review
 |
 +--> Documentation
 |
 v
Pull Request

This represents a broader software-engineering workflow rather than simple code generation.

59. Multi-Agent Software Engineering

Specialized agents can collaborate:

Architecture & Data Flow
 Supervisor
 |
 +--------------+--------------+
 | | |
 v v v
 Planner Coder Tester
 | | |
 +--------------+--------------+
 |
 v
 Reviewer

Possible roles:

  • planner
  • code explorer
  • implementer
  • test writer
  • security reviewer
  • documentation agent

Use this only when specialization provides real value.

60. Coding Agent Failure Modes

Common failures:

Hallucinated APIs#

The model invents a function that does not exist.

Wrong repository assumptions#

The model assumes architecture that is not present.

Over-editing#

The agent changes unrelated files.

Test gaming#

The agent changes tests to make code pass.

Infinite repair loops#

The agent repeatedly patches without convergence.

Security regressions#

Generated code introduces vulnerabilities.

Dependency sprawl#

The agent adds unnecessary packages.

61. Preventing Test Gaming

A dangerous behavior:

Architecture & Data Flow
Test fails
 |
 v
Agent modifies test
 |
 v
Test passes

Instead:

Architecture & Data Flow
Tests are protected
 |
 v
Agent modifies implementation
 |
 v
Existing tests
 |
 v
New tests

For trusted workflows, changes to tests should require additional review.

62. Preventing Scope Creep

Track:

text
Requested files
Expected files
Actual files

Example:

🐍 Python
allowed_files = {
 "app/api/employees.py",
 "app/services/employees.py",
 "tests/test_employees.py",
}

Reject unexpected modifications unless explicitly approved.

63. Agent Budgets

Set:

text
Maximum steps
Maximum model calls
Maximum tool calls
Maximum execution time
Maximum files changed
Maximum patch size

Example:

🐍 Python
MAX_STEPS = 25
MAX_FILES_CHANGED = 10
MAX_TOOL_CALLS = 50

Budgets protect against runaway behavior.

64. Production Coding Assistant Architecture

Architecture & Data Flow
+--------------------------------------------------------+
| Developer Experience |
| |
| IDE / CLI / Web |
| | |
| v |
| Task Manager |
| | |
| v |
| Model Router |
| | |
| v |
| Coding Agent |
| | |
| +--> Repository Retrieval |
| +--> Context Builder |
| +--> Planner |
| +--> Tool Gateway |
| | |
| v |
| Policy / Authorization |
| | |
| v |
| Sandbox |
| | |
| +--> Tests |
| +--> Linter |
| +--> Type checker |
| +--> Build |
| +--> Security scanner |
| | |
| v |
| Verification |
| | |
| v |
| Diff / Review |
| | |
| v |
| Human approval / CI |
+--------------------------------------------------------+

65. Observability

Record structured events:

text
task_started
repository_searched
file_read
patch_created
tool_executed
test_started
test_failed
repair_started
verification_passed
diff_generated
approval_requested

Track:

  • latency
  • model usage
  • tool usage
  • changed files
  • test results
  • failure categories
  • cost

Do not log secrets or unnecessary source content.

66. Evaluation of Coding Models

Evaluate several dimensions.

DimensionQuestion
CorrectnessDoes the code work?
Test successDoes it pass tests?
SecurityIs it safe?
MaintainabilityIs it understandable?
StyleDoes it follow project conventions?
EfficiencyIs it reasonably performant?
ScopeDid it change only what was required?
RobustnessDoes it handle edge cases?
ExplanationCan it explain the change?

67. Coding Benchmarks

Benchmark tasks can include:

text
Code completion
Bug fixing
Repository navigation
Feature implementation
Refactoring
Test generation
Documentation
Security repair

For repository tasks, evaluate:

text
Task success
+
Tests
+
Patch quality
+
Scope discipline
+
Security

68. Human Evaluation

Some dimensions require humans.

Reviewers can assess:

  • readability
  • architecture
  • maintainability
  • business correctness
  • appropriateness of abstractions

A useful process:

Architecture & Data Flow
Automated checks
 |
 v
Candidate changes
 |
 v
Human review
 |
 v
Final decision

69. Secure Coding Assistant

A production coding assistant should enforce:

Architecture & Data Flow
Identity
 |
Authorization
 |
Repository access
 |
Tool permissions
 |
Sandbox
 |
Security checks
 |
Human review

The AI model should not bypass these layers.

70. Enterprise Coding Assistants

Enterprise requirements may include:

  • private repositories
  • tenant isolation
  • access control
  • audit trails
  • data retention
  • model routing
  • code provenance
  • secret scanning
  • dependency scanning
  • secure execution

A strong architecture separates:

text
Developer data
+
Repository context
+
Model inference
+
Execution environment
+
Enterprise policy

71. Educational Coding Assistant

An educational coding assistant should optimize for learning, not just completion.

Instead of:

text
Student:
"Write the entire solution."

AI:
"Here is the complete answer."

use:

Architecture & Data Flow
Student attempt
 |
 v
Analyze
 |
 +--> Hint
 +--> Explanation
 +--> Smaller example
 +--> Debugging question
 |
 v
Student revises

This encourages active learning.

72. AI Tutor for Programming

A programming tutor can provide:

  • hints
  • error explanations
  • test cases
  • debugging guidance
  • conceptual explanations
  • code review
  • progressively stronger assistance

A useful assistance ladder:

text
Level 1: Ask a question
Level 2: Give a hint
Level 3: Explain concept
Level 4: Show a small example
Level 5: Suggest a patch
Level 6: Provide a full solution

The system can adapt assistance to learner needs.

73. Practical Project 1: AI Code Completion Assistant

Build a local coding assistant that:

  • accepts code context
  • predicts completion
  • supports multiple languages
  • measures latency
  • supports configurable models

Evaluate:

  • completion accuracy
  • latency
  • code validity
  • memory
  • user acceptance

74. Practical Project 2: Repository Q&A Assistant

Build:

Architecture & Data Flow
Repository
 |
 v
Parser / Indexer
 |
 v
Hybrid code search
 |
 v
Relevant context
 |
 v
Coding model
 |
 v
Answer + file/symbol references

Questions:

  • Where is authentication implemented?
  • Which service writes to the employee table?
  • Which tests cover this endpoint?

Require evidence for answers.

75. Practical Project 3: AI Bug-Fixing Agent

Build an agent that:

  1. receives a failing test
  2. inspects relevant code
  3. proposes a fix
  4. applies a patch
  5. runs tests
  6. repairs if necessary
  7. produces a final diff

Constraints:

  • maximum steps
  • maximum files changed
  • no production execution
  • protected tests

76. Practical Project 4: AI Test Generator

Build an assistant that generates:

  • unit tests
  • edge cases
  • negative tests
  • property-based tests

Pipeline:

Architecture & Data Flow
Function
 |
 v
Test analysis
 |
 v
Test generation
 |
 v
Test execution
 |
 v
Coverage / mutation analysis
 |
 v
Review

Do not measure success only by code coverage.

77. Practical Project 5: Secure Code Review Agent

Build an agent that reviews pull requests.

Check:

  • authentication
  • authorization
  • input validation
  • SQL safety
  • secrets
  • unsafe shell execution
  • dependency changes
  • logging
  • error handling

Output structured findings:

text
Severity
File
Symbol
Issue
Evidence
Recommendation

78. Practical Project 6: End-to-End Software Engineering Agent

Build:

Architecture & Data Flow
Issue
 |
 v
Planner
 |
 v
Repository Explorer
 |
 v
Coder
 |
 v
Tester
 |
 v
Debugger
 |
 v
Reviewer
 |
 v
Pull Request

Requirements:

  • structured state
  • tool permissions
  • sandbox
  • test gate
  • diff review
  • step budget
  • human approval

This can become a strong portfolio project.

79. Advanced Exercise 1: Repository RAG

Design a code RAG system for a repository with:

›100,000 files
10 million lines of code

Your design should include:

  • AST parsing
  • symbol indexing
  • dependency graph
  • embeddings
  • keyword search
  • hybrid retrieval
  • reranking
  • context compression

Explain how you avoid retrieving irrelevant code.

80. Advanced Exercise 2: Autonomous Bug Fixing

Design an agent that receives:

›Failing CI test

It must:

text
Inspect
 |
Hypothesize
 |
Patch
 |
Test
 |
Repair
 |
Verify

Define:

  • maximum iterations
  • test protection
  • patch-size limit
  • rollback
  • human escalation

81. Advanced Exercise 3: Secure Coding Agent

Design a coding agent with:

text
Read repository
Write code
Run tests
Run static analysis

but no:

text
Production deployment
Production credentials
Unrestricted network
Unrestricted filesystem

Explain how you implement the boundaries.

82. Advanced Exercise 4: AI Code Review Benchmark

Create 100 pull requests containing:

  • correctness bugs
  • security bugs
  • performance issues
  • style issues
  • harmless changes

Measure:

text
True positives
False positives
False negatives
Severity accuracy
Evidence quality

Create a release threshold for the review agent.

83. Advanced Exercise 5: Educational Coding Tutor

Design an AI tutor that refuses to immediately provide complete homework solutions.

Create an assistance policy:

Architecture & Data Flow
First attempt
 |
 v
Hint
 |
 v
Concept explanation
 |
 v
Debugging guidance
 |
 v
Partial example
 |
 v
Full solution when appropriate

Evaluate learning outcomes rather than only answer correctness.

84. Advanced Exercise 6: Multi-Agent Software Team

Design:

Architecture & Data Flow
Supervisor
 |
 +--> Planner
 +--> Coder
 +--> Tester
 +--> Security Reviewer
 +--> Documentation Agent

Define:

  • state schema
  • handoff rules
  • permissions
  • parallel tasks
  • verification
  • failure recovery
  • final approval

Compare this architecture with a single-agent baseline.

85. Common Mistakes

Mistake 1: Treating generated code as trusted#

Generated code is untrusted until verified.

Mistake 2: Giving agents unrestricted shell access#

Use narrow tools and sandboxes.

Mistake 3: Sending the whole repository to the model#

Use repository-aware retrieval.

Mistake 4: Using only vector search#

Combine semantic, lexical, symbol, and dependency-aware retrieval.

Mistake 5: Letting the agent modify tests to make them pass#

Protect tests and review test changes.

Mistake 6: No execution feedback#

Run tests and static analysis.

Mistake 7: Allowing unlimited repair loops#

Use budgets and convergence checks.

Mistake 8: Ignoring security#

Generated code can introduce serious vulnerabilities.

Mistake 9: Allowing unnecessary refactoring#

Prefer minimal, reviewable changes.

Mistake 10: Adding dependencies without review#

Evaluate security, licensing, maintenance, and compatibility.

Mistake 11: Using one model for every coding task#

Use model routing based on task complexity.

Mistake 12: Optimizing only for code generation speed#

Correctness and verification matter more than raw generation speed.

86. Final Mental Model

Think of Generative AI for Code as:

Architecture & Data Flow
Human Intent
 |
 v
Repository Context
 |
 v
Planning
 |
 v
Code Generation
 |
 v
Patch
 |
 v
Execution
 |
 v
Tests / Static Analysis
 |
 v
Observation
 |
 +---- Fail --> Repair
 |
 +---- Pass --> Review
 |
 v
 Approval
 |
 v
 Merge

The model is only one component.

A production coding system combines:

text
Model
+
Repository intelligence
+
Tools
+
Sandbox
+
Tests
+
Security
+
Policy
+
Human review

The central principle is:

The goal of AI-assisted software engineering is not to generate the most code. It is to produce correct, secure, maintainable, and reviewable software with reliable evidence that the change works.

87. Key Takeaways

  1. Code generation is a specialized form of generative modeling with strong executable feedback.
  2. Code completion, generation, transformation, explanation, and debugging are different tasks.
  3. Repository-level coding requires much more context than a single file.
  4. Code RAG should preserve symbols, dependencies, tests, and repository structure.
  5. Hybrid retrieval is often stronger than vector search alone.
  6. Coding agents should use narrow, typed tools.
  7. Generated code should execute inside controlled environments.
  8. Tests, linters, type checkers, and security scanners provide deterministic verification.
  9. Diff-based editing improves reviewability and rollback.
  10. Human approval remains important for high-impact changes.
  11. Agent budgets prevent runaway execution and cost.
  12. Repository conventions are important context for generated changes.
  13. Test gaming is a serious failure mode and tests should be protected.
  14. Security must be evaluated independently from functional correctness.
  15. Model routing can improve coding-system cost and latency.
  16. Educational coding assistants should optimize for learning, not merely answer completion.
  17. Multi-agent software engineering can provide specialization but increases system complexity.
  18. Production coding assistants require authorization, sandboxing, observability, evaluation, and governance.
  19. The strongest coding agents use execution feedback to iteratively improve their output.
  20. The objective is reliable software engineering, not merely impressive code generation.

88. Knowledge Check

Question 1#

Why is code particularly suitable for generative AI verification?

A. Code never contains errors.

B. Generated code can often be compiled, executed, and tested.

C. Code does not require context.

D. Code models are deterministic.

Answer: B

Question 2#

Why is repository-level coding harder than code completion?

Answer: Repository tasks require understanding architecture, dependencies, conventions, tests, configuration, and relationships across multiple files.

Question 3#

What is code RAG?

Answer: Retrieval-augmented generation designed to retrieve relevant source code, symbols, documentation, tests, and repository relationships for a coding task.

Question 4#

Why is symbol-aware retrieval useful?

Answer: It preserves coherent programming units such as functions and classes instead of arbitrary text chunks.

Question 5#

Why should generated code run in a sandbox?

Answer: Generated code is untrusted and can consume resources, access files, execute commands, or perform unsafe operations.

Question 6#

Why should coding agents use narrow tools instead of unrestricted shell access?

Answer: Narrow tools reduce the agent's attack surface and make permissions, validation, auditing, and testing easier.

Question 7#

What is the purpose of a test-gated coding workflow?

Answer: To require generated changes to satisfy deterministic checks before they can proceed toward approval or merge.

Question 8#

What is test gaming?

Answer: Changing or weakening tests so that incorrect implementation appears to pass.

Question 9#

Why is minimal-change behavior valuable?

Answer: It makes changes easier to review, test, debug, and roll back.

Question 10#

What is the central principle of AI-assisted software engineering?

Answer: Generate useful changes, then verify them through execution, testing, security checks, and review rather than trusting model output by itself.

89. Course Progression

The course has now moved from multimodal intelligence into AI-assisted software engineering.

Architecture & Data Flow
Advanced LLM Training
 |
 v
Post-Training & Alignment
 |
 v
Reasoning Models
 |
 v
Small Language Models & Edge AI
 |
 v
Advanced AI Agents & Computer Use
 |
 v
Advanced Multimodal AI
 |
 v
Generative AI for Code
 |
 v
Enterprise Generative AI
 |
 v
AI FinOps
 |
 v
AI Reliability / SRE
 |
 v
AI Red Teaming
 |
 v
Future AI Architectures
 |
 v
Full Generative AI Capstone

The next notebook moves into Enterprise Generative AI, covering enterprise architecture, private data, governance, multi-tenancy, security, enterprise RAG, workflow automation, AI platforms, model gateways, compliance, deployment strategies, adoption, and enterprise-grade use cases.

Knowledge Checkpoint

Generative AI for Code & Sandboxing Checkpoint

Q1.What is Fill-in-the-Middle (FIM) training in code foundation models (e.g. StarCoder, CodeLlama)?
ASplitting a source file into Prefix, Middle, and Suffix, and training the model to predict the Middle given the Prefix and Suffix.
BInserting comments between every line of code.
CFormatting code according to PEP 8 rules.
DReplacing function names with random variables.
Q2.What metric is standard for benchmarking code generation models on coding problems (e.g. HumanEval)?
Apass@k (e.g. pass@1, pass@10) measuring whether at least one of $k$ generated code samples passes all unit test assertions.
BBLEU score against human code.
CLine count of generated code.
DNumber of compiler warnings.
Q3.How do AI coding assistants perform Repository-Level context retrieval?
ABy parsing repository Abstract Syntax Trees (ASTs), indexing symbol definitions/references, and retrieving relevant imports using vector search and graph analysis.
BBy reading the entire hard drive into memory.
CBy guessing function names randomly.
DBy compiling the whole OS kernel.
Track Your Learning

Finished studying this notebook?

Mark this guide as completed to update your course progress roadmap.

Previous Lesson
LLM Reasoning & Reasoning Models
Next Lesson
Synthetic Data & Dataset Generation for Generative AI