PyCon DE 2025

HELLO 👋

Alessandro Romano

Your instructor for the day.

CO-HOST

My Data Guest — on Substack

Interviews with top AI & data experts, deep dives, and courses.

Follow along mydataguest.substack.com →

An AI Knowledge Assistant for company operations.

Ask it a question about the company; it decides where to look — internal docs, Slack, GitHub — retrieves what's relevant, and synthesises a grounded answer.

When it makes sense, it also acts: draft a Slack message, open a GitHub issue, suggest a next step. By the end of the day it runs live on the internet.

• Tools — give the model functions it can call to act on the world
• Memory / state — carry context forward across steps
• A loop — let it decide the next step, again and again, until the goal is met

That's the whole idea. An agent is an LLM placed inside a decision-making loop, with tools to act and state to remember.

This loop is the heart of everything we build today. Next: the most common way to structure it — ReAct.

from langchain.agents import create_agent

@tool
def web_search(query: str) -> str:
    """Search the web for current info."""
    return search_client.run(query)

agent = create_agent(llm, tools=[web_search])

agent.invoke({"messages": [("user", "...")]})

LangGraph ships the whole loop prebuilt. We use it today — then rebuild it by hand in Module 4 to see inside.

WHAT THOSE 5 LINES DO

• Wire the LLM to the tool
• Run the reason → act → observe loop for you
• Stop when the answer is ready
• Return the full message trace to inspect

WATCH FOR

• The three walls, live
• Structured output = trustable data
• The message trace: request → run → answer
• The agent acts — search breaks a wall

Both are teaching demos. The real app build starts in Module 4.

# the loop, stripped to its essence
while not done:
    thought = llm(prompt(history))
    if thought.is_final:
        return thought.answer
    # the model chose a tool + args
    result = tools[thought.tool](**thought.args)
    history.append(thought, result)   # observe

LangGraph will give us this loop with state, retries and routing handled properly — but this is all it is underneath.

⚠️ FAILURE MODES TO WATCH

• Looping — repeats the same action forever. Cap the steps.
• Hallucinated actions — invents a tool that doesn't exist. Validate the choice.
• Tool misuse — right tool, wrong arguments. Enforce a schema.
• Never stopping — no clear finish condition. Define "done".

No framework here — we hand-build the loop Module 2 hid, so it's never a black box again.

WATCH FOR

• The model chooses between two tools by their docstrings
• A multi-step question → the loop takes several turns
• The step cap firing when steps run out
• Thought → Action → Observation, in plain Python

class State(TypedDict):
    messages: Annotated[list, add_messages]

def call_model(state: State):
    return {"messages": [llm_with_tools.invoke(state["messages"])]}

graph = StateGraph(State)
graph.add_node("agent", call_model)
graph.add_node("tools", ToolNode(tools))

graph.add_edge(START, "agent")
graph.add_conditional_edges("agent", should_continue)
graph.add_edge("tools", "agent")   # loop back

app = graph.compile()

READ IT AS A LOOP

• START → agent — the model reasons over the messages
• should_continue — tool call? → tools. Else → END
• tools → agent — feed the result back and reason again
• add_messages — the reducer that appends to state

That's the ReAct loop from Section 3 — now explicit, inspectable, and safe to extend.

WATCH FOR

• The compiled graph — draw it, read the loop
• The tools → agent edge that makes it iterate
• State accumulating via add_messages
• The full message trace end-to-end

This is v1 of the app we grow & deploy — the same loop from Module 3, now a real graph.

from langchain_core.tools import tool

# @tool turns a typed, documented
# function into a callable the LLM sees
@tool
def search_github_issues(repo: str, query: str) -> str:
    """Search open issues in a GitHub repo (repo is 'owner/name')."""
    return gh.search(repo, query)   # your API call

WHAT THE MODEL RECEIVES

• name — search_github_issues
• args schema — from the type hints
• description — from the docstring

FOR OUR ASSISTANT

• Web search (Seltz)
• Slack: draft a message
• GitHub: draft an issue

# 1 · a schema pins down the arguments
class RefundArgs(BaseModel):
    order_id: str
    amount: float = Field(gt=0)              # must be > 0
    reason: Literal["damaged", "late"]

@tool(args_schema=RefundArgs)
def issue_refund(order_id, amount, reason) -> str:
    """Issue a refund for an order."""
    try:
        return billing.refund(order_id, amount, reason)
    except Exception as e:
        return f"error: {e}"   # readable, not a crash

• Tool schema — a Pydantic args_schema encodes real rules (ranges, enums). The model is told the constraints.
• Handle errors — wrap the risky call; return a readable string, never let it raise into the loop.
• Fallback — on failure, degrade gracefully: a cached value, a simpler tool, or "I couldn't do X, try Y".
• Structured output — a tool can return a Pydantic object, so downstream nodes get fields, not prose.

WATCH FOR

• Wrapping an external API as a @tool
• Docstring + try/except = a reliable tool
• The agent choosing search vs. an action
• Draft tools returning a payload, not sending

Adding tools didn't touch the graph — same v1 loop, richer hands.

WATCH FOR

• Chunk size / overlap shaping retrieval
• Unknown question → "I don't know" (not a guess)
• Sources cited in the answer
• Retrieval as a tool → agent routes docs vs. web

The prompt the model actually sees is assembled from many parts. Designing that assembly — what to include, in what order, and what to leave out — is context engineering.

• System instructions — role, rules, output format
• User input — the current question
• History — relevant prior turns, trimmed
• Retrieved data — RAG chunks, tool outputs
• State — passed cleanly through LangGraph

A tiny local stdio server — no hosting, no network. To the agent, MCP tools are just tools.

WATCH FOR

• A server defined in ~6 lines (FastMCP)
• Its functions loaded as LangChain tools
• One client, many servers possible
• The agent using them like any other tool

HOW THEY TALK

• Shared state — agents read/write the same graph state
• A supervisor — a router agent decides who acts next
• Handoffs — one agent passes control (and context) to another
• Structured messages — clear contracts between them

Standalone demo — our assistant stays a single agent. Swarm (peer-to-peer hand-off) is covered on the previous slide.

WATCH FOR

• The supervisor routing to the right specialist
• Each specialist has one job, one tool
• Specialists report back to the supervisor
• When the extra complexity is (and isn't) worth it

import streamlit as st
from assistant import build_assistant   # our v3.1 graph

# build once; one thread_id per session = memory
if "agent" not in st.session_state:
    st.session_state.agent = build_assistant()
    st.session_state.tid = str(uuid.uuid4())

for role, text in st.session_state.history:
    st.chat_message(role).write(text)

if prompt := st.chat_input("Ask…"):
    st.chat_message("user").write(prompt)
    cfg = {"configurable": {"thread_id": st.session_state.tid}}
    ans = st.session_state.agent.invoke({"messages":[("user",prompt)]}, cfg)
    st.chat_message("assistant").write(ans["messages"][-1].content)

THE ESSENTIALS

• chat_input / chat_message — the whole UI
• session_state — survives Streamlit's rerun-on-every-keystroke
• thread_id — per session → the memory from Module 6
• the UI just invokes the graph — all logic lives in assistant.py

The graph is imported, not rebuilt — same code the deploy step (Module 9) ships.

We leave Jupyter — this runs on localhost, the same files Module 9 deploys.

WATCH FOR

• Graph logic in assistant.py, UI in app.py
• Conversation in session_state
• thread_id → it remembers within the chat
• The tools it used, surfaced in the UI

An agent is a black box until you can see inside it. LangSmith records every run — prompts, tool calls, tokens, latency, errors — so you can debug and watch it in production.

The best part: zero code changes. Set four env vars and every run is traced automatically:

LANGSMITH_TRACING=true
LANGSMITH_API_KEY=lsv2_pt_...
LANGSMITH_PROJECT=knowledge-assistant
LANGSMITH_ENDPOINT=https://api.smith.langchain.com

WHAT YOU GET IN THE DASHBOARD

• Every trace — the full agent → tool → agent path, step by step
• Which tool ran, with its inputs & outputs
• Tokens, latency & cost per run
• Errors surfaced, so failures aren't invisible

Let's do it together: create a project on smith.langchain.com, generate an API key, and paste four lines into .env. Then chat with the assistant and watch the trace appear.

• 1New project — name it knowledge-assistant
• 2Pick the framework — LangGraph, then switch the language to Python (it opens on TypeScript)
• 3Generate API Key — copy it once, it won't show again
• 4Paste into .env — the four LANGSMITH_* vars
• 5Ask a question — refresh the dashboard, the trace is there

smith.langchain.com

Note the language tab: it lands on TypeScript. Click Python, then use the .env tab instead of Shell.

We never click "create a service." We commit a render.yaml that declares what should exist — and let the platform make reality match the file.

• Clicking through a dashboard — nobody remembers what you picked, nothing is reviewable, and it dies with your browser tab
• A file in git — diffable, reviewable, and it recreates the same service on a fresh account
• The repo is the source of truth — change the file, push, the infrastructure follows

RENDER, READING OUR BLUEPRINT

dashboard.render.com

We typed none of the left column. Render read it from the file.

Nothing to write — the blueprint is already in the repo. We point Render at it and paste our keys.

• 1New + → Blueprint — connect the GitHub repo
• 2Render reads render.yaml — build & start commands declared
• 3Paste the secrets — every sync: false key, in the dashboard
• 4Watch the build log — uv sync --frozen → Streamlit boots
• 5Open the live URL — ask a question, find that run in LangSmith

THE WHOLE DEPLOY CONFIG

# render.yaml (repo root)
services:
  - type: web
    name: knowledge-assistant
    runtime: python
    plan: free
    rootDir: src
    buildCommand: pip install uv && uv sync --frozen
    startCommand: uv run streamlit run app/solution/app.py ...
    healthCheckPath: /_stcore/health
    envVars:
      - key: OPENAI_API_KEY
        sync: false   # ← set in the dashboard
      - key: LANGSMITH_API_KEY
        sync: false

Same files from Module 8 — now traced and on the internet.

WATCH FOR

• A full trace appear in LangSmith
• render.yaml driving the deploy
• The live URL serving our agent
• A cold start on the first hit