Skip to content

Conversations

src.domains.conversations.endpoints

Conversation management API endpoints for chat/streaming functionality.

This module provides REST endpoints for: - Creating and managing conversations (chat sessions) - Fetching messages - Streaming AI responses with token-by-token generation - Starring/unstarring messages for bookmarking - Auto-generating conversation titles

Architecture

Conversation Flow: ┌────────────────────────────────────────────────────────────┐ │ POST /conversations/ │ │ └─> Create conversation with LLM + generation params │ └────────────────────────────────────────────────────────────┘ ↓ ┌────────────────────────────────────────────────────────────┐ │ POST /{conversation_id}/generate_title │ │ └─> Stream AI-generated title based on first message │ └────────────────────────────────────────────────────────────┘ ↓ ┌────────────────────────────────────────────────────────────┐ │ POST /{conversation_id}/query │ │ └─> Stream AI response token-by-token │ │ └─> Save user message + AI response to database │ └────────────────────────────────────────────────────────────┘ ↓ ┌────────────────────────────────────────────────────────────┐ │ GET /{conversation_id}/fetch_messages │ │ └─> Retrieve all messages in conversation │ └────────────────────────────────────────────────────────────┘

Streaming Pattern

All generation endpoints return StreamingResponse with: - Content-Type: application/x-ndjson - Token-by-token yields via async generator - Automatic database persistence after stream completes

Endpoints
  • GET / → List all conversations
  • GET /{conversation_id} → Get conversation with messages
  • POST / → Create new conversation
  • PATCH /{conversation_id} → Update conversation (name, LLM, params)
  • DELETE /{conversation_id} → Delete single conversation
  • POST /{conversation_id}/query → Stream AI response
  • POST /{conversation_id}/generate_title → Stream title generation
  • GET /{conversation_id}/fetch_messages → List messages
  • PATCH /messages/{message_id}/star → Star/unstar message
Example

Create conversation and generate response::

# 1. Create conversation
POST /erudi/conversations/
{
  "llm_id": 1,
  "temperature": 0.7,
  "top_p": 0.9,
  "max_tokens": 512,
  "custom_prompt": "You are a helpful assistant."
}
→ {"id": 42, "name": "New Conversation", ...}

# 2. Send message and stream response
POST /erudi/conversations/42/query
{
  "user_message": "Explain quantum computing"
}
→ StreamingResponse (application/x-ndjson)
→ "Quantum computing..." (token-by-token)

# 3. Fetch conversation history
GET /erudi/conversations/42/fetch_messages
→ [
    {"role": "user", "content": "Explain quantum computing"},
    {"role": "assistant", "content": "Quantum computing..."}
  ]
Note
  • StreamingResponse allows frontend to display tokens in real-time
  • Messages are saved to database only after streaming completes
  • Use run_in_threadpool() for sync repository/service calls
Warning

Streaming endpoints hold database sessions during generation. Long generations (>60s) may cause connection timeouts.

create_conversation(payload, service=Depends(get_conversation_service), db=Depends(get_db)) async

Create a new conversation with specified LLM and generation parameters.

Parameters:

Name Type Description Default
payload ConversationCreate

ConversationCreate schema with llm_id, temperature, top_p, max_tokens, and optional custom_prompt.

required
service ConversationService

Injected conversation service.

Depends(get_conversation_service)
db Session

Database session for transaction control.

Depends(get_db)

Returns:

Name Type Description
ConversationResponse

Created conversation with auto-generated name (e.g., "New Conversation") and assigned ID.

Example

::

POST /erudi/conversations/
{
  "llm_id": 5,
  "temperature": 0.8,
  "top_p": 0.95,
  "max_tokens": 1024,
  "custom_prompt": "You are a creative writer."
}
→ {
    "id": 123,
    "name": "New Conversation",
    "llm_id": 5,
    "temperature": 0.8,
    "top_p": 0.95,
    "max_tokens": 1024,
    "custom_prompt": "You are a creative writer."
  }
Note

Use POST /{conversation_id}/generate_title to auto-generate a meaningful name after the first message exchange.

Source code in backend/src/domains/conversations/endpoints.py
@router.post("/", response_model=ConversationResponse, status_code=201)
async def create_conversation(
    payload: ConversationCreate,
    service: ConversationService = Depends(get_conversation_service),
    db: Session = Depends(get_db),
):
    """Create a new conversation with specified LLM and generation parameters.

    Args:
        payload: ConversationCreate schema with llm_id, temperature, top_p,
            max_tokens, and optional custom_prompt.
        service: Injected conversation service.
        db: Database session for transaction control.

    Returns:
        ConversationResponse: Created conversation with auto-generated name
            (e.g., "New Conversation") and assigned ID.

    Example:
        ::

            POST /erudi/conversations/
            {
              "llm_id": 5,
              "temperature": 0.8,
              "top_p": 0.95,
              "max_tokens": 1024,
              "custom_prompt": "You are a creative writer."
            }
            → {
                "id": 123,
                "name": "New Conversation",
                "llm_id": 5,
                "temperature": 0.8,
                "top_p": 0.95,
                "max_tokens": 1024,
                "custom_prompt": "You are a creative writer."
              }

    Note:
        Use POST /{conversation_id}/generate_title to auto-generate a meaningful
        name after the first message exchange.
    """
    """Create a new conversation for a specific LLM (body JSON)."""
    try:
        conv = await run_in_threadpool(
            service.create_conversation,
            llm_id=payload.llm_id,
            temperature=payload.temperature,
            top_p=payload.top_p,
            max_tokens=payload.max_tokens,
            custom_prompt=payload.custom_prompt,
            web_search_enabled=payload.web_search_enabled,
            reasoning_effort=payload.reasoning_effort,
            compaction_threshold=payload.compaction_threshold,
        )
        db.commit()
        logger.info(
            f"Conversation created: id={conv.id}, llm_id={payload.llm_id}, "
            f"temperature={payload.temperature}, top_p={payload.top_p}, "
            f"max_tokens={payload.max_tokens}, "
            f"web_search_enabled={conv.web_search_enabled}, "
            f"reasoning_effort={conv.reasoning_effort}, "
            f"compaction_threshold={conv.compaction_threshold}"
        )
        return conv
    except Exception:
        db.rollback()
        raise

delete_conversation(conversation_id, service=Depends(get_conversation_service), db=Depends(get_db)) async

Delete a conversation and all associated messages (cascade delete).

Parameters:

Name Type Description Default
conversation_id int

ID of the conversation to delete.

required
service ConversationService

Injected conversation service.

Depends(get_conversation_service)
db Session

Database session for transaction control.

Depends(get_db)

Returns:

Name Type Description
dict

Success confirmation message.

Example

::

DELETE /erudi/conversations/42
→ {"message": "Conversation deleted successfully"}
Warning

This is a permanent deletion. All messages in the conversation will also be deleted due to cascade constraints.

Source code in backend/src/domains/conversations/endpoints.py
@router.delete("/{conversation_id}")
async def delete_conversation(
    conversation_id: int,
    service: ConversationService = Depends(get_conversation_service),
    db: Session = Depends(get_db),
):
    """Delete a conversation and all associated messages (cascade delete).

    Args:
        conversation_id: ID of the conversation to delete.
        service: Injected conversation service.
        db: Database session for transaction control.

    Returns:
        dict: Success confirmation message.

    Example:
        ::

            DELETE /erudi/conversations/42
            → {"message": "Conversation deleted successfully"}

    Warning:
        This is a permanent deletion. All messages in the conversation will
        also be deleted due to cascade constraints.
    """
    """
    Delete a conversation by its ID.
    """
    try:
        await service.delete_conversation(conversation_id)
        logger.info(f"Conversation deleted: id={conversation_id}")
        return {"message": "Conversation deleted successfully"}
    except Exception:
        db.rollback()
        raise

generate_title(conversation_id, payload, service=Depends(get_conversation_service)) async

Stream AI-generated conversation title based on first message.

Uses the loaded LLM to generate a concise title (1-5 words) summarizing the conversation topic. Streams title token-by-token for real-time display.

Parameters:

Name Type Description Default
conversation_id int

ID of the conversation to generate title for.

required
payload ConversationQuery

ConversationQuery with question (typically first user message).

required
service ConversationService

Injected conversation service.

Depends(get_conversation_service)

Returns:

Name Type Description
StreamingResponse

Text stream with generated title (text/plain).

Example

::

POST /erudi/conversations/42/generate_title
{
  "question": "Explain quantum entanglement in simple terms"
}
→ StreamingResponse: "Quantum Entanglement Basics"
Note

Title generation uses the same LLM as the conversation but with a specialized prompt: "Generate a 1-5 word title for: {question}"

Source code in backend/src/domains/conversations/endpoints.py
@router.post("/{conversation_id}/generate_title")
async def generate_title(
    conversation_id: int,
    payload: ConversationQuery,
    service: ConversationService = Depends(get_conversation_service),
):
    """Stream AI-generated conversation title based on first message.

    Uses the loaded LLM to generate a concise title (1-5 words) summarizing
    the conversation topic. Streams title token-by-token for real-time display.

    Args:
        conversation_id: ID of the conversation to generate title for.
        payload: ConversationQuery with question (typically first user message).
        service: Injected conversation service.

    Returns:
        StreamingResponse: Text stream with generated title (text/plain).

    Example:
        ::

            POST /erudi/conversations/42/generate_title
            {
              "question": "Explain quantum entanglement in simple terms"
            }
            → StreamingResponse: "Quantum Entanglement Basics"

    Note:
        Title generation uses the same LLM as the conversation but with
        a specialized prompt: "Generate a 1-5 word title for: {question}"
    """
    """Generate a title for the conversation based on the first message."""
    return StreamingResponse(
        service.generate_title_stream(conversation_id, payload.question),
        media_type="text/plain",
        headers={
            "Cache-Control": "no-cache",
            "X-Accel-Buffering": "no",
        },
    )

get_all_conversations(conversation_repo=Depends(get_conversation_repository)) async

Fetch all conversations from the database.

Parameters:

Name Type Description Default
conversation_repo ConversationRepository

Injected conversation repository.

Depends(get_conversation_repository)

Returns:

Type Description

List[ConversationResponse]: All conversations with metadata (id, name, llm_id, creation date, last update).

Example

::

GET /erudi/conversations/
→ [
    {
      "id": 1,
      "name": "Quantum Computing Discussion",
      "llm_id": 5,
      "created_at": "2025-10-24T10:30:00",
      "temperature": 0.7,
      "top_p": 0.9
    },
    ...
  ]
Note

Does not include messages. Use GET /{conversation_id} for full details.

Source code in backend/src/domains/conversations/endpoints.py
@router.get("/", response_model=List[ConversationResponse])
async def get_all_conversations(
    conversation_repo: ConversationRepository = Depends(get_conversation_repository),
):
    """Fetch all conversations from the database.

    Args:
        conversation_repo: Injected conversation repository.

    Returns:
        List[ConversationResponse]: All conversations with metadata (id, name,
            llm_id, creation date, last update).

    Example:
        ::

            GET /erudi/conversations/
            → [
                {
                  "id": 1,
                  "name": "Quantum Computing Discussion",
                  "llm_id": 5,
                  "created_at": "2025-10-24T10:30:00",
                  "temperature": 0.7,
                  "top_p": 0.9
                },
                ...
              ]

    Note:
        Does not include messages. Use GET /{conversation_id} for full details.
    """
    """
    Fetch all conversations.
    """
    logger.debug("Fetching all conversations")
    # Read-only operation, no commit needed
    return await run_in_threadpool(conversation_repo.get_all_conversations)

get_conversation_by_id(conversation_id, conversation_repo=Depends(get_conversation_repository)) async

Fetch a single conversation by ID with all associated messages.

Parameters:

Name Type Description Default
conversation_id int

ID of the conversation to retrieve.

required
conversation_repo ConversationRepository

Injected conversation repository.

Depends(get_conversation_repository)

Returns:

Name Type Description
ConversationWithMessagesResponse

Conversation metadata and full message history (user and assistant messages).

Example

::

GET /erudi/conversations/42
→ {
    "id": 42,
    "name": "Python Tutorial",
    "llm_id": 3,
    "messages": [
      {"role": "user", "content": "Explain decorators"},
      {"role": "assistant", "content": "Decorators are..."}
    ],
    "temperature": 0.7,
    "custom_prompt": "You are a Python expert."
  }
Source code in backend/src/domains/conversations/endpoints.py
@router.get("/{conversation_id}", response_model=ConversationWithMessagesResponse)
async def get_conversation_by_id(
    conversation_id: int,
    conversation_repo: ConversationRepository = Depends(get_conversation_repository),
):
    """Fetch a single conversation by ID with all associated messages.

    Args:
        conversation_id: ID of the conversation to retrieve.
        conversation_repo: Injected conversation repository.

    Returns:
        ConversationWithMessagesResponse: Conversation metadata and full message
            history (user and assistant messages).

    Example:
        ::

            GET /erudi/conversations/42
            → {
                "id": 42,
                "name": "Python Tutorial",
                "llm_id": 3,
                "messages": [
                  {"role": "user", "content": "Explain decorators"},
                  {"role": "assistant", "content": "Decorators are..."}
                ],
                "temperature": 0.7,
                "custom_prompt": "You are a Python expert."
              }
    """
    """
    Fetch a single conversation by its ID, including messages.
    """
    logger.debug(f"Fetching conversation {conversation_id}")
    # Read-only operation, no commit needed
    return await run_in_threadpool(
        conversation_repo.get_conversation_by_id,
        conversation_id,
    )

get_conversation_repository(db=Depends(get_db))

Provide a conversation repository instance for dependency injection.

Parameters:

Name Type Description Default
db Session

SQLAlchemy session injected by FastAPI Depends(get_db).

Depends(get_db)

Returns:

Name Type Description
ConversationRepository ConversationRepository

Repository for conversation CRUD operations.

Example

::

@router.get("/")
async def list_conversations(
    repo: ConversationRepository = Depends(get_conversation_repository)
):
    return await run_in_threadpool(repo.get_all_conversations)
Source code in backend/src/domains/conversations/endpoints.py
def get_conversation_repository(db: Session = Depends(get_db)) -> ConversationRepository:
    """Provide a conversation repository instance for dependency injection.

    Args:
        db: SQLAlchemy session injected by FastAPI Depends(get_db).

    Returns:
        ConversationRepository: Repository for conversation CRUD operations.

    Example:
        ::

            @router.get("/")
            async def list_conversations(
                repo: ConversationRepository = Depends(get_conversation_repository)
            ):
                return await run_in_threadpool(repo.get_all_conversations)
    """
    """Provide a conversation repository instance."""
    return ConversationRepository(db)

get_conversation_service(db=Depends(get_db), checkpointer=Depends(get_checkpointer))

Provide a conversation service instance for dependency injection.

Parameters:

Name Type Description Default
db Session

SQLAlchemy session injected by FastAPI Depends(get_db).

Depends(get_db)

Returns:

Name Type Description
ConversationService ConversationService

Service layer for complex conversation operations (streaming generation, title generation, message persistence).

Example

::

@router.post("/{conversation_id}/query")
async def query(
    conversation_id: int,
    payload: ConversationQuery,
    service: ConversationService = Depends(get_conversation_service)
):
    return StreamingResponse(
        service.query_conversation(...),
        media_type="text/event-stream"
    )
Source code in backend/src/domains/conversations/endpoints.py
def get_conversation_service(
    db: Session = Depends(get_db),
    checkpointer=Depends(get_checkpointer),
) -> ConversationService:
    """Provide a conversation service instance for dependency injection.

    Args:
        db: SQLAlchemy session injected by FastAPI Depends(get_db).

    Returns:
        ConversationService: Service layer for complex conversation operations
            (streaming generation, title generation, message persistence).

    Example:
        ::

            @router.post("/{conversation_id}/query")
            async def query(
                conversation_id: int,
                payload: ConversationQuery,
                service: ConversationService = Depends(get_conversation_service)
            ):
                return StreamingResponse(
                    service.query_conversation(...),
                    media_type="text/event-stream"
                )
    """
    """Provide a conversation service instance."""
    return ConversationService(db, checkpointer=checkpointer)

get_message_repository(db=Depends(get_db))

Provide a message repository instance for dependency injection.

Parameters:

Name Type Description Default
db Session

SQLAlchemy session injected by FastAPI Depends(get_db).

Depends(get_db)

Returns:

Name Type Description
MessageRepository MessageRepository

Repository for message CRUD operations.

Example

::

@router.get("/{conversation_id}/messages")
async def get_messages(
    conversation_id: int,
    repo: MessageRepository = Depends(get_message_repository)
):
    return await run_in_threadpool(
        repo.get_messages_by_conversation, conversation_id
    )
Source code in backend/src/domains/conversations/endpoints.py
def get_message_repository(db: Session = Depends(get_db)) -> MessageRepository:
    """Provide a message repository instance for dependency injection.

    Args:
        db: SQLAlchemy session injected by FastAPI Depends(get_db).

    Returns:
        MessageRepository: Repository for message CRUD operations.

    Example:
        ::

            @router.get("/{conversation_id}/messages")
            async def get_messages(
                conversation_id: int,
                repo: MessageRepository = Depends(get_message_repository)
            ):
                return await run_in_threadpool(
                    repo.get_messages_by_conversation, conversation_id
                )
    """
    """Provide a message repository instance."""
    return MessageRepository(db)

get_messages_by_conversation(conversation_id, message_repo=Depends(get_message_repository)) async

Fetch all messages for a specific conversation ordered by creation time.

Parameters:

Name Type Description Default
conversation_id int

ID of the conversation to fetch messages from.

required
message_repo MessageRepository

Injected message repository.

Depends(get_message_repository)

Returns:

Type Description

List[MessageResponse]: All messages (user and assistant) in the conversation.

Example

::

GET /erudi/conversations/42/fetch_messages
→ [
    {"id": 1, "role": "user", "content": "Hello"},
    {"id": 2, "role": "assistant", "content": "Hi there!"}
  ]
Source code in backend/src/domains/conversations/endpoints.py
@router.get(
    "/{conversation_id}/fetch_messages",
    response_model=List[MessageResponse],
)
async def get_messages_by_conversation(
    conversation_id: int,
    message_repo: MessageRepository = Depends(get_message_repository),
):
    """Fetch all messages for a specific conversation ordered by creation time.

    Args:
        conversation_id: ID of the conversation to fetch messages from.
        message_repo: Injected message repository.

    Returns:
        List[MessageResponse]: All messages (user and assistant) in the conversation.

    Example:
        ::

            GET /erudi/conversations/42/fetch_messages
            → [
                {"id": 1, "role": "user", "content": "Hello"},
                {"id": 2, "role": "assistant", "content": "Hi there!"}
              ]
    """
    """
    Fetch all messages for a specific conversation.
    """
    logger.debug(f"Fetching messages for conversation {conversation_id}")
    # Read-only operation, no commit needed
    return await run_in_threadpool(
        message_repo.get_messages_by_conversation,
        conversation_id,
    )

query_and_respond(conversation_id, payload, service=Depends(get_conversation_service)) async

Stream AI response to user query and persist message history.

This is the core conversation endpoint. It: 1. Loads conversation history and LLM configuration 2. Appends user message to history 3. Streams AI-generated response token-by-token 4. Saves both user message and AI response to database after streaming

Parameters:

Name Type Description Default
conversation_id int

ID of the conversation to query.

required
payload ConversationQuery

ConversationQuery with user_message and optional kb_id (knowledge base for RAG injection).

required
service ConversationService

Injected conversation service.

Depends(get_conversation_service)

Returns:

Name Type Description
StreamingResponse

AI response stream (text/plain), sent token-by-token for real-time display in frontend.

Example

::

POST /erudi/conversations/42/query
{
  "user_message": "What is the capital of France?",
  "kb_id": null
}
→ StreamingResponse: "The capital of France is Paris..."
Note
  • Supports RAG: If kb_id provided, injects relevant KB chunks into prompt
  • Messages are saved AFTER streaming completes (ensures full response captured)
  • Uses the conversation's temperature and top_p; the output budget is derived from the model's context window, not from the request
Source code in backend/src/domains/conversations/endpoints.py
@router.post("/{conversation_id}/query")
async def query_and_respond(
    conversation_id: int,
    payload: ConversationQuery,
    service: ConversationService = Depends(get_conversation_service),
):
    """Stream AI response to user query and persist message history.

    This is the core conversation endpoint. It:
    1. Loads conversation history and LLM configuration
    2. Appends user message to history
    3. Streams AI-generated response token-by-token
    4. Saves both user message and AI response to database after streaming

    Args:
        conversation_id: ID of the conversation to query.
        payload: ConversationQuery with user_message and optional kb_id
            (knowledge base for RAG injection).
        service: Injected conversation service.

    Returns:
        StreamingResponse: AI response stream (text/plain), sent token-by-token
            for real-time display in frontend.

    Example:
        ::

            POST /erudi/conversations/42/query
            {
              "user_message": "What is the capital of France?",
              "kb_id": null
            }
            → StreamingResponse: "The capital of France is Paris..."

    Note:
        - Supports RAG: If kb_id provided, injects relevant KB chunks into prompt
        - Messages are saved AFTER streaming completes (ensures full response captured)
        - Uses the conversation's temperature and top_p; the output budget is
          derived from the model's context window, not from the request
    """
    """Query the conversation and get a streaming response."""
    # NDJSON (#90): one JSON event per line (answer / thinking / tool_call /
    # tool_result / error / done). The renderer splits the body on newlines and
    # parses each line. generate_title and arena keep the plain-text protocol.
    return StreamingResponse(
        service.query_and_respond_stream(conversation_id, payload),
        media_type="application/x-ndjson",
        headers={
            "Cache-Control": "no-cache",
            "X-Accel-Buffering": "no",
        },
    )

star_message(payload, message_repo=Depends(get_message_repository), db=Depends(get_db)) async

Mark a message as starred (bookmarked for later reference).

Parameters:

Name Type Description Default
payload MessageStarRequest

MessageStarRequest with message_id to star.

required
message_repo MessageRepository

Injected message repository.

Depends(get_message_repository)
db Session

Database session for transaction control.

Depends(get_db)

Returns:

Name Type Description
dict

Success confirmation with state="success".

Example

::

POST /erudi/conversations/star_message
{
  "message_id": 345
}
→ {"state": "success", "message": "Message starred successfully"}
Note

Starred messages can be filtered/highlighted in frontend for quick access to important responses or bookmarked content.

Source code in backend/src/domains/conversations/endpoints.py
@router.post("/star_message")
async def star_message(
    payload: MessageStarRequest,
    message_repo: MessageRepository = Depends(get_message_repository),
    db: Session = Depends(get_db),
):
    """Mark a message as starred (bookmarked for later reference).

    Args:
        payload: MessageStarRequest with message_id to star.
        message_repo: Injected message repository.
        db: Database session for transaction control.

    Returns:
        dict: Success confirmation with state="success".

    Example:
        ::

            POST /erudi/conversations/star_message
            {
              "message_id": 345
            }
            → {"state": "success", "message": "Message starred successfully"}

    Note:
        Starred messages can be filtered/highlighted in frontend for quick access
        to important responses or bookmarked content.
    """
    """Star a message in the conversation."""
    try:
        await run_in_threadpool(message_repo.star_message, payload.message_id)
        db.commit()
        logger.info(f"Message starred: message_id={payload.message_id}")
        return {"state": "success", "message": "Message starred successfully"}
    except Exception:
        db.rollback()
        raise

store_error_message(conversation_id, service=Depends(get_conversation_service), db=Depends(get_db)) async

Store an error message when AI generation fails (fallback mechanism).

Used by frontend to record generation failures (OOM, timeout, model crash). Ensures conversation history remains consistent even after errors.

Parameters:

Name Type Description Default
conversation_id int

ID of the conversation where error occurred.

required
service ConversationService

Injected conversation service.

Depends(get_conversation_service)
db Session

Database session for transaction control.

Depends(get_db)

Returns:

Name Type Description
dict

ID of the created error message record.

Example

::

POST /erudi/conversations/42/store_error_message
→ {"error_message_id": 567}
Note

Error messages are stored with role="assistant" and content indicating failure (e.g., "Generation failed due to system error").

Source code in backend/src/domains/conversations/endpoints.py
@router.post("/{conversation_id}/store_error_message")
async def store_error_message(
    conversation_id: int,
    service: ConversationService = Depends(get_conversation_service),
    db: Session = Depends(get_db),
):
    """Store an error message when AI generation fails (fallback mechanism).

    Used by frontend to record generation failures (OOM, timeout, model crash).
    Ensures conversation history remains consistent even after errors.

    Args:
        conversation_id: ID of the conversation where error occurred.
        service: Injected conversation service.
        db: Database session for transaction control.

    Returns:
        dict: ID of the created error message record.

    Example:
        ::

            POST /erudi/conversations/42/store_error_message
            → {"error_message_id": 567}

    Note:
        Error messages are stored with role="assistant" and content indicating
        failure (e.g., "Generation failed due to system error").
    """
    """Store an error message in the conversation when generation fails."""
    try:
        error_message_id = await run_in_threadpool(
            service.store_error_message,
            conversation_id,
        )
        db.commit()
        logger.info(
            f"Error message stored: conversation_id={conversation_id}, "
            f"error_message_id={error_message_id}"
        )
        return {
            "message": "Error message stored successfully",
            "error_message_id": error_message_id,
        }
    except Exception:
        db.rollback()
        raise

unstar_message(payload, message_repo=Depends(get_message_repository), db=Depends(get_db)) async

Remove star/bookmark from a previously starred message.

Parameters:

Name Type Description Default
payload MessageStarRequest

MessageStarRequest with message_id to unstar.

required
message_repo MessageRepository

Injected message repository.

Depends(get_message_repository)
db Session

Database session for transaction control.

Depends(get_db)

Returns:

Name Type Description
dict

Success confirmation with state="success".

Example

::

POST /erudi/conversations/unstar_message
{
  "message_id": 345
}
→ {"state": "success", "message": "Message unstarred successfully"}
Source code in backend/src/domains/conversations/endpoints.py
@router.post("/unstar_message")
async def unstar_message(
    payload: MessageStarRequest,
    message_repo: MessageRepository = Depends(get_message_repository),
    db: Session = Depends(get_db),
):
    """Remove star/bookmark from a previously starred message.

    Args:
        payload: MessageStarRequest with message_id to unstar.
        message_repo: Injected message repository.
        db: Database session for transaction control.

    Returns:
        dict: Success confirmation with state="success".

    Example:
        ::

            POST /erudi/conversations/unstar_message
            {
              "message_id": 345
            }
            → {"state": "success", "message": "Message unstarred successfully"}
    """
    """Unstar a message in the conversation."""
    try:
        await run_in_threadpool(message_repo.unstar_message, payload.message_id)
        db.commit()
        logger.info(f"Message unstarred: message_id={payload.message_id}")
        return {"state": "success", "message": "Message unstarred successfully"}
    except Exception:
        db.rollback()
        raise

update_conversation(conversation_id, payload, service=Depends(get_conversation_service), db=Depends(get_db)) async

Update conversation metadata (name, LLM, generation parameters).

Parameters:

Name Type Description Default
conversation_id int

ID of the conversation to update.

required
payload ConversationUpdate

ConversationUpdate schema with optional name, llm_id, temperature, top_p, max_tokens, custom_prompt.

required
service ConversationService

Injected conversation service.

Depends(get_conversation_service)
db Session

Database session for transaction control.

Depends(get_db)

Returns:

Name Type Description
ConversationResponse

Updated conversation with new metadata.

Example

::

PATCH /erudi/conversations/42
{
  "name": "Python Advanced Topics",
  "llm_id": 7,
  "temperature": 0.5,
  "max_tokens": 2048
}
→ {
    "id": 42,
    "name": "Python Advanced Topics",
    "llm_id": 7,
    "temperature": 0.5,
    "max_tokens": 2048,
    ...
  }
Note

Only provided fields are updated. Omitted fields remain unchanged.

Source code in backend/src/domains/conversations/endpoints.py
@router.patch("/{conversation_id}", response_model=ConversationResponse)
async def update_conversation(
    conversation_id: int,
    payload: ConversationUpdate,
    service: ConversationService = Depends(get_conversation_service),
    db: Session = Depends(get_db),
):
    """Update conversation metadata (name, LLM, generation parameters).

    Args:
        conversation_id: ID of the conversation to update.
        payload: ConversationUpdate schema with optional name, llm_id,
            temperature, top_p, max_tokens, custom_prompt.
        service: Injected conversation service.
        db: Database session for transaction control.

    Returns:
        ConversationResponse: Updated conversation with new metadata.

    Example:
        ::

            PATCH /erudi/conversations/42
            {
              "name": "Python Advanced Topics",
              "llm_id": 7,
              "temperature": 0.5,
              "max_tokens": 2048
            }
            → {
                "id": 42,
                "name": "Python Advanced Topics",
                "llm_id": 7,
                "temperature": 0.5,
                "max_tokens": 2048,
                ...
              }

    Note:
        Only provided fields are updated. Omitted fields remain unchanged.
    """
    """Update conversation fields (name and llm_id)."""
    try:
        result = await run_in_threadpool(
            service.update_conversation,
            conversation_id=conversation_id,
            name=payload.name,
            llm_id=payload.llm_id,
            temperature=payload.temperature,
            top_p=payload.top_p,
            max_tokens=payload.max_tokens,
            custom_prompt=payload.custom_prompt,
            web_search_enabled=payload.web_search_enabled,
            reasoning_effort=payload.reasoning_effort,
            compaction_threshold=payload.compaction_threshold,
        )
        db.commit()
        updated_fields = [
            field
            for field, value in (
                ("name", payload.name),
                ("llm_id", payload.llm_id),
                ("temperature", payload.temperature),
                ("top_p", payload.top_p),
                ("max_tokens", payload.max_tokens),
                ("custom_prompt", payload.custom_prompt),
                ("web_search_enabled", payload.web_search_enabled),
                ("reasoning_effort", payload.reasoning_effort),
                ("compaction_threshold", payload.compaction_threshold),
            )
            if value is not None
        ]
        logger.info(f"Conversation updated: id={conversation_id}, fields={updated_fields}")
        return result
    except Exception:
        db.rollback()
        raise

src.domains.conversations.schemas

Pydantic schemas for conversation and message data validation.

This module defines request/response schemas for: - Message creation and responses (user/assistant messages) - Conversation CRUD operations (create, update, read) - Query parameters for streaming generation - Bulk operations (delete multiple conversations) - Message starring (bookmarking)

Schema Hierarchy

::

MessageBase → MessageResponse
ConversationBase → ConversationCreate, ConversationUpdate, ConversationResponse
ConversationResponse → ConversationWithMessagesResponse
Validation Rules
  • Message content: non-empty, no length cap
  • Message sender: Must be "user" or "assistant"
  • Temperature: 0.0-2.0 (default 0.2)
  • Top-p: 0.0-1.0 (default 0.5)
  • Max tokens: 1-32768 (default 1024)
  • Custom prompt: Max 4096 characters
Example

Create conversation request::

payload = ConversationCreate(
    llm_id=5,
    temperature=0.7,
    top_p=0.9,
    max_tokens=2048,
    custom_prompt="You are a helpful coding assistant."
)

Query conversation with message::

query = ConversationQuery(
    question="Explain asyncio in Python",
    temperature=0.5,
)
Note

All schemas use Pydantic v2 field validation with clear error messages. from_attributes=True allows ORM model conversion to schema.

ConversationBase

Bases: BaseModel

Base schema for conversations with required LLM ID.

Attributes:

Name Type Description
llm_id int

ID of the LLM model to use for this conversation.

ConversationCreate

Bases: ConversationBase

Schema for creating new conversations with generation parameters.

Attributes:

Name Type Description
llm_id int

Inherited from ConversationBase.

temperature Optional[float]

Sampling temperature (0.0-2.0, default 0.2).

top_p Optional[float]

Nucleus sampling threshold (0.0-1.0, default 0.5).

max_tokens Optional[int]

Legacy fallback ceiling, used ONLY when the engine cannot report a context window; with a window, the output budget is automatic and this value is ignored.

custom_prompt Optional[str]

Custom system prompt override (max 4096 chars).

Example

::

conv = ConversationCreate(
    llm_id=3,
    temperature=0.8,
    top_p=0.95,
    max_tokens=2048,
    custom_prompt="You are a creative storyteller."
)

ConversationQuery

Bases: BaseModel

Schema for querying conversations with streaming generation parameters.

Attributes:

Name Type Description
question str

User message/query to send to LLM.

images Optional[List[str]]

Optional base64 data-URL images attached to the question (vision models).

image_paths Optional[List[str]]

Local filesystem path of each image, parallel to images.

attachments Optional[List[str]]

Local filesystem paths of documents/folders attached to the question.

temperature Optional[float]

Optional temperature override for this query only.

top_p Optional[float]

Optional top_p override for this query only.

max_new_tokens Optional[int]

Accepted and ignored -- the output budget is computed per model call from the context window the model runs in (src.agents.output_budget), so nothing on the client sets it. The field stays on the schema so an older client is answered, not rejected with a 422.

custom_prompt Optional[str]

Optional system prompt override for this query only.

Example

::

query = ConversationQuery(
    question="Explain list comprehensions",
    temperature=0.3,
)

ConversationResponse

Bases: ConversationBase

Schema for conversation responses with full metadata.

Attributes:

Name Type Description
id int

Conversation database ID.

created_at datetime

Conversation creation timestamp.

last_message_time datetime

Timestamp of most recent message.

name str

Conversation display name.

temperature float

Current temperature setting.

top_p float

Current top_p setting.

max_tokens int

Current max_tokens setting.

custom_prompt str

Current system prompt.

ConversationUpdate

Bases: ConversationBase

Schema for partial conversation updates (PATCH operations).

All fields are optional to support partial updates. Only provided fields will be updated in the database.

Attributes:

Name Type Description
llm_id Optional[int]

Optional LLM ID override.

name Optional[str]

Optional conversation name.

temperature Optional[float]

Optional temperature override.

top_p Optional[float]

Optional top_p override.

max_tokens Optional[int]

Legacy fallback ceiling (window-less engines only); persisted but ignored whenever the engine reports a window.

custom_prompt Optional[str]

Optional system prompt override.

Example

::

update = ConversationUpdate(
    name="Python Best Practices",
    temperature=0.3  # Only update name and temperature
)

ConversationWithMessagesResponse

Bases: ConversationResponse

Extended conversation response including full message history.

Attributes:

Name Type Description
messages List[MessageResponse]

List of all MessageResponse objects in conversation.

Example

::

GET /conversations/42
→ ConversationWithMessagesResponse {
    id: 42,
    name: "Python Tutorial",
    messages: [
      MessageResponse(...),
      MessageResponse(...)
    ]
  }

MessageBase

Bases: BaseModel

Base schema for messages with sender and content validation.

Attributes:

Name Type Description
sender str

Message sender, must be "user" or "assistant".

content str

Message content, non-empty, no length cap.

validate_sender(v)

Ensure sender is either 'user', 'assistant', or 'llm'.

Parameters:

Name Type Description Default
v

Sender value to validate.

required

Returns:

Name Type Description
str

Validated sender value.

Raises:

Type Description
ValueError

If sender is not "user", "assistant", or "llm".

Source code in backend/src/domains/conversations/schemas.py
@validator("sender")
def validate_sender(cls, v):
    """Ensure sender is either 'user', 'assistant', or 'llm'.

    Args:
        v: Sender value to validate.

    Returns:
        str: Validated sender value.

    Raises:
        ValueError: If sender is not "user", "assistant", or "llm".
    """
    if v not in ["user", "assistant", "llm"]:
        raise ValueError('Sender must be either "user", "assistant", or "llm"')
    return v

MessageResponse

Bases: MessageBase

Schema for message responses with database fields.

Extends MessageBase with id, conversation_id, timestamp, starred.

Attributes:

Name Type Description
id int

Database message ID.

conversation_id int

Parent conversation ID.

timestamp datetime

Message creation timestamp.

starred bool

Whether message is bookmarked (default False).

trace Optional[List[dict]]

Optional list of the assistant turn's non-answer stream events (thinking / tool_call / tool_result, and the compaction done events, #614) for replay on reload (#90).

MessageStarRequest

Bases: BaseModel

Schema for starring/unstarring messages.

Attributes:

Name Type Description
message_id int

ID of the message to toggle star state.

Example

::

POST /conversations/star_message
{
  "message_id": 345
}

src.domains.conversations.services

Services for conversation management and streaming AI generation.

Business logic for the conversation lifecycle and streaming generation. Since the LangChain refactor, generation goes through AgentRunner (one create_agent per turn, history persisted in the LangGraph checkpointer) instead of the hand-rolled prompt/engine loop. The SQLAlchemy Message/Conversation tables remain the source of truth for display (fetch_messages, starred, timestamps); the checkpointer holds the agent's working state and rolling summary.

Layering: streaming methods are async generators (the FastAPI StreamingResponse consumes them directly on the event loop); all synchronous SQLAlchemy work is wrapped in run_in_threadpool so DB commits never block the loop.

ConversationService(db, checkpointer=None)

Service for managing conversations and message processing.

Initialize the service.

Parameters:

Name Type Description Default
db

SQLAlchemy session for repository operations.

required
checkpointer

LangGraph checkpointer (app-wide AsyncPostgresSaver) for stateful conversations. None runs the agent statelessly (used by tests that don't exercise persistence).

None
Source code in backend/src/domains/conversations/services.py
def __init__(self, db, checkpointer=None):
    """Initialize the service.

    Args:
        db: SQLAlchemy session for repository operations.
        checkpointer: LangGraph checkpointer (app-wide ``AsyncPostgresSaver``)
            for stateful conversations. ``None`` runs the agent statelessly
            (used by tests that don't exercise persistence).
    """
    logger.debug("Initializing ConversationService")
    self.conversation_repo = ConversationRepository(db)
    self.message_repo = MessageRepository(db)
    self.checkpointer = checkpointer
    self.runner = AgentRunner(checkpointer)
    self.db = db

create_conversation(llm_id, temperature=None, top_p=None, max_tokens=None, custom_prompt='', web_search_enabled=None, reasoning_effort=None, compaction_threshold=None)

Create a new conversation with the given LLM and generation params.

temperature / top_p / max_tokens left None resolve to the MODEL's defaults (#388: the captured generation_config, else the neutral constants, see src.database.generation_hints); an explicit value wins. Mirrors web_search_enabled=None, which copies the GLOBAL user-settings default at creation (#310), reasoning_effort=None, which copies the global default level the same way (1.1.2), and compaction_threshold=None, which copies the global default threshold (#607). The conversation owns its values afterwards -- later model/global changes never retro-affect it.

Source code in backend/src/domains/conversations/services.py
def create_conversation(
    self,
    llm_id: int,
    temperature: Optional[float] = None,
    top_p: Optional[float] = None,
    max_tokens: Optional[int] = None,
    custom_prompt: str = "",
    web_search_enabled: Optional[bool] = None,
    reasoning_effort: Optional[str] = None,
    compaction_threshold: Optional[int] = None,
) -> Conversation:
    """Create a new conversation with the given LLM and generation params.

    ``temperature`` / ``top_p`` / ``max_tokens`` left ``None`` resolve to the
    MODEL's defaults (#388: the captured generation_config, else the neutral
    constants, see ``src.database.generation_hints``); an explicit value
    wins. Mirrors
    ``web_search_enabled=None``, which copies the GLOBAL user-settings
    default at creation (#310), ``reasoning_effort=None``, which copies
    the global default level the same way (1.1.2), and
    ``compaction_threshold=None``, which copies the global default
    threshold (#607). The conversation owns its values afterwards -- later
    model/global changes never retro-affect it.
    """
    logger.info(f"Creating new conversation with LLM {llm_id}")
    if temperature is None or top_p is None or max_tokens is None:
        defaults = resolve_sampling_defaults(self.conversation_repo.get_llm_by_id(llm_id))
        temperature = defaults.temperature if temperature is None else temperature
        top_p = defaults.top_p if top_p is None else top_p
        max_tokens = defaults.max_tokens if max_tokens is None else max_tokens
    settings_repo = User_Settings_Repository(self.db)
    if web_search_enabled is None:
        web_search_enabled = settings_repo.get_web_search_enabled()
    if reasoning_effort is None:
        reasoning_effort = settings_repo.get_default_reasoning_effort()
    if compaction_threshold is None:
        compaction_threshold = settings_repo.get_default_compaction_threshold()
    return self.conversation_repo.create_conversation(
        llm_id=llm_id,
        name="New Conversation",
        temperature=temperature,
        top_p=top_p,
        max_tokens=max_tokens,
        custom_prompt=custom_prompt,
        web_search_enabled=web_search_enabled,
        reasoning_effort=reasoning_effort,
        compaction_threshold=compaction_threshold,
    )

delete_conversation(conversation_id) async

Delete a conversation and purge its checkpointer thread.

Both stores must be cleared: ids can be reused after a fresh start or a sequence reset, so a stale checkpointer thread would otherwise leak a deleted conversation's agent context into a future one with the same id (BLOCKER B3).

Source code in backend/src/domains/conversations/services.py
async def delete_conversation(self, conversation_id: int) -> None:
    """Delete a conversation and purge its checkpointer thread.

    Both stores must be cleared: ids can be reused after a fresh start or a
    sequence reset, so a stale checkpointer thread would otherwise leak a deleted
    conversation's agent context into a future one with the same id (BLOCKER B3).
    """
    logger.info(f"Deleting conversation {conversation_id}")
    await run_in_threadpool(self._delete_conversation_db, conversation_id)
    await self._purge_thread(conversation_id)

generate_title_stream(conversation_id, question) async

Stream an auto-generated 2–4 word title (stateless one-shot).

Source code in backend/src/domains/conversations/services.py
async def generate_title_stream(
    self,
    conversation_id: int,
    question: str,
) -> AsyncGenerator[str, None]:
    """Stream an auto-generated 2–4 word title (stateless one-shot)."""
    start_s = time.perf_counter()
    logger.info(
        f"Generating title for conversation {conversation_id}: "
        f"{truncate_for_log(question, 2000)}"
    )

    try:
        _conversation, llm = await run_in_threadpool(
            self._load_conversation_and_llm, conversation_id
        )
    except Exception:
        # Recovered with the default title; the traceback says why.
        logger.warning(
            f"Title generation: could not load conversation {conversation_id} or its LLM; "
            f"keeping the default title",
            exc_info=True,
        )
        await run_in_threadpool(self._save_title, conversation_id, "New Conversation")
        return

    if not question or not question.strip():
        await run_in_threadpool(self._save_title, conversation_id, "New Conversation")
        return

    prompt_text = self._build_title_prompt_text(question, llm.type)
    temperature = 0.5 if llm.type == "mistral" else 1.0
    top_p = 0.9 if llm.type == "mistral" else 0.95

    generated_title = ""
    try:
        async with contextlib.aclosing(
            self.runner.astream_oneshot(
                llm=llm,
                prompt_text=prompt_text,
                temperature=temperature,
                top_p=top_p,
                max_tokens=12,
            )
        ) as chunks:
            async for chunk in chunks:
                generated_title += chunk
                yield chunk
    finally:
        final_title = _sanitize_title(generated_title) or "New Conversation"
        duration_ms = (time.perf_counter() - start_s) * 1000
        logger.info(
            f"Title generation completed for conversation {conversation_id}: "
            f"duration_ms={duration_ms:.0f}, title={final_title!r}"
        )
        await run_in_threadpool(self._save_title, conversation_id, final_title)

query_and_respond_stream(conversation_id, payload) async

Stream the agent's turn as NDJSON events (application/x-ndjson).

One JSON object per line: answer (streamed answer text), thinking (streamed reasoning), tool_call / tool_result (agent activity), compaction (#614: phase: "start" while earlier messages are being summarized, phase: "done" with summarized, reason, summary and summary_lost once they were), memory_reload (the model reloads: the memory brake unloaded it, or, with reason: "idle", the idle timeout did) / memory_warning (this machine's memory), error (the mapped ERROR sentinel), then a terminal done (#90).

The answer text is accumulated exactly as before and persisted as the assistant message content; thinking, tool and compaction-done events are collected into an ordered trace (capped, drop-oldest, compactions pinned) and persisted alongside so the panel and the compaction marker replay on reload. compaction start and the memory events reach the wire only. Persists the user message up front (so it shows immediately); the agent restores prior history from the checkpointer.

Error mapping (#252/#225-D4): the runner yields its curated error as an answer event carrying the ERROR sentinel. Here that answer is (a) still appended to assistant_response so the persisted content is byte-for-byte the pre-#90 error turn, and (b) re-framed on the wire as an error event (the frontend renders the red bubble from error as it did from the sentinel substring). Error turns persist only their compaction events (the compaction is checkpointed even when the model call then fails).

Source code in backend/src/domains/conversations/services.py
async def query_and_respond_stream(
    self,
    conversation_id: int,
    payload: ConversationQuery,
) -> AsyncGenerator[str, None]:
    """Stream the agent's turn as NDJSON events (``application/x-ndjson``).

    One JSON object per line: ``answer`` (streamed answer text), ``thinking``
    (streamed reasoning), ``tool_call`` / ``tool_result`` (agent activity),
    ``compaction`` (#614: ``phase: "start"`` while earlier messages are being
    summarized, ``phase: "done"`` with ``summarized``, ``reason``, ``summary``
    and ``summary_lost`` once they were), ``memory_reload``
    (the model reloads: the memory brake unloaded it, or, with
    ``reason: "idle"``, the idle timeout did) / ``memory_warning`` (this
    machine's memory), ``error`` (the mapped ERROR
    sentinel), then a terminal ``done`` (#90).

    The answer text is accumulated exactly as before and persisted as the
    assistant message ``content``; thinking, tool and compaction-``done``
    events are collected into an ordered ``trace`` (capped, drop-oldest,
    compactions pinned) and persisted alongside so the panel and the
    compaction marker replay on reload. ``compaction`` ``start`` and the
    memory events reach the wire only. Persists the user message up front (so
    it shows immediately); the agent restores prior history from the
    checkpointer.

    Error mapping (#252/#225-D4): the runner yields its curated error as an
    ``answer`` event carrying the ERROR sentinel. Here that answer is (a) still
    appended to ``assistant_response`` so the persisted content is byte-for-byte
    the pre-#90 error turn, and (b) re-framed on the wire as an ``error`` event
    (the frontend renders the red bubble from ``error`` as it did from the
    sentinel substring). Error turns persist only their compaction events
    (the compaction is checkpointed even when the model call then fails).
    """
    start_s = time.perf_counter()
    # Local-first policy: log the question content (bounded), never image
    # bytes — only their count and encoded size.
    image_note = ""
    if payload.images:
        total_b64_chars = sum(len(url) for url in payload.images)
        image_note = f", images={len(payload.images)} ({total_b64_chars} base64 chars)"
    if payload.attachments:
        image_note += f", attachments={len(payload.attachments)}"
    logger.info(
        f"Processing query for conversation {conversation_id}{image_note}: "
        f"{truncate_for_log(payload.question, 2000)}"
    )

    try:
        conversation, llm = await run_in_threadpool(
            self._load_conversation_and_llm, conversation_id
        )
    except Exception:
        # The response has started (an NDJSON stream), so no exception
        # handler will see this: the record is written here.
        logger.exception(f"Failed to load conversation {conversation_id} or its LLM for query")
        # No conversation loaded -> nothing to persist; just surface the error.
        yield _ndjson({"t": "error", "text": ERROR_MESSAGE})
        yield _ndjson({"t": "done"})
        return

    assistant_response = ""
    trace: list = []
    persisted = False

    async def _persist_assistant_once() -> None:
        # The ``done`` event is the client's refetch signal (#303): the
        # assistant row must be committed before that event goes out, or
        # the refetch races the insert and returns rows without the answer.
        # Idempotent so the finally-block safety net (client disconnect
        # mid-stream) never double-writes.
        nonlocal persisted
        if persisted:
            return
        persisted = True
        # An error turn persists only its compactions: the compaction was
        # checkpointed before the model call failed, and its marker must
        # survive a reload.
        if ERROR_SENTINEL in assistant_response:
            persist_trace = _cap_trace([e for e in trace if _is_compaction(e)])
        else:
            persist_trace = _cap_trace(trace)
        await run_in_threadpool(
            self._persist_assistant_message,
            conversation_id,
            assistant_response,
            persist_trace,
        )

    try:
        # Documents attached to this question (#492): read on this machine
        # through the KB's DocumentReader and injected as delimited blocks
        # ahead of the user's words. Blocking parsing -> threadpool.
        attachments = await run_in_threadpool(resolve_attachments, payload.attachments)
        user_message = self._build_user_message(
            prepend_attachment_block(attachments.block, payload.question), payload.images
        )
        # Read BEFORE the user message is persisted: a new conversation
        # shows its first-reply hint, not the idle-reload notice.
        has_turns = await run_in_threadpool(self.message_repo.has_messages, conversation_id)
        await run_in_threadpool(
            self._persist_user_message,
            conversation_id,
            self._user_display_content(
                payload.question,
                payload.images,
                payload.image_paths or [],
                attachments.paths,
            ),
        )

        starred = await run_in_threadpool(
            self.message_repo.get_starred_messages, conversation_id
        )
        # Derive the turn's mode (plain / systematic-KB / agentic-KB) from
        # the model's tool-calling capability and build the runner bundle
        # (#84). Retrieval is injected so it runs only in systematic mode and
        # keeps the conversation's degrade-to-no-context policy.
        # How much this conversation lets the model deliberate (1.1.2). The
        # conversation OWNS its level, like its web toggle; the plan is
        # resolved against the ARTIFACT's lever, so the same level reaches
        # a gpt-oss and a Qwen2.5 through different mechanisms. Blocking
        # (a template probe may read the tokenizer) -> threadpool.
        effort_plan = await run_in_threadpool(
            plan_reasoning_effort, llm, conversation.reasoning_effort
        )
        plan = await run_in_threadpool(
            plan_turn,
            llm,
            question=payload.question,
            retrieve=lambda: self._retrieve_kb_excerpts(llm, payload.question),
            custom_prompt=payload.custom_prompt,
            starred_messages=starred,
            # #310: the conversation OWNS its web toggle (copied from the
            # global default at creation).
            web_search_enabled=bool(conversation.web_search_enabled),
            effort_section=effort_plan.prompt_section,
        )
        params = GenParams(
            temperature=payload.temperature
            if payload.temperature is not None
            else conversation.temperature,
            top_p=payload.top_p if payload.top_p is not None else conversation.top_p,
            # Only the FALLBACK output budget. What the model actually gets
            # is computed per model call from the window it is running in
            # (``src.agents.output_budget``), and that overrides this value
            # whenever the engine reports a window. This one is what an
            # engine that reports none still runs with -- today's behaviour,
            # unchanged. ``payload.max_new_tokens`` is deliberately not read:
            # the control it came from is gone (the field is still accepted
            # so an older client is not rejected).
            max_tokens=conversation.max_tokens or FALLBACK_MAX_TOKENS,
        )

        # Safety net (#133/#212): unless the model is positively vision-
        # capable, the runner strips images before inference — surface that
        # up front instead of silently dropping the attachment. The notice
        # is persisted with the assistant message (deliberate).
        supports_vision = await run_in_threadpool(detect_supports_vision, llm.link)
        if payload.images and supports_vision is not True:
            assistant_response += IMAGES_IGNORED_NOTICE
            yield _ndjson({"t": "answer", "text": IMAGES_IGNORED_NOTICE})

        # Same contract for documents (#492): a file the reader could not
        # turn into text, or text cut to fit the budget, is named up front
        # rather than dropped in silence. Persisted with the answer.
        attachment_notice = build_attachment_notice(attachments)
        if attachment_notice:
            assistant_response += attachment_notice
            yield _ndjson({"t": "answer", "text": attachment_notice})

        # ``aclosing``: whenever THIS generator is closed, the runner is
        # closed with it, inside the guard. Starlette never closes a body
        # iterator itself: a disconnect that cancels the stream unwinds
        # the chain through the cancellation (the runner then closes
        # LangGraph inside the guard, see ``src.agents.isolated_stream``),
        # but a disconnect noticed while ``send()`` is in progress leaves
        # this generator suspended at its yield, and closing it then still
        # depends on its finalization.
        async with contextlib.aclosing(
            self.runner.astream_text(
                llm=llm,
                user_message=user_message,
                system_prompt=plan.system_prompt,
                params=params,
                thread_id=str(conversation_id),
                summarize=True,
                kb_context_block=plan.kb_context_block,
                kb_language_line=plan.kb_language_line,
                tools=plan.tools,
                context=plan.context,
                supports_vision=supports_vision,
                effort_plan=effort_plan,
                # #607: the conversation OWNS its compaction threshold
                # (copied from the global default at creation); a lowered
                # one compacts on this very turn.
                compaction_threshold=conversation.compaction_threshold,
                emit_events=True,
                announce_idle_reload=has_turns,
            )
        ) as events:
            async for event in events:
                if event["t"] == "answer":
                    text = event["text"]
                    # Persistence is unchanged: the full answer text
                    # (including a sentinel error string, if any) is
                    # accumulated for the DB.
                    assistant_response += text
                    if text.startswith(ERROR_SENTINEL):
                        yield _ndjson(build_stream_error_event(event))
                    else:
                        yield _ndjson(event)
                elif event["t"] == "memory_warning":
                    # Wire only, NEVER the persisted trace (1.1.2): the
                    # warning describes the machine's memory NOW -- an old
                    # conversation reopened on a bigger machine must not
                    # replay it as if it still applied.
                    yield _ndjson(event)
                elif event["t"] == "memory_reload":
                    # Wire only, never persisted, for the same reason: the
                    # memory brake or the idle timeout (``reason``)
                    # unloaded the model, and this turn reloads it -- a
                    # fact about this machine at this moment, not part of
                    # the conversation. ``reason`` passes through as is.
                    yield _ndjson(event)
                elif event["t"] == COMPACTION_EVENT and event.get("phase") == "start":
                    # Wire only (#614): the pending answer says the
                    # conversation is being summarized; ``done`` (below)
                    # is what the trace keeps.
                    yield _ndjson(event)
                else:
                    # thinking / tool_call / tool_result / compaction done
                    # -> wire AND replay trace.
                    trace.append(event)
                    yield _ndjson(event)
        await _persist_assistant_once()
        yield _ndjson({"t": "done"})
    except Exception:
        logger.exception(f"Query streaming failed for conversation {conversation_id}")
        if not assistant_response:
            assistant_response = ERROR_MESSAGE
            yield _ndjson({"t": "error", "text": ERROR_MESSAGE})
        await _persist_assistant_once()
        yield _ndjson({"t": "done"})
    finally:
        duration_ms = (time.perf_counter() - start_s) * 1000
        logger.info(
            f"Query completed for conversation {conversation_id}: "
            f"duration_ms={duration_ms:.0f}, "
            f"response_chars={len(assistant_response)}, "
            f"preview={truncate_for_log(assistant_response, 500)}"
        )
        # Safety net: a client that disconnects mid-stream closes the
        # generator before either ``done`` path ran — persist here so the
        # partial answer is never lost (no-op when done already persisted).
        await _persist_assistant_once()

store_error_message(conversation_id)

Persist a fallback error message (called by the frontend on failure).

Source code in backend/src/domains/conversations/services.py
def store_error_message(self, conversation_id: int) -> int:
    """Persist a fallback error message (called by the frontend on failure)."""
    logger.info(f"Storing error message for conversation {conversation_id}")
    message = self.message_repo.create_message(
        conversation_id=conversation_id,
        content=ERROR_MESSAGE,
        sender="llm",
    )
    self.conversation_repo.update_last_message_time(conversation_id)
    return message.id

update_conversation(conversation_id, name=None, llm_id=None, temperature=None, top_p=None, max_tokens=None, custom_prompt=None, web_search_enabled=None, reasoning_effort=None, compaction_threshold=None)

Partial update of conversation metadata (only non-None fields).

Source code in backend/src/domains/conversations/services.py
def update_conversation(
    self,
    conversation_id: int,
    name: Optional[str] = None,
    llm_id: Optional[int] = None,
    temperature: Optional[float] = None,
    top_p: Optional[float] = None,
    max_tokens: Optional[int] = None,
    custom_prompt: Optional[str] = None,
    web_search_enabled: Optional[bool] = None,
    reasoning_effort: Optional[str] = None,
    compaction_threshold: Optional[int] = None,
) -> Conversation:
    """Partial update of conversation metadata (only non-None fields)."""
    logger.info(f"Updating conversation {conversation_id}")
    return self.conversation_repo.update_conversation(
        conversation_id=conversation_id,
        name=name,
        llm_id=llm_id,
        temperature=temperature,
        top_p=top_p,
        max_tokens=max_tokens,
        custom_prompt=custom_prompt,
        web_search_enabled=web_search_enabled,
        reasoning_effort=reasoning_effort,
        compaction_threshold=compaction_threshold,
    )

build_stream_error_event(answer_event)

The wire error event for a sentinel-carrying answer event.

The renderer has always read {"t": "error", "text": ...} and renders it as the red turn. A failure the engine could identify adds two keys the renderer acts on rather than displays: code (one of src.engines.cuda_compatibility.CUDA_FAILURE_CODES) and raw (the child's captured output, shown in the copyable block of the decision dialog). Both are omitted for every other failure, so the untyped path keeps exactly the shape it had.

Source code in backend/src/domains/conversations/services.py
def build_stream_error_event(answer_event: dict) -> dict:
    """The wire ``error`` event for a sentinel-carrying answer event.

    The renderer has always read ``{"t": "error", "text": ...}`` and renders it
    as the red turn. A failure the engine could identify adds two keys the
    renderer acts on rather than displays: ``code`` (one of
    ``src.engines.cuda_compatibility.CUDA_FAILURE_CODES``) and ``raw`` (the
    child's captured output, shown in the copyable block of the decision
    dialog). Both are omitted for every other failure, so the untyped path
    keeps exactly the shape it had.
    """
    error_event: dict = {"t": "error", "text": answer_event["text"]}
    for key in ("code", "raw"):
        value = answer_event.get(key)
        if value:
            error_event[key] = value
    return error_event