Skip to content

Docs: allow html responses from tool results to include a custom messages #1140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Haervwe wants to merge 2 commits into
base: dev
Choose a base branch
from
Draft

Docs: allow html responses from tool results to include a custom messages #1140

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7129d54
Document custom LLM context messages for tools
Haervwe Mar 15, 2026
60bd538
docs: enhance LLM context messages for HTMLResponse embeds with detai…
Haervwe Mar 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/features/extensibility/plugin/development/rich-ui.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,34 @@ def create_visualization_tool(self, data: str) -> HTMLResponse:
return HTMLResponse(content=html_content, headers=headers)
```

### Custom LLM Context Messages (Tools)

By default, when a Tool successfully returns an HTMLResponse embed, the LLM is sent a generic system message: "{tool_function_name}: Embedded UI result is active and visible to the user."

While this prevents the LLM from trying to read raw HTML, it also hides all the context of what actually happened. If you want the LLM to know specific details about the generated embed—like the parameters used, the status of the action, or what the user is seeing—you can override this generic message returning
a tuple of `(HTMLResponse, llm_message)` instead of just the `HTMLResponse`. The `llm_message` string or object will be sent to the LLM as context instead of the generic message, allowing you to provide rich semantic information about the embed:

```python
from fastapi.responses import HTMLResponse
def generate_music_tool(self, genre: str, duration: int) -> HTMLResponse:
"""
Generates music and embeds a player in the chat.
"""
# Your HTML player generation logic
html_content = f"<html><body><audio controls src='...'></audio></body></html>"
headers = {
"Content-Disposition": "inline",

}

llm_message = f"Audio player embedded successfully. A {duration}-second {genre} track was generated. The user can play or download it from the player above. {download_link}"

return HTMLResponse(content=html_content, headers=headers), llm_message
```

This ensures the LLM receives actionable, semantic feedback about the visualization or embedded UI, allowing it to provide relevant follow-up responses to the user.


## Action Usage

Actions work exactly the same way. The rich UI embed is delivered to the chat via the event emitter:
Expand Down