-
Notifications
You must be signed in to change notification settings - Fork 1.1k
[UNIT] Add draft Gradio bonus module for agents-course #120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
[UNIT] Add draft Gradio bonus module for agents-course #120
Conversation
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really like the ToC structure and it makes total sense.
Are you looking to get started on this now? If so, I'd recommend smaller PRs for each chapter off this branch.
- Showing visible thought process of agents | ||
|
||
- Hands-on Project: Building a Complete Agent UI | ||
- Implementing an Agentic UI : smolagents |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's amazing that you're planning to implement it in 3 libraries. But feel free to pick one, if you think it's difficult to maintain.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This suggestion makes sense, and the team is on board with it as well! I’m excited to focus only on smolagents for this particular chapter.
- Implementing an Agentic UI : llamaindex | ||
- Adding tool usage visualization | ||
- Customizing the interface | ||
- Bonus Module Final Quiz |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just so you know, I've also made a code quiz space that uses an LLM as a judge. I can help with setting up the questions if you like.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That'll be great, thanks!
- Introduction to Gradio for AI Agents | ||
- Why Gradio for Agent UIs? | ||
- Understanding the role of UI in Agent interactions | ||
- Setting up Gradio |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You might want to mention the difference between this and a gradio space being used as a tool for an agent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I missed this point in my latest submission, will include the distinction now. Thanks, Ben.
- Tool usage visualization | ||
- Quick Quiz 2 | ||
|
||
- Advanced Agent UI Features |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will be nice with lot's of examples and embedded gradio spaces.
|
||
*We are also planning to release some bonus units, stay tuned!* | ||
*We have one bonus unit available for you: Gradio module helps you create interactive interfaces for your agents. More bonus units coming soon!* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would not change that sentence. As it removes the emphasis on other future bonus Units
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed with @Jofthomas. Because we don't have only bonus units with gradio, for instance next week we have one that is not about gradio but fine-tune.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense. Thanks for the reviews.
Sounds exciting. |
Adds first three chapter files for Gradio module
I've added the first three chapters for the Gradio module. Third one is still work in progress. Would appreciate feedback on the content while I continue working on the remaining chapters. cc: @dawoodkhan82 for vis. |
units/en/unit5/1_introduction.mdx
Outdated
Before we dive deeper, let's set up Gradio in your environment: | ||
|
||
```bash | ||
pip install gradio |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just in case they have an older version of gradio installed
pip install gradio | |
pip install --upgrade gradio |
This creates a complete chat interface: | ||
|
||
<need to add an image of the generated ui with an example in the agents-course dataset at https://huggingface.co/datasets/agents-course/course-images> | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should add a sentence here explaining that while GradioUI(agent) launches a default Gradio UI, it doesn't offer the ability to customize the Gradio interface or to build larger UIs that the agent UI can be a part of, or use other frameworks besides smolagents, etc. which is why we'll understand how to build a UI using Gradio directly.
|
||
## Your First Agent UI | ||
|
||
Let's create a simple Agent that helps with weather information. Here's how to build it: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps we should mention to isntall lanchgain and langgraph
- Shows the Agent's thought process and tool-usage in real time | ||
- Keeps users engaged while waiting | ||
|
||
We are using `type="messages"` in gr.ChatInterface. This passes the history as a list of dictionaries with openai-style "role" and "content" keys. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should talk about the gr.ChatMessage
class briefly here and explain that we'll dive more deeply in the next section
I've just wrapped up the final three chapters for the Gradio module! I would really appreciate any feedback on the content. Would be great to get any help/comments in enhancing the course and fix any bugs. Also, I'm cc'ing @dawoodkhan82 for visibility. |
|
||
## What is Gradio? | ||
|
||
Gradio is a Python library that makes it easy to create **beautiful web interfaces** for your AI models and Agents. Think of it as a bridge between your Agent's capabilities and its users: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gradio is a Python library that makes it easy to create **beautiful web interfaces** for your AI models and Agents. Think of it as a bridge between your Agent's capabilities and its users: | |
[Gradio](https://github.com/gradio-app/gradio) is a Python library that makes it easy to create **beautiful web interfaces** for your AI models and Agents. Think of it as a bridge between your Agent's capabilities and its users: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like "web applications" instead of "web interfaces" personally.
- Display your Agent's thought process and tool usage | ||
- Handle file uploads and multimedia content that can be useful for your AI agents | ||
- Share your Agent with anyone via a URL | ||
- Deploy your Agent UI to Hugging Face Spaces |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Deploy your Agent UI to Hugging Face Spaces | |
- Deploy your Agent UI to [Hugging Face Spaces](https://huggingface.co/spaces/) |
# Import tool from Hub | ||
image_generation_tool = load_tool("m-ric/text-to-image", trust_remote_code=True) | ||
# initialize a model | ||
model = HfApiModel() | ||
# Initialize the agent with the image generation tool | ||
agent = CodeAgent(tools=[image_generation_tool], model=model) | ||
# launch the gradio agentic ui | ||
GradioUI(agent).launch(share=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove inline comments?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you referring to all of them? I noticed comments in the code examples from other units, so I just followed that pattern. However, I understand your point, it does appear overcrowded with the number of comments. Perhaps I could eliminate some of the unnecessary ones to reduce the clutter.
|
||
## Your First Agent UI | ||
|
||
We'll start by installing langchain and langgraph for this section. Additionally, set up the environment variable `OPENAI_API_KEY` to store your OpenAI API keys. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Btw it'd be great if we could showcase HF inference providers!
1. **Auto closing the tool usage accordion after the agent is done populating them.** | ||
|
||
Using the Metadata dictionary key called `status` to communicate to the gradio ui when the tool usage is complete. | ||
This also adds a spinner appears next to the thought title. If the `status` is "done", the thought accordion becomes closed. If not provided, the thought accordion is open and no spinner is displayed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This first point seems out-of-place, since we haven't introduced ChatMessage. I think it should go in the next chapter. Perhaps in this part, only focus on the UI tweaks listed in point (2) and explain them in a bit more detail?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it!
@@ -0,0 +1,580 @@ | |||
# Building Agent UIs with Model Context Protocol (MCP) | |||
|
|||
In previous units, we explored creating basic Agent interfaces with Gradio's ChatInterface and advanced visualization using ChatMessage. Now, let's take your Agent UIs to the next level by integrating with the Model Context Protocol (MCP), which enables seamless communication between Claude and external tools. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In previous units, we explored creating basic Agent interfaces with Gradio's ChatInterface and advanced visualization using ChatMessage. Now, let's take your Agent UIs to the next level by integrating with the Model Context Protocol (MCP), which enables seamless communication between Claude and external tools. | |
In previous units, we explored creating basic Agent interfaces with Gradio's ChatInterface and advanced visualization using ChatMessage. Now, let's take your Agent UIs to the next level by integrating with the Model Context Protocol (MCP), which enables seamless communication between your chatbot applicaiton and external tools. |
## What is Model Context Protocol (MCP)? | ||
The Model Context Protocol (MCP) is an open standard that allows LLMs to connect with external data sources and tools. Think of MCP like a "USB-C port for AI" - it provides a standardized way for AI models to interact with various tools and data sources. | ||
|
||
With MCP you can: | ||
- Connect Claude Desktop to specialized tools that extend its capabilities | ||
- Create a modular architecture where tools can be easily added or swapped | ||
- Use an LLM application like Gradio chatbot and use it as a MCP Client |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This intro seems to be written for Claude Desktop? Let's go through and update the references to be for your Gradio-based chatbot. I think this intro coudl also benefit from more concrete examples since it feels a bit abstract to me
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the second half of this chapter where we extend the smolagents UI, but the purpose of the first half is not clear. Maybe you should be clear at the beginning that the goal of this chapter is to reverse engineer the smolagents UI, show how you would create it with Gradio from scratch, which in turn lets you extend it in specific ways.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure we need the first half of this chapter. I like the idea of limiting this chapter to just more advanced gradio features and removing the smolagents UI deep dive.
@@ -72,8 +72,9 @@ Here is the **general syllabus for the course**. A more detailed list of topics | |||
| 2 | Frameworks | Understand how the fundamentals are implemented in popular libraries : smolagents, LangGraph, LLamaIndex | | |||
| 3 | Use Cases | Let's build some real life use cases (open to PRs 🤗 from experienced Agent builders) | | |||
| 4 | Final Assignment | Build an agent for a selected benchmark and prove your understanding of Agents on the student leaderboard 🚀 | | |||
| 5 | Bonus Gradio Module | Learn to build and deploy interactive AI agents with Gradio interfaces | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a suggestion, not sure if this makes more sense
| 5 | Bonus Gradio Module | Learn to build and deploy interactive AI agents with Gradio interfaces | | |
| 5 | Bonus Gradio Module | Learn to build and deploy web applications for AI agents using Gradio | |
Gradio is a Python library that makes it easy to create **beautiful web interfaces** for your AI models and Agents. Think of it as a bridge between your Agent's capabilities and its users: | ||
|
||
|
||
With Gradio, you can: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you mentioned adding gifs. Would be nice to have gifs here to show off these features.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I've been trying to create GIFs for the demos included in the course, but I haven't found any tool that produces good-resolution GIFs from short videos. I've used Windows' native ClipChamp and various free online tools like Ezgif, freeconvert etc, but none have provided satisfactory results. Do you have any recommendations for a free tool? Otherwise, I can also embed short videos.
|
||
## What's Next? | ||
Now that you have these Gradio skills, consider exploring: | ||
- Deploying your Agent UIs to Hugging Face Spaces for wider access |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Deploying your Agent UIs to Hugging Face Spaces for wider access | |
- Deploying your Agent UIs to [Hugging Face Spaces](https://huggingface.co/spaces) for wider access |
|
||
|
||
Finally, we would love **to hear what you think of the course and how we can improve it**. If you have some feedback then, please 👉 [fill this form](https://docs.google.com/forms/d/e/1FAIpQLSe9VaONn0eglax0uTwi29rIn4tM7H2sYmmybmG5jJNlE5v0xA/viewform?usp=dialog) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a call to action to share demos on twitter and tag @gradio
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah, good call, thanks!
This PR is a work-in-progress draft to include the gradio bonus module in the agents-course.
This is based on @burtenshaw's suggestion. We can discuss what to include in this bonus unit. To begin, I have uploaded a tentative plan outlining the chapters and their content. 😄