J’ai commencé réfléchir à la structure des phrases. J’ai pensé à l’ordre des pronoms avant le verbe actif. Si, il y a plus d’un, je veux savoir lequel est plus proche le verbe actif. Maintenant, je sais que le pronom direct est avant le pronom indirect, les deux avant le verb. C’est important à savoir ça.
French
English
Notes
Elle ne veut pas <<aucune question>>
She doesn’t want <<any questions>>
Aucune is always followed by a singular noun
Cette phrase veut dire quoi?
What does this sentence mean?
This is a common saying to know
Nous avons des <<milliers des livres>>
We have <<thousands of books>>
Il m’demande lui envoyer un message
He asked me to send him a message
How to compose with direct and indirect pronouns
Tu me donnes quoi?
What are you giving me?
Also good to know
Je veux <<parler au directeur>> tout de suite
I want <<to talk to the director>> right away
Je viens de finir ma lettre. Super, je te donne une enveloppe tout de suite.
I just finished my letter. Super, I am giving an envelope to you right now
Où est le bureau de la directrice?
Where is the director’s office?
Vous <<écrivez à la directrice>>
You are writing to the director.
Antoine n’est pas là. Tu sais où il est? Non, attends. Je lui écris un e-mail.
Antoine isn’t here. Do you know where he is? No, wait. I am writing an email to him.
Je suis fatiguée et j’ai besoin d’un café. Je <vais faire une pause>
I am tired and I need a coffee. I <<am going to take a break>>.
Tu peux lui expliquer ça au téléphone
You can explain this by telephone to him(her).
The word order
Je veux ce document <sur papier>. J’ai besoin d’une imprimante.
I want this document <<on paper>>. I need a printout.
Est-ce qu’<il y an encore du paper> dans l’imprimante?
Is there paper still in the printer.
Ces employés sont nouveaux et je leur <montre les bureaux>.
These employees are new and I am <<showing the offices to them>>.
Nous leurs expliquons <comment marche l’imprimante>.
We are explaining <<how the printer works>> to them.
Sentence structure and word order
Hélène, si tu ne connais pas mon voisin célèbre, je peux <(te) le présenter>
Helen, if you don’t know my famous neighbor, I can <<introduce him (to you)>>.
Pronouns appear in a specific order. The first one after the verb immediately precedes the verb with the second one further to the left of that pronoun
Cette employée est nouvelle alors je lui présente les autres membres de l’équipe
This employee is new so I’m introducing the other team members to him.
J’ai commencé réfléchir à la structure des phrases. J’ai pensé à l’ordre des pronoms avant le verbe actif. Si, il y a plus d’un, je veux savoir lequel est plus proche le verbe actif. Maintenant, je sais que le pronom direct est avant le pronom indirect, les deux avant le verb. C’est important à savoir ça.
French
English
Notes
Elle ne veut pas <<aucune question>>
She doesn’t want <<any questions>>
Aucune is always followed by a singular noun
Cette phrase veut dire quoi?
What does this sentence mean?
This is a common saying to know
Nous avons des <<milliers des livres>>
We have <<thousands of books>>
Il m’demande lui envoyer un message
He asked me to send him a message
How to compose with direct and indirect pronouns
Tu me donnes quoi?
What are you giving me?
Also good to know
Je veux <<parler au directeur>> tout de suite
I want <<to talk to the director>> right away
Je viens de finir ma lettre. Super, je te donne une enveloppe tout de suite.
I just finished my letter. Super, I am giving an envelope to you right now
Où est le bureau de la directrice?
Where is the director’s office?
Vous <<écrivez à la directrice>>
You are writing to the director.
Antoine n’est pas là. Tu sais où il est? Non, attends. Je lui écris un e-mail.
Antoine isn’t here. Do you know where he is? No, wait. I am writing an email to him.
Je suis fatiguée et j’ai besoin d’un café. Je <vais faire une pause>
I am tired and I need a coffee. I <<am going to take a break>>.
Tu peux lui expliquer ça au téléphone
You can explain this by telephone to him(her).
The word order
Je veux ce document <sur papier>. J’ai besoin d’une imprimante.
I want this document <<on paper>>. I need a printout.
Est-ce qu’<il y an encore du paper> dans l’imprimante?
Is there paper still in the printer.
Ces employés sont nouveaux et je leur <montre les bureaux>.
These employees are new and I am <<showing the offices to them>>.
Nous leurs expliquons <comment marche l’imprimante>.
We are explaining <<how the printer works>> to them.
Sentence structure and word order
Hélène, si tu ne connais pas mon voisin célèbre, je peux <(te) le présenter>
Helen, if you don’t know my famous neighbor, I can <<introduce him (to you)>>.
Pronouns appear in a specific order. The first one after the verb immediately precedes the verb with the second one further to the left of that pronoun
Cette employée est nouvelle alors je lui présente les autres membres de l’équipe
This employee is new so I’m introducing the other team members to him.
When learning a new language, as I’m currently pursuing with French, once you get past the basics of conjugation, grammar and pronunciation, becoming fluent requires a deeper understanding of the language. It is necessary to not only know how to speak the language but also know the nuances of a langauge including common usage patterns, useful related phrases, and linguistic and historical facts. How certain phrases are commonly use. What does it sound like spoken versus written. What language rules cause a change in spelling and word groupings. These are all insights that would be helpful to me as a second language learner to have when studying a language.
For example, in French the phrase “pas de” doesn’t change even if it is followed by a plural noun. It is used as “pas de viandes” as well as “pas de fromage”. This and other similar examples enrich and enhance french-language learning. A language learner has to be able to take notes on these variations and practice them.
Goals
During my studies, I often have to use multiple websites and apps to get the additional information I need to understand the grammar, spelling, and language rules to name a few. Often, It’s more helpful to get additional details and examples.
The goal of my project was to create a French language advisor to help me achieve my goal of being fluent in the French language. Currently, I am working to be fluent in French and I’m currently at CEFR 43 where I can read, write and speak about every things. At this point in my learning, it’s important to understand and explore the language nuances to foster thinking “in French”, versus just learning grammar, vocabulary, and conjugations.
I wanted to see if a chatbot could eliminate the need for multiple apps and websites to get the information I need.
The goals were to:
Create a model that can use an agent, Google Search grounding, and retrieval augmented generation (RAG) to supplement and support my French language studies
Have the model offer suggestions, ideas, examples and interesting information and facts about the French language
Include something interesting, funny and/or popular culture references
Features
The primary feature of the model was to give me the ability to ask any question and receive results that may refer to embedded texts (RAG) or Google search results. I needed the ability to input a word, phrase or sentence in French, and receive an appropriate response, preferably in French.
Chatbot features
Get interesting French language details including grammar, composition, and spelling with Google search grounding.
Create a domain-specific LLM: Add French language notes and documents
Use existing texts as input for RAG to reference French textbooks and documents
LangGraph to manage the conversation and its details
Implementation
This solution leverages LangGraph, Gemini’s Google Search, RAG and functions. The steps used to implement this solution are outlined here. The initial stages of my project were to create a basic graph that included Google searches, client instantiation and creating embeddings. Once that was tested then add and integrate RAG to provide a richer learning experience.
Prepare Data: The PDFs for RAG are French-language textbooks that were imported then converted to Documents (object type) before embedding them using Google’s models/text-embedding-004. Once the embeddings were created, they were stored in a Chroma vectorstore database.
defcreate_embeddings(text:str)-> List[float]:"""Create embeddings for a given text.""" response = client.models.embed_content(model=EMBEDDING_MODEL,contents=text,config=types.EmbedContentConfig(task_type="semantic_similarity"),)return response.embeddings
Build Agents & Functions: The chatbot consists of LangGraph nodes, functions, conditional edges and edges as seen in this mermaid graph.
Search grounding: Google search is used to return augmented results such as common usage examples and conjugations. The wrapper used is ChatGoogleGenerativeAI. It’s part of the langchain_google_genai package. It was configured with temperature and top_p.
LangGraph Nodes: This was the most challenging for me as I had to first learn LangGraph beyond the codelabs to understand it. Once I understood, I was able to create and sequence the nodes and edges to get the model behavior and output that I wanted.
# Create and display the graphworkflow.add_node("chatbot", chatbot)workflow.add_node("human", human_node)workflow.add_node("tools", tools_node)workflow.add_node("language_details", language_details_node)# Chatbot may go to tools, or human.workflow.add_conditional_edges("chatbot", maybe_route_to_tools)# Human may go back to chatbot, or exit.workflow.add_conditional_edges("human", maybe_exit_human_node)# Define chatbot node edgesworkflow.add_edge("language_details","chatbot")# Set graph start and compileworkflow.add_edge(START,"chatbot")workflow_with_translation_tools = workflow.compile()
Reflection and Learning
Prior to participating in the Google Gen AI 5-Day Intensive, I was unfamiliar with LangGraph, RAG, and agents. Now, I understand the flexibility and promise of LangGraph, RAG and agentic LLMs to allow models to be more flexible and relevant.
Some of what I’ve had to learn are following:
The types Gemini models I can use for embedding and the chatbot. In this case, I used the text-embedding-004 embedding model without experiencing any issues.
How to configure chatbot and embedding clients. Both clients were easy to configure and were set once and done.
How Google AI can be used in a LangGraph graph. I used two major packages, GoogleGenerativeAIEmbeddings and ChatGoogleGenerativeAI. Using either package was easy because the API and code documentaion was very good.
The difference between conditional edges and edges and how to create them. This was a bit tricky to decide what should be a tool versus a function for a conditional edge, versus a node. I also had to consider which logic should be place where and how the output should be handled. It took several graph iterations to get almost the way I like it. For the purposes of the capstone, it works as I intended.
How to import and extract text from PDFs. Importing and extracting text from PDF files was easy, as long as the PDF contained actual text and not images. To keep the scope of the project within my timeframe, I only wanted to work with PDFs that contained text.
How to embed documents for RAG contexts. Document embedding took several minutes for three decent-sized PDFs. It was the most time-consuming part of testing. It’s easy to see where as the set of documents I embed grows, more time and resources will be required.
Creating a Chroma vectorstore database. Creating and retrieving the vectorstore contents was fairly straightforward. It worked consistently locally and in the Kaggle Notebook.
Creating, updating and accessing a graph’s State object. I would have liked to have more time to master the state object but I was able to understand enough to make it work for my project. I would have liked to customize the state more but didn’t have the time to do so. I did find it to be a convenient resource to access messages no matter where it was invoked in the graph.
Create multiple tools and nodes to be used in the graph. I knew what tools I wanted to include based on the resources and tools I currently use when studying French. The goal was to consolidate the information I receive from multiple sources into a single one. But, there are other tools that would enrich my learning. For example, the ability to get rich text and image responses.
Using a prompt template to create and use basic prompts. I didn’t get as much time to investigate creating and using PromptTemplates. I know they could be useful for managing and transforming user input and I will be exploring them further beyond this project.
In addition, since I wrote the code locally in Zed, it took some time and effort to transform the code for a Kaggle notebook. One of the issues I had was that the dependencies initially installed in Kaggle’s environment caused import and dependency resolution errors. It took some time to make sure that the dependencies that worked locally were the same on Kaggle. In hindsight, I’m glad I developed it locally versus starting in a notebook
The Bigger Picture
Thinking beyond this capstone project, language learners need a wise companion to provide information and guidance. The companion would be a language note-taking app with an AI companion that would make suggestions about language rules, common usages, examples and comparisons. While taking notes, or when asked a question, an AI companion would be displayed in a sidebar with context rich details, and unique information and historical facts. This way, as they learn, they get to consider and contrast what they’ve learned. This would create a richer learning environment to motivate and encourage fluency. It can include images too.
This project is about taking the first steps to creating such a companion that will use the keywords, phrases and sentences I input to give me more context and nuance about it’s meaning, common usage patterns and interesting cultural and historical facts. It’s about going beyond the basics to to become immersed in French, especially if you don’t get to live in or visit a francophone country.
Ultimately, it can be developed into an AI-enabled, context-aware application. The AI integrated into the app would be a wise language advisor to help me learn the language. This would be similar to what happens currently with AI-enabled coding apps such as Zed, but more engaging. The AI assistant would remain aware of what I’m typing and offer context-rich suggestions. It would also allow the user to provide instruction at the start of the session for what their learning target is. Once the AI has this, it can then provide a richer learning experience.
This chatbot is the first step in realizing this vision. I can now input anything in french get a variety of details about it, including making additional requests for grammar, gender, and popular culture details.
The next steps for my French language assistant is to continue refine and update the graph, then create a user-friendly interface, before considering an app or website.
Je n’ai rien appris beaucoup de nouveaux mots aujourd’hui, mais j’ai appris quelque choses de grammaire. Je sais maintenant que l’ordre de plusieurs adjectifs avant le nom est très complexe. Il n’y pas des règles précises quand il y a plusieurs adjectifs. J’ai aussi appris comment utiliser un adverbe dans le passé composé.
Quand j’étais petite, j’ai écouté mes cousins parlent en francais. Depuis, j’ai adoré le langue est j’ai pris français à l’université. j’ai regardé aussi French In Action, une serie de PBS. J’ai visité une fois en 1999 à Paris.
J’adore la langue et la nourriture, et je veux visiter encore est habiter là. Avant, je veux apprendre français si je peux le parler courament. Je voudrai écrire et parle le français plus mieux qu’aujourd’hui.
Si maintenant, j’étudie le français tous les jours. J’utilise Duolingo et mes notes à l’apprendre. Je le recherche aussi les mots et grammaire avec sites web comme Lawless French, Reverso et French Language Stack Exchange.
Ici, je veux à partager mes notes de la langue parce qu’ils contiennent mes questions sur les leçons. Depuis, je peux les chercher quand je veux.
Codelab 2/2 from day one. The code lab is here in Kaggle.
"""
Prompting
Google Gen AI 5-Day Intensive Course
Host: Kaggle
Day: 1
Codelab: https://www.kaggle.com/code/reniseblack/day-1-evaluation-and-structured-output/edit
"""
import enum
import io
import os
from pprint import pprint
import typing_extensions as typing
from google import genai
from google.api_core import retry
from google.genai import types
from IPython.display import Markdown, clear_output, display
is_retriable = lambda e: ( # noqa: E731
isinstance(e, genai.errors.APIError) and e.code in {429, 503})
genai.models.Models.generate_content = retry.Retry(
predicate=is_retriable)(genai.models.Models.generate_content)
client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"])
# Create and send a single-turn prompt
MODEL = "gemini-2.0-flash"
prompt = "Explain AI to me like I'm a kid."
response = client.models.generate_content(
model=MODEL,
contents=prompt
)
# Get the response text then render as HTML (optional)
response_text = response.text
print("Model response: \n\n", Markdown(response.text))
section_break = "----------------------------"
print(section_break, "\n\nStarting a multi-turn chat ...\n\n")
# Start a multi-turn chat
chat_message = "Hello! My name is Zlork."
chat = client.chats.create(model=MODEL, history=[])
response = chat.send_message(chat_message)
print(response.text)
chat_message2 = "Can you tell me something interesting about dinosaurs?"
response = chat.send_message(chat_message2)
print(response.text)
chat_message3 = "Do you remember my name?"
response = chat.send_message(chat_message3)
print(response.text)
# See the list of available models
print("\n\nGetting a list of available models ... \n\n")
for model in client.models.list():
print(model.name)
print("\n\nGetting a list of gemini-2.0-flash models ... \n\n")
for model in client.models.list():
if model.name == 'models/gemini-2.0-flash':
pprint(model.to_json_dict())
break
# Set token limit to restrict output length
print("\n\nSetting a token limit ... \n\n")
short_config = types.GenerateContentConfig(max_output_tokens=200)
PROMPT_2 = ('Write a 1000 word essay on the importance of olives in modern '
'society.')
response = client.models.generate_content(
model=MODEL,
config=short_config,
contents=PROMPT_2
)
print(response.text)
print("\n\n")
PROMPT_3 = "Write a short poem on the importance of olives in modern society."
# config is where you specify top_k, temperature, etc
response = client.models.generate_content(
model=MODEL,
config=short_config,
contents=PROMPT_3
)
print(response.text)
print("\n\n")
short_config = types.GenerateContentConfig(max_output_tokens=100)
response = client.models.generate_content(
model=MODEL,
config=short_config,
contents='Write a 150 opinion about the importance of pine trees.')
print(response.text)
print("\n\n")
print("\n\nSetting a temperature limit ... \n\n")
# Set a temperature that helps manage the randomness
high_temp_config = types.GenerateContentConfig(temperature=2.0)
PROMPT_4 = "Pick a random colour... (respond in a single word)"
for _ in range (5):
response = client.models.generate_content(
model=MODEL,
config=high_temp_config,
contents = PROMPT_4
)
if response.text:
print(response.text, "-" * 25)
print("\n\n")
print("\n\nSetting a low temperature ... \n\n")
# Setting a low temperature
low_temp_config = types.GenerateContentConfig(temperature=0.0)
for _ in range(5):
response = client.models.generate_content(
model=MODEL,
config=low_temp_config,
contents = PROMPT_4
)
if response.text:
print(response.text, "-" * 25)
print("\n\n")
print("\n\nSetting top-p ... \n\n")
# Setting top-p that also controls output diversity
# It will determine when to stop selecting tokens that are least probable
TEMPERATURE_2 = 1.0
TOP_P1 = 0.05
model_config = types.GenerateContentConfig(
# These are default gemini-2.0-flash values
temperature=TEMPERATURE_2,
top_p=TOP_P1
)
STORY_PROMPT = ("You are a creative writer. Write a short story about a cat"\
"who goes on an adventure.")
response = client.models.generate_content(
model=MODEL,
config=model_config,
contents=STORY_PROMPT
)
print(response.text)
print("\n\n")
print("\n\nSetting top-p ... \n\n")
# Create a zero-shot prompt
TEMPERATURE_2 = 0.1
TOP_P2 = 1
MAX_OUTPUT_TOKENS = 5
model_config = types.GenerateContentConfig(
temperature=TEMPERATURE_2,
top_p=TOP_P2,
max_output_tokens=MAX_OUTPUT_TOKENS,
)
ZERO_SHOT_PROMPT = """Classify movie reviews as POSITIVE, NEUTRAL or NEGATIVE.
Review: "Her" is a disturbing study revealing the direction
humanity is headed if AI is allowed to keep evolving,
unchecked. I wish there were more movies like this masterpiece.
Sentiment: """
response = client.models.generate_content(
model=MODEL,
config=model_config,
contents=ZERO_SHOT_PROMPT)
print(response.text)
print("\n\n")
print("\n\nSetting top-p ... \n\n")
# Setting enum mode
class Sentiment(enum.Enum):
POSITIVE = "right on"
NEUTRAL = "meh"
NEGATIVE = "sucks lemons"
ENUM_TYPE = "text/x.enum"
response = client.models.generate_content(
model=MODEL,
config=types.GenerateContentConfig(
response_mime_type=ENUM_TYPE,
response_schema=Sentiment
),
contents=ZERO_SHOT_PROMPT)
print(response.text)
print("\n\n")
print("\n\nSetting top-p ... \n\n")
# Get the Sentiment returned as a Python object
enum_response = response.parsed
print(enum_response)
print(type(enum_response))
print("\n\n")
print("\n\nSetting top-p ... \n\n")
# Send one-shot and few-shot prompts
FEW_SHOT_PROMPT = """Parse a customer's pizza order into valid JSON:
EXAMPLE:
I want a small pizza with cheese, tomato sauce, and pepperoni.
JSON Response:
```
{
"size": "small",
"type": "normal",
"ingredients": ["cheese", "tomato sauce", "pepperoni"]
}
```
EXAMPLE:
Can I get a large pizza with tomato sauce, basil and mozzarella
JSON Response:
```
{
"size": "large",
"type": "normal",
"ingredients": ["tomato sauce", "basil", "mozzarella"]
}
```
ORDER:
"""
CUSTOMER_ORDER = "Give me a large with cheese & pineapple"
MAX_OUTPUT_TOKENS = 250
response = client.models.generate_content(
model='gemini-2.0-flash',
config=types.GenerateContentConfig(
temperature=TEMPERATURE_2,
top_p=TOP_P2,
max_output_tokens=MAX_OUTPUT_TOKENS,
),
contents=[FEW_SHOT_PROMPT, CUSTOMER_ORDER])
print(response.text)
print("\n\n")
print("\n\nSet to JSON response only ... \n\n")
# Set output to JSON format
class PizzaOrder(typing.TypedDict):
size: str
ingredients: list[str]
type: str
ORDER_1 = "Can I have a large dessert pizza with apple and chocolate"
response = client.models.generate_content(
model=MODEL,
config=types.GenerateContentConfig(
temperature=TEMPERATURE_2,
response_mime_type="application/json",
response_schema=PizzaOrder
),
contents=ORDER_1
)
print(response.text)
print("\n\n")
print("\n\nUse Chain of Thought (CoT) ... \n\n")
# Use Chain of Thought (CoT) to show reasoning to check hallucinations
CHAIN_OF_THOUGHT_PROMPT = """When I was 4 years old,
my partner was 3 times my age. Now, I am 20 years old.
How old is my partner? Return the answer directly."""
response = client.models.generate_content(
model=MODEL,
contents=CHAIN_OF_THOUGHT_PROMPT
)
print(response.text)
print("\n\n")
print("\n\nUse Chain of Thought (CoT) with step-by-step ... \n\n")
# Use Chain of Thought (CoT) to show reasoning to check hallucinations
CHAIN_OF_THOUGHT_PROMPT_2 = """When I was 4 years old, my partner was 3 times
my age. Now, I am 20 years old. How old is my partner?
Let's think step by step."""
response = client.models.generate_content(
model=MODEL,
contents=CHAIN_OF_THOUGHT_PROMPT_2
)
Markdown(response.text)
print("\n\n")
print("\n\nReAct prompt examples ... \n\n")
# Examples of ReAct prompts
model_instructions = """
Solve a question answering task with interleaving Thought, Action, Observation
steps. Thought can reason about the current situation,
Observation is understanding relevant information from an Action's output and
Action can be one of three types:
(1) <search>entity</search>, which searches the exact entity on Wikipedia and
returns the first paragraph if it exists. If not, it will return some
similar entities to search and you can try to search the information from
those topics.
(2) <lookup>keyword</lookup>, which returns the next sentence containing
keyword in the current context. This only does exact matches, so keep your
searches short.
(3) <finish>answer</finish>, which returns the answer and finishes the task.
"""
example1 = """Question
Musician and satirist Allie Goertz wrote a song about the "The Simpsons" ch\
aracter Milhouse, who Matt Groening named after who?
Thought 1
The question simplifies to "The Simpsons" character Milhouse is named after \
who. I only need to search Milhouse and find who it is named after.
Action 1
<search>Milhouse</search>
Observation 1
Milhouse Mussolini Van Houten is a recurring character in the Fox animated \
television series The Simpsons voiced by Pamela Hayden and
created by Matt Groening.
Thought 2
The paragraph does not tell who Milhouse is named after, maybe I can look u\
p "named after".
Action 2
<lookup>named after</lookup>
Observation 2
Milhouse was named after U.S. president Richard Nixon, whose middle name wa\
s Milhous.
Thought 3
Milhouse was named after U.S. president Richard Nixon, so the answer is Ric\
hard Nixon.
Action 3
<finish>Richard Nixon</finish>
"""
example2 = """Question
What is the elevation range for the area that the eastern sector of the
Colorado orogeny extends into?
Thought 1
I need to search Colorado orogeny, find the area that the eastern sector
of the Colorado orogeny extends into, then find the elevation range
of the area.
Action 1
<search>Colorado orogeny</search>
Observation 1
The Colorado orogeny was an episode of mountain building (an orogeny) in Colora
do and surrounding areas.
Thought 2
It does not mention the eastern sector. So I need to look up eastern sector.
Action 2
<lookup>eastern sector</lookup>
Observation 2
The eastern sector extends into the High Plains and is called the Central Plain
s orogeny.
Thought 3
The eastern sector of Colorado orogeny extends into the High Plains. So I need
to search High Plains and find its elevation range.
Action 3
<search>High Plains</search>
Observation 3
High Plains refers to one of two distinct land regions
Thought 4
I need to instead search High Plains (United States).
Action 4
<search>High Plains (United States)</search>
Observation 4
The High Plains are a subregion of the Great Plains. From east to west, the
High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130m).
Thought 5
High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1
,800 to 7,000 ft.
Action 5
<finish>1,800 to 7,000 ft</finish>
"""
question = """Question
Who was the youngest author listed on the transformers NLP paper?
"""
# You will perform the Action; so generate up to, but not including,
# the Observation.
react_config = types.GenerateContentConfig(
stop_sequences=["\nObservation"],
system_instruction=model_instructions + example1 + example2,
)
# Create a chat that has the model instructions and examples pre-seeded.
react_chat = client.chats.create(
model=MODEL,
config=react_config,
)
response = react_chat.send_message(question)
print(response.text)
print("\n\n")
print("\n\nAdding an observation with author names ... \n\n")
# The model won't find the authors, so supply them with a followup
observation = """Observation 1
[1706.03762] Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan
N. Gomez, Lukasz Kaiser, Illia Polosukhin
We propose a new simple network architecture, the Transformer, based solely on
attention mechanisms, dispensing with recurrence and convolutions entirely.
"""
response = react_chat.send_message(observation)
print(response.text)
print("\n\n")
print("\n\nUse Gemini 2.0 Flash Thinking Mode ... \n\n")
# Examples of ReAct prompts
THINKING_MODE_MODEL = "gemini-2.0-flash-thinking-exp"
THINKING_MODE_PROMPT = ("Who was the youngest author listed on the "
"transformers NLP paper?")
response = client.models.generate_content_stream(
model=THINKING_MODE_MODEL,
contents=THINKING_MODE_PROMPT
)
buffer = io.StringIO()
for chunk in response:
buffer.write(str(chunk.text))
# Display the response as it is streamed
print(chunk.text, end='')
# Render the response as formatted markdown
clear_output()
Markdown(buffer.getvalue())
print("\n\n")
print("\n\nUse Gemini 2.0 to generate code ... \n\n")
# Generate code wth Gemini
# The Gemini models love to talk, so it helps to specify they stick to the code
# if that is all that you want.
CODE_PROMPT = """Write a Python function to calculate the factorial of a number.
No explanation, provide only the code."""
response = client.models.generate_content(
model=MODEL,
config=types.GenerateContentConfig(
temperature=1,
top_p=1,
max_output_tokens=1024,
),
contents=CODE_PROMPT)
Markdown(response.text)
print("\n\n")
print("\n\nUse Gemini 2.0 to execute code ... \n\n")
# Execute code wth Gemini
config = types.GenerateContentConfig(
tools=[types.Tool(code_execution=types.ToolCodeExecution())],
)
code_exec_prompt = """
Generate the first 14 odd prime numbers, then calculate their sum.
"""
response = client.models.generate_content(
model=MODEL,
config=config,
contents=code_exec_prompt)
for part in response.candidates[0].content.parts:
pprint(part.to_json_dict())
print("-----")
print("\n\n")
print("\n\nSee more parts of the response ... \n\n")
# See executable_code and code_execution_result
for part in response.candidates[0].content.parts:
if part.text:
display(Markdown(part.text))
elif part.executable_code:
display(Markdown(f'```python\n{part.executable_code.code}\n```'))
elif part.code_execution_result:
if part.code_execution_result.outcome != 'OUTCOME_OK':
display(Markdown(f'## Status {part.code_execution_result.outcome}'))
display(Markdown(f'```\n{part.code_execution_result.output}\n```'))
print("\n\n")
print("\n\nExplaining code ... \n\n")
# Get an explanation for code
# file_contents = !curl https://raw.githubusercontent.com/magicmonty/bash-git-prompt/refs/heads/master/gitprompt.sh
explain_prompt = f"""
Please explain what this file does at a very high level. What is it,
and why would I use it?
```
{file_contents}
```
"""
response = client.models.generate_content(
model='gemini-2.0-flash',
contents=explain_prompt)
Markdown(response.text)
Codelab 1/2 from day one. The code lab is here in Kaggle.
"""
Evaluation and Structured Output
Google Gen AI 5-Day Intensive Course
Host: Kaggle
Day: 1
Kaggle: https://www.kaggle.com/code/markishere/day-1-evaluation-and-structured-output
"""
import enum
import os
from google import genai
from google.api_core import retry
from google.genai import types
from IPython.display import Markdown, display
client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"])
# Automated retry
is_retriable = lambda e: (
isinstance(e, genai.errors.APIError) and e.code in {429, 503}
)
genai.models.Models.generate_content = retry.Retry(predicate=is_retriable)(
genai.models.Models.generate_content
)
# if not hasattr(genai.models.Models.generate_content, '__wrapped__'):
# genai.models.Models.generate_content = retry.Retry(
# predicate=is_retriable)(genai.models.Models.generate_content)
# Evaluation
# Understand model performance
# Get the file locally first
# !wget -nv -O gemini.pdf https://storage.googleapis.com/cloud-samples-data/generative-ai/pdf/2403.05530.pdf
document_file = client.files.upload(file="/Users/renise/Documents/Python/gen_ai/day_one/gemini.pdf")
print("\n")
print(document_file)
print("\n")
print("\nSummarize a document\n")
# Summarize a document
def summarize_doc(request: str) -> str:
"""Execute the request on the uploaded document."""
# Set the temperature low to stabilize the output.
config = types.GenerateContentConfig(temperature=0.0)
response = client.models.generate_content(
model="gemini-2.0-flash",
config=config,
contents=[request, document_file],
)
return response.text
request = "Tell me about the training process used here."
summary = summarize_doc(request)
# display(Markdown(summary + "\n-----"))
print("\n\n")
# Define an evaluator
SUMMARY_PROMPT = """\
# Instruction
You are an expert evaluator. Your task is to evaluate the quality of the responses generated by AI models.
We will provide you with the user input and an AI-generated responses.
You should first read the user input carefully for analyzing the task, and then evaluate the quality of the responses based on the Criteria provided in the Evaluation section below.
You will assign the response a rating following the Rating Rubric and Evaluation Steps. Give step-by-step explanations for your rating, and only choose ratings from the Rating Rubric.
# Evaluation
## Metric Definition
You will be assessing summarization quality, which measures the overall ability to summarize text. Pay special attention to length constraints, such as in X words or in Y sentences. The instruction for performing a summarization task and the context to be summarized are provided in the user prompt. The response should be shorter than the text in the context. The response should not contain information that is not present in the context.
## Criteria
Instruction following: The response demonstrates a clear understanding of the summarization task instructions, satisfying all of the instruction's requirements.
Groundedness: The response contains information included only in the context. The response does not reference any outside information.
Conciseness: The response summarizes the relevant details in the original text without a significant loss in key information without being too verbose or terse.
Fluency: The response is well-organized and easy to read.
## Rating Rubric
5: (Very good). The summary follows instructions, is grounded, is concise, and fluent.
4: (Good). The summary follows instructions, is grounded, concise, and fluent.
3: (Ok). The summary mostly follows instructions, is grounded, but is not very concise and is not fluent.
2: (Bad). The summary is grounded, but does not follow the instructions.
1: (Very bad). The summary is not grounded.
## Evaluation Steps
STEP 1: Assess the response in aspects of instruction following, groundedness, conciseness, and verbosity according to the criteria.
STEP 2: Score based on the rubric.
# User Inputs and AI-generated Response
## User Inputs
### Prompt
{prompt}
## AI-generated Response
{response}
"""
# Define a structured enum class to capture the result.
class SummaryRating(enum.Enum):
VERY_GOOD = 5
GOOD = 4
OK = 3
BAD = 2
VERY_BAD = 1
def eval_summary(prompt, ai_response):
"""Evaluate the generated summary against the prompt."""
chat = client.chats.create(model="gemini-2.0-flash")
# Generate the full text response
response = chat.send_message(
message=SUMMARY_PROMPT.format(prompt=prompt, response=ai_response)
)
verbose_eval = response.text
# Coerce into desired structure
structured_output_config = types.GenerateContentConfig(
response_mime_type="text/x.enum",
response_schema=SummaryRating
)
response = chat.send_message(
message="Convert the final score.",
config=structured_output_config
)
structured_eval = response.parsed
return verbose_eval, structured_eval
text_eval, struct_eval = eval_summary(
prompt=[request, document_file],
ai_response=summary
)
Markdown(text_eval)
# Play with the summary prompt
new_prompt = "Explain like I'm 5 the training process"
# Try:
# ELI5 the training process
# Summarise the needle/haystack evaluation technique in 1 line
# Describe the model architecture to someone with a civil engineering degree
# What is the best LLM?
if not new_prompt:
raise ValueError("Try setting a new summarization prompt.")
def run_and_eval_summary(prompt):
"""Generate and evaluate the summary using the new prompt."""
summary = summarize_doc(new_prompt)
display(Markdown(summary + "\n-----"))
text, struct = eval_summary([new_prompt, document_file], summary)
display(Markdown(text + "\n-----"))
print(struct)
run_and_eval_summary(new_prompt)
Les leçons de français était un peu difficile aujourd’hui parce que je n’ai concentré pas bien. Encore, j’ai fait asses bien. J’ai appris de passé composé avec les verbes voir, lire, terminé, et emprunté. J’ai appris aussi comment il faut dire “il y a ? ans” quand je parle du passé. Par exemple, il faut dire “Je l’ai vu il y a deux ans” pour quelque chose tu as vois deux ans du passé.
French
English
Notes
Il <<en>> mange beaucoup
He eats a lot of <<it>>
goût
taste
Tu l’a lu?
Tu veux <<encore voir>> ce flim?
Je vais <<emprunter>> ce livre à la bibliothèque
I am going to <<borrow>> this book at the library
Il veut être <<auteur>>
He wants to be <<an author>>
auteur (masc);
Ils l’ont <<vraiment adoré>>
They <<really loved>> it.
Notice “vraiment” is before the past tense of the action verb
Il a vraiment aimé <<le dernier livre de cet auteur>>
He really liked <<this author’s last book>>
Possessive nouns always come after what it’s possessing
J’ai <<déjà lu>> ce livre. Je ne veux pas encore le lire.
I <<already read>> this book. I don’t want to read it again.
Il l’a déjà lu
He has already seen it
Je <<n’ai pas terminé>> ce livre
I <<didn’t finished>> this book
Je <<l’ai emprunté à>> Marc le mois dernier
I <<borrowed it from>> Marc last month
To borrow from is expressed with “à” instead of “de”
J’<<ai presque terminé>> ce livre
I <<almost finished>> this book
J’<<ai presque tout lu>>.
I’ve <<almost read everything>>
Tu as vu <<la fin du film>>?
Have you seen <<the end of the film >>?
J’ai trente ans. <<Il y a dix ans>>, j’avais vingt ans
I am thirty years old. <<Ten years ago>>, I was 20 years old.
Je vais <<commander>> ce livre <<sur Internet>> aujourd’hui.
I am going to <<order>> this book <<online>> today.
Aujourd’hui, j’ai apris beaucoup des mots et grammaires. Je comprends comment à dire “il y a du vent dehors”, “une araignée”, “le toit”, “à l’étage”, “la pastèque”, y autres. J’ai passé du temps rechercher comment utiliser les prépositions “à”, “par” et “dans”. Je le sais plus mieux qu’avant.
French
English
Notes
Elle est <<une femme de ménage>>
She is a <<cleaning lady>>
“Ménager” means “to clean”.
Il y a <<une araignée>> <<au plafond>>
There’s <<a spider>> <<on the ceiling>>
How to say both “spider” and “on the ceiling”. Notice “on the” is “au/à la” in french.
le toit
the roof (of a building)
Ils sont venue <<à vélo>>?
They came <<by bike>>.
The preposition “à” implies “riding on” something
Il est venu hier
He came yesterday.
L’oiseau est entré <<par la fenêtre>>
The bird entered <<through the window>>
Use “par” when going through something or indicating direction
Elles sont entrées <<dans le musée>>.
They entered <<into the museum>>
<<Hier soir>>, il est parti à neuf heures.
<<Last night>>, she left at nine o’clock
Il <<est descendu>> maintenant.
He <<came>> down now
Descendre can also mean to go down.
Ils <<sont montés>> <<à l’étage>>.
They <<went>> <<upstairs>>.
How to say “upstairs”
<<L’homme de ménage>> est venu nettoyer la maison
<<The cleaning man>> came to clean the house
Il ne faut pas monter <<sur le toit>>.
One must not climb up <<on the roof>>
“Monter” means to “mount”, “climb”, “ride”, “go up”, “rise” depending on the context
Les araignées sont montées <<sur le mur>>
The spiders climbed up <<(on) the wall>>
To make it explicit that an insect climbs “on” the wall versus “up” the wall.