Q&A Application on Videos#
1. Connect to EvaDB#
import evadb
cursor = evadb.connect().cursor()
2. Register Functions#
Register speech-to-text whisper model from HuggingFace
cursor.query("""
CREATE UDF SpeechRecognizer
TYPE HuggingFace
'task' 'automatic-speech-recognition'
'model' 'openai/whisper-base';
""").execute()
Note
EvaDB allows users to register any model in HuggingFace as a function.
Register OpenAI LLM model
cursor.query("""
CREATE UDF ChatGPT
IMPL 'evadb/udfs/chatgpt.py'
""").execute()
# Set OpenAI token
import os
os.environ["OPENAI_KEY"] = "sk-..."
Note
ChatGPT function is a wrapper around OpenAI API call. You can also switch to other LLM models that can run locally.
3. Summarize Video in Text#
Create a table with text summary of the video. Text summarization is generated by running speech-to-text Whisper model from HuggingFace.
cursor.query("""
CREATE TABLE text_summary AS
SELECT SpeechRecognizer(audio) FROM ukraine_video;
""").execute()
This results a table shown below.
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text_summary.text |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| The war in Ukraine has been on for 415 days. Who is winning it? Not Russia. Certainly not Ukraine. It is the US oil companies. US oil companies have reached $200 billion in pure profits. The earnings are still on. They are still milking this war and sharing the spoils. Let us look at how Exxon mobile has been doing. In 2022, the company made $56 billion in profits. Oil companies capitalized on instability and they are profiting from pain. American oil companies are masters of this art. You may remember the war in Iraq. The US went to war in Iraq by selling a lie. The Americans did not find any weapons of mass destruction but they did find lots of oil. And in the year since, American officials have admitted this. And this story is not over. It's repeating itself in Ukraine. They are feeding another war and filling the coffers of US oil companies. |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
4. Q&A using ChatGPT#
We can now embed the ChatGPT prompt inside SQL with text summary from the table as its knowledge base.
cursor.query("""
SELECT ChatGPT('Is this video summary related to Ukraine russia war', text)
FROM text_summary;
""").df()
This query returns a projected DataFrame.
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| chatgpt.response |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Based on the provided context, it seems that the video summary is related to the Ukraine-Russia war. It discusses how US oil companies are allegedly profiting from the war in Ukraine, similar to how they allegedly benefited from the war in Iraq. |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+