In this tutorial, we demonstrate how to combine the power of SerpAPI’s Google search capabilities with Google’s Gemini-1.5-Flash model to create an advanced, end-to-end research and analysis workflow within a Google Colab notebook. By defining an AdvancedSerpAPI Python class, users gain access to enhanced search methods that cover general web results, news articles, and images, while also leveraging Gemini to perform in-depth analyses of those results. The code provides specialized utilities for targeting Marktechpost tutorials, aggregating content across categories like LangChain, ChatGPT, and MLOps, and then synthesizing actionable insights using a carefully constructed prompt.
!pip install google-search-results langchain-community langchain-core google-generativeai -q
import os
import json
from serpapi import GoogleSearch
import google.generativeai as genai
from datetime import datetimeWe install the required Python packages for SerpAPI searches, LangChain utilities, and Google’s Gemini SDK. The subsequent imports bring in standard modules (os, json, datetime) for environment configuration, JSON handling, and timestamps, as well as SerpAPI’s GoogleSearch class for making API calls and genai for interacting with the Gemini model.
SERPAPI_API_KEY = "Use Your API Key Here"
GEMINI_API_KEY = "Use Your API Key Here"
os.environ["SERPAPI_API_KEY"] = SERPAPI_API_KEY
genai.configure(api_key=GEMINI_API_KEY)
We assign placeholder strings for your SerpAPI and Gemini API keys, then set the SerpAPI key as an environment variable (so SerpAPI calls authenticate automatically) and configure the Gemini client with its API key so you can invoke the Gemini model.
class AdvancedSerpAPI:
def __init__(self, serpapi_key, gemini_key):
self.serpapi_key = serpapi_key
self.gemini_model = genai.GenerativeModel('gemini-1.5-flash')
def search_google(self, query, num_results=5, location="United States"):
"""Enhanced Google search with multiple parameters"""
params = {
"engine": "google",
"q": query,
"api_key": self.serpapi_key,
"num": num_results,
"location": location,
"hl": "en",
"gl": "us"
}
search = GoogleSearch(params)
results = search.get_dict()
return self.extract_search_results(results)
def search_news(self, query, days_back=7):
"""Search for recent news articles"""
params = {
"engine": "google_news",
"q": query,
"api_key": self.serpapi_key,
"gl": "us",
"hl": "en"
}
search = GoogleSearch(params)
results = search.get_dict()
return self.extract_news_results(results)
def search_images(self, query, num_images=10):
"""Search for images with metadata"""
params = {
"engine": "google_images",
"q": query,
"api_key": self.serpapi_key,
"num": num_images
}
search = GoogleSearch(params)
results = search.get_dict()
return self.extract_image_results(results)
def extract_search_results(self, results):
"""Extract and clean search results"""
cleaned_results = []
if 'organic_results' in results:
for result in results['organic_results']:
cleaned_results.append({
'title': result.get('title', ''),
'link': result.get('link', ''),
'snippet': result.get('snippet', ''),
'position': result.get('position', 0)
})
return cleaned_results
def extract_news_results(self, results):
"""Extract news articles with timestamps"""
news_results = []
if 'news_results' in results:
for article in results['news_results']:
news_results.append({
'title': article.get('title', ''),
'link': article.get('link', ''),
'snippet': article.get('snippet', ''),
'date': article.get('date', ''),
'source': article.get('source', '')
})
return news_results
def extract_image_results(self, results):
"""Extract image results with metadata"""
image_results = []
if 'images_results' in results:
for img in results['images_results']:
image_results.append({
'title': img.get('title', ''),
'original': img.get('original', ''),
'thumbnail': img.get('thumbnail', ''),
'source': img.get('source', '')
})
return image_results
def analyze_with_gemini(self, search_results, analysis_prompt):
"""Use Gemini Flash to analyze search results"""
results_text = json.dumps(search_results, indent=2)
full_prompt = f"""
{analysis_prompt}
Search Results Data:
{results_text}
Please provide a comprehensive analysis based on the search results.
"""
try:
response = self.gemini_model.generate_content(full_prompt)
return response.text
except Exception as e:
return f"Gemini analysis failed: {str(e)}"
def search_marktechpost_tutorials(self, topic="", num_results=10):
"""Search specifically for trending tutorials from Marktechpost"""
queries = [
f"site:marktechpost.com {topic} tutorial guide how-to 2024 2025",
f"site:marktechpost.com trending {topic} tutorial",
f"site:marktechpost.com top {topic} books frameworks"
]
all_results = []
for query in queries:
params = {
"engine": "google",
"q": query,
"api_key": self.serpapi_key,
"num": num_results // len(queries),
"hl": "en",
"gl": "us"
}
search = GoogleSearch(params)
results = search.get_dict()
extracted = self.extract_search_results(results)
all_results.extend(extracted)
unique_results = []
seen_links = set()
for result in all_results:
if result['link'] not in seen_links:
unique_results.append(result)
seen_links.add(result['link'])
return unique_results[:num_results]
def get_trending_marktechpost_content(self, categories=None):
"""Get trending content from Marktechpost across different categories"""
if categories is None:
categories = ["AI", "LLM", "Machine Learning", "Python", "Tutorial", "Framework"]
trending_content = {}
for category in categories:
print(f"🔍 Searching for trending {category} content...")
results = self.search_marktechpost_tutorials(category, num_results=5)
trending_contentAgentic AI = results
print(f"✅ Found {len(results)} {category} tutorials")
return trending_content
def smart_research(self, topic, research_depth="medium", focus_marktechpost=True):
"""Intelligent research combining multiple search types with Marktechpost focus"""
print(f"🔍 Starting smart research on: {topic}")
if focus_marktechpost:
marktechpost_results = self.search_marktechpost_tutorials(topic, num_results=8)
print(f"✅ Found {len(marktechpost_results)} Marktechpost tutorials")
web_results = self.search_google(f"{topic} tutorial guide", num_results=3)
print(f"✅ Found {len(web_results)} additional web results")
all_web_results = marktechpost_results + web_results
else:
all_web_results = self.search_google(f"{topic} overview facts", num_results=5)
print(f"✅ Found {len(all_web_results)} web results")
news_results = self.search_news(topic)
print(f"✅ Found {len(news_results)} news articles")
analysis_prompt = f"""
Analyze the search results about '{topic}' with focus on Marktechpost content and provide:
1. Key tutorials and guides available
2. Trending topics and frameworks
3. Learning resources and books mentioned
4. Recent developments and updates
5. Practical implementation guides
6. Recommended learning path
Focus on actionable insights and learning resources.
"""
all_results = {
"marktechpost_results": marktechpost_results if focus_marktechpost else [],
"web_results": all_web_results,
"news_results": news_results,
"search_topic": topic,
"timestamp": datetime.now().isoformat()
}
gemini_analysis = self.analyze_with_gemini(all_results, analysis_prompt)
return {
"topic": topic,
"marktechpost_tutorials": marktechpost_results if focus_marktechpost else [],
"web_results": all_web_results,
"news_results": news_results,
"ai_analysis": gemini_analysis,
"total_sources": len(all_web_results) + len(news_results)
}
This class, AdvancedSerpAPI, encapsulates SerpAPI-based search methods (web, news, and images) and helper functions to clean the resulting JSON data. It also integrates a Gemini-1.5-Flash model, via analyze_with_gemini, to generate an AI-driven summary of any collected search data. Additional utilities include specialized Marktechpost tutorial lookups, a “get trending” routine across categories, and a combined “smart research” workflow that stitches together tutorials, web results, news, and Gemini analysis.
def demo_marktechpost_tutorials():
"""Demo specifically focused on Marktechpost tutorials"""
searcher = AdvancedSerpAPI(SERPAPI_API_KEY, GEMINI_API_KEY)
print("🚀 Marktechpost Trending Tutorials Finder")
print("=" * 50)
print("\n📚 Demo 1: Trending Marktechpost Tutorials by Category")
trending_content = searcher.get_trending_marktechpost_content([
"LangChain", "ChatGPT", "Python", "AI", "MLOps"
])
for category, tutorials in trending_content.items():
print(f"\n🔥 Trending {category} Tutorials:")
for i, tutorial in enumerate(tutorials[:3], 1):
print(f" {i}. {tutorial['title']}")
print(f" 📎 {tutorial['link']}")
if tutorial['snippet']:
print(f" 📝 {tutorial['snippet'][:100]}...")
print("\n🎯 Demo 2: Deep Dive - LangChain Tutorials")
langchain_research = searcher.smart_research("LangChain", focus_marktechpost=True)
print(f"\n📊 Research Summary:")
print(f"Topic: {langchain_research['topic']}")
print(f"Marktechpost Tutorials Found: {len(langchain_research['marktechpost_tutorials'])}")
print(f"Total Sources: {langchain_research['total_sources']}")
print(f"\n🤖 AI Analysis Preview:")
print(langchain_research['ai_analysis'][:600] + "..." if len(langchain_research['ai_analysis']) > 600 else langchain_research['ai_analysis'])
print("\n📰 Demo 3: Latest AI Trends from Marktechpost")
ai_trends = searcher.search_marktechpost_tutorials("AI trends 2024 2025", num_results=5)
print("Recent AI trend articles:")
for i, article in enumerate(ai_trends, 1):
print(f"{i}. {article['title']}")
print(f" 🔗 {article['link']}")
def demo_advanced_serpapi():
"""Comprehensive demo of SerpAPI capabilities"""
searcher = AdvancedSerpAPI(SERPAPI_API_KEY, GEMINI_API_KEY)
print("🚀 Advanced SerpAPI Tutorial with Gemini Flash")
print("=" * 50)
print("\n📊 Demo 1: Smart Research on AI Technology")
research_results = searcher.smart_research("artificial intelligence 2024 trends")
print(f"\n🔍 Research Summary:")
print(f"Topic: {research_results['topic']}")
print(f"Total Sources: {research_results['total_sources']}")
print(f"\n🤖 AI Analysis Preview:")
print(research_results['ai_analysis'][:500] + "..." if len(research_results['ai_analysis']) > 500 else research_results['ai_analysis'])
print("\n📰 Demo 2: Recent News Search")
tech_news = searcher.search_news("technology breakthrough", days_back=7)
print(f"Found {len(tech_news)} recent tech news articles:")
for i, article in enumerate(tech_news[:3], 1):
print(f"{i}. {article['title'][:80]}...")
print(f" Source: {article['source']} | Date: {article['date']}")
print("\n🖼️ Demo 3: Image Search")
space_images = searcher.search_images("space exploration 2024", num_images=5)
print(f"Found {len(space_images)} space-related images:")
for i, img in enumerate(space_images[:3], 1):
print(f"{i}. {img['title'][:60]}...")
print(f" Source: {img['source']}")
demo_marktechpost_tutorials() initializes the AdvancedSerpAPI class and prints trending tutorials from Marktechpost for a list of categories (LangChain, ChatGPT, Python, AI, MLOps). It then performs a “deep dive” smart research on “LangChain,” showing counts of tutorials and a preview of Gemini’s AI analysis. Finally, it retrieves and lists the top five recent “AI trends 2024–2025” articles from Marktechpost.
Also, demo_advanced_serpapi() creates an AdvancedSerpAPI instance but focuses on a broader workflow: it runs smart research on “artificial intelligence 2024 trends” and prints the topic summary and AI analysis snippet. It then performs a news search for “technology breakthrough,” lists the first three articles with sources and dates, and concludes by fetching and displaying a handful of “space exploration 2024” image results.
if __name__ == "__main__":
if SERPAPI_API_KEY == "your_serpapi_key_here" or GEMINI_API_KEY == "your_gemini_key_here":
print("⚠️ Please set your API keys before running the demo!")
print("1. Get SerpAPI key from: https://serpapi.com")
print("2. Get Gemini API key from: https://makersuite.google.com")
else:
print("🎯 Running Marktechpost-focused demo...")
demo_marktechpost_tutorials()
print("\n" + "="*50)
print("🌟 Running general demo...")
demo_advanced_serpapi()
def compare_search_engines(query, engines=['google', 'bing', 'duckduckgo']):
"""Compare results across different search engines"""
results = {}
for engine in engines:
params = {
"engine": engine,
"q": query,
"api_key": SERPAPI_API_KEY
}
try:
search = GoogleSearch(params)
results[engine] = search.get_dict()
except Exception as e:
results[engine] = {"error": str(e)}
return results
def trending_searches(location="United States"):
"""Get trending searches"""
params = {
"engine": "google_trends_trending_now",
"api_key": SERPAPI_API_KEY,
"geo": location
}
search = GoogleSearch(params)
return search.get_dict()
print("✅ Advanced SerpAPI Tutorial with Marktechpost Focus loaded successfully!")
print("🔑 Remember to set your API keys before running demos")
print("📚 New Functions: search_marktechpost_tutorials, get_trending_marktechpost_content")
print("🎯 Marktechpost-specific features: LangChain, ChatGPT, Python, AI, MLOps tutorials")
print("\n🚀 Quick Start Examples:")
print("searcher = AdvancedSerpAPI(SERPAPI_API_KEY, GEMINI_API_KEY)")
print("langchain_tutorials = searcher.search_marktechpost_tutorials('LangChain')")
print("trending_ai = searcher.get_trending_marktechpost_content(['AI', 'Python'])")
print("research = searcher.smart_research('ChatGPT', focus_marktechpost=True)")Finally, the section includes a Python “main” guard that first verifies your SerpAPI and Gemini keys, prompting you to obtain them if they’re still placeholders, and otherwise runs the Marktechpost‐focused and general demos in sequence. It also defines two utility functions: compare_search_engines, which queries multiple search engines (Google, Bing, DuckDuckGo) via SerpAPI and returns their raw JSON results or errors, and trending_searches, which fetches today’s trending topics using the Google Trends endpoint. After these definitions, the script prints a brief status message confirming that the tutorial loaded successfully, reminds you to set your API keys, and highlights newly added methods for fetching Marktechpost tutorials and trending content.
In conclusion, by following this tutorial, users will have a reusable, modular Python class that streamlines web research and analysis, from performing keyword-driven searches to automatically summarizing findings using Gemini-powered AI. The combination of SerpAPI’s reliable search endpoints and Gemini’s natural language understanding enables a seamless “research-to-insights” workflow, ideal for content creators, developers, and technical teams who need to stay up-to-date with the latest tutorials and industry trends.
Check out the Notebook here. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 95k+ ML SubReddit and Subscribe to our Newsletter.
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.
Leave a comment