Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka

Waayo, waxaa laga yaqaan 'Episode ofQalabka LabHakuHaku: Web Scraping Club, waxaan isticmaali karo Firecrawl si ay u qiyaasta this newsletter, a vector DB sida Pinecone si ay u aragto markdown articles ', iyo API OpenAI si ay u aragto info iyo si ay u qiyaasta in our prompt si ay u aragto ugu fiican.

Taageerada waa mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah.

Sida loo yaqaan 'AI landscape' waxaa laga yaabaa in ka mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah.Sida loo yaabaa, waxaa laga yaqaan MCP (Model Context Protocol).Sida loo yaabaa, waxaan ka soo bandhigi karaa in ay ka mid ah MCP waxaa loo isticmaali karaa for web scraping.

Sidee waa mid ka mid ah model kontekst protocol?

EE.UUModel Context Protocol (MCP)Isticmaalka ugu horeysay ee Anthropic, waxaa loo isticmaali karaa in la isticmaali karaa macluumaadka ugu badan ee la isticmaali karaa macluumaadka iyo data ka mid ah macluumaadka iyo data.universal connectorQalabka dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha iyo dhismaha.

Sida loo isticmaali karaa?

MCP waxaa ku saabsan shuruudaha client-server: shuruudaha host (e.g., IDE ama assistant AI) waxay ku xiran MCP.client(Claude, Cursor, iwm.) oo ku xira MCP ka mid ah mid ka mid ah MCPserversSida loo yaabaa, waxaa laga yaabaa in ay ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah.

Macluumaadka MCP waxaa loo isticmaali karaaresources(Lex-only data kontekst analog to GET endpoints) iyotools(waxanka ama shuruudaha model waa in la isticmaali karaa analog to POST endpoints) ee LLM.

MCP-ka ugu horeysay ee 2024 ee loo yaqaan "Anthropic Open-Source MCP" ee loo yaqaan "Anthropic Open-Source MCP" ee loo yaqaan "Anthropic Open-Source MCP" ee loo yaqaan "Anthropic Open-Source MCP" ee loo yaqaan "Anthropic Open-Source MCP" ee loo yaqaan "Anthropic Open-Source MCP".common framework for AI-to-tool communicationSida loo isticmaali karaa, waxaa loo isticmaali karaa mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah.

Sida loo isticmaali karaa, waxaa loo isticmaali karaa mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ka mid ah.

Ma rabtaa MCP?

Sida loo yaabaa in la soo bandhigay RAG ka hor, mid ka mid ah wax soo saarka la soo bandhigay in la soo bandhigay in la soo bandhigay ka mid ah xafiisyada waa la soo bandhigay, sida loo yaabaa in la soo bandhigay in ka mid ah macluumaadka on Pinecone iyo ka dib markii la soo bandhigay in la soo bandhigay OpenAI si ay u soo bandhigay.

Sida loo isticmaali karaa MCP, in ay u isticmaali karaa in IDE Cursor, sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale.

Markaad ka mid ah wax soo saarka ah ee loo isticmaali karaa MCP waa in uu ku yaalaa programming iyo injiilada prompt. Haddaa loo isticmaali karaa tool MCP waa sida loo isticmaali karaa function in our code: ka dib markii loo isticmaali karaa prompt in IDE, waxaa loo isticmaali karaa oo loo isticmaali karaa iyo waxaa loo isticmaali karaa in ay isticmaali karaa in aad isticmaali karaa in aad isticmaali karaa in aad isticmaali karaa in aad isticmaali karaa.

Last but not least, MCP is a standard, iyo in theoretically, ka dib markii aan ka mid ah server oo la sameeyaan, waxaan sidoo kale la socdaan in la mid ah mid ka mid ah model iyo qalabka in ay ku socdaan.

Sida loo isticmaali karaa MCP

Waxaan u isticmaali karaa two key technologies for implementing our solution:MCP Python SDKiyoCamoufox.

MCP Python SDK.The MCP Python SDK waa library rasmi ee loo soo saarka MCP server ( iyo client) ee Python. Waxaa la aasaasay dhismaha MCP dhismaha oo dhan, si ay u isticmaali karaa dhismaha dhismaha dhismaha dhismaha dhismaha dhismaha dhismaha dhismaha dhismaha dhismaha dhismaha dhismaha.

Sida loo isticmaali karaa macluumaadka macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka Macluumaadka MacFastMCPQalabka dhismaha iyo dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha, dhismaha dhismaha, dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha@mcp.tool()Sida loo isticmaali karaa in ay isticmaali karaa in ay isticmaali karaa in ay isticmaali karaa.

Shuruudaha Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka

Camoufox for HTML retrieval.In web scraping, soo saarka HTML ah ka mid ah ka mid ah mid ka mid ah macluumaadka anti-scraping ama JavaScript dhismaha) waa mid ka mid ah warshadaha. Waxaan ka mid ah loo isticmaalo Camoufox, oo waa mid ka mid ah macluumaadka stealth open-source ee loo isticmaalaa in ay u baahan yahay sidaas, si kastaba ha ahaatee si ay u baahan yahay in ay ka mid ah in ay ka heli karaa HTML ka mid ka mid ah page. This is true because the MCP runs locally on my machine, so I won't need any proxy. On top of its stealth capabilities, I wanted to use Camoufox to build a MCP logic from 0. If you want to save time, you can useMacluumaadka MCP ServerMarkaHaku: Hyperbrowser 1Waxaad ka mid ah macluumaadka ugu soo saarka ah sida loo soo saarka data iyo si ay u isticmaali karaa page, si ay u isticmaali karaa in aad u isticmaali karaa.

Shuruudaha Technical ee MCP Server for Writing a Camoufox scraper

hadda, waxaan soo saarka MCP server oo ay ka caawinay in la xitaa a Camoufox scraper in three steps.

• fetch_page_content will be a tool that opens Camoufox and stores the HTML of the target page on a file.

• generate_xpaths will be a tool that reads the HTML file and, given a template of output data, creates the selectors, again saving them to a file. That’s why we want it flexible enough to handle different page types (for example, a product listing page vs. a product detail page in e-commerce).

• write_camoufox_scraper will be a tool that reads the selectors and a template of Camoufox spider (Camoufox_template.py) and creates a new one based on that and the selectors just created.

  
  

Qalabka Qalabka (xpath_server.py) waxay ku salaysan in repository ee folder 79.MCP/MCPFiles.

Bilood 1: Dhammaan oo ku habboon

First, sidoo kale u baahan tahay in aadPython 3.10+Waxaad u baahan tahay MCP SDK iyo Camouf, oo aad u isticmaali karaa via pip.

pip install mcp camoufox 

EE.UUmcpQalabka waxay ku jiraan MCP Python SDK iyo CLI tool.CamoufoxWaxaad u baahan tahay in ay ka mid ah wax soo saarka ah si ay u isticmaali karaa browser-ka (tusaale,python -m camoufox fetchku soo dejisan Firefox stealth - si aad u aragto docs Camoufox for details). Markaad ka mid ah waxaa la aasaasay, sidoo kale aad si ay u aragto codka server.

Bilood 2: Qalabka MCP Server

Shuruudaha Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabkaxpath_server.pyShuruudaha ugu horeysay ee MCP waa mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah.

from mcp.server.fastmcp import FastMCP
import asyncio
from camoufox.async_api import AsyncCamoufox
import time
import os

mcp = FastMCP("Scrapy XPath Generator")


.....
coded tools
....

if __name__ == "__main__":
	# Initialize and run the server
	mcp.run(transport='stdio')

Sida loo isticmaali karaa MCP Server.

Step 3: Waqtiga HTML Retrieval via Camoufox

Waayo, waxaan u baahan tahay in ay u isticmaali karaa URL-ka, si ay u isticmaali karaa content HTML-ka. Waxaan loo isticmaali karaa Camoufox in ay u isticmaali karaa in ay u isticmaali karaa.toolHakufetch_page_content:

HTML_FILE_PATH = "/Users/pierluigivinciguerra/TWSC/Code/TheWebScrapingClub/79.MCP/downloaded_page.html"  # File to store the downloaded HTML
CAMOUFOX_FILE_PATH = "/Users/pierluigivinciguerra/TWSC/Code/TheWebScrapingClub/79.MCP/camoufox_template.py"

@mcp.tool()
async def fetch_page_content(url: str) -> str:
	global latest_html

	"""Fetch page HTML using Camoufox stealth browser."""
	print(f"[DEBUG] Fetching URL: {url}")
	try:
		async with AsyncCamoufox(humanize=True) as browser:
			page = await browser.new_page()
			await page.goto(url)
			time.sleep(10)
			latest_html = await page.content()
			with open(HTML_FILE_PATH, "w", encoding="utf-8") as f:
				f.write(latest_html)
			print("[DEBUG] HTML stored for later use")
			return "HTML fetched and stored successfully."
	except Exception as e:
		print(f"[ERROR] {e}")
		return f"Error fetching page: {str(e)}"

Ka dib oo ka mid ah macluumaadka, waxaa laga yaqaan Camoufox scraper ah oo ka dib u aragto URL-ka ah oo aad ka mid ah Cursor iyo ku salaysan codka HTML ee file.

Step 4: Xiriiradda XPath Selectors ku salaysan template

Waayo, waxaa loo isticmaali karaa in aad u isticmaali karaa in aad isticmaali karaa in aad isticmaali karaa in aad isticmaali karaa.generate_xpaths,Qalabka dhismaha iyo dhismaha dhismaha iyo dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha dhismaha iyo dhismaha dhismaha.

Sida loo yaabaa, waxaa laga yaabaa in ay ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah.PLPSida loo yaabaa, waxaa loo yaabaa in ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah mid ah mid ah mid ah mid ka mid ah mid ah mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah.PDPSidee waxaa laga yaabaa in ka mid ah wax soo saarka ah.

@mcp.tool()
def generate_xpaths(template: str) -> dict:
	"""Write XPATH selectors for the requested fields using the downloaded HTML file."""

	if not os.path.exists(HTML_FILE_PATH):
		return {"error": f"No HTML file found. Run fetch_page_content() first."}
	
	if template.lower() == "plp":
		fields = "product title, product link, product price, product image, product code"
	elif template.lower() == "pdp":
		fields = "product title, product price, product description, product image, product color, product size, product code"
	else:
		return {"error": "Unknown template type"}

	# Return the HTML and requested fields so Cursor can analyze them
	return {
		"message": "Print the XPath expressions for the requested fields using the variable latest_html.",
		"requested_fields": fields
	}

Sida loo yaabaa, waxaan ka soo xiriir in ay u baahan tahay in ay ka mid ah ka mid ah macluumaadka ka mid ah macluumaadka ka mid ah macluumaadka.

Step 5: Si loo yaqaan Camoufox Scraper

Ma rabtaa in aad Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada Cudarada (Camoufox_template.py) iyo selectors si ay u qaadi karo scraper la soo saarka wax soo saarka ah oo aad u baahan yahay. Waxaan ka heli karaa in loo isticmaali karaa template Camoufox Scraper waa in ay u baahan yahay, sidaas, codka loo soo saarka waa mid ka mid ah hallucination iyo ma ahaato.

@mcp.tool()
def write_camoufox_scraper(template: str, url: str) -> dict:
	print(f"[DEBUG] Writing scraper for template: {template} and URL: {url}")
	"""Reads file Camoufox_template.py and uses it to write a new Camoufox scraper with the requested fields and starting from the url"""
	with open(CAMOUFOX_FILE_PATH, "r", encoding="utf-8") as f:
		latest_html = f.read()	
	return{"message": "Using this template, write a working scraper with the requested fields and starting URL"}
	
	

Waa in; hadda waxaan u baahan tahay kaliya si ay u baabuurta Cursor si ay u iibsato this MCP server iyo u xusuusato in loo isticmaali karaa qalabka.

Sida loo isticmaalo MCP Server ee Cursor

Sida loo yaqaan "Cursor" (IDE-ka AI-ka ah) waxaa loo isticmaali karaa in ay ku saabsan MCP Server oo loo yaqaan "Intelligent Intelligence Server" (MCP Server) oo loo yaqaan "Intelligent Intelligent Intelligence" (AI-ka IDE-ka ah) waa in ay u isticmaali karaa in ay u isticmaali karaa macluumaadka macluumaadka macluumaadka ee webka (eyna loo isticmaali karaa macluumaadka XPath).

Sida loo soo bandhigay server ee Cursor, sidoo kale si ay u isticmaali karaa:

1. Open Cursor Settings – In Cursor IDE, go to the settings menu.

2. Navigate to MCP Servers – Under the Features section, find MCP Servers. (This requires Cursor v0.45.6+ or later, which supports custom MCP server integration.)

3. Add a New MCP Server – Click the “+ Add New MCP Server” button. This will prompt you to enter details for the server.

4. Configure the server – Give it a name like “Scrapy XPath Server” (any friendly name). For Type, select "command" (since we will run a local command to start the server). For Command, enter the command to launch your server. For example, if your script is saved at /path/to/xpath_server.py, you might put it in the box.

5. Save and verify – After entering the details, save the configuration. You should see your MCP server listed in Cursor’s MCP Servers section. Cursor will now launch this server (via the given command) and maintain a connection to it. You might see a green light next to the MCP server’s name.

   
   

   

   
   

Sida loo isticmaali karaa server,Cursor’s AI assistant (the “Composer” agent)Waxaa la isticmaali karaa qalabka cusub oo la isticmaali karaa in la isticmaali karaa.

Markaas oo ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah.fetch_page_contentwax soo saarka iyo ka dibnagenerate_xpathOona

Haku

Ka dib markii ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah mid ah mid ahMarka loo yaqaan Marco Vinciguerra ee ScrapeGraphAI ee ku yaalinta ugu weynee:

“Xaaxa URL @https://www.gucci.com/it/it/ca/women/shoes-for-women-c-women-shoesThen, xaaxa selector XPath loo isticmaalo template PLP. Si loo isticmaalo HTML downloaded (downloaded_page.html file) si loo isticmaalo XPATH selectors.Waaxa selector oo loo isticmaalo.Waaxa file Camoufox_template.py si loo isticmaalo template PLP oo loo isticmaalo template Camoufox scraper. If in doubt, read the selectors.txt file to understand how many fields and which selectors you have to use. Use the file Camoufox_template.py for building the scraper and customize with it the fields in output read by the PLP template.”

Cursor waxaa la socdaan tab Camoufox, waxaa la socdaan URL oo ku yaalaa, waxaa la socdaan HTML ee file, waxaa la socdaan, iyo ka dib markii ay loo soo saarka selectors XPATH iyo la socdaan in file kale. Waxaan ka heli karaa in ay ku salaysan resultada oo dhan ka mid ah file in ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid.

Shuruudaha ugu horeysay ee loo isticmaali karaa, sidaas waxaa laga yaqaan "technically correct" iyo wax soo saarka waa ugu fiican.

Shuruudaha, oo ka mid ah loo yaqaan 'website' oo ka mid ah, waxaa laga yaqaan 'all fields', oo waa ugu fiican oo ka mid ah in ay u hesho badan oo ku saabsan shuruudaha.

Waxaan sidoo kale u baahan tahay in ay ka mid ah wax soo saarka, oo aan ka mid ah wax soo saarka. Waxaan sidoo kale ka mid ah wax soo saarka, oo ka mid ah wax soo saarka, oo ka mid ah wax soo saarka.

Sida loo yaabaa, waxaa laga yaabaa in ay ka mid ah wax soo saarka ah oo ka mid ah wax soo saarka ah oo ka mid ah wax soo saarka iyo wax soo saarka.

Waxaad ka mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah.

The article is part of “The Lab” series by Pierluigi Vinciguerra. Check out his Substack page for more knowledge on Web Scraping.