Too Long; Didn't Read
Scrapy is an application framework for crawling web sites and extracting structured/unstructured data which can be used for a wide range of applications such as data mining, information processing or historical archival. It is not only able to scrap data from websites, but it is able to scrape data from web services such as Amazon API, Twitter/Facebook API as well. It provides a global key-appings that the code can use to pull configuration values from. It is a global directory/crawlers which contains all spiders which. The. project structure which scrapy creates for a user has: scrapy.cfg, test_project.py, items.py, pipelines.py and spiders.