scrapy操作流程_bootstrap还有必要学吗

需要说明的是，Scrapy其实已经提供了许多Downloader Middleware，比如负责失败重试、自动重定向等功能的Middleware，它们被DOWNLOADER_MIDDLEWARES_BASE变量所定义。

DOWNLOADER_MIDDLEWARES_BASE变量的内容如下所示：

{ 
    'scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware': 100, 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware': 300, 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware': 350, 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware': 400, 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': 500, 'scrapy.downloadermiddlewares.retry.RetryMiddleware': 550, 'scrapy.downloadermiddlewares.ajaxcrawl.AjaxCrawlMiddleware': 560, 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware': 580, 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware'