20,741
edits
Howto:Processing d-tpp using Python: Difference between revisions
Jump to navigation
Jump to search
Howto:Processing d-tpp using Python (view source)
Revision as of 19:25, 28 November 2017
, 28 November 2017→Scraping
| Line 35: | Line 35: | ||
=== Scraping === | === Scraping === | ||
{{Note|This will download roughly | {{Note|This will download roughly 4gb of data in ~17k files, for each AIRAC cycle!}} | ||
Alternatively, use a media pipeline <ref>http://sergeis.com/web-scraping/downloading-files-scrapy-mediapipeline/</ref> | Alternatively, use a media pipeline <ref>http://sergeis.com/web-scraping/downloading-files-scrapy-mediapipeline/</ref> | ||
<syntaxhighlight lang="python"> | <syntaxhighlight lang="python"> | ||