Extract Summit 2021 Coding Contest

Get your Scrapy Cloud account, your Scrapy Discord access, and your registration in place beforehand.

Once you have these in place, you're ready to begin as soon as the contest begins at 2 pm GMT on 30th September.

Here's how to proceed on the day:

• We will reveal through Discord the URL of the target website and a specification of item fields that need to be extracted.
• You must write a spider that extracts all items with the specified fields, and run it in Scrapy Cloud.
• Once the Scrapy Cloud job finishes, you must submit the job ID to a bot in the Scrapy Discord server.
• The bot will let you know whether or not you managed to extract all items with complete data.
• If you failed, update your code and try again with a new Scrapy Cloud job. The bot will accept unlimited job submissions for the duration of the competition.
• To win, be the first to submit a job that successfully extracts all expected data.

The website will not ban any client for any reason, so you will not need any proxy. But crawling the website and extracting item data will not be straightforward, do not expect to get a working spider on your first run.