Web crawl / Scrapy developers for Corporate360 - Kerala startup jobs
Web crawl / Scrapy developers
  • Full Time
  • Applications have closed


We are looking for Scrapy developers to join our data science team to help us build and maintain web crawlers. This is the perfect job for someone seeking to work on interesting and challenging data mining problems, with a team that is truly passionate about what they do.


  • Web crawler development with Scrapy
  • Deploying and maintaining web crawlers
  • Process large amounts of crawled data
  • Text processing in python


  • Python guru
  • Linux experience is must
  • Experience in Scrapy is a must
  • Knowledge in NLTK, pandas, scikit-learn, mapreduce, nosql, etc will be a plus
  • Should know how to deal with anti-scraping measures
  • Should be able to configure proxies/IP rotators etc
  • Should have crawled heavy Javascript/Ajax websites
  • Familiarity with AWS (S3, EC2, SWF) or similar is a plus