My week Scraping out sites with my favourie languages nodejs(casperjs) ,python
September 06, 2013
It was a great scrapping sites for fun.First i wanted some gui tool already build to solve this.later found it better to scrap using my favourite languages.You can see my github repo for code i haven’t explained much of it here.
website i scraped:
The best thing to scrap a site because with this i can scrap any site i want.This uses phantomjs to run which is a headless webkit browser.
I dont think there is a use case where i cant scrap a site with this.
The below repo contains a scraper for cfainstitute and turkisleather brands.
This is a great project for scrapping with. Though i could have scraped with casperjs i just want to try out whatever i could.
Gui tool which i used:
Its not probably worth a mention because it was waste of my time.Due to inefficiency to scroll pages automatically and most of the time scraping fails in midway and i have to restart it again which is a waste of time.