Bit of a problem with the headline: they didn't crawl anything because Common Cr... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		wmf on Oct 16, 2012 \| parent \| context \| favorite \| on: Data Mining 3.4 billion Web pages for $100 of EC2 Bit of a problem with the headline: they didn't crawl anything because Common Crawl already did that.

chime on Oct 16, 2012 [–]

Data Mining != Crawling. I don't see a problem with that.

Steko on Oct 17, 2012 | [–]

Submitted Title used to say "Crawled"

sjg007 on Oct 17, 2012 | | [–]

Is grepped a better choice? You can crawl in memory from a repository, or "crawl" across the net.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact