Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The interesting thing is I counted about 17 instances of "make something people want" in pg's essays and the number of times the individual words are used is much much higher:

    742 make
    753 something
    1589 people
    719 want
fyi, here's the Ruby code for grabbing the text of all the essays:

  require 'open-uri'
  require 'hpricot'
  
  text = ""
  
  listing = Hpricot(open("http://www.paulgraham.com/articles.html"))
  
  (listing/"font"/"a").each do |a|
    url = a.attributes["href"]
    
    puts url
    
    essay = Hpricot(open("http://www.paulgraham.com/"+url))
    essay.search("script").remove
    essay.search("link").remove
    essay.search("meta").remove
    essay.search("style").remove
    
    text += essay.inner_text.gsub(/[\n\r]/, " ") + " "
  
  end
  
  File.open("pg.txt", 'w') { |f| f.write(text) }


It is polite to sleep for a second between requests when scraping.


Ruby code is so damn elegant. :)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: