【Python】 httpgetの実装

import　urllib2
def httpget(address, user_agent='myagent'):
  opener = urllib2.build_opener()
  opener.addheaders = [('User-agent', user_agent)]
  doc = opener.open(address).read()
  return doc

使用例

doc = httpget('http://www.yahoo.com')

使用例ShiftJISをUTF-8に変換

doc = unicode(httpget('http://www.amazon.co.jp/'), 'cp932', 'ignore')