Project

General

Profile

Defect #86

Page scraping timeouts are not handled properly

Added by Deoren Moor over 5 years ago.

Status:
Assigned
Priority:
Normal
Assignee:
Category:
Scraping
Target version:
Start date:
11/29/2011
Due date:
% Done:

0%

Affected Version:
Resolution:

Description

Traceback (most recent call last):
  File "/usr/local/bin/email_ebook_deals.py", line 467, in <module>
    main()
  File "/usr/local/bin/email_ebook_deals.py", line 426, in main
    site_content = fetch_page(site)
  File "/usr/local/bin/email_ebook_deals.py", line 239, in fetch_page
    html_page = urllib2.urlopen(site['url'])
  File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1170, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1145, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>

Related issues

Related to Defect #103: socket.error: [Errno 104] Connection reset by peer Assigned 09/23/2012

Also available in: Atom PDF