Project

General

Profile

Feature #24

Add option to skip invalid links on future runs

Added by Deoren Moor almost 7 years ago. Updated almost 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Cache Handling
Target version:
Start date:
12/05/2010
Due date:
% Done:

0%

Estimated time:
Resolution:
Fixed

Description

Basically add them to cache.ini so a future retrieval attempt won't be made.


Related issues

Related to Feature #26: Add option to skip "not found" content on future runsClosed2010-12-05

Associated revisions

Revision 25 (diff)
Added by Deoren Moor almost 7 years ago

[issue #19,20,23,24,25,26,27]
Unfortunately this a big commit, instead of broken up into one commit per defect/feature like I prefer them to be.

Implemented functions:

  • SanitizeName (#20)
  • DownloadFile (#19, somewhat)
  • RemoveFile (#23)

Implemented features:

  • Spider content (#25)
  • Skip invalid and missing content (#24, #26)
  • Remove incomplete files (#23)
  • Added socket.timeout value (#27). May tweak in the future.

History

#1 Updated by Deoren Moor almost 7 years ago

  • Status changed from Assigned to Closed
  • Resolution set to Fixed

Implemented in r25.

Also available in: Atom PDF