Uploaded image for project: 'DSpace'
  1. DSpace
  2. DS-2463

iplists.com-non_engines.txt is outdated and slow

    XMLWordPrintable

    Details

    • Attachments:
      0
    • Comments:
      0
    • Documentation Status:
      Complete or Committed

      Description

      config/spiders/iplists.com-non_engines.txt apparently hasn't been updated in seven years. A number of entries refer to things which no longer exist, and some of them cause long timeouts when initializing SpiderDetector.

      I suggest removing this file (and the config. setting which updates it). Some of the information in it might be more useful for augmenting the user-agent patterns.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              mwood Mark H. Wood
              Reporter:
              mwood Mark H. Wood
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: