Results 1 to 4 of 4

Thread: spidering this site

  1. #1
    Senior Member registered user
    Join Date
    Nov 2002
    Location
    Auckland, New Zealand
    Posts
    818

    spidering this site

    Plese don't.

    - Firstly, why?

    I found this in my logs :
    Quote Originally Posted by my log file
    80.133.143.XX - - [24/Feb/2003:00:36:37 +1300] "GET /docs/index.php/KnoppixKDE/K
    noppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knop
    pixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppix
    KDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE
    /KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kn
    oppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knopp
    ixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixK
    DE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/
    KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kno
    ppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppi
    xKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKD
    E/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/K
    noppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knop
    pixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppix
    KDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE
    /KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kn
    oppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knopp
    ixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixK
    DE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/
    KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kno
    ppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppi
    xKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKD
    E/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/K
    noppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knop
    pixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppix
    KDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE
    /KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kn
    oppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knopp
    ixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixK
    DE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/
    KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kno
    ppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppi
    xKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKD
    E/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/K
    noppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knop
    pixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppix
    KDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE
    /KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kn
    oppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knopp
    ixKDE/KnoppixKDE/KnoppixKDE/TextFormattingRules?action=LikePages HTTP/1.0" 200 6
    4585 "http://www.knoppix.net/docs/index.php/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kno
    ppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppi
    xKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKD
    E/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/K
    noppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knop
    pixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppix
    KDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE
    /KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kn
    oppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knopp
    ixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixK
    DE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/
    KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kno
    ppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppi
    xKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKD
    E/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/K
    noppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knop
    pixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Knoppix
    KDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE
    /KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/Kn
    oppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/KnoppixKDE/TextFormattingRu
    les" "Wget/1.7" 3 www.knoppix.net
    That is ONE request from a broken wget. This wastes my bandwidth, and there isn't any reason for someone to wget my site anyway.

    Do not do this, or risk being banned. A few people have already been nullrouted. and I thought I should give some warning.

  2. #2
    Senior Member registered user
    Join Date
    Feb 2003
    Posts
    199
    What exactly is that that you quoted?

  3. #3
    Senior Member registered user
    Join Date
    Nov 2002
    Location
    Auckland, New Zealand
    Posts
    818
    one line of my apache access log

  4. #4
    Senior Member registered user
    Join Date
    Nov 2002
    Location
    Auckland, New Zealand
    Posts
    818
    Someone using httrack requesting 10 pages a Second, just got nullrouted.
    if you are the person from *.dsl.pol.co.uk

    Shame on you!

Similar Threads

  1. Site upgrade
    By eadz in forum The Lounge
    Replies: 2
    Last Post: 12-04-2004, 07:45 AM
  2. Can't get to this site in Linux
    By rrfish72 in forum General Support
    Replies: 5
    Last Post: 08-04-2004, 03:00 PM
  3. Fast download site
    By walwal in forum General Support
    Replies: 3
    Last Post: 07-14-2004, 07:22 PM
  4. Site speed
    By eadz in forum The Lounge
    Replies: 9
    Last Post: 09-30-2003, 01:32 PM
  5. site will be down
    By eadz in forum The Lounge
    Replies: 2
    Last Post: 05-07-2003, 05:22 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Samsung Galaxy Tab A8 10.5

Samsung Galaxy Tab A8 10.5" SM-X200 128GB Wifi Only Tablet Open Box

$149.99



Samsung USB 3.1 Flash Drive Bar Plus 256gb TITAN Gray picture

Samsung USB 3.1 Flash Drive Bar Plus 256gb TITAN Gray

$24.10



Samsung Galaxy Tab A9 (X110) 64GB 4GB RAM International Version (New) picture

Samsung Galaxy Tab A9 (X110) 64GB 4GB RAM International Version (New)

$129.99



Samsung Galaxy Chromebook Go, 14-inch, 32GB, Unlocked All Carriers - Silver picture

Samsung Galaxy Chromebook Go, 14-inch, 32GB, Unlocked All Carriers - Silver

$99.95



Samsung Galaxy Tab E 8

Samsung Galaxy Tab E 8" 16GB Black SM-T377V (Verizon) Android Tablet IG1000

$30.35



Samsung - S Pen Creator Edition - White picture

Samsung - S Pen Creator Edition - White

$40.00



Samsung Galaxy Tab A T387V 8

Samsung Galaxy Tab A T387V 8" 32GB Black Android WiFi + Verizon - Acceptable

$25.99



Samsung Galaxy 12.4 Inch Slim Keyboard For S7+ | S7 FE | S8+ | S8+ 5G EF-DT730 picture

Samsung Galaxy 12.4 Inch Slim Keyboard For S7+ | S7 FE | S8+ | S8+ 5G EF-DT730

$49.99



Samsung 16GB 2Rx4 PC3-12800R M393B2G70BH0-CK0 DDR3 RDIMM - SERVER RAM picture

Samsung 16GB 2Rx4 PC3-12800R M393B2G70BH0-CK0 DDR3 RDIMM - SERVER RAM

$8.90



Samsung 22 In LED-backlit LCD monitor, S22E450D picture

Samsung 22 In LED-backlit LCD monitor, S22E450D

$45.00