robots.txt: important if you’re hosting passwords

This is going to be a fun post that's related to some of my password work. Some of the text may not be PG13, so parental discretion is advised.

As most of you know, I've been collecting password lists. In addition to normal password lists that are useful in bruteforcing, I have a (so far) lame collection of non-hacking dictionaries. Things like cities, English words, etc.

There was a time when the biggest dictionary I had, weighing in at 6.4mb, was a German wordlist. 6.4mb doesn't sound like much, but at the time I was on a DSL connection; with about 400kbit upstream (on a good day), I could feel every download.

After awhile, I started realizing that german.txt was being downloaded pretty regularly. Far more regularly than any other lists. This didn't make any sense to me -- were German hackers stockpiling their tools before 202(c) came into effect? Were people interested in what Germans were using for passwords? Were people trying to cheat at German Boggle? I wasn't sure!

At that point, I removed the files and didn't worry too much about it. It wasn't worth hosting.

Fast forward a year or two. A friend of mine, who we affectionately call "The German", was doing some research into referer logs for different sites, and asked if he could have the referer log skullsecurity.org. I happily obliged, and asked him if he could figure out why so many people were downloading my German password list (he's German himself, after all). The referer entries he found made us laugh. There were thousands, but here are a few:

  • http://www.google.de/search?q=1%20cm%20dicker%20anleimer%20wie%20hinlegen%20beim%20trocknen -> German_list.txt
  • http://www.google.de/search?q=Du%20singst%20shake%20your%20ass%20und%20wackelst%20mit%20dem%20kopf -> German_list.txt
  • http://www.google.de/search?q=schuh%20bode%20cowboy%20stiefel -> German_list.txt
  • http://www.google.de/search?q=Drillinge%2BIchstedt -> German_list.txt
  • http://www.google.de/search?q=porn+bemastung -> German_list.txt
  • http://www.google.de/search?q=masth%c3%bchner+von+gut+deutsch-nienhof -> German_list.txt
  • http://www.google.de/search?q=teuerste+sexpuppe+Real+Dolls+shop -> German_list.txt
  • http://www.google.de/search?q=lolita+sexfilm+ohne+jeglichen+geb%C3%BCrhen -> German_list.txt

And, of course, my absolute favourite:

  • http://www.google.de/search?q=porno+ porn+ comics+ cartoon+ hardcore+ gropers+ raped+ Asian+ porn,Asian+ porn+ movies,Asian+ porn+ video,Asian+ idol+ movies,Asian+ idol+ porn+ wild+ japan+ porn,+ asian+ porn+ videos,+ free+ japanese+ porn,+ asian+ sex+ movies,+ orienta+ porn,+ japan+ porn,+ asian+ porn,+ asian+ sex -> German_list.txt

Now, I don't know German, and I'm pretty they aren't all questionable. Google Translate tells me that one is about cowboy boots. I do, however, recognize some somewhat more naughty words; words that really shouldn't be associated with my site.

So, the moral of the story is: hosting wordlists can get you some pretty interesting search results. If that's what you're into, let me know and I'll send you a list of keywords to put on your site. :)

Now? I host my passwords on a separate domain with a robots.txt file. No more wacky referers!

3 thoughts on “robots.txt: important if you’re hosting passwords

  1. Reply

    Jagermo

    haha, nice post. I speak German, so if you are interested in a translation, I could help you :-)
    just drop a E-Mail or DM @jagermo

    1. Reply

      Ron Post author

      @Jagermo Thanks! Anything interesting? I know I had some really bad stuff in my logs originally, but I only posted stuff that LOOKED bad here :)

  2. Reply

    michal

    hi could you tell me how to hack in to facebook i looked on many pages today and i searched in pirate bay facebook then i found your torrent i am realy intrested in hacking and i have just started so i want to start from websites like facebook i found meany pages that give you passwords on one of them i would have to pay 90$ for one password or 140$ for unlimited and the website would get passwords for free.
    that website gave you coded files in md5
    i tried uncoding web sites they worked with coding and uncoding but with the password they dont work at all.
    could you point me some programs or programs for making hacking program that could help me pelase could you send me an email to michal9034902@wp.pl

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>