Title: Internet geeks here! Who can determine how many web spiders/crawlers are on 4um?? Source:
none URL Source:http://none Published:Nov 20, 2009 Author:X-15 Post Date:2009-11-20 03:04:08 by X-15 Keywords:None Views:2783 Comments:170
"Web spiders/crawlers: programs that search websites looking for specific words or patterns to compile into a database."
A popular gun website I visit had 20 running, if 4um has less then I assume it has a lower profile in the eyes of FedGov.
Internet geeks here! Who can determine how many web spiders/crawlers are on 4um??
You'd need access to christine's server logs to get a good idea. However, there are many kinds of spiders, some quite difficult to detect.
Here is a quickie spider that I wrote. It runs on the Mac, OS X 10.5 Leopard. However, it is a standard Bash script and should work easily on Linux or Unix systems, probably in a Cygwin setup on Windows too.
The script uses Lynx, a venerable text-only browser, to fetch my Comments page to a file called htmlsource1. It then uses the stream editor Sed to parse this captured HTML file by scanning the right column for news stories, capturing the thread names and URLs at 4um to a file called htmlsource2.
It then uses Lynx to capture each thread to a separate file by thread number in a subdirectory called '4um'.
You could build a database or use text search tools like grep to mine the stored threads for info.
No doubt, various federal agencies and people like ADL or SPLC use scripts like this to capture many forums and use grep and other search tools to scan each thread captured for relevant keywords to flag them for review by human beings.
I'm presenting this so that 4um folks can get some idea of how these agencies and busybodies operate. Geeks know this stuff but the average person has no idea how easy it is. People should know how easy it is to database their every remark since we no longer live in a free country.
I'm presenting this so that 4um folks can get some idea of how these agencies and busybodies operate. Geeks know this stuff but the average person has no idea how easy it is. People should know how easy it is to database their every remark since we no longer live in a free country.
BTW, by changing only a few lines in the above code, I could slowly download your entire database and reconstruct each poster's remarks. Essentially, your mySQL database could be replicated by downloading all the threads and parsing the user comments into a new mySQL database. I'm sure Neil could point this out as well, probably better than I can. This is why watching your server logs for an IP that downloads every thread or an IP that downloads every thread in the database sequentially is good to do.
Anyway, this seemed a good thread to point this stuff out.
You only have to parse for the HTML tags and CSS classes. Not at all difficult.
I once wrote a Firefox extension that allowed me to entirely replace the look and feel of TOS, add backgrounds, insert YouTubes to replace the YouTube links, implement my own browser-based bozo filter, etc.
It's quite easy. You have to have good server log analytic software to find out if your site is being mined. Now, 4um isn't really high traffic so you can probably get a good idea by looking at IP addresses. You should watch for IP addresses that only read threads (and that read every thread) and never post. Those lurkers can just as easily be spiders for ADL, FBI, SPLC, NetNanny, Google, etc. In fact, you should assume that you are being spidered this way until you can prove otherwise.
And the spidering of your threads could just as easily come from multiple IP addresses. You can deter some of this by requiring the use of cookies but a competent programmer can fake that too.
You should assume that every word you put online will be recorded. The Feds are building huge new datacenters to store the content of the entire internet and all cell phones and landlines. They may make no use of that unless and until they detect (or need) to chase a domestic threat for terrorism or hate crimes. It is at that point that you will receive a subpoena for your server logs to obtain IP addresses (if they don't already have them by being on the backbone and sniffing everything) and an included order that you will not tell anyone that they are assembling evidence. After that, the ISPs for those requested IP addresses will get national security directives and subpoenas to provide their logs to identify the IP address and they would also be silenced.
This is is the new Soviet Amerika. Welcome to the gulag, comrade.
She owns both domain names and points both of them to her site. If you use libertypost.net as your URL, it mostly works but you won't have the same browser cookies (it seems) so it may look or act a little differently on a few screens. You might have to sign in again or whatnot.
Libertypost.net points to the server for libertypost.org. Libertypost.com and libertypost.us are owned by the USPS and that isn't sinister either.
Pointing multiple domains at a single server or server farm is a very common practice. This is how places like Amazon and Newegg work too.
Because noone would ever think of something so outrageous as duplicating a website. The idea is simply preposterous. For anyone to even think of such a thing or question it is clearly and indication of well, that they have a problem.
Forgive me Maam for my ignorance or paranoia or stupidity or whatever my problem is, but isn't the fact that there are two different urls mean that there are two different websites? Regardless of how similar they look?
Forgive me Maam for my ignorance or paranoia or stupidity or whatever my problem is, but isn't the fact that there are two different urls mean that there are two different websites? Regardless of how similar they look?
How snarky you are, wudi. Nevertheless, I'll answer you, noting the irony of you calling me rude.
All website servers and all computers (and devices like addressable routers) occupy a dotted-IP address on the internet, just as your own home network has its own range of addresses. So your home network might use the common reserved range of addresses 192.168.0.1 - 192.168.255.255. In other words, a range of 65536 addresses (roughly) are reserved from any server or other LAN using them. So you might use an IP address of 192.168.0.1 for your DSL modem, the router you connect to it might use 192.168.1.1 and then all your home network computers could use addresses in the range 192.168.1.2 - 192.168.1.254.
So a place like libertypost.org has a server with a dotted-IP address too. The dotted-IP address is what you access, the name is just a convenient way to access the actually address.
You can point one domain name to a webserver. Or you can point a million names to a webserver. Or you can point different domain names to different subdirectories of the same server. Or whatever. Domain names don't care what you point them at. And the domain registrars are happy when you buy multiple names so they make more money.
So, using common utilities, you can look up the actual dotted-IP address of LibertyPost.
libertypost.net. 236 IN A 69.55.11.240 libertypost.org. 236 IN A 69.55.11.240
So you can see for yourself that the two names are the same. You can also just use the dotted-IP address since that is what you always end up using anyway. The name only translates to the number.
So try clicking it: 69.55.11.240. Once again, you arrive at LibertyPost. Unless Goldi really is the supersecret hacker of all time.
You can verify these domain names are at these numbers as I've told you by using your Command Prompt on Windows XP (or its equivalent on Vista or Win7). It's found in the Programs->Accessories folder. Once you open it, you can type nslookup to find your own primary, probably a DSL modem or router in your house. You can then enter libertypost.net or libertypost.org or whatever to check for yourself what the IP address really is. Type 'exit' to leave nslookup and then type 'exit' again to dismiss the command prompt and return to Windows. You can also google for 'nslookup' and use the online versions of these but you can't really trust those nslookup sites either because it might be Goldi running them all on her secret Evil Servers. Obviously, you can trust no one. It's really sad.
Of course, Goldi's site, like 4um, runs from a single mySQL database using a bunch of Perl scripts and PHP pages, the most common kind of interactive site. So it is possible that she has two "copies" of her site, one with Good Decent Godfearing code, the other with the Supersecret Evil Goldi-Lox Killer code. So that if you go to libertypost.org, you get the Good Decent Godfearing version and if you go to libertypost.net, you get the Supersecrect Evil Goldi-Lox Killer version.
Now, this whole notion of a good Goldi-server and an evil Goldi-server is somewhat daft. First, Goldi would have to be a lot more sophisticated than she is about servers and writing code. She doesn't do any of that stuff and hasn't even upgraded her older Pinguinite code (written by Neil and available for free) to the newer Pinguinite code that christine uses here which has more features, is more secure, etc. Beyond that, the idea of Goldi-two-servers is rather odd because the first principle of Being Evil is: Don't let anyone figure out that you really are evil.
Now, I can go to any of the three known methods of reaching LibertyPost via libertypost.org or libertypost.net or 69.55.11.240 (among who knows how many other evil versions of LP there are), and the pages are the same, the most recent articles are the same, the most recent comments are the same. And again, I can sign in to any of them and they behave identically.
Now, since everyone but you and a bunch of old cat ladies posting on Usenet or chatting on IRC already knows the above info, then what sense would it make to have the Goldi-two-servers thing? None whatsoever. What would be more interesting would be to have a single server with the two names pointed at it but monitor the access to it by the IP addresses of those you have banned from it. This way, you can be Evil to banned or disliked users but never look Evil to the Chosen People of LibertyPost (those who Goldi hasn't banned, this being 4um, I thought that was fun way to express it). And people on Goldi's ban list are always going to be assumed to be nursing a grudge anyway so no one will believe their accusations. So you might choose to be paranoid and think that Goldi is only running her Supersecret Evil Goldi-Code if you visit her server but she does show no signs of such code or admin sophistication and sneakiness. It really is just her chat board, I think.
But you can believe or make up anything you like if you enjoy that more. But I see no reason to believe that Goldi or Christine are quite so powerful or obsessed over their chat boards as that would require. Now, me posting this much text to you, that might in and of itself seem Quite Evil to some folks despite my obvious benevolent intentions. But then, perhaps paranoid nihilism really is the safest policy and you have to assume that every website and every server and every person is out to get you and that they are also only figments of your own imagination.
Of course, maybe Goldi is Christine and 4um/LP are just two of her servers. Maybe Goldi/Christine is an employee of FBI/NSA/ADL/SPLC and these sites are just honeypots, beginning to end. Let your imagination run wild.
Or maybe Goldi and Christine are just two ladies who run chat forums, as they appear to be.
You started being rude to me by asking 'What is your problem anyway?' at post #43.
You first started off with liberty-post.org and liberty-post.net (which are entirely different domain names from libertypost.org and libertypost.net since domain names can include only lower-case letters, numerals, hyphens).
Then you switched to complaining about libertypost.net, demanding attention and complaining that no one was posting about them. Despite the fact that the rest of us already knew what was up with those domains.
Now go fuck yourself, you stupid, ass-kissing cunt.
(Edited by wudidiz for grammatical errors.)
A perfect example of why you tend to get banned from a lot of websites. What a martyr.
You first started off with liberty-post.org and liberty-post.net (which are entirely different domain names from libertypost.org and libertypost.net since domain names can include only lower-case letters, numerals, hyphens).
No I didn't. Bullshitting moron.
You're really turning out to be quite the piece of work, aren't you.
Forgive me Maam for my ignorance or paranoia or stupidity or whatever my problem is, but isn't the fact that there are two different urls mean that there are two different websites? Regardless of how similar they look?
How snarky you are, wudi. Nevertheless, I'll answer you, noting the irony of you calling me rude.
All website servers and all computers (and devices like addressable routers) occupy a dotted-IP address on the internet, just as your own home network has its own range of addresses. So your home network might use the common reserved range of addresses 192.168.0.1 - 192.168.255.255. In other words, a range of 65536 addresses (roughly) are reserved from any server or other LAN using them. So you might use an IP address of 192.168.0.1 for your DSL modem, the router you connect to it might use 192.168.1.1 and then all your home network computers could use addresses in the range 192.168.1.2 - 192.168.1.254.
So a place like libertypost.org has a server with a dotted-IP address too. The dotted-IP address is what you access, the name is just a convenient way to access the actually address.
You can point one domain name to a webserver. Or you can point a million names to a webserver. Or you can point different domain names to different subdirectories of the same server. Or whatever. Domain names don't care what you point them at. And the domain registrars are happy when you buy multiple names so they make more money.
So, using common utilities, you can look up the actual dotted-IP address of LibertyPost.
libertypost.net. 236 IN A 69.55.11.240 libertypost.org. 236 IN A 69.55.11.240
So you can see for yourself that the two names are the same. You can also just use the dotted-IP address since that is what you always end up using anyway. The name only translates to the number.
So try clicking it: 69.55.11.240. Once again, you arrive at LibertyPost. Unless Goldi really is the supersecret hacker of all time.
You can verify these domain names are at these numbers as I've told you by using your Command Prompt on Windows XP (or its equivalent on Vista or Win7). It's found in the Programs->Accessories folder. Once you open it, you can type nslookup to find your own primary, probably a DSL modem or router in your house. You can then enter libertypost.net or libertypost.org or whatever to check for yourself what the IP address really is. Type 'exit' to leave nslookup and then type 'exit' again to dismiss the command prompt and return to Windows. You can also google for 'nslookup' and use the online versions of these but you can't really trust those nslookup sites either because it might be Goldi running them all on her secret Evil Servers. Obviously, you can trust no one. It's really sad.
Of course, Goldi's site, like 4um, runs from a single mySQL database using a bunch of Perl scripts and PHP pages, the most common kind of interactive site. So it is possible that she has two "copies" of her site, one with Good Decent Godfearing code, the other with the Supersecret Evil Goldi-Lox Killer code. So that if you go to libertypost.org, you get the Good Decent Godfearing version and if you go to libertypost.net, you get the Supersecrect Evil Goldi-Lox Killer version.
Now, this whole notion of a good Goldi-server and an evil Goldi-server is somewhat daft. First, Goldi would have to be a lot more sophisticated than she is about servers and writing code. She doesn't do any of that stuff and hasn't even upgraded her older Pinguinite code (written by Neil and available for free) to the newer Pinguinite code that christine uses here which has more features, is more secure, etc. Beyond that, the idea of Goldi-two-servers is rather odd because the first principle of Being Evil is: Don't let anyone figure out that you really are evil.
Now, I can go to any of the three known methods of reaching LibertyPost via libertypost.org or libertypost.net or 69.55.11.240 (among who knows how many other evil versions of LP there are), and the pages are the same, the most recent articles are the same, the most recent comments are the same. And again, I can sign in to any of them and they behave identically.
Now, since everyone but you and a bunch of old cat ladies posting on Usenet or chatting on IRC already knows the above info, then what sense would it make to have the Goldi-two-servers thing? None whatsoever. What would be more interesting would be to have a single server with the two names pointed at it but monitor the access to it by the IP addresses of those you have banned from it. This way, you can be Evil to banned or disliked users but never look Evil to the Chosen People of LibertyPost (those who Goldi hasn't banned, this being 4um, I thought that was fun way to express it). And people on Goldi's ban list are always going to be assumed to be nursing a grudge anyway so no one will believe their accusations. So you might choose to be paranoid and think that Goldi is only running her Supersecret Evil Goldi-Code if you visit her server but she does show no signs of such code or admin sophistication and sneakiness. It really is just her chat board, I think.
But you can believe or make up anything you like if you enjoy that more. But I see no reason to believe that Goldi or Christine are quite so powerful or obsessed over their chat boards as that would require. Now, me posting this much text to you, that might in and of itself seem Quite Evil to some folks despite my obvious benevolent intentions. But then, perhaps paranoid nihilism really is the safest policy and you have to assume that every website and every server and every person is out to get you and that they are also only figments of your own imagination.
Of course, maybe Goldi is Christine and 4um/LP are just two of her servers. Maybe Goldi/Christine is an employee of FBI/NSA/ADL/SPLC and these sites are just honeypots, beginning to end. Let your imagination run wild.
Or maybe Goldi and Christine are just two ladies who run chat forums, as they appear to be.
TooConservative posted on 2009-11-22 8:06:06 ET Reply Trace Private Reply