Deep Web, Dark Web, and the Difference.

     Firstly, what this article is not. The terms deep web and dark web are not synonymous! The dark net is a subset of the deep web. You'll understand this a bit better later. I'm also not going to tell you how to access the dark web, how to find those sites or how to use the software involved. I'm simply telling you what it all is, and what its about. This is a conscious decision for two reasons. Firstly, I'm not an expert, and therefore I don't want anyone relying on my knowledge to attempt to use these softwares. Secondly, the Government lists and watches people who use this software, so if your not an expert you'd be an idiot to try and access the dark web sites as you'll be listed. What this article is, it just states what the deep web, and it's subset, the dark web is in more technical detail beyond what you've seen in the news.

     The deep web, at a basic sense, is any information that is not accessible through standard search engines or links. This type of data includes Twitter and Facebook posts, buried links (as to that search engines can't find them - like half my website), databases, file servers, bank accounts, pay walls, or any information that isn't meant to be seen by a simple search from the general public without authorization. Search engines don't have access to these places because they usually require direct access to a database and search engines aren't built for that, some websites tell engine crawlers to ignore the website (not all engines conform to this), and others just aren't exposed to the net live, as in you may need credentials like a two-factor authentication or a login to get in.

     If you realize what this, essentially, is, you'll not be surprised to learn that the deep web is about 4000 times larger than the surface web, the part we generally experience everyday, as you are now reading this. It mostly consists of the storage of data and infrastructure to bring the surface web to life. This doesn't imply that it's invisible, its not, a great deal of it is there to see if you just know how. Its just simply not indexed, as it were, by a search engine. In it's simplest form its the infrastructure of the internet and its the content that was not meant to be generally public - that's it!

     Now, what is the dark web? This is where things get interesting, dangerous and possibly illegal (but wasn't intended to be). The first thing to understand when talking about the dark web is what's called an overlay network. These are networks that are built on top of (overlay) another network.  These hardwired servers, the ones that your modem cables eventually connect to for example, can have virtual links that travel over the internet. That is an overlay network. You've very likely used a overlay network before. Anytime you access your bank account and make a request of it, you use an overlay network. It is a virtual connection that is encrypted to prevent unauthorized communication from anyone else but the authorized.

     Dark webs are built on this technology. These sites are hidden from the general internet and require very specific methods to access. These methods generally consist of specific software, configurations and authorizations to get to. The vast majority are heavily encrypted, but one isn't, Clearnet. This "dark web" is simply connections that exist outside the traditional world-wide-web. It provides some low-base anonymity, with sites routinely identified by their IP addresses only. I'm sure there are illicit activities going on through those sites but when your live like that, it seems stupid to do so.

     There are many dark webs but the most well know is likely Tor, often referred to as Onionland because of it's onion routing anonymization technique. These sites have .onion prefixes and are not indexed by any search engine so its a word of mouth system in essence and they are only accessible from within the Tor network. The Tor network is operated on the backs of servers who wish to participate in the network and is accessed over a Tor Browser or Tor Linux. WARNING: You can google these softwares but the Government (NSA), does in fact flag people who download this software. However, they are inherently not illegal. Many legitimate uses exist for good reason. High-risk journalists in some nations are required to use such networks in order to avoid punishment. Tor servers that purpose of freedom of speech and freedom of press. Tor is also vulnerable, if the entry node can be identified, your anonymity is lost. Also, the exit nodes decrypt Tor traffic, a black hat could potentially find you depending on the information passed. Another, more interesting vulnerability is that if you have Government level resources and setup a fake site, its possible to actually trace back through the nodes to your entry node. For more, see article from Naked Security or this article from Vice.

     As you have already likely seen, many illicit uses also exist. That's how many terrorists communicate, botnets, illegal porn, drugs, you name it can be bought and accessed on the dark web sites. People. Do not think however that the Government isn't watching these things. They are. The NSA has some very impressive tools to make this happen and users who intend to access these illicit dark web sites (I forgot to mention there are legit dark web sites too) have to jump through many many hoops to try and avoid detection. Some techniques for monitoring are even based upon weather or not someone resizes their damn browser window. In general, as with everything on the internet, absolutely NOTHING is entirely anonymous and neither are dark nets - if you hit a dark net site you will be flagged if your ISP or the Government knows about it. I mentioned previously that there are legitimate uses for dark nets and many dark net sites have legit uses. Another example beyond the journalist are whistle blower sites, although I suppose that depends upon your perspective.

     However, as this article in Wired points out, the dark web perception is generally a myth. Don't get mistaken, plenty of illicit activities going on but much of the traditional web is just as dark. Many radicalization and Terrorist web forums are in plain view for anyone to google. Law enforcement has found that just 0.2% of the URLs linking to child pornography are hosted on the dark net. There are live sites to buy and sell someone's SSN (don't do it). The Dark Net itself is very tiny and its not impenetrable as I've already stated. Going undercover on these dark sites is still incredibly effective against criminals.

     So, to summarize, the deep web, is simply any content not meant to be linked from a search engine or accessible to just anyone, more akin to the infrastructure of the net. The dark web, is a subset for the expressed intent of being anonymous. The dark net is not inherently good or bad, its all about your intent. Doctors on the dark net can give real advice to drug users who come out of the woodwork because they are hidden, people can circumvent the Great Chinese Firewall to see what the rest of the world is like and discuss their own Government without being persecuted - but there is some seriously depraved stuff on that same network because of the same anonymity. In essence, it exists as a glimmer of the original intent of the web, "a space beyond control of individual states" (Wired).  Consider the potential as freedoms diminish.


© 2016 Zachary G Wolfe -- Remember to turn your brain off for a reboot sometimes...