Recon - My Way Or High Way

19 min readJun 7, 2020

Hello everyone its been a long time I haven’t published any blog because I was busy in making a blog on my recon technique. I hope you guys will like it.

Summary :

Reconnaissance (recon) is an important technique for penetration testing and the beginning point of many data breaches. It involves gathering of information about the target which can be useful for finding flaws or vulnerabilities. Many people never do proper reconnaissance and start attacking the target which is a wrong way.

Recon gives you many juicy information and many a times it happens that people get critically sensitive information only by doing recon. Recon is important to know that how the target works and what are the technologies they use.

Everyone knows their are two types of recon techniques :
1. Active
2. Passive

1. Active :

Active reconnaissance is a type of attack in which an intruder engages with the targeted system to gather information about vulnerabilities. This may be through automated scanning or manual testing using various tools. In an active recon an attacker needs to interact with the target. This recon is fast and more accurate, however it also makes much more noise. Since the attacker have to interact with the target to gain information, there is an increased chance that the recon will get caught by a firewall or one of the network security devices.

2. Passive :

Passive reconnaissance is an attempt to gain information about targeted computers and networks without actively engaging with the systems. Both types of reconnaissance are sometimes referred to as passive attacks because the purpose is simply to obtain information, rather than to actively exploit the target.

What we are going to use ?

1. WayBack Machine
2. Knock.py and Sublister
3. Dnsdumpster
4. Netcraft
5. crt.sh
6. Altdns
7. Yougetsignal
8. IP range finder
9. Whois
10. Censys.io
11. Domain Profiler
12. ViewDNS
13. VirusTotal
14. Photon
15. LinkFinder
16. Retire.js
17. Dig to check for CNAME
18. Ping to check OS
19. Nmap
20. DirSearch
21. Wappalyzer
22. Whatweb
23. Finding Hidden Parameters (Arjun)
24. S3 Bucket Recon
25. TiDos

WayBack Machine :

The Wayback Machine is a digital archive of the World Wide Web, founded by the Internet Archive. It allows the user to go “back in time” and see what websites looked like in the past.

How to use WayBack Machine ?

Go to https://archive.org/web/
Enter the URL to check

3. Click on Browse History to check the results

As you can see here the data available here is from May 26, 2004 to May 31, 2020 which means there has been some updates between this dates, Let’s see

4. Click on any date and select the snapshot

WayBack Machine - Selecting Gmail in 2006

You can also use come filters for searching in wayback machine like, using https://web.archive.org/web/*/website.com/* will pull down a list of paths that the wayback machine has crawled. We can then use the filter to search for specific files.

Why should we use WayBack Machine ?

Using wayback machine could be a great advantage because you can go to the past and check for the sensitive files that existed before. For eg. I have a website that I have made in 2010 which have frequent updates till 2020, so what an attacker can do is he/she can use the wayback machine to check my past website of 2010 and then he/she can use the filter to searches, can use the directory brute force attack etc. to have sensitive files

2. Knock.py & Sublist3r :

This tools are used to find subdomains of any website.

Knock.py is a python tool designed to enumerate subdomains on a target domain through a wordlist.

Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu and Ask.

How to use Knock.py ?

Download it from github : https://github.com/guelfoweb/knock
Download all the requirements as suggested in the repository
Open it in your console and run it

Command : python knock.py example.com -w <YourCustomWordList>

How to use Sublist3r ?

Download it from github : https://github.com/aboul3la/Sublist3r
Download all the requirements as suggested in the repository
Open it in your console and run it

Command : ./sublist3r.py -d example.com

Now we have the subdomains of the websites but have you ever thought that a subdomain can also have a subdomain ? Many websites have 3 - 4 levels of subdomains.

Say for example I got a subdomain named mail.example.com and after finding a subdomain of mail.example.com I got a subdomain my.mail.example.com which might be untested. So always look for subdomains and subdomains of subdomains.

3. DNSdumpster :

DNSdumpster is a free domain research online tool that can discover hosts related to a domain. It helps to find out subdomains, HTTP headers, banner grabbing, MX Records, DNS Servers , TXT Records etc. It helps a lot in gathering information about the target.

How to use it ?

Simple visit https://dnsdumpster.com/ and enter your target domain

2. It will give the results as shown below

As you can see there are 3rd and 4th level of subdomains on which you can perform your attacks.

3. You will notice some symbols below the subdomain name, click on it and you will have some different results

4. Netcraft :

Netcraft is an Internet monitoring company that monitors uptimes and provides server operating system detection as well as a number of other services. Netcraft has an online search tool that allows users to query its databases for host information.

Netcraft gives you some more information about the websites like NetBlocks, OS name, Site reports which includes site title, site rank, site description and many more things. Netcraft is also a good online tool for recon.

How to use it ?

Go to https://searchdns.netcraft.com/ and enter your target domain with asterisk symbol like *.example.com

Using * (asterisk) symbol will search all the possible results of subdomain that a domain contains.

2. After that you’ll get a following result

3. Here you can see the Netblock, OS, and Site report is available.

NetBlock - A netblock is a range of consecutive IP addresses. Netblocks are sometimes displayed in Classless Inter-Domain Routing (CIDR) notation.

4. Check for the site report by clicking on it

5. Crt.sh

crt.sh is a web interface to a distributed database called the certificate transparency logs. It gives some detailed information about certificates, issuer name, matching identities and many more interesting stuff.

How to use it ?

Go to https://crt.sh/ and enter your target domain and click on search

2. Now you’ll see crt.sh ID, click on it and you’ll get some more interesting information

6. Altdns

Altdns is a DNS recon tool that allows for the discovery of subdomains that conform to patterns. Altdns takes in words that could be present in subdomains under a domain (such as test, dev, staging) as well as takes in a list of subdomains that you know of.

From these two lists that are provided as input to altdns, the tool then generates a massive output of “altered” or “mutated” potential subdomains that could be present. It saves this output so that it can then be used by your favorite DNS brute forcing tool. This is another good tool to enumerate subdomains.

How to use it ?

Go to https://github.com/infosec-au/altdns and clone it
Open the terminal and run this command : altdns -i subdomains.txt -o data_output -w words.txt -r -s results_output.txt

-i : list of subdomains

o : output location for altered domains

w : list of words to alter the subdomains with resolve all altered domains

-r : resolves each generated, permuted subdomain

-s : tells altdns where to save the results of the resolved permuted subdomains

NOTE : Altdns Installation

→ If it gives an error while running it (Error : No Module Named ‘Queue’ )

→ Just perform following steps : (Linux)

i) Go to /usr/local/lib/python3.6/dist-packages/altdns

ii) Open __main.py__ using command : nano __main.py__

iii) Do the changes as show in the image

7. Yougetsignal :

It is a collection of uncomplicated network tools. It helps a lot in gathering information about the target. There are many networking tools provided by Yougetsignal but we are going to use “Reverse IP Domain Check” .

What is Reverse IP Domain Check ?

A reverse IP domain check takes a domain name or IP address pointing to a web server and searches for other sites known to be hosted on that same web server. Data is gathered from search engine results. Basically it will give the list of domains that are hosted on same server.

How to use it ?

Go to https://www.yougetsignal.com/ and select Reverse IP Domain Check

2. Now enter your target domain and look at the results

Now we have enough of domains, subdomains and their subdomains. Let’s find out IP range.

8. IP Range Finder

An IP address is always a set of four numbers like that. Each number can range from 0 to 255. So, the full IP addressing range goes from 0.0. 0.0 to 255.255. 255.255. Finding an IP range in recon is equally important because it tells us that how many unique IP addresses block a particular domain has.

How to find it ?

Go to https://bgp.he.net/ and enter your target IP

2. Click on search and after the result appears click on Whois

After we get the range it is mandatory to perform whois check to ensure that the IP belongs to the same target domain

9. Whois :

A Whois lookup is a way for you to search the public database for information about a specific domain, such as the expiration date, current registrar, registrant information, etc.

How to perform it ?

Go to your linux terminal and type the command whois <IPaddr>

10. Censys.io :

Censys is a public search engine that enables researchers to quickly ask questions about the hosts and networks that compose the Internet.

Censys is another great online tool that will give you juicy information about your target. You can also find censys github tool here : https://github.com/christophetd/censys-subdomain-finder

How to use Censys.io ?

Go to https://censys.io/ and select Search IPv4

2. Enter your target IP in search box and click enter

3. See the results

11. Domain Profiler :

Domain profiler is a tool that uses information from various sources (Whois, DNS, SSL, ASN) to determine what decisions have been made regarding a domain or list of domains.

The best thing about domain profiler is that, that it will give you the details of Email hosting, DNS hosting and Domain registrar of the target website.

How to use it ?

Go to https://github.com/jpf/domain-profiler and download it
Use the following command to run it

Command : ./profile <targetName>

12. ViewDNS :

ViewDNS is an online tool to check DNS information and it is equally useful in many things. We will be using it for two main purpose :

Reverse Mail Exchange Lookup (Reverse MX Lookup)
IP History

Use of Reverse MX Lookup :

It takes a mail server (e.g. mail.google.com) and quickly shows all other domains that use the same mail server. Useful for identifying domains that are used as email aliases.

Use of IP History :

We will be using this feature to know how active our target website is because using IP History will show you the last seen of the particular IP address.

How to use ViewDNS for Reverse MX Lookup ?

Go to https://viewdns.info/ and you’ll find many options

2. Enter your target IP or name in Reverse MX Lookup

3. Click on go and see the results

How to use ViewDNS for IP History ?

Go to https://viewdns.info/
Enter your target IP or name in IP History

3. Click on go and see the results

Now we have the list of domains, subdomains, subdomains of subdomains, IP range, DNS information, domain registrar information etc. Now what to do for further recon ? Let’s see a new thing that this domains and subdomains communicates to which other external domains. VirusTotal will help in finding this this out.

13. VirusTotal :

VirusTotal is an online service that analyzes files and URLs enabling the detection of viruses, worms, trojans and other kinds of malicious content using antivirus engines and website scanners.

But it also helps us to detect about the details and other communication links of the websites. So we will be using it for checking communication links

How to use it ?

Go to https://www.virustotal.com/gui/home/search and search for your target domain

2. After the search you’ll get 4 options Detection, Details, Links and Community, click on Links and it will show you all the outgoing links of your target website

Outgoing Links - An outgoing link, also called an external link, is a link from your website to a different website. An outbound link for you is an inbound link for someone else. If a link takes readers to another page on the same website, it’s called an internal link.

Let’s find some API keys and some third party application used by the target. We will be using Photon tool for that.

14. Photon :

Photon is basically a data extraction tool which helps us find many useful information from the websites.

How to use it ?

Download it from https://github.com/s0md3v/Photon and install all the requirements as described in git repository
Go to terminal and run the following command

Command : python3 photon.py -u “<yourTargetName>” --keys --dns

3. Go to the created directory, its name will be yourtarget.com

4. Open yourtarget.com.png file and you’ll be able to see all the third party application that your target communicates with

5. Open the keys.txt file

NOTE : Photon Installation

→ If it gives an error while running (Error : No Module Named ‘tld’)

→ Just perform following steps : (Linux)

i) curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

ii) python3 get-pip.py --force-reinstall

iii) sudo pip3 install tld

So that’s all with photon, now let’s find out some js files and some more endpoints from those js files from the websites using LinkFinder.

15. LinkFinder :

LinkFinder is a python script written to discover endpoints and their parameters in JavaScript files. This way penetration testers and bug hunters are able to gather new, hidden endpoints on the websites they are testing. Resulting in new testing ground, possibility containing new vulnerabilities. It does so by using jsbeautifier for python in combination with a fairly large regular expression.

How to use it ?

Go to https://github.com/GerbenJavado/LinkFinder and download it, then complete all the requirements mentioned in the git repository
Use the following command to run it

Command : ./linkfinder.py -i https://yourTarget.com -o cli

-i : input your target name with http or https protocol
-o : will show you the output, cli means it will be shown on your command line

You can also find endpoint from js files by simply passing the js URLs obtained by the LinkFinder

Finding endpoint from JS files - LinkFinder

Now after we have the list of js files we can find the vulnerable JavasScript libraries using Retire.js

16. Retire.js :

Retire.js is a burp suite plugin to find vulnerable JavaScript libraries. It passively look at JavaScript files loaded and identify those vulnerable based on various signature types (URL, filename, file content or specific hash).

You’ll find this plugin under Extender → BApp Store in burp suite

How to use it ?

Install the plugin in the burp suite as shown above
Visit the target website or the JS file endpoint obtained using LinkFinder and Retire.js will automatically do a passive scan for you

NOTE : Here in the above screenshot you will see Results tab but in new burp version it will be Issue activity tab.

17. Dig to check for CNAME (For Subdomain Takeover) :

Dig stands for (Domain Information Groper) is a network administration command-line tool for querying Domain Name System (DNS) name servers. We will be using this command to check if we could takeover any subdomain or not.

CNAME is a Canonical Name record is a type of resource record in the Domain Name System which maps one domain name to another.

How to identify it is vulnerable ?

Domain name (e.g. sub.example.com) uses a CNAME record to another domain (e.g., sub.example.com CNAME anotherdomain.com).
At some point in time, anotherdomain.com expires and is available for registration by anyone.
Since the CNAME record is not deleted from example.com DNS zone, anyone who registers anotherdomain.com has full control over sub.example.com until the DNS record is present.

How to use dig command ?

It is really easy to use

Go to terminal and type dig yourTarget.com

Here if your target.com.edgekey.net expires after some point of time and it is still their in CNAME, then yourTarget.com is prone to subdomain takeover. You just need to register yourself at target.com.edgekey.net (Domain in CNAME).

18. Ping to check OS :

Ping is a computer network administration software utility used to test the host is live or not on an Internet Protocol network. You can also identify which OS is being used by your target by using ping command.

How to use it ?

Go to your terminal and type ping yourTarget.com

Here you can see ttl, which means time to live. TTL helps us in identifying which OS our target has.

ttl=128 or ttl=127 or ttl=126 : Windows OS

ttl=63 or ttl=64 or ttl=65 : Linux OS

19. Nmap :

Nmap is used to discover hosts and services on a computer network by sending packets and analyzing the responses. We will be using nmap for port scanning of our target.

How to use it ?

Go to your terminal and run the following command for more informative scan

Command : nmap -sC -sV -T4 -A yourTargetIP

-sC : equivalent to --script=default

-sV : Probe open ports to determine service/version info

-T4 : Sets timing template (0–5) (higher is faster)

-A : enable OS detection, version detection, script scanning, and traceroute

20. DirSearch :

Now we will be searching for the directories present on your target using dirsearch.py a github tool. We also have dirb/dirbuster/gobuster tools pre-installed on kali system but I like dirsearch.py so I’ll be using it.

How to use it ?

Go to https://github.com/maurosoria/dirsearch and download it
Use the following command to run it

Command : cd dirsearch

./dirsearch.py --url <yourTargerURL> -w <wordlist> -e <extension>

After directory search we will be using wappalyzer and whatweb to find out which technologies our target website is using and what are the request headers.

Wappalyzer is a similar tool to WhatWeb in that it also performs analysis on the HTTP response in order to determine technologies in use. The results from the analysis tend to be not as verbose and it does not dig into the version detection to the same degree as WhatWeb.

21. Wappalyzer :

Wappalyzer is a browser extension that uncovers the technologies used on websites. It detects content management systems, eCommerce platforms, web servers, JavaScript frameworks, analytics tools and many more.

How to use it ?

You can add it on your firefox from here : https://addons.mozilla.org/en-US/firefox/addon/wappalyzer/

2. It is really simple to use, just add the extension to your browser and visit the target website

3. It will automatically detect everything, just click on the symbol to see the results

22. Whatweb :

WhatWeb identifies websites. WhatWeb recognizes web technologies including content management systems (CMS), blogging platforms, statistic/analytics packages, JavaScript libraries, web servers, and embedded devices. WhatWeb has over 1700 plugins, each to recognize something different. WhatWeb also identifies version numbers, email addresses, account IDs, web framework modules, SQL errors, and more. It also detects all the request headers.

How to use it ?

It comes pre-installed in kali linux
Open your terminal and type the following command

Command : whatweb <yourTargetname>

As of now we have many information, but I need some more so let’s dive more deeper into recon. Let’s find out some hidden parameters on target website.

23. Arjun :

Arjun is a great tool for finding hidden parameters on a website. It has its own parameter payload list which contains 25890 parameters.

How to use it ?

Go to https://github.com/s0md3v/Arjun and download it
Run it using following command

Command : python3 arjun.py -u https://<yourTarget.com>

2. Check the results

24. S3 Bucket Recon :

An Amazon S3 bucket is a public cloud storage resource available in Amazon Web Services’ (AWS) Simple Storage Service (S3), an object storage offering. Amazon S3 buckets, which are similar to file folders, store objects, which consist of data and its descriptive metadata. Many buckets leak API keys which is an information disclosure.

How to use it ?

Go to https://github.com/clarketm/s3recon and download it, complete all the mentioned requirements
Now run the following command to find the buckets

Command : s3recon “<wordlist.txt>” -o “results.json” --public

wordlist.txt : your target names

-o : for output in JSON format

--public : will find all the publicly available buckets

3. After it runs you’ll see the following on the screen

4. Open any link

NOTE : Opening any private bucket URL will give you Access Denied

Now I would like to show you an awesome tool that helps in Recon & OSINT, Scanning & Enumeration, Vulnerability Analysis, Exploitation, Auxiliary Modules. It is one of my favorite tool and I would suggest you guys to use it atleast once. I’m sure you guys will love this tool called TiDos.

25. TiDos :

TiDos is an offensive web application framework with lot of modules. It helps in many penetration testing task from performing recon to attacking a web application. It is built in python and is fully automated tool.

How to use it ?

Go to https://github.com/0xInfection/TIDoS-Framework and download it, complete all the mentioned requirements
Go to your terminal and run the following command to run it

Command : ./tidos.py

The framework will start automatically and you will have the following screen