CONTENT DISCOVERY

Content discovery within the context of web application security is the act of finding content such as files, pages and features that aren't obviously apparent or aren't intended to be publicly available. Content discovery can be achieved via manual discovery, open source intelligence (OSINT) or by using automated tools. 

Manual Discovery is a method of finding content by exploring a web app manually. Some examples of places to look manually are:

  • Robots.txt
  • Sitemap.xml
  • HTML Headers
Robots.txt is a file that web applications have to specify which parts of the website should be included or excluded as part of search engine results. It also Specifies which search engines are allowed to show results for their website. This can be a useful file as a penetration tester to have access to as it can provide a list of pages that were not intended to be viewed.

In the below example, the robots.txt file showed a "/staff-portal" page which could be of interest.



Sitemap.xml is a file that displays all the essential pages to ensure search engines can crawl them and show them in their search results. It can be used as a penetration tester to access older or harder to reach parts of a web app.

HTML Headers Can be viewed by using curl http://[IP] -v, this will display the source code of a site and any HTML headers which can contain meta data, such as webserver software and any back end programming language. 
Favicon is the small icon next to the URL in a browsers address bar. If the developers of the site have not edited it, it may still be the favicon from the sites framework. This can be checked using curl http://[adress of favicon] | md5sum and the resulting output checked against the OWASP favicon database. With the information about the framework the web app is built on, it may open up new avenues for exploitation. 


Open Source Intelligence (OSINT) is a way of gathering information that is publicly available. Some examples of avenues to explore when looking for content are:
  • Google hacking/dorking - Using google operators to help search for content that may be otherwise difficult to find.
  • Wappalyzer - is an online tool for identifying what technologies were used to build a site.
  • Wayback Machine - Is a site that archives other websites historical versions.
  • GitHub - A version control system which allows users to share updates to software. It can be useful for finding source code or information that may not be intended to be publicly available.
  • S3 Buckets - A storgae device provided by Amazon that may be left public.
Automated Tools such as dirb, gobuster and ffuf are tools that are designed to quickly test a website for pages according to a wordlist provided by the user. If there are matches for these pages within a web app, the user will be told which matches have been made. It can be useful for quickly searching for common pages that aren't necessarily intended for public use.





Comments

Post a Comment

Popular posts from this blog

BURPSUITE IN-DEPTH

Image
NOTE: This post will be a much deeper and in-depth dive in to some of the functionality of Burp Suite. If you need a quicker overview, please visit my  Burp Suite Basics  post.  Repeater The repeater function of Burp Suite allows us to take a request captured by Proxy, edit it and send the same request repetitively. This makes repeater ideal for any manual poking around at an endpoint. Basic Usage: The repeater interface can be split in the 6 main sections. Refer to the image above: List of requests that are going through Repeater. Controls for the current request, these allow us to send/cancel a hanging request and go forwards/backwards in a request history. Request/Response view. Requests are edited in the Request view and Responses will show up in the Response view. Allows us to change the layout for the Request and Response view. The inspector allows us to break requests apart and analyse and edit them in a more intuitive way than the raw editor. The IP address o...
Read more

CROSS-SITE SCRIPTING

Image
Cross-Site Scripting (XSS) is an injection attack where malicious JavaScript gets injected into a web application with the intention of being executed by other users.  XSS Payloads In XSS the payload is the JavaScript code we wish to be executed on the targets machine. The payload is comprised of two parts, the intention and the modification. The intention is what the attacker intends the JavaScript to execute and the modification is the changes the attacker needs to do on the code to make it execute in each specific scenario.  Proof of Concept This is a simple payload in order to prove that an attacker can actually achieve XSS on a website: <script>alert('XSS');</script> Session Stealing Details of a users session, such as log in tokens are often kept in cookies on the targets machine. The below JavaScript takes the target's cookie, base64 encodes it to ensure successful transmission and then posts it to a website under the attackers control <script>fet...
Read more

NMAP ADVANCED SCANS

Image
In this post we will be covering techniques to evade firewalls and IDS systems ad well as more advanced scans and scan options, such as: Null Scan FIN Scan Xmas Scan Maimon Scan Ack Scan Window Scan Custom Scan We will also cover the following: Spoofing IP Spoofing MAC Decoy Scan Fragmented Packets Idle/Zombie Scan Null, FIN and XMAS Scans These scans can be efficient when scanning a target behind a stateless (non-stateful) firewall. A stateless firewall will check if the incoming packet has the SYN flag to detect a connecting attempt. Using a flag combination that does not match the SYN packets could make it possible to deceive a firewall and reach the system behind it. However, a stateful firewall will practically block all such crafted packed and render this kind of scan useless. Null Scan Null scan does not set any flag at all; all six flag bits are set to zero. We can use this option with -sN . A TCP Packet with no flags set will not trigger a response when it ...
Read more