Recently a member from the Trustwave SpiderLabs team created an nmap NSE script that could be used to take a screenshot of webpages as it scanned the network. Working for a top 10 accounting firm, I conduct a lot of internal penetration tests for clients that operate on very large networks, and sometimes I’m required to audit entire counties. Having the ability to view all the webpages on the internal network without being required to manually type in each addresses into the browser sounded amazing. This was very exciting news now that there was a way to automate this process and have the ability to scale. I dove in right away to get started by installing the script based on the instructions in the link listed below:
http://blog.spiderlabs.com/2012/06/using-nmap-to-screenshot-web-services.html
I highly suggest you look over the article above as I wrote this article in hopes that it would help assist anyone when having issues getting the http-screenshot NSE script to function properly with the latest version of nmap.
I encountered some adversity while following the tutorial since I wasn’t running BT5. I was running Ubuntu 10.04.04 amd64 and had just installed the latest version of nmap (6.02) via svn and compiled from source.
The first problem with the instructions was the version of wkhtmltoimage-i386 didn’t work on my system, most likely due to my x64 architecture. Be sure to test your version of wkhtmltoimage before continuing. The command should display a help menu with options when ran without any arguments.
Not working:
root@lockout:~# file ./wkhtmltoimage-i386 ./wkhtmltoimage-i386: ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, stripped root@lockout:~# ./wkhtmltoimage-i386 root@lockout:~#
Make sure you download the correct version that will work with your OS, and test it before continuing. wkhtmtoimage downloads can be found here.
In my case I tweak the tutorial to this:
wget https://wkhtmltopdf.googlecode.com/files/wkhtmltoimage-0.11.0_rc1-static-amd64.tar.bz2
tar -jxvf wkhtmltoimage-0.11.0_rc1-static-amd64.tar.bz2
cp wkhtmltoimage-amd64 /usr/local/bin/
This was just the beginning of my problems as I attempted to get this NSE script to function properly. I continued to follow the instructions like the tutorial had listed
git clone git://github.com/SpiderLabs/Nmap-Tools.git
cd Nmap-Tools/NSE/
cp http-screenshot.nse /usr/local/share/nmap/scripts/
nmap –script-updatedb
The problems started when I tried to update my nmap database that keeps track of the NSE scripts. Here is what the output looked like on my initial run:
root@lockout:~/Nmap-Tools/NSE# nmap --script-updatedb Starting Nmap 6.02 ( http://nmap.org ) at 2012-07-11 23:36 CDT NSE: Updating rule database. NSE: Failed to load /usr/local/bin/../share/nmap/scripts//http-screenshot.nse: /usr/local/bin/../share/nmap/scripts//http-screenshot.nse:30: variable 'shortport' is not declared stack traceback: [C]: in function 'error' /usr/local/bin/../share/nmap/nselib/strict.lua:80: in function '__index' /usr/local/bin/../share/nmap/scripts//http-screenshot.nse:30: in function </usr/local/bin/../share/nmap/scripts//http-screenshot.nse:1> NSE: failed to initialize the script engine: /usr/local/bin/../share/nmap/nse_main.lua:460: could not load script stack traceback: [C]: in function 'error' /usr/local/bin/../share/nmap/nse_main.lua:460: in function 'new' /usr/local/bin/../share/nmap/nse_main.lua:1113: in main chunk [C]: in ? QUITTING! root@lockout:~/Nmap-Tools/NSE#
The first problem that I noticed right away was “variable ‘shortport’ is not declared on live 5 above”. I opened up the script to try and make some changes to the http-screenshot NSE script.
On line #28 you should see the line below:
require "shortport"
replace this line with the following to declare the variable.
local shortport = require "shortport"
Once you have saved your changes, lets try to update the nmap database. If everything went successfully, you should have some output like below that shows the database updated. If you still have issues related to variables, place the string ‘local’ in front of the variable causing issues, and that should fix your problem.
root@lockout:~/Nmap-Tools/NSE# nmap --script-updatedb Starting Nmap 6.02 ( http://nmap.org ) at 2012-07-11 23:43 CDT NSE: Updating rule database. NSE: Script Database updated successfully. Nmap done: 0 IP addresses (0 hosts up) scanned in 1.04 seconds root@lockout:~/Nmap-Tools/NSE#
Before attempting to run the script, we will need to update a few items to make the http-screenshot NSE script to run properly. The first item we needed to modify was the line that called wkhtmltoimage-i386 to wkhtmltoimage-amd64 since we downloaded and installed a different version above.
On line #48 you should see the line below:
local cmd = "wkhtmltoimage-i386 -n " .. prefix .. "://" .. host.ip .. ":" .. port.number .. " " .. filename .. " 2> null >null"
replace the line above with the following to changes listed below. We first want to change the quality that wkhtmtoimage uses to create screenshots of the webpage. This will save disk space on your hdd and speed up the downloading of screenshots as well. In my testing, I noticed that a quality of 20 (default 94) was not very noticeable to the eye, but decreased the size of the screenshots significantly.
local cmd = "wkhtmltoimage-amd64 --quality 20 -n " .. prefix .. "://" .. host.ip .. ":" .. port.number .. " " .. filename .. " 2> null >null"
Make sure you update the nmap database everytime you make a change to the NSE script.
root@lockout:~/Nmap-Tools/NSE# nmap --script-updatedb
At this point I thought I was smooth sailing now that nmap was scanning properly, and I wasn’t getting any errors. Well, I was wrong. Nmap would continually hang when scanning the network, and specifically when running the NSE portion of the namp scan. It was getting hung on the http-screenshot NSE script specifically, and I needed to find out what line of code was causing the issue.
http-screenshot.nse hanging
... Completed SYN Stealth Scan against 192.168.51.14 in 45.47s (10 hosts left) Completed SYN Stealth Scan against 192.168.51.12 in 45.50s (9 hosts left) Completed SYN Stealth Scan against 192.168.51.0 in 45.51s (8 hosts left) Completed SYN Stealth Scan against 192.168.51.5 in 45.51s (7 hosts left) Completed SYN Stealth Scan against 192.168.51.22 in 45.53s (6 hosts left) Completed SYN Stealth Scan against 192.168.51.2 in 45.53s (5 hosts left) Completed SYN Stealth Scan against 192.168.51.1 in 45.55s (4 hosts left) Completed SYN Stealth Scan against 192.168.51.3 in 45.55s (3 hosts left) Completed SYN Stealth Scan against 192.168.51.15 in 45.55s (2 hosts left) Completed SYN Stealth Scan against 192.168.51.19 in 45.61s (1 host left) Completed SYN Stealth Scan at 23:49, 47.00s elapsed (38000 total ports) NSE: Script scanning 38 hosts. Initiating NSE at 23:49
I noticed that the script called the binary wkhtmltoimage-amd64, but never specified a timeout if the program never returned a result in a timely fashion. I started looking for options in wkhtmtoimage to see if a timeout value could be specifed that would give up after set amount of time. That was when I stumbled up this article here that essential states the pre packaged binary that I downloaded did not have a timeout by default.
I knew linux had a ‘timeout’ package that I could download right from the Ubuntu repositories. Since the NSE script was making a system call, I could install the ‘timeout’ package and use it directly in the NSE script.
apt-get install timeout -y
Now that I have the timeout utility installed on my OS, I was ready to modify the script for one last time.
local cmd = "wkhtmltoimage-amd64 --quality 20 -n " .. prefix .. "://" .. host.ip .. ":" .. port.number .. " " .. filename .. " 2> null >null"
local cmd = "timeout 20 wkhtmltoimage-amd64 --quality 20 -n " .. prefix .. "://" .. host.ip .. ":" .. port.number .. " " .. filename .. " 2> null >null"
Make sure that when you run the http-screenshot.nse script that you utilize the -sV flag in nmap to properly enumerate http vs https services.
root@lockout:~/Nmap-Tools/NSE# nmap -sV -p 80,443 --script=http-screenshot google.com
If your nmap output continues to tell you “failed (verify wkhtmltoimage-i386 is in your path)” but the script is still output images to your current working directory, this is because there is a flaw in the programs logic. The code below will always make result = “failed (verify wkhtmltoimage-i386 is in your path)” because ret is actually a boolean value and not a string or decimal. This means that the ‘ret’ variable will never == 0 and therefore never give you the output you wish to see such as “Saved to screenshot-nmap-74.125.225.7:443.png”
-- If the command was successful, print the saved message, otherwise print the fail message local result = "failed (verify wkhtmltoimage-i386 is in your path)" if ret == 0 then result = "Saved to " .. filename end
In order to make the code function how it was intented we will change the code to the following:
-- If the command was successful, print the saved message, otherwise print the fail message local result = "failed (verify wkhtmltoimage-i386 is in your path)" if ret then result = "Saved to " .. filename end
I tested these changes and it helped my performance significantly. Nmap was scanning entire class C networks in a timely fashion and I was no longer getting hung on the NSE scripts!
Hope this helps some of you out there who may have had issues getting the http-screenshot NSE script to run properly initially.
The post Using Nmap to Screenshot Web Services Troubleshooting appeared first on Pentest Geek - Penetration Testing - Infosec Professionals.