Getting started with "The Sleuth Kit" pt. 2 + Volumes & Partitions

Intro

My last post involved mostly the installation and imaging process one would first need to get out of the way before using either TSK or Autopsy. In this post, I will introduce the concepts of hard drive storage and file system data structures, then begin showing a few TSK tools. Since it's mostly looked down upon to be a tool monkey and only know how to manage a forensic tool, without actually knowing the details of the actual evidence itself, i find it necessary to talk more than just about the tools. I understand this causes the title of these posts to be somewhat misleading, but I will eventually focus directly on the tools after i get all the relating information out of the way. I'm not used to blogging, and i'm coming to realize the importance of beforehand planning.

Formatting Hard Drives

Before a filesystem can be even put onto a hard drive, the drive needs to go through a process called low level formatting. This is a process that marks out the cylinders and tracks and then divides those tracks into multiple sectors. These marks on the surfaces of the platters are then the addressable blocks that exist before MBR partitioning or GPT partitioning are applied.

A sector is a unit of bytes (512 bytes to be exact), that acts as a container for holding your data, and a cluster is a container for sectors. Sectors are assigned an address that make locating a sector via the bios or drive controller even possible. There are two addressing schemes used; the original method is the cylinder/head/sector (CHS) addressing scheme, and the more modern logical block addressing scheme (LBA). MBR used to have an 8 gb drive limitation with CHS, but then started employing LBA, to obtain the 2 tb limit. But don't focus too much on MBR, because GPT is used more so anyways, but we'll get more into theses addressing methods later on. So clusters are made up of a collection of sectors, and cluster sizes, depending on the operating system you're using, range in sizes from 512 bytes to 64 kb's. 64 kb's is equal to 64000 bytes, which can contain 125 sectors. Inside these sectors and clusters, is where your files are contained. The magnetic material on the surface of these drives are positioned in a particular physical state to represent either on or off.  These ones and zeros are what are used to make up all your files, folders and executables on your hard drive.

Sectors are sequential, but clusters can be non-contiguous. A file that has multiple clusters, can have its cluster scattered all over a drive, and their locations are retained with the power of FAT or MFT in rtfs. We'll get into those details a couple blog posts from now :^).

Then after the low level formatting of a drive, then comes the partitioning of the drive. Examples of these partitioning mechanisms include the MBR (Master Boot Record) partitioning scheme, (which you'll see with older older more smaller drives. Then there is the more modern partitioning scheme called GPT (GUID Partition Table), which is a partitioning method designed to work with larger storage devices). Here are two invaluable images depicting both from microsoft.com:

Figure 2.1a

tech net.microsoft.com

technet.microsoft.com

The MBR is located at the first physical sector of a hard drive, and contains the boot code, four partition table entries, and a signature value. The MBR is the only place that holds the partition information. In the beginning part of the MBR is the "boot sector", or boot code. This part of the MBR is looked at by the BIOS, to see which partition on your hard drive, contains your primary operating system to boot up with. The rest of the MBR contains 4 entries, and these entries are like little book marks that contain information about the possible four partitions that you may have on your hard drive. I say "may" because you could design you hard drive to only have one partition with one operating system inside of it, which is what most people end up doing. I'll get into what a partitions and volumes are later on. For now, back to these four entries. These entries, (each being 16 bytes long), keep a record of the first and starting sectors of each partition on your drive. These are indicators that are provided when the hard rive says, "hey how many and where are your partitions located." MBR partitions have a size limitation of up to 2 TB's of storage, meaning your drive can only go up to size of about 2 TB's for each of the 4 primary partitions on the drive. 

Down below in figure 2.2, is a carved out sector of my MBR. The imaged copy that i am using as my dd image which is a 298GB NTFS drive, is an MBR partitioned drive, not a GPT partition drive. (It's old i guess). You can find out on your windows system by typing DISKPART in your cmd, the type LIST DISK in your new cmd window. If you see under Gpt an empty space below it, that means you don't have one. You can also see that here in figure 2.1b

Figure 2.1b

Figure 2.2

My MBR carved with dd and the xxd tool;
dd if= ntfscopy.img.001 bs=512 skip=0 count=1 | xxd

The yellow is 446 bytes of the boot code. The Green is my first partition entry, followed by the locations of the slots for the rest of the possible partition "entries", (remember you can have up to four possible partitions), and as you can see, i have two partitions on my drive. Also remember, this is MBR is before my GPT partition table, because this is an image of an up to date NTFS drive, that is partitioned with GPT. The number 80 in the beginning of the green table is where the flag indicator tell be where to boot from, 80 boot flag, means this is your bootable partition. The 07 you can see in the green bar is an indicator for an NTFS partition. The last two bytes 55 AA, are the boot record signature. All this is in Carriers FSFA book. I'm sorry for the brevity, but i must move on.

GPT supports up to 128 partitions on a drive, so there is not 4 partition limitation, nor is there the need for an extended partition like there is in MBR. In MBR, the location where the partition sizes are stored are only 4 byte long indicators (32 bits), located in the MBR partition headers. This is what causes the drive limitation of 2 TB's. GPT uses 64 bits for storing logical block addresses in the GPT header, allowing them to have many more sectors on a drive, up to 9 petabytes i think. These directory entries, (like in MBR), exist inside the partition table. Directory entries gives a description of each of the partitions on disk. At LBA0 is ab MBR partition, which is used for backward compatibility reasons, and then there is a GPT header at LBA1, which gives a general layout of the disk volume itself, and contains pointers to where certain information is located on the drive. At the end of the drive is a backup partition that contains a backup of the header and partition entries. 


Figure 2.3

wikipedia.org

Partitions & Volumes

What is a Partition is? What is a Volume?.

What are Active primary partitions, primary partitions, logical partitions, extended partitions? :^)

Active Primary Partition:
Primary Partitions:
Logical Partitions:
Extended Partitions:

When someone says "volume", (also called a logical drive), they are referring to any accessible storage areas with a hard drive. If your entire drive is just one large partition that encompasses the entire physical drive, well than that whole area of storage is also considered a volume. When you take a drive and start partitioning it, each partition IS also considered a volume. The reason why the two general aren't identical is because a partition is a data structure that is designed to house an entire operating system. While this partition is ALSO a volume, inside this partition, multiple VOLUMES segment into multiple specific storage facilities.

You could design your drive to have one partition, while inside this partition, you have a volume to hold the OS data, and another volume to hold user data. Or, you could install more than one file system on a drive, one OS for each partition. Primary partitions are what hold your bootable OS, and extended partitions are used for things like storing a VM, since a VM file system can be a dynamic virtual file system. I know that was confusing. Sorry, gotta move on.

So you can have one partition, and two hard drives. When you combine the three, by creating a partition that spans two drives, this is considered a logical partition. A good way to remind yourself of how a volume or logical volume exists, just think of how RAID works.

You'll also see the abstraction of logical and physical not just with hard drives, but you'll also see this abstraction in memory storage with page files. This is when your physical memory is just too small, so it creates a page file storage space on your hard drive, so now the CPU thinks it's looking at a memory card that is larger than it actually is.

Figure 2.4

(Brian Carrier, File system forensic analysis.)

Now when you're working with Windows, under disk utility you'll see some drive letters. Usually there are three drive letters by default on a windows hard drive, and these drive letters are also the partitions that you have inside your drive. For example, drive letters like C: is usually your primary drive, and if you have any extended partitions in your drive they will be labeled from D: to Z:. So if you created a new partition on your drive, it will be assigned the next available letter between D to Z.

Lets get to it

Navigate to your image

My image is a dd copy of an NTFS hard drive. Size is 298gb. I'm working in a VM of Ubuntu where my TSK tools are located. The dd image is contained on my external hard drive, so in or to run my tools against it, i need to change my directory with cd /media to the directory where my image is located. When you want to know the path to a files location before using this command, all you have to do is drag and drop the file into the terminal and voila. Terminal will tell you. (As long as the external hard drive where your image is located, is mounted in your VM).

The location of my image is: /media/digitalresidue/My Passport/1NTFS image/ntfscopy.img.001. 

My Passport - is the name of my ext drive
1NTFS image - is the name of my folder
ntfscopy.img.oo1 - is the name of my dd image

I want to be in the parent directory where my image is located. The parent directory is 1NTFS image, and to get there i type: cd /media/digitalresidue/My\ Passport/1NTFS\ image. 

Notice that i had to enter a backslash "\" in between the words that had a space. I'm not sure the reasoning, but if you're trying to navigate to a directory, if there are words that have a space, you need to insert a backlash there.

Now run a dir and you will see where my .img is located.

Figure 2.5

Here's a simple sudo fdisk -l, a native command that is comparable to TSK's mmls:

Figure 2.6

The asterisk * next to ntfscopy.img001p1 is where your partition of "focus" starts, at sector 2048. The reason why your partition table starts at 2048, is because the drive itself was partitioned this way, with something like older method of partitioning with something like fdisk will start you at 2048, and this is the default starting sector location where your ACTIVE partition begins. Normally you will see your partition start at sector 63, but in this example, it simply isn't the case

1) The BIOS loads a bootstrap program that loads the MBR (located at physical sector 0), into memory.
2) The MBR then looks to the partition table to see which partition on disk is the primary active partition to boot into, (which will be at sector 0 of the volume boot sector of said partition).
3) Your MBR loads you into that active partition.
4) You are now located in a GUI of where your Operating System is located.

In figure 2.6 you'll see ntfscopy.img.001p1 partition starts at sector 2048 and ends at 206847. In Windows 7 filesystem, this p1 partition is your System Reserved, which technically is also your "Active Primary Partition", which includes your systems boot loader, along with other configuration files, and has your system jump to the bootable partition, which in this case, is ntfscopy.img.001p2. This partition is your "Primary Partition", that contains your formatted NTFS partitioned filesystem. Before sector 2048 is empty space, and also where your MBR and boot code is located.


***As a cautionary side note to the material I am presenting, there may be many places where i am saying incorrect things, so please just leave corrections in the comments section below :). After the p1 active primary partition, there is the partition 2 'p2', that you can see what they represent, at the top of this blog post under "corrections".


Here is a list including some of the main commands in TSK without their associated switches. Some of these commands listed here are not included in Carriers book FSFA, so we won't really touch on what isn't discussed in his book:

Figure 2.7

wiki.sleuthkit.org

Each one of these commands is assigned to a layer of abstraction. TSK simplifies their identification by giving them a prefix that are initialed for each layer.

Each tool name has two parts, where the first part identifies its group and the second part identifies its function. For example, fls is a file name category tool (the f) that lists (the ls), and the istat tool is in the metadata category (the i) that displays statistics (the stat). (Carrier, 2005. p.g 377).

1. mm - tools that operate at the media management layer.
1. mmls
2. fs - tools that operate on the file system layer.
1. fsstat 
3. blk - tools that operate at the "block" or data unit layer.
1. dcat
2. dls
3. stat
4. dcalc
4. i - tools that work at the anode layer. Also known as the metadata layer.
1. icat
2. ifind
3. ils
4. stat
5. f - file name layer.
1. ffind 
2. fls

There is a list of other tools that come bundled with TSK. It's hard for me to figure out a better way to organize these tools, plus, these are the main tools we will be focusing on anyways.

Forensic layers of abstraction and their respective tools:

1) At the lowest level there is the Disk layer. Also known as the physical layer. We acquire our images at the disk level with something like dd. A potential place of interest for an analyst would be the HPA or DCO slack spaces of a drive. Something like diskstat can analyze for potential HPA hiding, but since i used FTK to image my drive, i can't show an example output of this. FTK imager allows you to use the dd tool as an imaging option, and unfortunately this method of imaging will not include the HPA of your drive if one does exist. In order to image the HPA area of a hard drive with FTK or dd, you will need to reformat and reset the maximum addressable sectors the hard drive. I think there are some tools that allow you to image HPA without reformatting; such X-ways replica.
A native command in linux will tell you if you even have an HPA area, hdparm -N, against your image.

2) The next layer is the Volume layer. This is an area of abstraction that exists between the physical drive itself, and the filesystem layer. mmstat ran against your image will tell you what partition type you're working with, in my case, it's a dos based disk.
mmls is one of the first go to tools an investigator turns to when running TSK. This command carves out your partitions, regardless of where they are located on disk, and gives you the starting and ending sector details. Here will be my command: mmls -t dos tfscopy.img.001. (-t is a switch that is followed by which partitioned drive you have, in my case, is dos.)

Figure 2.8











As you can see in the blue circles, are rows of information that tells you where your partitions are located, their size, their beginning and ending sectors, and their offsets. My image has A primary partition, from slot 00 to 01, which is the primary table #0, our MBR. Then from there we have unallocated space until our next partition, which is our NTFS partition. Our NTFS partition offset starts at 2048, then the volume ends at 206847, then another volume begins at 206848. 2048 to 206847 is our reserved partition in NTFS, and is home to many boot files. The second volume is simply our file system data volume :^).

Getting started with "The Sleuth Kit" pt 1 + dd imaging

This post will be an overview involving some of the capabilities a user can except when using "The Sleuth Kit", a forensic library of C tools accessible through the CLI, and "Autopsy", the GUI version of TSK, all brought to you by a Mr. Brian Carrier. The approach Mr. Carrier takes in his book, "File system forensic analysis," is a layered approach that looks at a file system from a couple different perspectives. This abstraction simply helps an investigator to organize a computer system as a layered construction, and discussing the possible artifacts that can be obtained from each layer.

I've downloaded TSK into Ubuntu using commands (sudo apt-get install sleuthkit), and created my forensic images using the dd utility, which is a native command to the linux OS.  TSK analyzes and supports raw data images from Encases .E01 and raw (dd) files disk images, generated with linux's dd command or FTK's imager. TSK and Autopsy are used for analyzing raw images, not creating them, so we're going to have to use dd to image a drives partition. Whereas something like dc3dd, which is a third party utility in Linux, also creates raw data files, but are not compatible with TSK. If you want to try out dc3dd, you will have to type sudo apt-get install dc3dd.

Commands like dd and dc3dd can be combined with switches like log=, skip=, hash=, but PRIMARILY, dd and dc3dd are used with the operands if= of= before using anything else. (if=<the device you want to image> of=<where you want the imaged data to end up>). This arrangement is very important. If you accidentally swap these to, you could mistakenly be over writing data where you wanted data to be copied from. This concern is more relative to when you are "cloning" a drive. Cloning is the process of copying a hard drives data and copying this information to another hard drive. While imaging involves the process of copying a hard drives data and saving it as copied image file. Both processes create an exact record of your drive or partition as (raw images). Raw images include allocated and unallocated files, swap spaces, host protected area HPA and device configuration overlay DCO, along with the other data structures that make up the filesystem. dd and dc3dd creates these raw images.

Here is how you would use dd; sudo dd if=<source> of=<target> bs=<byte size> 

Source is the drive you want to image. Target is where you want the image copy to be sent to . And byte size option should be 512 bytes, if you leave bs= out of the equation, dd should assign blocks as 512 bytes by default. 

You can pipe your results with the "|" command, b giving your dd results to a remotely connected machine using nectat, for example: sudo dd if=<source> of=<target> bs=<byte size> | netcat < targethost-IP > 1234. 

But if you wanted to send your dd .img results to the local desktop you will use; dd if=/dev/sda of=/home/desktop/test.img bs=<byte size>.

So how do i find out the path names of my devices to fill in the if= of= operands? What is my target, and what is my source? For this information, all you'll need to do is just run sudo fdisk -l 


















So as you can see, i have a 64gb USB drive, and a 1 tb drive concede to my workstation. The red boxes signify the primary path to each device, its home directory so to speak, and in the green boxes are the primary partitions that exist for each drive.

dd is capable of imaging the entire drive, or if time is of the essence, image specific locations of the drive, or just a drives main partition. You will have more forensic value of a full disk image, but if you simply don't have the space or time, just image a single primary partition. To image a full drive to a target drive you include the root path name of the device. For example: my picture above shows that /dev/sdc is the root path location for the 64 gb USB drive, whereas /dev/sdc2 is where the primary partition table is located. So, as you can see above, the red boxes indicate the root path name of the devices file location, whereas in the green boxes are partitions inside that devices location. For this exercise i will be interested in imaging the full USB drive, since space and time is not a concern of mine, and imaging the boot sector offers more assisted evidence to our investigation.

So you would assume to image the entire USB drive and send it to my external hdd, all i would have to tap is: sudo dd if=/dev/sdc of=/dev/sdb1 bs=512. Well this isn't going to work, because in order to send files to an ex-hdd, you need to designate to the /media directory. /media is usually where and ext-hdd ends up, and there a thousand different ways in which to write commands and scripts that send files to an external hard drive, and since i'm not a Linux power user, i only know this one method. The second method i'm familiar with is creating a directory inside the ext-hdd with mkdir, and then specifying output to that directory, creating a "mount point". But we'll just stick with the first method. 

You need to navigate your shell to a new directory. In other words, your terminal can move what neighborhood it is living in. By default, when you open the terminal, you're simply brought to the desktop of the current user that is logged in. But you can, you can pick up shop, move into a new directory that exists within your filesystem (like an ext-hdd), and then take control of where you're located. 

You change your terminals directory with the cd command. So look up the path to your ext-hdd, (usually /media/user/nameofdrive). 

So the entire command will be: cd /media/digitalresidue/UNTITLED 

And this will redirect you to the inside of the ext-hdd. From there, you're going to command an image be done of your USB, and simple give a name to the imaged file, an not specify a destination, since you're already in the destination in which you want it to end up.

So my imaging command is now: sudo dd if=/dev/sdc of=./ddtest.img

And i just got a warning symbol (see below), because i just realized my ext-hdd is Read Only. I'll leave this part in, since it's helpful to discuss how to change a disks read/write settings.










Come to find out, the reason for my ext-hdd is in Read Only mode, is because it is an hfs+ journaled partitioned hard drive (since my macbook is the primary system it is connected to). "Ubuntu only has support for writing to non-journaled HFS+ volumes." One way around this is to repartition the drive to something else, um nope, then i'll lose all my data. So i'm kinda screwed here. I could install hfsprogs, but i really wanted this exercise to be a "native run", without third party installs. Oh well. Most people have a ton of tools installed on their workstations anyway. So sudo apt-get install hfsprogs :-/. After installed, make sure your drive is still mounted, which is should be, and start navigating back to your ext-hdd directory with the cd commands that we explained earlier, then presto, your imaging of a FAT USB drive with a pain in the butt HFS+ partitioned ext-hdd is under way:








As you can see, after pressing enter, there is no "progress" data, (infos that shows you the imaging taking place), so you're left to sit and wait, not know how much has been completed nor know when the download will be completed. You do however get this information with dc3dd and prog=.

With my image complete, it is a file that can now be referenced with my TSK commands. This seems like a good place to stop part 1, and my apologies for there not being much to do with working with TSK just yet. Unfortunately i wanted to give a step by step guide, starting from the ground up, and beginning with what one would expect to create in order to even use the TSK toolkit. So in part two, i will get down to the wondrous magic, a forensicator can accomplish with TSK.