Encrypted Container File Recovery

Scenario:
During a technical interview, I was told definitively, categorically, unequivocally that it was impossible to recover deleted files from within an encrypted container, even if you possess the key. Windows was the OS of choice, and he insisted that he knew from personal experience as they use encrypted containers. He stated that all the file data would be garbled due to the encryption, so it didn't matter if you recovered it or not; the content would not be readable. File wiping was not part of the equation (I asked). This did not sit quite right with me, but I was in the middle of an interview and did not already know the answer, as I had not ever encountered that type of situation. I expressed surprise that that would be the case, but left it at that.

My thoughts:
Pondering it over on the way home, more and more it just didn't make sense to me. If they're created on the file system, the files themselves are not encrypted; the container is encrypted. It has to be mounted in order to access the files. When a file is deleted, the container is mounted and the file is in a readable state. Therefore it should exist in a readable state. If the container is deleted, that's a different matter, although it is feasible that the container could be recovered. All you should have to do is mount the encrypted container, and search within it for the file, which would still exist until overwritten.

I determined to test my theory as soon as possible. Of course, that took a little longer than I initially wanted, as I was “side-tracked” by continuing Dropbox research. However, I outlined a plan, and recently sat down to work through it. Initially, I thought of looking to the host file system, as the OS would provide access to the files through the encrypted container. As I continued the thought process and began testing, I realized this would not be exactly true.

The host OS' file system (ie, MFT) would not reference these files, as they only exist inside the container. The container has its own file system (and MFT), which would be the home for the files' information (and possibly the files themselves, if they were small enough to be resident. Then of course, there's unallocated space. The host OS' could potentially have residue in places like the pagefile or memory, but that should be it.

Now, I don't know what type of encryption they use, but I'm thinking in the end, an encrypted container is going to work the same, regardless of flavor. If I'm wrong, well, then the following is true at least for Truecrypt 7.0a. I approached this research from the standpoint that the files would have been created directly in the container, rather than on the host file system and subsequently moved into the container. Had it been the other way around, there could be artifacts or residue left on the host. I also assumed a fixed container size; with dynamic, the process might end up being a bit more complex.

The process:
The basic idea I had was to create some files inside an encrypted container. Confirm they existed there methodologically, confirm they weren't on the host file system. Then remove/delete them from the container, and try to identify their remains, attempt recovery. So here's the basic outline of my steps and actions; I've tried to retain some order to it and hope it makes sense:

1. Create a 500MB TrueCrypt container.

As a side note, I did this inside Dropbox and had no difficulty; some people have had trouble, and I think it might be related to creating a dynamic container, rather than fixed-size.

2. Create four (4) text files inside the container, filling with specific text from Altheide & Carvey's excellent Digital Forensics With Open Source Tools, since I had just read that.

3. Confirm files' association to host file system
a. Without mounting the container, extracted host system MFT using FTK Imager, mounted with Sanderson Forensics MFTView, exported to XLS & TXT, and searched for filenames. None present, as anticipated.
b. With container mounted, extracted host system MFT, mounted, exported and searched. Still nothing.

4. Confirm files' association to container file system
a. With container mounted, extracted logical drive's MFT (ie, for the container), exported, and searched. Files were identified, as anticipated.
b. As these files were small enough to be resident in the MFT, all content is visible.

5. Move all 4 files out to new location outside of the container(not securely). Obviously, the container is mounted.
a. Extract logical drive's MFT, mount with MFTView, locate resident files.
b. Viewing the hex/text of the entry in MFTView, was able to recover all 4 files to matching hash value.

6. What about larger files not contained in the MFT?
a. Copy entries.log file (4.23KB) - a Dropbox artifact – into container.
b. MFT contains entry but no contents for non-resident file.

7. Delete files (no wiping) from within mounted container and attempt recovery.
a. Mount logical file system in FTKi.
b. INFO2 file contains list of files.
c. Export "Dt1" etc files w/FTKi, hashes match original.
d. Exported MFT and mount in MFTView. Files are resident at “root” of MFT and in Recycler; fully recoverable from both locations.

As a side note, sometimes MFTView seemed to have difficulty displaying the file contents correctly, and thus the extraction of that data to recover the file would result in a hash not matching. This was did not occur all the time, and was observed to happen whether the file was live or deleted. Obviously, the file contents weren't actually stored that way, so it was some programmatic issue. I don't know whether MFTView is a current application or not, as I don't see it listed on the website any longer. I have not (yet) contacted Sanderson Forensics about it, as I don't think it matters for the purpose of this research. An example is below (first with issue, then normal):

Now back to the process...

8. Search host filesystem for files (pagefile, RAM)
a. Copied files back into container, reboot (this flushes my pagefile), open each file
b. Used FTKi, exported host system pagefile.sys and RAM
c. Ran Sysinternals strings (5-character minimum) and output to text files.
d. Loaded & searched in notepad++. Found all 4 test files' contents, plus large portions of entries.log.

Cut away from the outline for a moment. I re-deleted the files, imaged the logical FS and mounted in ProDiscover Basic to search for the files. Found them as they existed in Recycler, to be expected. I purged Recycler by drilling down, selecting all files and deleting again. Keyword searches in Unallocated space weren't turning up the files, and it finally dawned on me that it was because the files were resident in the MFT, so as long as the entries existed there, they wouldn't show up in Unallocated. I needed larger files across the board. So I copied back in the entries.log file, along with a DOC, XLS, and PDF. Deleted (the PDF was too large and was “permanently deleted”) and re-imaged.

9. Load image, search for files (using ProDiscover)
a. Content search: DOC, XLS, and TXT were in Recycler. PDF was in "Deleted" (too big for Recycler). INFO2 listed filenames. The .~Lock file for the DOC was also in Deleted.
b. Cluster search: Two hits on "SIFT" that appear to be related to a PDF file, and contain the filename.

10. Dig a little further (using ProDiscover)
a. Emptied Recycler (drilled down, selected all files and rt-click delete)
b. Re-imaged w/FTKi, reloaded, and Content search. All files found in "Deleted."
c. Searched Clusters, same PDF hit, found TXT file, possibly DOC/XLS (since binary).

11. Final push
a. Carved image with photorec, using default settings.
b. 4 files were recovered, the PDF, TXT, DOC, and XLS. The TXT file is ~1/2 the size, but since there's no header info, it's pretty incredible anyway. All except the text file hash match to the original.

Summary:
This was a fun little exercise, and I think I can categorically, definitively, unequivocally state that it absolutely is possible to recover deleted files from within an encrypted container when you have the key to the container. Obviously, there are variables. If the container size is dynamic, for instance, this could impact things, but I think the odds are still fairly good, and the process is essentially the same. The amount of time that has passed – as with any investigation – is important, but with close proximity it may even be possible to find the files' content (if previously viewed) in pagefile or RAM.

But the core is that it is possible to recover. Knowing the content and the filenames, I was able to easily recover deleted and purged (but not wiped) files from within an encrypted container. I was also able to carve the files without any use of filenames, contents, or type by an automated process. The process could be done more manually by using Sleuthkit or other utilities. Anyway, it can be done, and that's that.

Dropbox Forensics Follow-Up

Several months ago I started on a quest to research locally-created artifacts related to the use of Dropbox on Windows systems. This took several months of work as time allowed, in order to complete the outline I was following. This culminated in a blog post on SANS, a more complete article hosted on Forensic Focus, and a summary of artifacts on Forensic Artifacts. However, that's not all I have to offer on the subject. Yes, folks, for a limited time only, when you buy all three you get a fourth for free! That's a $19.95 value, included at no extra cost! (shipping & handling not included; residents of the UK must pay VAT - I know, it sucks)

At the end of the article (hosted on Forensic Focus), I wrapped up with some outstanding items, or possible other things to research. I have spent some more time going over some (only some, not all) of those; this follow-up post will cover my additional research:
1. Does unlinking (local or web) change the registry?
2. What impact does uninstallation have on the registry?
3. What are the various “hash” values; what do they signify?
4. Do the IP addresses vary with geographic area?
5. What data is transferred across the unencrypted connection?
6. Do the SQLite databases contain deleted entries, and how can those be parsed?
7. Are file/system IDs or encoded info stored in the databases, 'entries.log' or elsewhere?

1. Instead of doing ProcMon or RegMon by Sysinternals, I ran regshot 1.8.2 to create snapshots before & after each unlinking. Initially I kept getting BSOD'd every time it would scan the registry but switching systems eliminated that issue. Ultimately there were no obvious registry changes related to the unlinking (local or web).

2. I used regshot before & after the uninstallation as well, and quickly identified 49 deleted entries (truncated here; complete on Forensic Artifacts):

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\ShellIconOverlayIdentifiers\DropboxExt1\: "{FB314ED9-xxxx-xxxx-xxxx-xxxxxxxxxxxxx}"
HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\ShellIconOverlayIdentifiers\DropboxExt2\: "{FB314EDA-xxxx-xxxx-xxxx-xxxxxxxxxxxxx}"
HKU\S-1-5-21-xxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxx\Software\Dropbox\InstallPath: "C:\Documents and Settings\username\Application Data\Dropbox\bin"
HKU\S-1-5-21-xxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxx\Software\Microsoft\Windows\CurrentVersion\Shell Extensions\Approved\{FB314ED9-xxxx-xxxx-xxxx-xxxxxxxxxxxxx}: "DropboxExt"
HKU\S-1-5-21-xxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxx\Software\Microsoft\Windows\CurrentVersion\Shell Extensions\Approved\{FB314EDA-xxxx-xxxx-xxxx-xxxxxxxxxxxxx}: "DropboxExt"
HKU\S-1-5-21-xxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxx\Software\Microsoft\Windows\CurrentVersion\Uninstall\Dropbox\UninstallString: ""C:\Documents and Settings\username\Application Data\Dropbox\bin\Uninstall.exe""
HKU\S-1-5-21-xxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxx_Classes\CLSID\{FB314ED9-xxxx-xxxx-xxxx-xxxxxxxxxxxxx}\: "DropboxExt"
HKU\S-1-5-21-xxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxx_Classes\CLSID\{FB314EDA-xxxx-xxxx-xxxx-xxxxxxxxxxxxx}\: "DropboxExt"
HKU\S-1-5-21-xxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxx_Classes\CLSID\{FB314EDB-xxxx-xxxx-xxxx-xxxxxxxxxxxxx}\: "DropboxExt"
HKU\S-1-5-21-xxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxx_Classes\CLSID\{FB314EDC-xxxx-xxxx-xxxx-xxxxxxxxxxxxx}\: "DropboxExt"

I've x'd out some of the SIDs to (hopefully) make it easier to focus, and because I didn't want to post the full SIDs on the internet. I left the first segment for some of the SIDs since that part makes a noticeable, incremental change.

3. There is actually a correlation between "hash" values in the various config files. It should be noted that Dropbox hashes the files in 4MB chunks, and stores the hashes the same way (base64 encoded). Thus, there may be multiple hash values for a single file (but only when it's larger than 4MB). Here's where I've followed the trail of hash:
filecache.db block hash field
entries.log 5th section is hash
sigstore.db stores hash (and size in bytes)

4. I know that some application updates will reach out to different servers based on geographic location, and I wondered if this was the same for Dropbox. Using NirSoft CurrPorts, it was easy to gather the active connections here in Texas. I had reason to take a trip to California, so I did the same thing there. Finally, I established a VPN connection to another country and checked the connections that way as well.

There were some minor variations between the locations for IP addresses, although host names remained largely the same. The one thing that did not change in any of these, was the IP and host name for the sole HTTP (unencrypted to port 80) connection.

5. So then there's the question of this single unencrypted connection. I had not previously examined the content of this traffic, but I have now, using Netwitness Investigator to isolate the connection stream of interest and exporting that out for posterity and more review.

It's basically a "Hello, here I am" and "Let's keep the connection going" type of conversation. Of course, it's in clear text. My only concern is that it transmits the namespace ID (from config.db, root_ns), and possibly that of shared directories as well (there's a second entry that follows the namespace format, but I haven't been able to confirm that yet). With some of the Dropbox-related security issues that have recently come to the surface, I'm a little concerned about this data being transmitted in the clear, especially when I don't know for sure if it can be exploited (and since the IP address and host name are always the same).

6. Deleted entries within the SQLite database files can indeed be recovered. I suspected as much, but I'm not a DB (or SQLite) guru. Historically I've relied on others to develop a tool I can use for this purpose, and I've stuck to my guns in this instance. CCL-Forensics has a product designed for this purpose, called epilog; while it's a commercial product, there is a 7-day trial available.

I must say, it works quite nicely. I removed some files from my Dropbox folder just for this test (relocated to another directory), and then downloaded (have to register, but no sales personnel have contacted me yet), installed, and ran epilog. They have some videos on YouTube, but I found the info I needed in their Help file. There are some different methods to recover deleted entries, but I simply focused on the "Free Page Analysis" which parses the link list or freelist within the database. It very definitely did what I needed it to do.

Edit: I intended to note that to export a report-type of info from Epilog you basically have the option of going to an XML file, which may not be directly what you need. For me, I wanted to look at the data in a spreadsheet. Most methods to convert XML to CSV revolve around going through a couple steps (ie, XSLT), I found XSlicer to be very helpful.

7. And yes, other encoded data does exist within different config files. Dropbox makes use of base64 encoding, and one of the key places is the "entries.log" file located within the ".dropbox.cache" directory inside the user's Dropbox folder. (This set of artifacts is discussed in more detail in the Forensic Focus article.) By cross-referencing with the various parsed database files, I was able to decipher the entries.log (pipe-delimited) file:
1st section is filename (as it exists in .dropbox.cache directory)
2nd section is root_ns/path
3rd section is unix epoch timestamp
4th section is size (bytes)
In addition, the host.db file, 2nd row is user's Dropbox path.

So that pretty much wraps things up. I did not do any research into alternate file transfer methods (I think Dropship has addressed that rather well), but I did note that if you share a file (Public folder) you can get the link to that file; that link can be transferred via email, IM, etc, and the file downloaded by whomever has the link.

Some other resources:
I've already mentioned epilog, which I think has great potential.

There's also Dropbox Reader by ATC-NY; it's a set of python scripts to parse the SQLite files (they pull from the Dropship project). In addition to something like a SQLite Browser this can be very helpful to gather and cross-reference information.

Derek Newton has done some good research, hosted on his blog.
Forensic Artifacts
Security Issues

Great paper on cloud security (with focus on Dropbox) by SBA-Research; the actual download is here

I've mentioned the Dropship project a couple times, but it has been "officially" shut down. Research determined that it was possible to "share" files without using the Public folder, thus potentially facilitating illegal file-sharing. Although Dropship is no longer developed (by the originator) other forks can be found.

I think that's about it, folks. Unless something else comes up to pique my interest (I'm open to suggestions), I think I'm about done with Dropbox research for now. It's been a lot of fun going through this process, and I've learned a lot, which is also good. Hopefully this will all prove useful - to myself and others - in our forensicating efforts.