Making a native Windows bootable USB stick (Part 1)

This is in fact incredibly easy and I find it invaluable to keep a bootable USB stick to hand for various purposes including:

  1. Deleting temporary files, etc. that would otherwise be locked or difficult to access
  2. Taking offline backups of files, e.g. “software” or “system” registry hives, or whole systems (make your backup USB hard drive bootable too using the exact same technique)
  3. Resetting forgotten passwords (via temporary replacement of sethc.exe)

In part 1, I’ll cover creating a basic bootable image that can be either built into an ISO for use in virtual environments, or burnt to disc, or copied to USB stick and made bootable. In later parts, I’ll show how we can make the image much more customisable and user friendly – much like the old, but certainly not forgotten, BartPE which is how we had to do things before Microsoft made the WinPE tools available to the masses.

The steps:

  • Install the Windows ADK (Assessment and Deployment Kit, formerly the Automated Installation Kit (AIK)) which is available here
  • Run the “Deployment and Imaging Tools Environment” shortcut that gets created in the Start Menu. Run this with administrative privileges which should give a command prompt.
  • Now we need to copy the required files to a working area so that we can customise them if required. So locate the “copype.cmd” batch file, which on a 64 bit system with the default installation path is in “C:\Program Files (x86)\Windows Kits\8.1\Assessment and Deployment Kit\Windows Preinstallation Environment” and run it with the following parameters:
copype.cmd   x86   c:\winpe
  • This should then have created the following folders, with files in some of them, in the destination you selected (c:\winpe in the example above):
fwfiles
media
mount
  • Now we have the files that can be copied onto a USB stick or drive, which doesn’t have to be empty, and then we just need to make it bootable so copy all folders and files from the “c:\winpe\media” folder to the root of your chosen USB stick, maintaining the same folder structure. The files are barely 200MB in total so you don’t need a particularly large capacity USB stick. Note that there are a fair few locale folders, e.g. “en-gb”, so you should only need to copy the one(s) you need, which is typically “en-us” for US English. For instance, the root of my USB stick looks like this:

Image

  • Once copied, we just need to make sure the USB stick is bootable which is achieved by running the following command in the administrative command prompt, where “G:” is the drive letter of my USB stick:
bootsect /nt60  G: /mbr

If you have multiple partitions on your USB stick, you will also need to ensure that the partition where you’ve copied the boot files to is the active partition. This can be done via the Disk Management snapin (diskmgmt.msc), right clicking on the partition and selecting “Mark Partition as Active”.

That’s it, you should now have a bootable USB stick, as long as your system is USB bootable – sometimes this has to be enabled in the BIOS, the boot order changed so USB comes before the hard disk(s) or, frequently, there is an option immediately after power on to get a one-time boot menu where you can then select to boot off the USB stick.

If you don’t mind losing the contents of the USB stick then instead of copying the files to the stick and making it bootable, you can just use the “MakeWinPEMedia” command in the same command prompt with the /ufd paramter.

To make a bootable ISO image, that can then either be used to boot up a virtual machine or burnt to CD to boot a physical system, run the following command in the same command prompt:

MakeWinPEMedia   /ISO  c:\winpe c:\temp\winpe.iso

When you boot though, you will eventually just get a command prompt, albeit one that has access to all the local file systems and tools built in to the image such as diskpart.exe and net.exe. Whilst a lot can be done from this command prompt, as long as you know the commands you need, such as to copy files off an unbootable system, it’s not very intuitive so in the next instalment I’ll cover how to customise the boot image to give a quasi start menu.

Troubleshooting Unknown Devices and Drivers that Refuse to Install

Ever had an “Unknown Device” in Device Manager and ended up spending a long time trying to find drivers for the device? This can be particularly difficult on laptops where the vendor has put in hardware that you don’t even know about and weren’t necessarily expecting.

Fortunately, it is actually quite easy, usually, to troubleshoot these situations and know what drivers to search for by looking at the hardware id in the details in Device Manager and then searching the web for that term.

Image

Note that the “VEN_” tells us the vendor, “NVIDIA” in the example above, as can be found at http://www.pcidatabase.com and the “DEV_” is the device. Incidentally, vendor “VEN_8086” is Intel which always makes me chuckle! I usually don’t need to bother searching for the “SUBSYS_” or “REV_” strings since “VEN_” and “DEV_” is enough to identify the vendor and device and thence driver. Obviously it is always best to get the drivers from the Vendor’s website rather than a third party site since these latter drivers could contain malware, etc.

However, sometimes even with the seemingly correct driver package it still does not cure the problem. For instance, I recently had a battle with the drivers for an Epson printer. I knew I’d win but it was how long it would take and how much hair I would lose in the process!

I started with getting the latest driver package from the vendor’s web site rather than using the bundled CD as you never quite know how long these have been sat in a stock room somewhere. However, after installing the drivers, the USB attached printer was still showing as an unknown device. Even trying to update the driver, via Device Manager, and pointing it at the folder where the .inf file resided did not rectify the situation.

I then looked in the .inf file for the hardware id since the way Windows finds the right .inf file for a device is to look for an .inf file with that hardware id in it (see http://msdn.microsoft.com/en-us/library/windows/hardware/ff549520(v=vs.85).aspx). What I noticed was that the hardware id wasn’t listed despite the .inf file being the right one according to the descriptions in the file, i.e. had the name of the printer model I was trying to use in it. An easy way to search .inf files for a given id from a cmd prompt is:

          findstr /i /c:”PCI\VEN_10DE^&DEV_0A6C” %systemroot%\inf\*.inf

Note the escaping of the ampersand (&) above otherwise the batch processor will treat the command line as two separate commands and will not run properly. Due to a limitation in “findstr” (and “find”), the above won’t work for Unicode files so instead I use Strings.exe from SysInternals/Microsoft in the following way which works for both Unicode and ASCII files:

          strings.exe %systemroot%\inf\*.inf | find /i “PCI\VEN_10DE&DEV_0A6C”

I use “find” instead of “findstr” as I frequently get “out of memory” errors when using findstr in this pipelined way.

What I then did was to take a copy of the .inf file and add in the “missing” hardware id such that Windows now accepted it as being the “correct” .inf file for this device. It did then fail on some of the subsequent copying of driver files but that was soon rectified by creating a new, temporary, folder with a copy of this .inf file in, and then copying into there all of the missing files, which were dotted around the system in such places as the driver installation package folder and %systemroot%\ System32\spool. Identifying the missing files was achieved with a mixture of SysInternals/Microsoft Process Monitor running during the driver install and entries in the file %systemroot%\inf\setupapi.dev.log.

This setupapi.dev.log is “quite interesting” in that it keeps a record of all device installations, and statuses, from the birth of the specific computer. Think someone may have inserted a USB stick to “steal” something when you left your system unlocked (tut, tut)? Then either search for that date/time in the setupapi.dev.log file or look for the hardware id in this file, e.g. search for “Cruzer” if you think that the device was a SanDisk Cruzer flash drive, for instance.

Displaying text from a (log) file at a specific offset

When analysing product behaviour here at AppSense we often run SysInternals Process Monitor (procmon) to understand what is going on behind the scenes. In the procmon trace, we will see our log files being written to but all that is shown in the trace is the offset into the file that the data is written to and how much is written which, on the whole, is not that useful. What we sometimes need to know is exactly what text is written to the log file at this point as it will help us to correlate what procmon tells us with what our product’s log file tells us.

In order to be able to perform the correlation, I wrote a utility, called FindFileEntry.exe, which takes an offset (-o), a length (-l) and text log file name (-f) and will output the line number of the file and the text (up to “length” characters) . It will work with both Unicode and ANSI/ASCII text files.

For instance, if we consider the following, somewhat contrived, snippet of a procmon trace:

procmon of log file

We can see that the AppSense Application Manager Agent process (AMAgent.exe) has written 88 bytes to the log file “C:\Temp\amlogs\demo\Application Manager_AGENT_06_11_2013_21_52_28.log” at an offset of 22,252,941 bytes. We therefore run the following options with the FindFileEntry utility:

c:\>FindFileEntry.exe -f "c:\temp\amlogs\demo\Application Manager_AGENT_06_11_2013_21_52_28.log" 
-o 22252941 -l 88

And it outputs the following where the number immediately before the first colon character is the line number within the file

206504:     T031268 693093390 21:54:21.502 [AMPipeServer::Run] Event signalled. waitValue = 22

Given that we now have the line number as well as the text written, we can open the log file in a text editor such as Notepad++ so that we can see what other lines of (useful) debugging information were written around that same period.

notepad++ showing log line

The utility will work for any text log file, not just AppSense’s, and can be downloaded from here. Please refer to the README within the download for more details.

PS. Wouldn’t it be nice if procmon had some kind of customisable capability to allow an event to be right clicked on, or similar, and an arbitrary program be selected to run, with parameters such as parts of the procmon trace line, e.g. to launch the FindFileEntry program seamlessly?

Capturing transitory files before they are deleted

Sometimes you need to take a copy of a file that another program creates when it runs and subsequently deletes but it may well have deleted it before you get a chance to take a copy. A colleague was struggling with such an issue recently so I suggested temporarily changing the permissions on the folder where the file was created such that the file could be created but not deleted. This should then allow the file to be copied at your leisure since the creating program should be unable to delete it (unless it manipulates the file’s Access Control List (ACL) itself but this is unlikely).

Owing to my age, I like command lines, although I shudder if anyone dares to call a cmd prompt a DOS prompt, and in this instance it’s much easier to show what we need to do to secure the folder which is:

icacls %temp% /deny appsense\ivan:(CI)(OI)(DE,DC)

This creates a deny Access Control Entry (ACE) in the ACL for the folder, sub folders and files where the file is created, which you can figure out with good old Process Monitor (procmon). The “OI” and “CI” are Object Inherit and Container Inherit respectively and without these specified the ACE would only be added to the folder and not be inherited by anything created in there. The “DE” and “DC” are Delete and Delete Child respectively which are the special permissions which we are explicitly denying and as we all know in Windows security it’s the most restrictive permission that wins so a Deny ACE beats an Allow ACE. We assume that there is already an ACE in the folder’s ACL that allows the user, “ivan” in this example, to create files – the user’s %temp% folder, for instance, would normally have a Full Control Allow ACE for the user to which it relates.

Once the required file(s) have been moved elsewhere, the ACE added above should be removed by running:

icacls %temp% /remove:d appsense\ivan

Note that as long as the user “ivan” already has permission to change the ACL on the required folder then the above icacls commands do not need be run elevated. You can run icacls with just the folder name as an argument to see what the current ACL contains.

Before the more pedantic amongst you contact me, I should point out that I know I ought not refer here to an ACL but instead to to a DACL (Discretionary Access Control List) but most of the time when we talk about security of Windows objects the “D” is silent. After all, the utility is not called icdacls.exe! Also, don’t confuse a DACL with a SACL (System Access Control List) which  contains audit settings, which you viewin Explorer’s security dialogues for NTFS objects for example.

See here and here for further discussions on DACLs, SACLs, ACEs and other Windows security concepts.

Where do registry settings in user profiles come from – part 1?

Trying to figure out where a particular registry setting that a user ends up with in their session can be tricky since there are a myriad of places that registry settings can come from. These can include:

·         The default user profile

·         A mandatory profile

·         An existing local profile

·         An existing roaming profile

·         Group policy

·         Logon scripts (of which there are many types and ways to launch)

·         Terminal Server “shadow” key

·         Applications (including application virtualisation products)

The first hurdle to cross when a user is not getting the correct setting can be to see what the setting really is in order to confirm that it is indeed the registry which is “at fault”. The rest of this article will explain the various methods that can be employed to see what particular registry setting a user has since this isn’t always as straightforward as it sounds.

If the user has logged off then using regedit.exe to load their ntuser.dat, which is what becomes their HKEY_CURRENT_USER hive when they logon, can be the simplest method. Their ntuser.dat file should be in the location specified for the roaming profile in their Active Directory account properties although on Terminal Services there are a number of group policy settings that can override this to set a computer wide roaming profile path or to force a mandatory profile. If a roaming profile is not configured then try the locally cached profile on the computer they were using via its C$ share. Group Policy Objects (GPOs) may be in place to delete locally cached profiles after logoff although this is usually only used in conjunction with roaming profiles since otherwise a user’s settings would be discarded (which can sometimes be desirable). I always take a copy of the ntuser.dat file and work on this in order not to disturb timestamps and to prevent issues if the user logs on again whilst you are performing investigations. To load the hive in regedit.exe, put the focus on to “HKEY_USERS” key and select the “Load Hive” option from the File menu. When prompted for a key name, enter any arbitrary, but vaguely meaningful, name. Remember also, when your investigations are complete, to unload the hive from the File menu.

If the user is still logged on, it may just be a case of running regedit.exe remotely and connecting to the user’s machine. Obviously when doing this HKEY_CURRENT_USER is not directly accessible so you will have to go via the remote HKEY_USERS and select the correct key name based on the user’s SID. Rather than having to ascertain the user’s SID, simply look in the “Volatile Environment” sub key of each SID key for clues as to which user it relates to, such as the “USERPROFILE” or “APPDATA” values. If remote access is not possible, regedit could be run locally although this may well be prevented via group policy and you probably want to run regedit yourself rather than letting a user loose with it so a remote control session or physical visit will be required. If regedit is denied via group policy, there are a couple of temporary ways around this short of changing GPO’s for this user. The first, which must obviously be done as an administrator, e.g. via runas, since users do not have write rights to their own group policy keys in HKEY_CURRENT_USER, is to temporarily change the registry value which stops regedit from working. This is the “DisableRegistryTools” in “HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\System” which should be (temporarily) set to a zero. No logoff and logon is required, which would probably refresh the value anyway, since this registry value is read dynamically at run time by regedit.exe itself. The second method is to find another tool that will expose the registry for examination – note that reg.exe is also subject to the above GPO applied registry setting. It is actually possible to make binary modifications to a copy of the regedit.exe executable so that it looks for another, non-existent, value but the method will not be detailed here. If you do this, you should guard this executable carefully and ideally use another security product which prevents execution of unauthorised code since this can wreak havoc on your carefully crafted computers.

Next time we will cover in more detail the multitude of places where these registry settings can come from.