Difference between revisions of "Hard Drive"

From xboxdevwiki
Jump to: navigation, search
(Fill in config area partition path)
 
(5 intermediate revisions by 5 users not shown)
Line 7: Line 7:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
! Drive Letter
+
! Drive Letter{{FIXME|reason=This is under control of running application / variation even in official products?}}
 
! Description
 
! Description
 
! Offset (bytes)
 
! Offset (bytes)
 
! Size (bytes)
 
! Size (bytes)
 
! Filesystem
 
! Filesystem
 +
! Device Object (MS Retail Kernel)
 
|-
 
|-
 
| N/A
 
| N/A
| Config Area
+
| [[Config Sector|Config Area]]
 
| 0x00000000
 
| 0x00000000
 
| 0x00080000
 
| 0x00080000
 
| Fixed Structure
 
| Fixed Structure
 +
| \Device\Harddisk0\Partition0
 
|-
 
|-
 
| X
 
| X
Line 24: Line 26:
 
| 0x2ee00000
 
| 0x2ee00000
 
| FATX
 
| FATX
 +
| \Device\Harddisk0\Partition3
 
|-
 
|-
 
| Y
 
| Y
Line 30: Line 33:
 
| 0x2ee00000
 
| 0x2ee00000
 
| FATX
 
| FATX
 +
| \Device\Harddisk0\Partition4
 
|-
 
|-
 
| Z
 
| Z
Line 36: Line 40:
 
| 0x2ee00000
 
| 0x2ee00000
 
| FATX
 
| FATX
 +
| \Device\Harddisk0\Partition5
 
|-
 
|-
 
| C
 
| C
Line 42: Line 47:
 
| 0x1f400000
 
| 0x1f400000
 
| FATX
 
| FATX
 +
| \Device\Harddisk0\Partition2
 
|-
 
|-
 
| E
 
| E
Line 48: Line 54:
 
| 0x131f00000
 
| 0x131f00000
 
| FATX
 
| FATX
 +
| \Device\Harddisk0\Partition1
 +
|}
 +
 +
::::::::::::::::::::::''side note: CD/DVD Drive "D:" <=> "\Device\CdRom0"''
 +
::::::::::::::::::::::''and usually: added Drive "F:" <=> "\Device\Harddisk0\Partition6"''{{FIXME|reason=Mark as unofficial / homebrew}}
 +
:::::::::::::::::::::::::&nbsp;&nbsp;&nbsp;''added Drive "G:" <=> "\Device\Harddisk0\Partition7"''{{FIXME|reason=Mark as unofficial / homebrew}}
 +
 +
'''Debug/Devkit HDD:'''
 +
{| class="wikitable"
 +
|-
 +
! Drive Letter
 +
! Description
 +
! Offset (bytes)
 +
! Size (bytes)
 +
! Filesystem
 +
! Device Object (MS Retail Kernel)
 +
|-
 +
| [FIXME]
 +
| [FIXME]
 +
| [FIXME]
 +
| [FIXME]
 +
| [FIXME]
 +
| [FIXME]
 +
|-
 
|}
 
|}
 +
  
 
'''FIXME:'''
 
'''FIXME:'''
 
* Add info on how extended partitions are added.
 
* Add info on how extended partitions are added.
  
== Locking ==
+
== Locking Mechanism and Basics ==
The hard drives in the Xbox are locked with a key which is unique to the specific Xbox. The drive is unlocked by the kernel at boot.
+
The hard drives in the Xbox are standard IDE drives locked with a key (referred to as the HDKey). The drive is unlocked by the Kernel at boot. The XBox uses a User Password with the Maximum security mode (see below).
 +
The password is generated in two distinct phases:
 +
* Extract the HDKey from the [[EEPROM]] which is unique to each Xbox making this phase dependent only on the [[EEPROM]].
 +
* Generate a drive specific password (keyed to the model and serial number of the drive) with the extracted HDKey from the [[EEPROM]].
  
=== Unlocking for Backups ===
+
The security feature of the hard drive can be enabled and disabled by sending special ATA commands to the drive. If a device is locked, it will refuse all access until it is unlocked.
 +
 
 +
=== Locking Mechanism ===
 +
The ATA/ATAPI Command Set - 2 (ACS-2) specification <ref>[http://www.t13.org/documents/UploadedDocuments/docs2011/d2015r7-ATAATAPI_Command_Set_-_2_ACS-2.pdf Draft: ATA/ATAPI Command Set - 2 (ACS-2) - T13/2015-D - Revision 7, June 22, 2011] (t13.org); Note: T13 drafts are freely available, only the final standards are behind a paywall.</ref> defines an optional ''SECURITY'' feature subset (chapter 7.43 - 7.48) which allows to limit access to the drive's data behind a hardware implemented locking mechanism:
 +
 
 +
* SECURITY DISABLE PASSWORD (Chapter 7.43)
 +
* SECURITY ERASE PREPARE (Chapter 7.44)
 +
* SECURITY ERASE UNIT (Chapter 7.45)
 +
* SECURITY FREEZE LOCK (Chapter 7.46)
 +
* SECURITY SET PASSWORD (Chapter 7.47)
 +
* SECURITY UNLOCK (Chapter 7.48)
 +
 
 +
==== The Password ====
 +
A device can have two passwords; either or both may be set with a maximum of 32 byte each. A device can be locked in two modes: High security mode or Maximum security mode. If the User password is not available, the only remaining way to get at least the bare hardware back to a usable state is to issue the SECURITY ERASE PREPARE command, immediately followed by SECURITY ERASE UNIT. In Maximum security mode, the SECURITY ERASE UNIT command requires the Master password and will completely erase all data on the disk.
 +
 
 +
===== User Password =====
 +
If a ''User Password'' is set (SECURITY SET PASSWORD), the drive blocks the access on a reboot again so you have to re-enter the password (SECURITY UNLOCK). Setting the ''User Password'' will also set the ''Master Password Capability''.
 +
 
 +
===== Master Password =====
 +
If the ''Master Password Capability'' is set the drive could be locked in one of two modes. Bit 8 in word 128 of the IDENTIFY response shows which mode the drive is in:
 +
* 0 = High: the ''Master Password'' could be used to unlock (SECURITY UNLOCK) just like the ''User Password'' or to deactivate the ''User Password'' (SECURITY DISABLE PASSWORD).
 +
* 1 = Maximum: the ''Master Password'' could only be used to wipe the drive (SECURITY ERASE UNIT). This makes the drive usable again but all data on it is lost.
 +
 
 +
=== Drive Data ===
 +
During the second phase, the serial and model numbers are needed. These values are available in the response data from the DEVICE_IDENTITY ATA command. However, the data needs to be reorganized. It is read in Big Endian words, and needs to be byte swapped first to get the byte ordering correct. Then, starting from the end of the data (serial == 20 bytes, model == 40 bytes) ignore ASCII spaces (byte value of 0x20) at the end of the data. Zeros are *not* trimmed, *only* spaces. Do not be fooled into believing that this data is a string. On some drives this is the case, but on others there are non-ASCII values in the fields.
 +
 
 +
=== Basic Security Algorithms ===
 +
There are three primary cryptography routines/functions needed when generating an XBox drive password:
 +
* SHA1: Hashing algorithm. It's primary purpose is to take an input message and create a (relatively) small signature (called a digest) which is unique to the original message. One of the goals of SHA1 is to make it difficult to alter the input message in such a way as to result in the same output digest.
 +
* RC4: Symmetric cipher. This means that the algorithm for encryption is the same as that for decryption. The purpose is to make one key work in both directions.
 +
* HMAC: Uses a hashing algorithm (in this case SHA1) to generate a cryptographically "strong" signature.
 +
 
 +
==== Password Algorithm ====
 +
- Key data is shown entering functions from the side
 +
- Data is shown entring from above or below in order of presentation from left to right
 +
 
 +
<pre>                                      RC4_key &gt;--(second)--&gt;--,
 +
                                        /|\                  |
 +
                                          |                    |
 +
.-&lt;--|__eeprom_key__|--&gt;-----------&gt; HMAC_SHA1                |
 +
|                                      /|\                  |
 +
|                                        |                    |
 +
|                        .---&gt;-----------'                    |
 +
|                        |                                    |
 +
|  eeprom_data = |__data_hash___|__enc_conf__|__enc_data__|  |
 +
|                        |            |          |          |
 +
|                        |            \|/          |          |
 +
|                        |        rc4_decrypt &lt;----|---------&lt;|
 +
|                        |            |          |          |
 +
|                      \|/            |          |          |
 +
|                (must be equal)      |          \|/        |
 +
|                      /|\            |      rc4_decrypt &lt;---'
 +
|                        |            |          |
 +
|                        |            \|/        \|/
 +
|                        |      |_confounder_|____data____|
 +
|                        |      /            /    |
 +
|                        |      /            /    |
 +
|                        |    /            /      |
 +
|                        |    /            /      |
 +
|                        |  \|/          /      \|/
 +
`---&gt;-----------------&gt; HMAC_SHA1        /  |__HDKey__|__|
 +
                                /|\      /        |
 +
                                \______/          |
 +
                                                  |
 +
              .-------------------------&lt;--------'
 +
              |
 +
              |              model_number  serial_number
 +
              |                      \        /
 +
              |                      \      /
 +
              `---&gt;-----------------&gt; HMAC_SHA1
 +
                                          |
 +
                                          \|/
 +
                                      HD_password
 +
</pre>
 +
This seems to be the easiest way to show the required calculations.
 +
 
 +
Basically there are several intermediate steps. First, generate the RC4_key from the eeprom_key and the data_hash (first 20 bytes of eeprom_data). Use the RC4_key to decrypt the encrypted confounder (8 bytes 20 bytes into eeprom_data) and the encrypted data (20 bytes 28 bytes into eeprom_data). Now generate an HMAC_SHA1 hash from the eeprom_key and the decrypted confounder and data. Verify that this hash matches the data_hash stored in the eeprom. If they don't match then the eeprom data is not correct. If the hashes match then the first 16 bytes of the decrypted data field is the HDKey.
 +
 
 +
Once you have the HDKey get the model and serial number from the drive. Generate an HMAC_SHA1 hash from the HDKey, model and serial numbers. The resulting 20 bytes are the HD password. The remaining 12 bytes needed for the password are zeros.
 +
 
 +
== Unlocking ==
 
Before connecting an Xbox HDD to a PC for a backup or modification, the drive must first be unlocked. This can be done with alternative dashboards (such as EvoX). But beware, once you unlock the disk you cannot use it with an official BIOS until you re-lock the disk! For this reason it is suggested to use a patched BIOS which does not require the disk to be locked. If you are unable to run unsigned code (needed to unlock the HDD before powering off), it is possible to hot-swap the drive after the Xbox has started. This is not a suggested method, but it has been known to work. The idea is that you start the Xbox and wait for the dashboard, at which point the drive will be unlocked. Then, while the Xbox is running, you disconnect the IDE cable (but not the power!), and then connect the drive to your PC. Then the drive can be mounted for read/write (using XboxHDM), or imaged directly.
 
Before connecting an Xbox HDD to a PC for a backup or modification, the drive must first be unlocked. This can be done with alternative dashboards (such as EvoX). But beware, once you unlock the disk you cannot use it with an official BIOS until you re-lock the disk! For this reason it is suggested to use a patched BIOS which does not require the disk to be locked. If you are unable to run unsigned code (needed to unlock the HDD before powering off), it is possible to hot-swap the drive after the Xbox has started. This is not a suggested method, but it has been known to work. The idea is that you start the Xbox and wait for the dashboard, at which point the drive will be unlocked. Then, while the Xbox is running, you disconnect the IDE cable (but not the power!), and then connect the drive to your PC. Then the drive can be mounted for read/write (using XboxHDM), or imaged directly.
 +
 +
=== Unlock via Serial for Seagate drives ===
 +
Look here: http://www.os2museum.com/wp/seagate-serial-talk/
 +
 +
=== Universal Unlock Method(s) ===
 +
'''TODO'''
  
 
'''FIXME:'''
 
'''FIXME:'''
Line 81: Line 201:
  
 
== Further Reading ==
 
== Further Reading ==
* [http://hackipedia.org/Disk%20formats/Partition%20tables/X-Box/Xbox_Partitioning_and_Filesystem_Details.htm Xbox Partitioning and Filesystem Details]
+
* [https://web.archive.org/web/20150502210033/http://hackipedia.org/Disk%20formats/Partition%20tables/X-Box/Xbox_Partitioning_and_Filesystem_Details.htm Xbox Partitioning and Filesystem Details]
 +
* [https://web.archive.org/web/20020617181617/http://www.tardis.ed.ac.uk:80/~lucien/computing/projects/xbox/XBOX-disk-layout.htm XBOX DISK LAYOUT v0.13]

Latest revision as of 04:55, 22 March 2022

The original Xbox hard disk drive was 8 GB in size. Later releases, 10 GB drives; however, only the first 8 GB of the drive was used. See Hardware Revisions for more information.

Partitions

The Xbox hard disk contains multiple partitions. Unlike a PC, which typically contains either a Master Boot Record or GUID Partition Table to specify the partition information, the Xbox kernel uses a fixed partition layout. The file system used on the Xbox is FATX, a variant of FAT16/32 developed by Microsoft specifically for the Xbox. [FIXME]

Drive Letter[FIXME] Description Offset (bytes) Size (bytes) Filesystem Device Object (MS Retail Kernel)
N/A Config Area 0x00000000 0x00080000 Fixed Structure \Device\Harddisk0\Partition0
X Game Cache 0x00080000 0x2ee00000 FATX \Device\Harddisk0\Partition3
Y Game Cache 0x2ee80000 0x2ee00000 FATX \Device\Harddisk0\Partition4
Z Game Cache 0x5dc80000 0x2ee00000 FATX \Device\Harddisk0\Partition5
C System 0x8ca80000 0x1f400000 FATX \Device\Harddisk0\Partition2
E Data 0xabe80000 0x131f00000 FATX \Device\Harddisk0\Partition1
side note: CD/DVD Drive "D:" <=> "\Device\CdRom0"
and usually: added Drive "F:" <=> "\Device\Harddisk0\Partition6"[FIXME]
   added Drive "G:" <=> "\Device\Harddisk0\Partition7"[FIXME]

Debug/Devkit HDD:

Drive Letter Description Offset (bytes) Size (bytes) Filesystem Device Object (MS Retail Kernel)
[FIXME] [FIXME] [FIXME] [FIXME] [FIXME] [FIXME]


FIXME:

  • Add info on how extended partitions are added.

Locking Mechanism and Basics

The hard drives in the Xbox are standard IDE drives locked with a key (referred to as the HDKey). The drive is unlocked by the Kernel at boot. The XBox uses a User Password with the Maximum security mode (see below). The password is generated in two distinct phases:

  • Extract the HDKey from the EEPROM which is unique to each Xbox making this phase dependent only on the EEPROM.
  • Generate a drive specific password (keyed to the model and serial number of the drive) with the extracted HDKey from the EEPROM.

The security feature of the hard drive can be enabled and disabled by sending special ATA commands to the drive. If a device is locked, it will refuse all access until it is unlocked.

Locking Mechanism

The ATA/ATAPI Command Set - 2 (ACS-2) specification [1] defines an optional SECURITY feature subset (chapter 7.43 - 7.48) which allows to limit access to the drive's data behind a hardware implemented locking mechanism:

  • SECURITY DISABLE PASSWORD (Chapter 7.43)
  • SECURITY ERASE PREPARE (Chapter 7.44)
  • SECURITY ERASE UNIT (Chapter 7.45)
  • SECURITY FREEZE LOCK (Chapter 7.46)
  • SECURITY SET PASSWORD (Chapter 7.47)
  • SECURITY UNLOCK (Chapter 7.48)

The Password

A device can have two passwords; either or both may be set with a maximum of 32 byte each. A device can be locked in two modes: High security mode or Maximum security mode. If the User password is not available, the only remaining way to get at least the bare hardware back to a usable state is to issue the SECURITY ERASE PREPARE command, immediately followed by SECURITY ERASE UNIT. In Maximum security mode, the SECURITY ERASE UNIT command requires the Master password and will completely erase all data on the disk.

User Password

If a User Password is set (SECURITY SET PASSWORD), the drive blocks the access on a reboot again so you have to re-enter the password (SECURITY UNLOCK). Setting the User Password will also set the Master Password Capability.

Master Password

If the Master Password Capability is set the drive could be locked in one of two modes. Bit 8 in word 128 of the IDENTIFY response shows which mode the drive is in:

  • 0 = High: the Master Password could be used to unlock (SECURITY UNLOCK) just like the User Password or to deactivate the User Password (SECURITY DISABLE PASSWORD).
  • 1 = Maximum: the Master Password could only be used to wipe the drive (SECURITY ERASE UNIT). This makes the drive usable again but all data on it is lost.

Drive Data

During the second phase, the serial and model numbers are needed. These values are available in the response data from the DEVICE_IDENTITY ATA command. However, the data needs to be reorganized. It is read in Big Endian words, and needs to be byte swapped first to get the byte ordering correct. Then, starting from the end of the data (serial == 20 bytes, model == 40 bytes) ignore ASCII spaces (byte value of 0x20) at the end of the data. Zeros are *not* trimmed, *only* spaces. Do not be fooled into believing that this data is a string. On some drives this is the case, but on others there are non-ASCII values in the fields.

Basic Security Algorithms

There are three primary cryptography routines/functions needed when generating an XBox drive password:

  • SHA1: Hashing algorithm. It's primary purpose is to take an input message and create a (relatively) small signature (called a digest) which is unique to the original message. One of the goals of SHA1 is to make it difficult to alter the input message in such a way as to result in the same output digest.
  • RC4: Symmetric cipher. This means that the algorithm for encryption is the same as that for decryption. The purpose is to make one key work in both directions.
  • HMAC: Uses a hashing algorithm (in this case SHA1) to generate a cryptographically "strong" signature.

Password Algorithm

- Key data is shown entering functions from the side - Data is shown entring from above or below in order of presentation from left to right

                                       RC4_key >--(second)-->--,
                                         /|\                   |
                                          |                    |
 .-<--|__eeprom_key__|-->-----------> HMAC_SHA1                |
 |                                       /|\                   |
 |                                        |                    |
 |                        .--->-----------'                    |
 |                        |                                    |
 |  eeprom_data = |__data_hash___|__enc_conf__|__enc_data__|   |
 |                        |             |           |          |
 |                        |            \|/          |          |
 |                        |        rc4_decrypt <----|---------<|
 |                        |             |           |          |
 |                       \|/            |           |          |
 |                 (must be equal)      |          \|/         |
 |                       /|\            |      rc4_decrypt <---'
 |                        |             |           |
 |                        |            \|/         \|/
 |                        |      |_confounder_|____data____|
 |                        |       /            /    |
 |                        |      /            /     |
 |                        |     /            /      |
 |                        |    /            /       |
 |                        |   \|/          /       \|/
 `--->-----------------> HMAC_SHA1        /   |__HDKey__|__|
                                /|\      /         |
                                 \______/          |
                                                   |
               .-------------------------<--------'
               |
               |              model_number   serial_number
               |                      \        /
               |                       \      /
               `--->-----------------> HMAC_SHA1
                                           |
                                          \|/
                                      HD_password

This seems to be the easiest way to show the required calculations.

Basically there are several intermediate steps. First, generate the RC4_key from the eeprom_key and the data_hash (first 20 bytes of eeprom_data). Use the RC4_key to decrypt the encrypted confounder (8 bytes 20 bytes into eeprom_data) and the encrypted data (20 bytes 28 bytes into eeprom_data). Now generate an HMAC_SHA1 hash from the eeprom_key and the decrypted confounder and data. Verify that this hash matches the data_hash stored in the eeprom. If they don't match then the eeprom data is not correct. If the hashes match then the first 16 bytes of the decrypted data field is the HDKey.

Once you have the HDKey get the model and serial number from the drive. Generate an HMAC_SHA1 hash from the HDKey, model and serial numbers. The resulting 20 bytes are the HD password. The remaining 12 bytes needed for the password are zeros.

Unlocking

Before connecting an Xbox HDD to a PC for a backup or modification, the drive must first be unlocked. This can be done with alternative dashboards (such as EvoX). But beware, once you unlock the disk you cannot use it with an official BIOS until you re-lock the disk! For this reason it is suggested to use a patched BIOS which does not require the disk to be locked. If you are unable to run unsigned code (needed to unlock the HDD before powering off), it is possible to hot-swap the drive after the Xbox has started. This is not a suggested method, but it has been known to work. The idea is that you start the Xbox and wait for the dashboard, at which point the drive will be unlocked. Then, while the Xbox is running, you disconnect the IDE cable (but not the power!), and then connect the drive to your PC. Then the drive can be mounted for read/write (using XboxHDM), or imaged directly.

Unlock via Serial for Seagate drives

Look here: http://www.os2museum.com/wp/seagate-serial-talk/

Universal Unlock Method(s)

TODO

FIXME:

  • Provide more info on locking/unlocking procedure.
  • Provide details about the key and how it can be derived from the EEPROM data.

How To: Backup an HDD

There are two general methods to back up your HDD: copying the files, or creating a byte-for-byte image of the drive.

Method 1: File Copy

This is an acceptable backup method, but it is not as accurate an exact copy. This method requires less work to create the backup, but more work to re-create a usable disk image. The dashboard files (found in C:) are the most essential part of a backup, and a complete disk image can be re-created (with some effort) with a copy of the dashboard files using a tool such as XboxHDM.

Remote

Simply run an XBE on your Xbox that provides an FTP server. This is a standard feature for alternative dashboards (such as EvoX). Then connect to your Xbox from another system and copy all files in C: and E:.

Direct

Unlock the HDD, connect it to your PC, mount the drive (see FATX), copy the files.

Method 2: Exact Copy

This is the most accurate method to backup your hard disk. This method requires more work to create the backup, but does not require any effort to create a usable disk image like the first method. There are multiple ways to implement this method, one is provided here.

Unlock the HDD, connect it to your PC using a USB-IDE adapter (available for ~$20USD). In GNU/Linux and other *NIX variants, DD can be used to perform the block copy. For example: sudo dd if=/dev/sdb of=xbox_hdd.raw bs=512. append status=progress to see the progress during copying if you run a recent distro, like so: sudo dd if=/dev/sdb of=xbox_hdd.raw bs=512 status=progress.If you're dumping an original Xbox HDD (capacity 8G or 10G), this will finish pretty quickly. The files can be extracted by mounting the filesystems in the image (see FATX).

Further Reading

  • Draft: ATA/ATAPI Command Set - 2 (ACS-2) - T13/2015-D - Revision 7, June 22, 2011 (t13.org); Note: T13 drafts are freely available, only the final standards are behind a paywall.