eMule uses various means to make sure that files in the network are
shared and downloaded without errors. Should an error occur, called
corruption, eMule has advanced functions to correct this corruption
with a minimum of additional data to redownload.
File Hash and ICH - Intelligent Corruption Handling
File Hash, Part Hashes & Hashset
For every file shared in the network an unique identification value is
created using the mathematical crypto algorithm MD4. This value is
called file hash and is contained in every standard eD2k link, e.g.
where 6744FC42EDA527B27F0B2F2538728B3E is the file hash making this
file uniquely identified all over the network.
This File Hash is calculated by dividing the entire file into
parts of 9.28 MB. For each of the parts a Part Hash is calculated using
the same MD4 algorithm. These Part Hashes, called
Hashset, are then used to calculate the final File Hash. For
example a 600 MB file would be divided into 65 parts each with its own
Part Hash which are then used to create the final File
To make sure that eMule always receives the correct Hashset a special
link can be created containing this, e.g.
where the p= value denotes the Hashset. Each Part
Hash is divided by a ":". This file has a size of 12043984 Bytes
(=11.49 MB) which means it has one full 9.28 part and the rest left to
11.49 MB resulting in two Part Hashes.
ICH Intelligent Corruption Handling
Whenever eMule finishes such a part it will be checked if the
downloaded data matches the Part Hash value for this finished part. If
yes, this part is offered for upload to help spreading it.
If no, a corruption has occurred and the part has to be downloaded
again. To avoid downloading the full 9.28 MB, ICH redownloads 180 KB
from the beginning of the part and then check the whole part again to
see if the Part Hash is now correct. If not, the next 180 KB are
downloaded, check again etc. until the part hash is correct again. In
the best case eMule has to download only 180 KB again if the corruption
is right at the beginning of the part. In the worst case the entire
part has to be downloaded again if the corruption is somewhere near the
end of the part. On average ICH saves 50% of redownloading in case of
AICH - Advanced Intelligent Corruption Handling
The standard ICH is pretty effective although it has its limitations as
only the whole 9.28 MB can be verified and no finer blocks. If more
than one position is corrupted or if malicious clients spread corrupted
data over and over again or even fake an entire Part Hash, ICH
is no long effective.
Here AICH will care for complete data integrity with a minimum cost of
redownloading or overhead by creating much finer hashes.
Root Hash, Block Hashes & AICH Hashset
This time our starting point are the 9.28 MB parts in a file. Each part
is divided into 180 KB blocks, resulting in 53 blocks per part and for
each block a hash value is calculated using the SHA1 hash algorithm.
These values are called Block Hashes and form the lowest level
of a complete AICH Hashset.
The image above shows how such a full hash tree is build upon the
blocks of a full 4 parts file. Each part contains 53 blocks resulting
in 212 Block Hashes which builds up a hash tree of further 7
Levels until the Root Hash is reached. The entire tree is
called AICH Hashset.
The green and yellow dots show the mathematical dependencies of the
smallest Block Hash to the Root Hash. This means if
we have a trusted Root Hash the entire tree can be verified against
eMule can create links containing the Root Hash, e.g.
where h= is the Root Hash. For releases this value should be
included as it significantly improves the corruption resistance of the
file by providing a trusted Root Hash. Read Trusting the
Recovering from a corruption
Whenever eMule detects a corruption in a part it requests a Recovery
Packet from a random client with a complete AICH Hash Set. This
Recovery Packet contains all 53 Block Hashes of the corrupted
part and a number of Verifying Hashes of the entire hash tree.
The image above shows a Recovery Packet for a 4 parts file. The number
of Verifying Hashes is determined by the part count of the
file (2^x >= 'part count', with x = Number of Verifying Hashes).
After receiving the Recovery Packet eMule checks the Verifying
Hashes against its trusted Root Hash. If they match, eMule checks
all 53 blocks of the corrupted part against the Block Hashes
from the Recovery Packet. AICH then restores all blocks which match
against their Block Hash to leave only the corrupted block(s)
In the Log a successful recovery will look like:
09.09.2004 02:43:43: Downloaded part 6 is corrupt ([file])
09.09.2004 02:43:46: AICH successfully recovered 8.22 MB of 9.28 MB
from part 6 for [file]
Trusting the Root Hash
The best thing is downloading from a link with a Root Hash.
Assuming that the source of this link is trustworthy the Root Hash is
trusted at once and saved to disk for this file.
If no Root Hash is provided in the link eMule has to trust the Root
Hash, which the sources for the file send. It only trusts a Root
Hash if at least 10 different sources send the same value and if
at least 92% of all sources agree to this value. Because this Root
Hash is not as trustworthy it is only valid for the current
session and does not get saved nor can links with Root Hash be
Once eMule has built an entire AICH Hashset, i.e. the file is
finished, it starts propagating the Root Hash to other
- New releases or rare files will probably not have enough full
sources to generate a trusted Root Hash. It is recommended to
Release files with attached hash.
- If there is no Root Hash or even a faked one eMule will be
able to successfully download and finish the file under normal
conditions. The AICH feature cannot be used in this case.
- As AICH Hashsets can be very large they are not stored in memory
but in the known2.met and are only read upon demand.
- AICH will only be effective for eMule clients v.44a and above but
retains backward compatibility to older clients.
Applies to: v.44a+
Update on: 2004-09-11 by Monk