Possible problem with cpu00700F01_ver07000110_2018-02-09_1F2B007F.bin
mikebdp2 opened this issue · 11 comments
@platomav , I am worried that cpu00700F01_ver07000110_2018-02-09_1F2B007F.bin microcode for AMD fam16h Kabini CPU, which you have released as a part of MCE DB r65 commit of AMD microcodes , might have been accidentally cut at its' tail. Why I think so?
1) I've noticed that the older microcode for the same CPU ID "700F01" , that is inside the released-by-AMD "microcode_amd_fam16h.bin" container with just 1 microcode (available here - https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/amd-ucode) - was 0x0D82 size - as its written at 0x30 offset! - but when you copied it to your repository as cpu00700F01_ver0700010F_2014-02-19_EBB9EFA6.bin - as a part of MCE DB r52 commit - you cut its' tail at 0x0D60, ending with 0xC519377C value. Although, I see that inside AMD's "microcode_amd_fam16h.bin", after 0xC519377C value there were only zeroes: 0x0D82 - 0x0D60 = 0x22 of zeroes, and maybe it was OK to get rid of them (or maybe those zeroes should have remained? since AMD microcodes are closed source, we can't know it for sure)
2) However, you have released a new cpu00700F01_ver07000110_2018-02-09_1F2B007F.bin microcode, but its' size is also this trimmed 0x0D60. But what data was at the next 0x22 data, until the real 0x0D82 size? Were it zeroes again, or there were some not-zero values? Trying to find an answer to this question by myself, I've attempted to find any UEFI/BIOS update containing this microcode - so that I could verify by myself - but sadly I couldn't find
Please, could you tell the source of this microcode, from which UEFI/BIOS file it was extracted? ( I've registered at win-raid forums, if this information is private - you could tell it in a private message here https://www.win-raid.com/u15307_mikebdp.html ) Or, if you couldn't tell your source even privately, please could you verify by yourself - that in the end of this new microcode there also were 0x22 all zeroes?
Best regards,
Mike Banon
AMD, in their infinite wisdom, do not include a size or checksum at the microcode header after 2010. We cannot thus verify their integrity but the size is static per generation. For CPUID 70 and 73, the size is 0xD60, as can be seen at MCE. Anything after that is padding.
Because AMD lack basic standards, OEMs do whatever they want with the microcode size. At some BIOS you may see them with 0xD60 size (next starting at 0xD61) or with variable padding based on what the OEM likes (0x800, 0x1000, 0x700 etc).
I don't keep logs of where everything came from, that would be completely impractical and a huge waste of time.
@platomav Thank you very much for reply and your hard work. Actually AMD includes a size of microcode at the section header, which is right before the microcode header inside the AMD container with microcodes. microcode.c file from coreboot open source BIOS sources - is explaining AMD container format in a great detail; in particular:
* Container Header
* Section Header
* Microcode Header
* Microcode "Blob"
* Section Header
* Microcode Header
* Microcode "Blob"
* ...
* ...
* (end of file)
and
* SECTION HEADER (offset += 12+n)
* Total size = 8 bytes
* [0:3] Unique identifier signaling start of section (0x00000001)
* [4:7] Total size (m) of following microcode section, including microcode header
Size of any microcode (consists of " Microcode Header + Microcode "Blob" ") - is given at the Section Header that precedes this, and it is 0x0D82 in our case - not 0x0D60. I guess its just stupid OEMs do not copy the Section Headers when they extract the microcodes from AMD container.
Finally I found one of the possible sources of that microcode - the latest update for 15-af124ur laptop. Just checked, there are also 0x22 of zeroes after 0x0D60 offset, same as it was before at AMD's ...fam16.bin container - which tells that a real size of this microcode is 0x0D82 at its' section header
I don't know if its' a coincidence that the last 0x22 are zeroes also at this new microcode as well, but I believe it is a good idea to add 0x22 of zeroes to the end of cpu00700F01_ver07000110_2018-02-09_1F2B007F.bin and change the size of the microcode for this CPU ID 00700F01 at MCE DB from 0x0D60 to 0x0D82. Because thats what AMD is telling of its' size at the section header, at 0x30 offset from the beginning of ...fam16.bin container file
you could see 0x0D82 bytes
The AMD container is not useful to us because the BIOS images hold the microcode binaries only. The microcodes themselves do not unfortunately have a size and checksum field at their header, as I said earlier. AMD, once again in their infinite wisdom, mentions the actual microcode static size only at confidential OEM documentation.
I had the same reservation at the Zen microcodes which always have 0x80 padding (0xFF) at the end for no reason. At older versions of MCE, it would actually keep these as well and check if they were indeed 0x80, a form of sanity check on my part. At some point, I happened to find an official AMD mention of the actual Zen microcode size and guess what, it really did end before that stupid padding.
As for what you suggested, MCE must not extract more data because a lot of times you will actually merge 0x22 irrelevant bytes, either from the next microcode or the BIOS code with follows afterwards. As I explained, some OEM BIOS may keep each 70/73 microcode at 0xD60 or maybe 0x800, 0x1000, 0x2000 etc. Honestly, it is not the OEMs fault but rather AMDs.
From my experience these past few years, AMD really struggles when it comes to firmware structuring, design and standards. Intel for example has had the same good main microcode header since 1995 with sizes, checksum, encryption, platform support etc. On the other had AMD seems to have changed their mind around 10 times since 2002 when the first K8 microcodes appeared. Not only that, they made their own bad header worse every time.
@platomav Thank you again for detailed explanation. Now I understand that, although AMD's official size for this microcode is 0x0D82 (as its declared by them at the container's header), if the last 0x22 are zeroes - indeed there's a danger that OEM could cut away those zeroes and we accidentally get a small irrelevant part of BIOS image if we would still be extracting 0x0D82 size, so maybe you are right in keeping it 0x0D60
Only reason I'm still concerned about this, is - I can't know if those 00's at the end are just null data, or there is some hidden meaning behind them... At some files it is FF's which is really a null data, and if this microcode is being applied to some CPU's internal area FF's, by cutting those last 00's of the microcode we could affect the end result ?
The only way this AMD container was useful to me, is that from its' header I learned whats the real size of this microcode. But I agree with you : it would have been much more convenient if this information always was in the header of microcode, and not in the container header which is always stripped away...
Out of curiosity I checked the CPU 610F01 official microcode size inside the AMD's fam15h microcode container, and it is 0x0A20 - exactly matches your size of cpu00610F01_ver0600111F_2018-03-05_AC55EB96.bin. Maybe only for ...fam16h.bin vs your file, it is 0x0D82 vs 0x0D60 - not matching
These 00 are definitely padding, thus useless. No secret meaning there. As for whether the CPU will accept the microcodes, it will. Thing is, at some generations, the AMD microcodes are not even encrypted. Combine that with the fact that there is no checksum and you'll realize that the CPU Microcode Loader has no idea if the input microcode is proper. One could literally add garbage data and the CPU would accept it. You might see some funny stuff or not even a bootable system, but there is nothing to stop the CPU from accepting such binaries. I've seen a few papers in which researchers have actually done that to see how the CPU would behave. That is not possible at Intel systems, even from 1995.
Another example of stupidity. AMD decided in 2002 that 4 bytes was too much for them to reserve to hold the microcode size so they implemented one only. Upon finding some old patents, you will see that this byte is a multiple of triads (who thought of that?) so you need to multiply its hex value with 32 (0x10 * 32 = 0x200). But wait, these is more. At the next generation, they decided that the multiplier should be 30, not 32 (0x20 * 30 = 0x3C0). I mean, why? Was it really that much to keep it at 32 and just make the binary 0x400 bytes in size? No, it has to be 0x3C0. But wait, these is more. At the next generation they removed the size byte altogether (still reserved) and resorted to this "static" microcode binary size scheme.
I could go on and on about how bad AMD microcode structure, selection, integrity and security is. What I've learned is that I cannot trust AMD and need to manually verify things. So I can't really trust them when they indicate the 70/73 size as being 0xD82 when we see them everywhere as 0xD60. It doesn't mean that that's the true size of the microcodes of that generation unfortunately, even if the rest of them match to what we've seen on the field. All the sizes in MCE were determined manually.
@platomav Found a very interesting paper - https://www.dcddcc.com/docs/2014_paper_microcode.pdf On page 13 it tells:
Fault injection tests on the 15h microarchitecture clearly demonstrate the presence of encryption, as the updates must first be decrypted before metadata felds can be verified. These results show that the microcode update mechanism takes on average 753913 cycles, with a sample standard deviation of 114841 of cycles
In contrast, fault injection tests on the 12h microarchitecture indicates that encryption is not present. On average, the microcode update mechanism takes 1116 cycles, with a sample standard deviation of 132 cycles.
Also,
On the 15h microarchitecture, binary comparison of multiple microcode revisions for a single processor produces no useful similarities, indicating that a chained block or stream cipher is likely being used for encryption. Since changes to offsets 672 through 2719, measuring exactly 2048 bits in size, result in early termination, it is possible that this segment contains a hash that is used to verify the integrity of the encrypted update data, in an encrypt-then-MAC approach. Although it is possible to use the reverse, MAC-then-encrypt, this is rather unlikely, as not only is it less secure, but 2048 bits is also rather long for the block size of a block cipher.
So it looks like AMD has added the encryption to their microcodes as well as a hash ("checksum") to verify their integrity, sometime between the fam12h and fam15h architectures
Yes I am aware of that paper, it is included at the Readme of MCExtractor as well. This is why I said "at some generations" and not "all". For example, 60-67 are encrypted, but not 68.
Indeed, there is a table at the same page:
| Patch Data ID | Microarchitecture | Encryption | Memory Space |
|---|---|---|---|
| 8000h | K8, 0fh, 10h, 11h | N | 000h - C3Fh |
| 8001h | 14h | Y | ? |
| 8002h | 15h | Y | ? |
| 8003h | 12h | N | 00000000h - FFFFFFFF |
Can't find any info about 68 CPUs. Maybe they are the early engineering samples - just like the CPUs which have their IDs ending with 00 - e.g. cpu00630F00 and cpu00700F00, early core stepping
I think that fam16h (70/73) is also without encryption: not just because of its' 8003h patch data ID, but also its indicated at the coreboot BIOS sources, e.g. this file for fam15h - F15TnEquivalenceTable.c - contains both encrypted and unencrypted equivalence tables, while a similar file for fam16h - F16KbEquivalenceTable.c - doesn't have anything like that. By the way, I really wonder how they got the unencrypted equivalence table, did they just decrypt it
Yes I remember. His 2014 research is kind of old at this point so it lacks 16h and 17h family findings. But is still is one of the most prominent papers for AMD microcode analysis.
I think the equivalence tables can be found at the AMD containers for each generation. The OEMs also keep them somewhere at the BIOS so that it knows what microcode CPUID to load in relation to the CPU reported one. AMD makes their lives harder by doing this whole single microcode for multiple CPUIDs scheme.
You can tell if a microcode is encrypted or not just by viewing it with a hex editor. If you see a lot of padding in between such a zeros or FF, it is not encrypted.