lzjb compression in ZFS

sushma_pawar · November 23, 2009, 2:47am

Hi all,

I am working on a project based on ZFS. The aim of this project is to read the raw data on disk for data recovery purpose. I started by reading the ZFS documentation of Sun Solaris. I somehow reached UBERBLOCK array of first partiton, and now as given in the document I am reading the block pointers and trying to reach the meta object set. I am getting some structure after I read the Block pointers offset in UBERBLOCK and doing some calculation on that, but I am not sure of the structure. I Want to know about the exact calculations required to reach meta object set and reading the compressed bytes of meta object set. Thanks in advance.

jlliagre · November 23, 2009, 4:19am

The best place to look would be ZFS source code like:
Cross Reference: /onnv/onnv-gate/usr/src/uts/common/fs/zfs/lzjb.c
By the way, latest OpenSolaris code includes a new zpool feature (ueberblock rollback) that allows recovery in many corruption situations.

sushma_pawar · November 23, 2009, 4:41am

Hi,

Thanks a lot for your reply. Actually I want to traverse the hard disk to retrieve the raw data and so I am going through Uberblock block pointers. I have been through the site u have given, but unable to get the clear idea as how to read bytes from Meta Object Set and how to decompress the bytes to reach the dnode array for data.

jlliagre · November 23, 2009, 4:48am

Although partially outdated, this is the best document describing how to access data on ZFS:
Welcome to OpenSolaris.org (Main.WebHome) - XWiki

sushma_pawar · November 24, 2009, 12:12am

Hi,

I have visited the site which u gave, but dit not get any kind of information.

---------- Post updated 11-24-09 at 12:12 AM ---------- Previous update was 11-23-09 at 11:54 PM ----------

Hi,

I want to know how to reach meta object set from UBERBLOCk. I mean what calulation are required to be done on block pointers we get in UBERBLOCK.

jlliagre · November 24, 2009, 5:05am

This is quite an overstatement. Anyway, here are other documents that will help you understanding ZFS internals:
http://www.osdevcon.org/2009/slides/zfs\_internals\_uli_graef.pdf
http://www.eecs.harvard.edu/cs261/papers/beebe09.pdf
http://www.bruningsystems.com/osdevcon_draft3.pdf

If you still fail to find an answer to that question with the help of the previous documents and despite having the source code available, I'm afraid you'd rather give up and switch to a simpler project.

sushma_pawar · November 24, 2009, 5:33am

Hi,

I understood the concept very well, I have already been through all this documents. The major hindrance is compression, as given in document the meta object set is compressed and the first byte of the meta object set is always "type", but the sector where I am reaching is having a structure with first byte as 00 (DMU_OT_NONE) which means "unallocated object". I have verified the structure through all the UBERBLOCK, but I am getting the same type of structure at different sectors.As per the document the structure should always contain block pointers, but I am not getting any sort of pointers, may be this dnode is compressed. Attached is the jpg of the sector where I am reaching through UBERBLOCK's block pointers. I want to know if this the right sector, and if YES how to reach the next level dnode through this sector.

jlliagre · November 24, 2009, 5:45am

I can't do that project for you. If the documents aren't detailed or accurate enough, the source code will show the actual way things are done. Source code is obviously authoritative.