C++ Code to Access Linux Hard Disk Sectors (with a LoopBack Virtual Hard Disk)

Hi all,

I'm kind of new to programming in Linux & c/c++. I'm currently writing a FileManager using Ubuntu Linux(10.10) for Learning Purposes. I've got started on this project by creating a loopback device to be used as my virtual hard disk. After creating the loop back hard disk and mounting it has the following configuration.

$> sudo fdisk -l /dev/loop0

Disk /dev/loop0: 10 MB, 10977280 bytes
255 heads, 63 sectors/track, 1 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/loop0 doesn't contain a valid partition table

Now what I want to do is develop a c++ program to read & write files to this loop back device,which I'm using to simulate an actual hard disk,at the blocks & sectors level. So far I've come up with the following code. But I'm still unable to read files from the hard disk one block at a time.

#include <iostream>
#include <stdio.h>

using namespace std;

int main()
{

    char block[512];
    int length=0;
    cout<<"Implementation of the File Handler Read Method..."<<endl;

    FILE *f = fopen("/dev/loop0", "r");
    if(f == NULL)
    {
        cout<<"Error In Opening the HardDisk File Retuning Error..."<<endl;
        return -1;
    }

    //Read One Block of Data to Buffer
    length = fread(block, 1, sizeof(block), f);

    /* Do something with the data */
    cout<<"Length : "<<length<<endl;

    return 0;
}

When I run this Program All what I get is the message for NULL.
"Error In Opening the HardDisk File Retuning Error...".
Could you please help me by pointing what am I doing wrong here ?. So I could open the loopback device as a file an access it at the sectors & block level.

Check the perms on /dev/loop0

ls -l /dev/loop0

HI dsw,

I changed the permissions of the /dev/loop0 to chmod 777 & then ran the command you gave and got the following results :

brw-rw---- 1 root disk 7, 0 2011-01-26 16:39 /dev/loop0

But now when I run the above Program I've written using the command prompt the following output appears :

Implementation of the File Handler Read Method...
Segmentation fault

Well it looks like you've got past the initial problem of the fopen() by changing perms.

I suspect you need to setup the fd first by doing an ftell() & fseek() as per the following example which should help you.

fread - C++ Reference

Failing that, add the -g flag to your g++ line and run it through gdb.

Hi dsw,

Thanks for the reply. I will check that.

Personally I'd use the open()/read()/write() calls instead of the stdio fopen()/fread()/fwrite() calls. You'll get more direct, unbuffered access; more useful error messages via perror() or errno; and a real file descriptor to call ioctl() on should you end up needing it. (which you might, dealing with block devices.)

Hi Corona688,

Could you please give me an example code sample using the above 03 methods. A pointer to get me started on them ?. Because I feel I'll have to deal with the file descriptor at some point & I feel it wouldn't be optimum to do so using the <stdio.h> methods. could you give me some help for me to get started with the : open()/read()/write() please ??.

These commands all have man pages accessible with 'man 2 open', 'man 2 read', 'man 2 write'. Also lseek.

#include <sys/types.h>
#include <unistd.h>
#include <sys/fcntl.h>
int main(void)
{
        char buf[512];
        ssize_t rb,wb;
        int fd=open("/path/to/dev", O_RDWR);
        if(fd < 0)
        {
                perror("Couldn't open device");
                return(1);
        }

        rb=read(fd, buf, 512);
        fprintf(stderr, "Read %d bytes\n", (int)rb);

        if(lseek(fd, 0L, SEEK_SET) != 0L)
        {
                perror("Couldn't seek");
                close(fd);
                return(1);
        }

        wb=write(fd, buf, rb);
        fprintf(stderr, "Wrote %d bytes\n", (int)wb);

        close(fd);
        return(0);
}

Hi Corona688,

I used your code & it works like a charm. I organized the code into simple separate functions as shown below.

#include <sys/types.h>
#include <unistd.h>
#include <sys/fcntl.h>
#include <cstdio>
#include <iostream>

#define DISK_NAME "/dev/loop0"

using namespace std;


/**
Read a set of Blocks into the Buffer
Block_SIZE=512bytes
*/
int readFromDisk(char *buffer,int length)
{
  ssize_t readBytes;
  int fd=open(DISK_NAME, O_RDWR);

   if(fd < 0)
    {
        perror("Couldn't open device");
        return 1;
    }

    readBytes=read(fd, buffer, length);
    cout<<"Read Bytes : "<<(int)readBytes<<endl;
	close(fd);

 return 0;
}

/**
Write the Buffer Content to HDD
Block_SIZE=512bytes
*/
int writeToDisk(char *buffer,int length)
{
    ssize_t writeBytes;
    int fd=open(DISK_NAME, O_RDWR);

	writeBytes=write(fd,buffer,length);
    cout<<"Wrote Bytes : "<<(int)writeBytes<<endl;
	close(fd);

 return 0;
}

/**
Seek out a Particular Block In the Disk
Block_SIZE=512bytes
*/
int seekDisk()
{
    cout<<"Seeking the File Disk..."<<endl;
    int fd=open(DISK_NAME, O_RDWR);
	if(lseek(fd, 0L, SEEK_SET) != 0L)
        {
                perror("Couldn't seek");
                close(fd);
                return(1);
        }
    close(fd);
    return 0;
}



int main(void)
{

cout<<"\n"<<"Running the OS File Manager....Main"<<endl;
/*
Test The File Manager
*/
char buffer[1024];
readFromDisk(buffer,512);
seekDisk();
writeToDisk(buffer,300);

return 0;
}

Now I'm having trouble figuring out a way to create a file using the above code. Could you please help me out here by giving me a point to start on working please ?.

---------- Post updated at 06:54 AM ---------- Previous update was at 01:41 AM ----------

Could Anyone Please Help me with this ?. I need to use the following code I've come up so far & figure out a way to create a file inside the block device. Following is my code.

#include <sys/types.h>
#include <unistd.h>
#include <sys/fcntl.h>
#include <cstdio>
#include <iostream>

#define DISK_NAME "/dev/loop0"

using namespace std;


/**
Read a set of Blocks into the Buffer
Block_SIZE=512bytes
*/
int readFromDisk(char *buffer,int length)
{
  ssize_t readBytes;
  int fd=open(DISK_NAME, O_RDWR);

   if(fd < 0)
    {
        perror("Couldn't open device");
        return 1;
    }

    readBytes=read(fd, buffer, length);
    cout<<"Read Bytes : "<<(int)readBytes<<endl;
	close(fd);

 return 0;
}

/**
Write the Buffer Content to HDD
Block_SIZE=512bytes
*/
int writeToDisk(char *buffer,int length)
{
    ssize_t writeBytes;
    int fd=open(DISK_NAME, O_RDWR);

	writeBytes=write(fd,buffer,length);
    cout<<"Wrote Bytes : "<<(int)writeBytes<<endl;
	close(fd);

 return 0;
}

/**
Seek out a Particular Block In the Disk
Block_SIZE=512bytes
*/
int seekDisk()
{
    cout<<"Seeking the File Disk..."<<endl;
    int fd=open(DISK_NAME, O_RDWR);
	if(lseek(fd, 0L, SEEK_SET) != 0L)
        {
                perror("Couldn't seek");
                close(fd);
                return(1);
        }
    close(fd);
    return 0;
}



int main(void)
{

cout<<"\n"<<"Running the OS File Manager....Main"<<endl;
/*
Test The File Manager
*/
char buffer[1024];
readFromDisk(buffer,512);
seekDisk();
writeToDisk(buffer,300);

return 0;
}

Can anyone lend me a helping hand here please ?.

'man 2 creat'

creat(2) operates on file descriptors as does the rest of the stdio.h calls - give the manpage a view.

-or-

you can OR the O_CREAT flag with O_RDWR in open(2) to create the file if it doesn't already exist.

O_RDWR | O_CREAT

Note the omitted trailing 'e' to creat(2).

Why are you opening and closing the file every time you do something?

This especially becomes useless:

/**
Seek out a Particular Block In the Disk
Block_SIZE=512bytes
*/
int seekDisk()
{
    cout<<"Seeking the File Disk..."<<endl;
    int fd=open(DISK_NAME, O_RDWR);
	if(lseek(fd, 0L, SEEK_SET) != 0L)
        {
                perror("Couldn't seek");
                close(fd);
                return(1);
        }
    close(fd);
    return 0;
}

...because the seek position gets reset every time you open it anyway. You should open it once and keep using the number it gives you. Your read code would end up looking like int readFromDisk(int fd, char *buffer,int length) { ... } iow exactly like normal read...so you might as well get used to ordinary read call instead.

You should also be doing something more with the number of bytes written and read than just printing it. read() and write() will occasionally surprise you with less than you asked for.

Hi Corona688,

I just reorganized my code & came up with the following structure & it works like a charm. But I'm still having trouble creating a file inside the block device using the

int writeToDisk(int fileDisId,char *buffer,int length)

method. Could you please point me from where I could get started. I want to simulate the working of the real os File Manager,which does it at the sector's level when writing a file to the disk. So I was wondering how could I use the above write method to achieve this. Could you please advice me on this as how may I proceed ?. Will I need to maintain a FAT like table when creating files so I could know in which sectors of the virtual hard disk the file is stored ?.

#include <sys/types.h>
#include <unistd.h>
#include <sys/fcntl.h>
#include <cstdio>
#include <iostream>
#include <string>

#define DISK_NAME "/dev/loop0"

using namespace std;


/**
Initialize the Virtual Disk File
*/
int initDisk()
{
    int fd=open(DISK_NAME, O_RDWR);

    if(fd < 0)
    {
        perror("Couldn't open device");
        return 1;
    }
    return fd;
}


/**
Read a set of Blocks into the Buffer
*/
int readFromDisk(int fileDisId,char *buffer,int length)
{
    ssize_t readBytes;
    readBytes=read(fileDisId, buffer, length);
    close(fileDisId);
    return (int)readBytes;
}

/**
Write the Buffer Content to HDD Blocks
*/
int writeToDisk(int fileDisId,char *buffer,int length)
{
    ssize_t writeBytes;
    writeBytes=write(fileDisId,buffer,length);
    close(fileDisId);
    return (int)writeBytes;
}

/**
Seek out a Particular Block In the Disk
Block_SIZE=512bytes
*/
int seekDisk(int fileDisId)
{
    cout<<"Seeking the File Disk..."<<endl;

    if(lseek(fileDisId, 0L, SEEK_SET) != 0L)
    {
        perror("Couldn't seek");
        close(fileDisId);
        return(1);
    }
    close(fileDisId);
    return 0;
}



int main(void)
{

    cout<<"\n"<<"Running the OS File Manager....Main"<<endl;

    /*
    Test The File Manager
    */

    int diskId;


    char buffer[1024];
     diskId = initDisk();
    cout<<"No of Bytes Read : "<<readFromDisk(diskId,buffer,1024)<<endl;

     diskId = initDisk();
     seekDisk(diskId);

    diskId = initDisk();
    cout<<"No of Bytes Written : "<<writeToDisk(diskId,buffer,1024)<<endl;

    return 0;
}

---------- Post updated at 11:48 AM ---------- Previous update was at 11:23 AM ----------

Hi dsw,

Thank you very much for your response. But what I need is to use the following method.

int writeToDisk(int fileDisId,char *buffer,int length)

So I could simulate the working of the read file manager by creating a file @ the sector's level. I know I might sound ridiculous. But this is for a learning purpose & I would really appreciate if you could correct my approach if I'm wrong & guide me here :).

The way you're using it, it contains no "files". A block device is nothing but a giant pile of undifferentiated blocks, ordered from first to last.

If you want to create a file in it, you have to know what type of partition it is, and follow that partition system's rules for how blocks are arranged into directories and files.

Hi Corona688,

So you mean I should find how the ext3 (as I've formatted my loop back disk with ext3 with mkfs tool) creates files & directories on the hard disk drive ?. Is that approach correct ?. If yes where do you suggest I start reading on programming material for me to get started please ?.

Yes, that's what I mean, you have to understand how ext3 works to mess with it raw. With the bits of it you've overwritten now, it probably needs a reformat to be valid ext3 again.

Do you need to use ext3? If not, I'd reformat it with /sbin/fsck.msdos -F 16 filename /sbin/mkfs.msdos -F 16 filename as the MSDOS filesystem is far, far simpler: just a big table, instead of a tree.

You might find the hexdump utility handy to show you what data you should expect to find in what places. Here's part of a hex dump of a flash drive I had lying around:

$ hexdump -C 8gflash-2011-01-27.vfat | head
00000000  eb 58 90 29 5e 76 56 4b  49 48 43 00 02 08 90 08  |.X.)^vVKIHC.....|
00000010  02 00 00 00 00 f8 00 00  20 00 10 00 80 1f 00 00  |........ .......|
00000020  80 40 ef 00 b8 3b 00 00  00 00 00 00 02 00 00 00  |.@...;..........|
00000030  01 00 08 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000040  00 00 29 53 6a af d5 20  20 20 20 20 20 20 20 20  |..)Sj..         |
00000050  20 20 46 41 54 33 32 20  20 20 fa 33 c9 8e d1 bc  |  FAT32   .3....|
00000060  f8 7b 8e c1 bd 78 00 c5  76 00 1e 56 16 55 bf 22  |.{...x..v..V.U."|
00000070  05 89 7e 00 89 4e 02 b1  0b fc f3 a4 8e d9 bd 00  |..~..N..........|
00000080  7c c6 45 fe 0f 8b 46 18  88 45 f9 38 4e 40 7d 25  ||.E...F..E.8N@}%|
00000090  8b c1 99 bb 00 07 e8 97  00 72 1a 83 eb 3a 66 a1  |.........r...:f.|

If you want to use a proper Linux filesystem, try ext2 instead of ext3, it's pretty much the same as ext3 but with a lot of journalling fluff cut out.

These links are good starting points on FAT and ext2.

Hi Corona688,

I think going with the MS-DOS filesystem (FAT-16) is a much better Idea. I formatted my current loop back device using the command you gave as follows :

$ sudo fsck.msdos -F 16 /dev/loop0

But I get the following error.

 
fsck.msdos: invalid option -- 'F'

Could you please verify how can I do this correctly ?. So I could format my loop back device with FAT-16 (for DOS file system) mode ?.

Did I say fsck!? :wall: Sorry! I meant /sbin/mkfs.msdos -F 16 filename

Hi Corona688,

I found the following article through a Google search(while awaiting your reply) on creating a floppy drive with the loop back device and got it mounted.

CREATING A Virtual MS-DOS Floppy Inside Linux.

Is the above approach correct ?.
Anyways now I'm back where I began as of how can I create files & folders inside the loop back device. Now with the MS-DOS FAT-16 file system. If you have any direct pointers to get me started on implementing this it would be a great help :).

Floppies use the FAT12 filesystem, which is good at not wasting space, but very hard to use. Its FAT entries are, as the name suggests, twelve bits in size. That's one and a half bytes, like this:

00 10 15 05 80 90

You can't make an array of that. You'd have to tease the data out the hard way, figuring out where the entry starts, which whole bytes and which half bytes should be retrieved, etc, etc.

You don't need to use the loop device at all to make the DOS filesystem, just to mount it. mkfs.msdos will warn you a file's not a device but write to it anyway.

Be sure to run losetup -d /dev/loop0 when you're done with that tutorial, or you'll find the loop device still busy when you try to use it for something else.

The instructions I posted in the post before yours should work fine for creating a FAT16 filesystem on a file 500 megabytes or smaller. FAT16 entries fit nicely in an array of 16-bit integers.

You should start making data structures to hold information so you don't have to find absolutely everything by byte offsets all the time:

#include <stdint.h> // standard sized integer types
// to make sure the compiler doesn't insert extra space in the structure
#pragma pack(push,1)
typedef struct bootsector
{
        uint8_t jump[3];
        uint8_t oemname[8];
        uint16_t bytes_per_sector;
        uint8_t bytes_per_cluster;
        uint16_t reserved;
        uint8_t fats;
        uint16_t root_entries;
        uint16_t total_sectors;
        uint8_t media_descriptor;
        uint16_t sectors_per_fat;
        uint16_t sectors_per_track;
        uint16_t heads;
        uint32_t hidden;
        uint32_t total;
} bootsector;
// reset structure definitions to normal
#pragma pack(pop);

...

bootsector bs;
read(fd, &bs, sizeof(bs));

fprintf(stderr, "%d bytes per sector\n", bs.bytes_per_sector);

All that's gleaned from information on the wiki, and it appears to match the dump I posted above(the OEM name is indeed filled with garbage for some reason!)

Look if the values you get in that structure seem normal. Try to use the information in it to find other structures. See if you can find the information you need to piece the partition together. You have to understand what's going on in those structures before you can alter them properly.

I hope this gives you the idea as I don't have the time to do absolutely everything for you.

Hi Corona688,

Thank you very very much for the immense support you've given me throughout this thread. I've learned a lot from you since I started this thread :). I understand what I should do and how I should proceed. Thanks you very much sir.

I've tried the HexDump Command you gave me on my Virtual MS-DOS(FAT-16) Floppy Drive & as follows :

sudo hexdump -C /dev/loop0 | head

and following is the output I got :

00000000  eb 3c 90 6d 6b 64 6f 73  66 73 00 00 02 01 01 00  |.<.mkdosfs......|
00000010  02 e0 00 40 0b f0 09 00  12 00 02 00 00 00 00 00  |...@............|
00000020  00 00 00 00 00 00 29 26  7a d7 50 20 20 20 20 20  |......)&z.P     |
00000030  20 20 20 20 20 20 46 41  54 31 32 20 20 20 0e 1f  |      FAT12   ..|
00000040  be 5b 7c ac 22 c0 74 0b  56 b4 0e bb 07 00 cd 10  |.[|.".t.V.......|
00000050  5e eb f0 32 e4 cd 16 cd  19 eb fe 54 68 69 73 20  |^..2.......This |
00000060  69 73 20 6e 6f 74 20 61  20 62 6f 6f 74 61 62 6c  |is not a bootabl|
00000070  65 20 64 69 73 6b 2e 20  20 50 6c 65 61 73 65 20  |e disk.  Please |
00000080  69 6e 73 65 72 74 20 61  20 62 6f 6f 74 61 62 6c  |insert a bootabl|
00000090  65 20 66 6c 6f 70 70 79  20 61 6e 64 0d 0a 70 72  |e floppy and..pr|

I hope this information will also be useful as I proceed as you've mentioned earlier. Thanks again for all your help & time :).