HUGE mmo files

I'm storing some images on a HFSQL database.
On disk, the images ara about 350MB

The resulting .mmo file is 3.7 GB (more than 10 times bigger!!)


PCSoft is not answering (as always)…

I'm doing something wrong?

The images are stored in a image/binary field and loaded with fieldname = dLoadImage(filename)

Ok, is dLoadImage that adds a lot of scrap.
If I use fLoadBuffer the size is what I'm expecting it to be (350 MB on disk, 350 MB on the .mmo file).

And I can use the record exactly as an image field
So, what's the difference between dLoadImage and fLoadBuffer? Or, better, why dLoadImage has a memory occupation of about 10 times of the loaded image?

Hi. I think it's completely normal. dLoadImage is used to load the (uncompressed) image into memory. So you were saving exactly the same information that you see when loading the image, but without compressing it according to the type of image file.

Rubén

Thank you for the answer.
I'll try with uncompressed images (BMP should do the work).

Hi Stefano.

Your problem is precisely that the images are stored as decompressed, so it increases the size so much. You should read the image in its original state (as it is on the disk) with fLoadBuffer and store that buffer in the memo field of your database.

In any case I don't know if storing 350 MB images in the database will be very efficient because in order to use them you will have to take them out of the database and then load them into the image field. It is a very big size. By reading the image directly from a file on the disk, the reader is able to read the information in an optimized way, presenting the image even without having read it completely, so the load seems faster.

On the other hand, if something affects the database and if it is corrupted will be affected many more images than if the images are each separately.

Rubén

Hi Ruben,

just to mention: there are some good reasons for storing images & PDFs in a database.

1 - in a multiuser environment all users have direct access to the images / documents. No local storages required. For multiuser environment one would have to provide access to a web site by using a link. Things do change fast .. is the link still valid? Are images / documents still there?

2 - Hopefully, the server has a regular routine to do a backup of the database. This makes sure that no image/document is lost.

Kind regards,
Guenter Predl
office@windev.at

We also store images inside the database. Works like a charm. In one project we already have over 1 million photos, coming from field service workers using a tablet. MMO file is over 150GB.

In all those years I had 1 or 2 issues with a corrupt image. But it turns out just the particular image was broken. It did not have an effect on the remaining part of the MMO file.

One nice thing of HFSQL btw is the option to automatically generate thumbnails. Works great and it helps speed up the application, while the thumbnails are loaded instead of the full image in for instance a looper control with a small image control inside.

I also have a document scanning project with an MSSQl server. Again no problems with that amount of images, but it lacks the thumbnail option.

As Gunter mentioned. Make sure you have proper backups, as always. That should not be a problem nowadays, even with huge files. Be ware though that you can NOT backup files when they are in use. With HFSQL it looks like you can copy the individual files, but the backup is not consistent in the first place.
Second: I did some test with large MMO files and although it looks like they can be copied (backup file had the same size), sometimes a great part of the file is empty!! You should either use the hot backup option of HFSQL (but it is very slow if you ask me) or you should shut down the database first and THEN do a file copy. On virtual servers like VMWare things are different btw.

Arie

Hi Gunter, I know, I use it often. I just mentioned his particular case of 350MB per photo. I don't think it's very efficient to store or retrieve images with those sizes.

Rubén

Hello Stefano,

I am not sure how effective it is, but there is also an option in the analysis definition of memo fields to force them to be compressed. It may help if space is a concern or it could just be something else to go wrong

Regards
Al