Re: storing a million of small files Oracle (or other db) vs. File System

I would say that as a rough guide, that file systems are greater at looking after a small number of large files, and databases are great at looking after a large number of small files...

hth

--
==============================
Connor McDonald

http://www.oracledba.co.uk

"Some days you're the pigeon, some days you're the statue..."
<cypher_at_punk.net> wrote in message news:9h71tb$b92$3_at_pita.alt.net...


> In comp.unix.solaris Rich Teer <rich_at_rite-group.com> wrote:

> #   On 23 Jun 2001 cypher_at_punk.net wrote:

> #

> #  > How about keeping the files in one or more directories

> #  > and manipulating symlinks instead?

> #  >

> #  > It might speed things up a bit.

> #

> #   Hmm, but wouldn't the directory that holds the symlinks then become

> #   a bottleneck?  If a directory has a million entries in it, searching

> #   it is gonna be slow (linear) whether the entries are files or

 symlinks.
>


> Yes, that's why I said:

>


>     "How about keeping the files in one or more directories"

>


> Because neither of us would put a million files

> in one directory.

>


> Since the initial setup is to have the files in once

> directory, it seems like it doesn't matter where they

> are as long as they can be found. So the hash-filename

> suggestion is a good one.

>


> When he ran a "64K files" delete, how exactly was

> it implemented? One thing you don't want to do is

> run 'ls' on the directory.

>


> ----

>


> Let's say hard links are used.

>


> The 64K (or whatever) deletion now means just removing

> the link, not having to do the UFS cleanup for each file.

>


> At some chosen cleanup time (2AM?), run (use a non-ls

> program) a scan for all files in the "real" directories

> with only one reference to them and delete them.

>


> That is a good shift for daily performance.