Filewatcher File Search File Search
Catalog
Content Search
» » » » » fpart-0.4.tbz » Content »
pkg://fpart-0.4.tbz:20229/share/doc/fpart/  info  downloads

fpart…  more info»

README

Fpart - README

What is fpart ?
***************

Fpart is a tool that helps you sort file trees and divide them into bags (called
"partitions"). It is developped in C and available under the BSD license.

It splits a list of directories and file trees into a certain number of
partitions, trying to produce partitions with the same size and number of files.
It can also produce partitions with a given number of files or a limited size.

Once generated, partitions are either printed as file lists to stdout (default)
or to files. Those lists can then be used by third party programs.

Examples :
**********

The following will produce 3 partitions, with (approximatively) the same size
and number of files. This can be used (for example) to launch several rsync
commands in parallel. 3 files, "var-parts.[0-2]", are generated as output :

$ fpart -n 3 -o var-parts /var

The following will produce partitions of 4.3 GB, containing music files ready
to be burnt to a DVD (for example). Files "music-parts.[0-n]", are generated
as output :

$ fpart -s 4617089843 -o music-parts /path/to/my/music

The following will produce partitions containing 10000 files each by examining
/usr first and then /home and display only partition 0 on stdout :

$ find /usr ! -type d | ./fpart -f 10000 -i - /home | grep '^0:'

The following will produce two partitions by re-using du(1) output. Fpart will
not examine the file system but instead re-use arbitrary values printed by du(1)
and sort them :

$ du * | fpart -n 2 -a

Installing :
************

To install fpart, get the 'Makefile' symlink point to the right Makefile,
e.g. :

# rm Makefile
# ln -s Makefile.linux Makefile

then, type in :

# make
# make install

See also :
**********

See fpart(1) for more details.

The partition problem is detailed here :

http://en.wikipedia.org/wiki/Partition_problem

I am sure you will also be interested in :

https://github.com/jbd/packo

which was developped by jbd as the original proof of concept.

Author / Licence :
******************

Fpart has been written by Gana´┐Żl LAPLANCHE <ganael.laplanche@martymac.org>
and is available under the BSD license (see COPYING for details).

Thanks to jbd for having given me the idea of this program !

Contributions :
***************

FTS code comes from FreeBSD sources (8.2-RELEASE) :
    lib/libc/gen/fts.c -> fts.c
    include/fts.h -> fts.h
It is available under the BSD license.

General process :
*****************

Here is a technical overview of how the program works :

Start
Read program arguments and path list
(or arbitrary values, either from stdin or as argument)
    /path/to/dir1  -> double-linked list of files found
    /path/to/file1 -> double-linked list of files found (linked to previous)
    /path/to/dir2  -> double-linked list of files found (linked to previous)
    /path/to/file2 -> double-linked list of files found (linked to previous)
If fixed-size number of partitions requested (option -n) :
    Creation of an array of pointers to file entries
    Sort that array given file size (qsort)
    Create a double-linked list of <n> partitions holding partitions
        information (total number of files, total size per partition)
    Associate each file entry to one of those partitions
    As a second pass, redispatch empty files (to try to get partitions with
        the same number of files)
Else (partitions limited to a given size or file number, options -s and -f) :
    Dynamically create our partitions as necessary (by linking new partitions
        to our double-linked list). Associate each file entry to one of
        those partitions
Print result
Cleanup
End
Results 1 - 1 of 1
Help - FTP Sites List - Software Dir.
Search over 15 billion files
© 1997-2017 FileWatcher.com