I have this large text file that when unzipped has about 2GB. I split this one into multiple(more than 5 million) files and now I have a folder of about 20GB, how is this possible?
Asked
Active
Viewed 60 times
1 Answers
8
Filesystem overhead.
Every time you create a file, the filesystem reserves space on disk to store the file name, and other metadata like its permissions, creation date, etc. It also has to maintain a sort of index of all the (possibly sparse) blocks occupied by each file. The index might be a simple list or a tree, depending on the filesystem.
Finally, note that most filesystems can not allocate only a portion of a block for a file, so the allocated space needs to be rounded up to the block size. This rounding affects both the file blocks, and the index blocks.
chi
- 14,704
- 1
- 31
- 40