28

tar on a directory mydir will archive hidden files and hidden subdirectories, but tar from within mydir with a * wildcard will not. Is this a known inconsistency or bug?

Edit: Additional information. tar from within mydir with a * wildcard will not "see" nor archive hidden files and hidden subdirectories in the immediate directory. However, in the non-hidden subdirectories of mydir hidden files and hidden subdirectories will be archived. In other words, deeper in the directory tree the hidden objects will be archived.

H2ONaCl
  • 10,644
  • 14
  • 70
  • 114

7 Answers7

42

With wildcard it will not work. You have to specify . (current directory) if you mean full directory including hidden files. You can do

tar -cvpzf test.tgz .
Madhur Ahuja
  • 22,211
  • 14
  • 71
  • 124
  • 10
    But be careful of placing the new tar inside the same directory you are tarring. You can get into a nasty loop. – Keith Jan 01 '11 at 02:22
  • 15
    tar cvpfz test.tgz --exclude=*.tgz . – H2ONaCl Jan 01 '11 at 02:53
  • 7
    Or just put it somewhere else: `tar czpf /var/tmp/mytar.tgz .`. – Keith Jan 01 '11 at 04:18
  • Putting it somewhere else might mean using sudo which might cause additional work (which you might want to avoid) in dealing with the GUI-based scheduler or crontab. – H2ONaCl Jan 01 '11 at 04:46
  • 1
    But you can always write to /var/tmp. That's how I usually do it, in fact. – Keith Jan 01 '11 at 05:12
  • I wrote the solution `tar cvpfz test.tgz --exclude=*.tgz .` in a comment above because it solves the problem of not wanting to tar the new tar file but I do not recommend it. I think it's better to invoke tar from outside (that is from the parent directory) so you explicitly name the directory to tar and avoid the `.` (dot). That has the benefit of being explicit and the benefit that you avoid needing the `--exclude` option. – H2ONaCl Jun 17 '18 at 07:16
13

The answer is that the * wildcard is handled by the shell and it just does not expand to things that start with a dot. The other wildcard ? also does not expand to things that start with a dot. Thanks to Keith for pointing out it is the shell that does the expansion and so it has nothing to do with tar.

If you use shopt -s dotglob then expansion will include things like .filename. Thanks to Andy.

Use shopt -u dotglob to turn it off.

Switching the dotglob option does not change ls itself. Rather it just changes expansion behaviour as exhibited in something like ls *.

Edit: My recommendations are in a comment below.

H2ONaCl
  • 10,644
  • 14
  • 70
  • 114
  • That's interesting but could you show it in an example command? – felwithe Sep 09 '16 at 15:38
  • 2
    @felwithe By using `shopt -s dotglob`, one could easily do something like `tar -cvpzf final.tar.gz *` inside of the directory whose content you wanna to tar – Rod Elias Apr 19 '18 at 14:08
  • @felwithe I did not show an example because I decided non-obvious behaviour happens when you tar with the * wildcard. I also think that changing the default behaviour with shopt is suboptimal. That is, avoid the non-obvious and avoid non-defaults. I decided it is better to explicitly name what you want to tar, if at all possible. – H2ONaCl Jun 17 '18 at 07:09
8

You can use:

tar -cvpzf test.tgz * .??*

But, this works only for hidden files with names > 2 (to prevent '.' and '..')

Markoj
  • 233
  • 2
  • 7
3

You can compress all the files / folders in your current directory (including .hidden) by using:

tar -zcvf compressed.tgz `ls -A -1`

The last argument are the folders you want to compress. If you pass it ls -A -1 , you are passing it all folders in your current directory but . and .. . When it comes to subdirectories, .hidden files are already included in the compression by default.

zurfyx
  • 31,043
  • 20
  • 111
  • 145
2

The shell expands the wildcards so tar doesn't even see it. You have to add them explicitly if you want to do that. (.*). However, it's most common to tar a single directory so that when you untar it all goes to the same place.

Keith
  • 42,110
  • 11
  • 57
  • 76
  • Actually tar on .* will go up (probably as high as the root) so you probably don't want to do that. – H2ONaCl Jan 01 '11 at 02:47
  • Correction: not as high as the root. – H2ONaCl Jan 01 '11 at 04:08
  • 2
    Oh right. It would be safer to use something like `.[a-zA-Z]*`. But, I always prefer to just tar one directory (parent, if need be). – Keith Jan 01 '11 at 04:16
  • This comment discusses both tar and the ls (list) command which is hard to see in this font. The shell might be responsible for expanding the * wildcard. However, for the . wildcard tar and ls behave differently so I don't know if the shell is providing a consistent behaviour in the case of the dot. ls . ignores hidden stuff (not consistent with tar) and ls -a . can see the .. (above) directory which is also not consistent with what tar will archive. tar only goes down with a . unless you do .* which we already discussed. – H2ONaCl Jan 01 '11 at 04:30
  • The dot(.) is not a wildcard in shell globbing. It is the name of the current directory. Old systems even had a physical directory entry of `.`. It is therefore consistent since the shell doesn't expand it. – Keith Jan 01 '11 at 05:12
  • However, to confuse things further, if you're using bash shell you can set the `dotglob` option (shopt -s dotglob) to disable the special handling of names starting with dot. – Keith Jan 01 '11 at 06:02
0
shopt -s dotglob

this will make the

Vatine
  • 20,782
  • 4
  • 54
  • 70
andy
  • 9
  • 1
-2

This will tar all hidden files and directories

tar -cvzf hidden.tgz `ls -a | egrep [.][^.]+`
brannala
  • 1