13

I'm hacking around in some scripts trying to parse some data written by Javas DataOutputStream#writeLong(...). Since java always seems to write big endian, I have a problem feeding the bytes to od. This is due to the fact that od always assumes that the endianess matches the endianess of the arch that you are currently on, and I'm on a little endian machine.

I'm looking for an easy one-liner to reverse the byte order. Let's say that you know that the last 8 bytes of a file is a long written by the aforementioned writeLong(...) method. My current best attempt to print this long is

tail -c 8 file | tac | od -t d8

, but tac only seems to work on text (fair enough). I've found some references to dd conv=swab, but this only swaps bytes in pairs, and cannot reverse these eight bytes.

Does anyone know a good one-liner for this?

jww
  • 97,681
  • 90
  • 411
  • 885
Alexander Torstling
  • 18,552
  • 7
  • 62
  • 74
  • Please give a sample input / output and remove Java from the equation. Do you want to invert every 8 bytes, or all bytes? For all bytes: http://stackoverflow.com/questions/11461625/bashreverse-the-order-of-characters-in-a-string Similar: http://unix.stackexchange.com/questions/13137/in-bash-how-to-convert-8-bytes-to-an-unsigned-int-64bit-le – Ciro Santilli OurBigBook.com Oct 06 '15 at 20:51
  • 1
    @CiroSantilli六四事件法轮功纳米比亚威视: Nope, won't do. Problem already solved. – Alexander Torstling Oct 07 '15 at 12:25

8 Answers8

13

You could use objcopy:

$ objcopy -I binary -O binary --reverse-bytes=num inputfile.bin outputfile.bin

where num is either 2 or 4.

malat
  • 12,152
  • 13
  • 89
  • 158
Marc
  • 159
  • 1
  • 4
9

Resorted to Perl in the end. Used a one-liner which I found at PERL One Liners:

tail -c 8 file | perl -0777e 'print scalar reverse <>' | od -t d8

The 0777 separator char was a bit puzzling to me, but this page at debian admin seems to suggest that it is a placeholder for 'no record separator', triggering a complete reverse byte-per byte.

Other suggestions are welcome.

EDIT: Found another command in a comment to tac.c, which I downloaded from GNU coreutils:

Copy each FILE, or the standard input if none are given or when a FILE name of "-" is encountered, to the standard output with the order of the records reversed. The records are separated by instances of a string, or a newline if none is given. By default, the separator string is attached to the end of the record that it follows in the file.

Options: -b, --before The separator is attached to the beginning of the record that it precedes in the file. -r, --regex The separator is a regular expression. -s, --separator=separator Use SEPARATOR as the record separator.

To reverse a file byte by byte, use (in bash, ksh, or sh): tac -r -s '.\| ' file

Alexander Torstling
  • 18,552
  • 7
  • 62
  • 74
8

Used dd, Luke!

dd if=sourcefile of=resultfile conv=swab
Anton Chevychalov
  • 161
  • 1
  • 2
3

Note the next version of GNU coreutils (>= 8.23) will add the --endian={little,big} option to the od command

pixelbeat
  • 30,615
  • 9
  • 51
  • 60
2

I came up with this Perl one-liner to convert 4-byte integers from one endianness to another:

$ perl -e 'open F,shift; do { read(F,$a,4); print scalar reverse($a);} while(!eof(F));' bigend.bin > littlend.bin

That probably works fine on real Linux machines, but Cygwin bit me in the end, treating the binary file as text and inserting a 0x0D (aka CR) before each 0x0A byte (aka newline). But if you pipe to cat -, it seems to leave it alone. This works for me:

$ perl -e 'open F,shift; do { read(F,$a,4); print scalar reverse($a);} while(!eof(F));' bigend.bin | cat - > littlend.bin
Mark Lakata
  • 19,989
  • 5
  • 106
  • 123
2

BASH:

od -b -v -w8 | while read pfx b8 ; do [ "$b8" ] && echo -n 12345678 | tr 87654321 \\${b8// /\\} ; done

To be a bit more robust depending on the output style of od, it may need to compress spaces ( insert "| sed 's/ */ /g'" after the w8).

Artjom B.
  • 61,146
  • 24
  • 125
  • 222
Brian Carcich
  • 391
  • 2
  • 8
1

xxd has two flags -e and -g for your purpose.

    -e          little-endian dump (incompatible with -ps,-i,-r).
    -g          number of octets per group in normal output. Default 2 (-e: 4).

This way, you can do:

tail -c 8 file | xxd -e -g8
matovitch
  • 1,264
  • 11
  • 26
0

A simple Python approach that reverses every 4 bytes. Uses newer Python 3.8+ walrus operator:

import sys
while word := sys.stdin.buffer.read(4):
    sys.stdout.buffer.write(bytes(reversed(word)))

The above is simple to understand, but if you want a more compact oneliner, you can modify the above script into:

python3 -c $'import sys\nwhile word := sys.stdin.buffer.read(4):\n sys.stdout.buffer.write(bytes(reversed(word)))'

Note the $'' notation assumes you're using bash. It allows you to use newlines in the Python command.

As an example, swapping some chars on stdin:

# echo ABCDEFGH | python3 -c $'import sys\nwhile word := sys.stdin.buffer.read(4):\n sys.stdout.buffer.write(bytes(reversed(word)))'
DCBAHGFE
Brian
  • 2,172
  • 14
  • 24