Post Sat Oct 21, 2006 1:47 pm

Skillz Sept 06 Winning Entry - Technical

Aaron Hertz

Here's my answers for the September 2006 Skillz Challenge:

1)  "Thing One" is dd, the file-manipulation swiss-army knife and fine companion of...
2)  "Thing Two", nc, the all purpose network swiss army knife.

3)  On, we run the command:

dd skip=8227126 | nc -l -p 12345

This tells dd to output to read from our data file, skipping the first
8,227,126 512-byte blocks, and write to stdout.  Next in the pipeline, nc will listen for a connection to port 12345, and then spew the rest of the file down to whatever client connects.

Then, to receive the file on, we run:

nc 12345 >>

This will connect to the listener we just created, and write the data it receives to the end of the data file.

(Of course, if we want to be really cute, and we're running a modern version of bash or ksh, we can do:

cat /dev/tcp/ >>

But, that would just be silly.  :)    )

This way we only transfer the missing piece of the file over the slow wireless link, minimizing the amount of data we transfer.  Why create the listener on  Well, if the upstairs office has an open WAP, it's likely a cheap residential wireless router.  So, it's likely that we won't be able to get a TCP connection into  Plus, we don't even know what our outside IP address is, and this way we don't have to bother to find out.

This does assume that the firewall in front of will allow connections to TCP port 12345.  If it doesn't, we can send our connection through an SSH tunnel.  When we connect from lumbergh to horton, we'd do something like:

ssh -L

We'd then run the same command on horton, but our command on lumbergh would then be:

nc localhost 12345 >>

That way, the only incoming connection to is over port 22, which we know is allowed.

4)  The most straightforward solution is to adjust the block size used by dd.  We'd change the command on horton to be:

dd bs=1 skip=4212288525 | nc -l -p 12345

This will, though, be more inefficient.  We'll be reading only one byte at a time from the file.  But, at least Nagle's algorithm will ensure that we don't send one-byte packets, which would be _horribly_ inefficient.

A more efficient way is to use tail.  The command:

tail -c +13 -

will skip the first 13 bytes of stdin, and print the rest.  That's exactly what we want.  So, our command on horton becomes:

dd skip=8227126 | tail -c +13 - | nc -l -p 12345

- -----

This was a really fun challenge - and your Seussian poetry is brilliant.  I look forward to next month.  :)