Limited updates to frisbee/imagezip to increase image size and improve distribution speed
The goal here would be to make it practical to use frisbee to distribute 10+GB images for Cloudlab3 and possibly Powder. In some situations right now, you could probably tar, scp, untar a big filesystem of data faster than using frisbee to lay down an image. Some of the tasks:
-
Support 64-bit block numbers in imagezip. Right now, you cannot image a filesystem larger than 2TB, no matter how little valid data it has in it. This is mostly an issue of maintaining backward compatibility and I have largely completed this step.
-
Tune frisbee for something more modern than a 100Mb link. With a 10Gb server link, it would be reasonable to try to achieve 1Gbs per image. That would allow us to distribute a 10GB image in something like 80-90 seconds. Ideally, we could crank it higher than that, say up to 10Gbs. There are any number of problems here that might bleed into the imagezip image format. Look at increasing the chunksize (from 1MB) or blocksize (1KB) to get more data on the wire faster. We have a number of constants related to allowing multiple outstanding requests, they need to be adjusted or eliminated. Flow control has been a bugaboo for a long time. Revisit (again!) maybe considering taking advantage of a lossless Ethernet control network and ECN.
-
Improve frisbee server. We should multithread so that it can pre-read image data and it should cache that data. On our current servers, it would be reasonable to read and cache an entire 1GB-ish image.
-
Improve unicast support. Right now, an image server cannot support more than a single unicast client at a time due to the naive way we support unicast. There are many cases where unicast distribution of images is reasonable. We still have situations today where a multicast client may take 20-30 seconds to hook up with the server, we could unicast the image 10x over in that time period!
Emphasis on limited here, since I have oodles of TODO files describing things that could/should be done to imagezip and frisbee.