Site Navigation:
Craig's twitter feed:
Tweets by @gevmage
Draft number two of this post.
How big is data? Human minds often have trouble encompassing more than a few orders of magnitude at a time. When the amount of somethings goes from thousands to millions to billions, it's difficult to really keep the whole picture in your grasp. This is particularly true of the sizes of quantities of data, which vary by many orders of magnitude even within a single computing device.
I sat down one time a few years ago and tried to come up with an analogy for the difference between the largest and smallest useful units of data to the largest and smallest animals on the earth. Unfortunately, animals don't have nearly enough range of sizes to accomplish this. More recently, I realized I could use length as a proxy for data size and give a fairly good idea of how big things are.
The bottom of the scale is 8 kilobytes, or 8000 bytes. This picked for both historical and practical reasons. ENIAC, the first electronic digital computer, only had registers but didn't have anything like a "memory". However, its immediate successor did; the EDVAC had a memory of about 8k in the 1950s (it was a room-sized computer). The Apollo Guidance Computer (super-miniaturized computer built in the mid-1960s about the size of a small suitcase) had an erasable memory of 2000 15-bit words or about 4k. The level-1 cache of a single integer core of an AMD Interlagos processor is 16k. So across the 3, which covers, 50 years of computing hardware, 8k is a reasonable starting point for the smallest amount of data storage worth bothering with. In our length analogy, we'll make 8k equal to 1 inch, 2.5 cm, or about the width of a person's thumb at the first knuckle.
A high-density 3.5-inch floppy disk is 1.44 MB. The analagous length is 180 inches, or about the length of a mid-sized sedan.
A digital photo is usually 2 to 3 MB; the analogous length is about that of a bus or a truck.
A CD is 650 MB. The analogous length is about 81,000 inches, 6700 feet, or a mile and a quarter.
A DVD is 4.7 GB, analagous length is 588,000 inches, or 9.3 miles. It's about the length of Manhattan island; you can drive that in 10 minutes on a highway.
As of this writing, the middle size option of both the middle iPad Mini and iPhone is 32GB; this is also a fairly common size of SD card. The analagous length is 4 million inches, or 63 miles; about an hour's drive.
A reasonable hard drive in a new laptop is 250 GB. The analogous length is 31 million inches which is about 500 miles. That's the distance between Boston and Washington DC, or between London and Edinburgh; about what you can drive in a car in a very long day.
Around the biggest single hard drive that you can buy in a retail store today (as of early 2013) is 4 Terabytes. The analagous length is 500 million inches or 7900 miles, or about 1/3 of the circumference of the earth.
So the ratio of the sizes of the smallest useful amount of data and the largest reasonable single chunk of data is the same as the difference between the width of someone's thumb and 1/3 of the way around the earth. There you have it.
I'm a pack rat; this comes as no surprise to anyone who's ever been in my house or garage or office. I tend to collect things that I want to use later. Often the net effect is that I end up having to throw stuff out later, frequently enough if I want something I can fine the one I used to have and put it to work.
I'm like that with physical things and with data. I have data disks that I was using regularly almost 25 years ago. So this blog post by data preservationist Jason Scott about the end of floppies came as a bit of a rude shock. I'd been carrying lots of old floppies with stuff that I used to use, and it never occured to me that I wouldn't be able to just throw the disk in a machine and read it.
I read that post last fall, when it was more than a year old. That really scared me, so I've been sort of vaguely tryin to get set up to pull data off of disks since then. I had the setup ready about a month ago but since then I've been rather busy. Today my wife is in Knoxville and my plane is in Lexington and broken, so I had time to start seriously feeding floppies into the machine.
The results were actually pretty good. The 5 1/4 floppies actually did better; there were only a few that I couldn't get a full image from. The 3.5 disks didn't do as well, but I was able to get full images off of about 2/3 of them, and partial images off of several more. I have 10 or so that just don't want to read in the drive, so I may work harder on those, or I may not.
Most of what I'd like to preserve are game files, and I got at least one clean copy of all of those. The other thing is images. A lot of the photos that I took in graduate school were on floppies, because the camera that I used was a Sony Mavica that recorded onto floppies. I'm pleased that I pulled images from several of those disks, so I have images of stuff that I haven't seen in 10 years.
Feeding disks into the machine
read read read...
Here's me, much younger and thinner with more hair. This is on the
balcony of my last graduate school apartment. The laptop is the one
my group bought me. It's an original iBook. I lived in this
apartment August of 2000 to July of 2002.
I didn't realize I had any photos of this car. This is my 1986 Ford
Escort EXP that I drove from the fall of 1996 to the spring of 2002.
Fantastic road car. It died the day that I interviewed for my
(current) job. It's parked in front of a storage unit. I'm about to
leave for the big summer run of my graduate school experient in the
spring of 2000.
Apparently, even then I was prone to taking photos of the highway.
This is probably driving towards West Virginia.
These 11 cards (and the corresponding ones in the other crate) were
what made my graduate school experiment possible. These cards
implement a wire-OR of the results of 352 comparators and put the
result out the orange cable on the right. My biggest worry between
1998 and 2000 was making sure that this piece of equipment did its
job.
Here's me in the Hall B control room at Jefferson Lab sometime in the
summer of 2000. I find this photo terribly amusing; I'm doin stuff on
my laptop surrounded by larger and much more capable screens. So that
hasn't changed at least. :-)
As I wrote in the previous entry, I was pretty sure that the Apollo GPS in the plane actually worked, but before using that unit as guidance, I wanted to verify that and get some practice.
So I went out to the hangar and set things up so that I could run the
GPS without draining the battery. I put a DC power supply on the left
wing and wired it into the supply wiring.
You can see below that the positive lead is clamped on the supply side
of the starter solenoid and the ground clamp is clamped on a handy
engine mount bolt:
The first test--when I turn the avionics on, had the Apollo enunciator
been left in Nav mode?
Yes! Now, with it switched into GPS mode
The GPS actually works and drives the CDI! Yay!
Here are all the lights on the announciator panel, by the way (shown
by pushing the "push to test" button). What
they all are and what they do is a topic for another post.
Now to put the GPS in simulator mode and "fly" it around. Here's
the map screen leaving Knoxville to the west toward Rockwood.
and the nav screen
Here's "flying" an approach, which is a flight plan flying to the
initial point of the approach and then a sharp turn left.
The screen tech on the Apollo is apparently suseptible to screen-burn,
as you can see by the variations in brightness here.
One other thing I did while I was out at the hangar that day was to
put on "step here" stickers. The flap at the back of the wing won't
support a person's weight and stepping on it is hard, so it's marked
with "NO STEP" stickers:
The flap itself is smooth, however, until that day I hadn't been able
to get anything to stick to the actual upper wing surface which has
no-slip stuff on it (it's fairly bumpy and stickers don't like it).
Well I sanded two spots smooth enough that the STEP HERE stickers
would stick to those spots, so now the combination gets the point
across.
My plane has two GPSs. This is not the way I would have designed the panel going in. Typically the planes I've flown have been one aviation GPS unit (which typically has a radio in it) and a communication radio with land-based radio navigation. That gives a certain redundancy; GPS for its nice features and the navigation radio for its.
Up until 10-odd years ago, I suspect my plane had the usual radio-based navigation devices in it. At some point someone installed an Apollo GX-60. The GX60 is a communication radio (but not a NAV radio) and it's an IFR-rated GPS, including being able to do non-precision GPS IFR approaches. It was a very very nice piece of gear for 1999.
At some later point, someone else (I think the guy I bought it from) also put in a Garmin 430W. Like the Apollo, the Garmin is a com radio, it can do IFR en-route and non-precision IFR approaches. Unlke the Apollo, it can use radio navigation both for en-route (VOR) and precision approaches (ILS). The Garmin also has a slightly bigger and color screen, and is just generally more sophisticaed and has more features.
My general intention when buying the plane and when initially flying it was to use the Garmin as my primary guidance and have the Apollo for backup. The Garmin is the nicer GPS, plus several of the club airplanes that I've flown have had a 430 or its big brother the 530, so I'm very familiar with how that family of GPSs operate. The Apollo was new gear to me. My plan was to mostly fly with the Garmin and ignore the Apollo.
However--there's something wrong with the Garmin, it seems. Four times now, it's done a full reboot for no reason that I could discern. It didn't stay powered down at all, just went to the initial startup screen. It happened twice as I was doing my initial training; they tighened the connects, and thought that was it. Then it failed again tens of hours later. I thought perhaps it was one of the memory cards...but now it's failed with the other one.
After the third failure I realized that I really couldn't depend on the Garmin for primary guidance for IFR flying. When it resets the currrentl flight plan gets erased, so if your'e flying IFR if it resets you're suddenly in the clouds and your magenta line is gone. Fortunately, the Apollo can ALSO be used for IFR flight as primary guidance. When I was having the realization about the Garmin, unfortunately, there had been something wrong with the Apollo as well (or so I thought).
Traditional radio navigation instruments typically had guidance in one dimension; they would tell you how far you were from the center of a cerrtain line (typically extending out from the center of a radio beacon). The instrument that shows how far you are from the center of where you want to be and which direction is called a "Course Deviation Indicator" or CDI. GPSs have carried that tradition forward. Aircraft panel-mount GPS units typically have a CDI that tell you how far you are from the center of the intended track.
You can see my two GPSs and their corresponding CDIs in this photo:
The Garmin 430W GPS is on top on the right, marked with the green "A",
and the dotted line goes to its CDI in the upper left. The Apollo is
by the green "B" and its CDI is in the lower left.
Flying down to Florida (to drop off my Mooney instructor) and flying the plane back home, the CDI for the Apollo didn't work. Since I was using the Garmin, I didn't really care. However, once I realized I was going to have to start usin the Apollo for primary guidance, the CDI not working was a problem. Aside from being nice to have, the CDI is REQUIRED to work if you're using that GPS to fly approaches. So without the CDI for the Apollo working correctly, I was looking at having an airplane with no safe legal way to fly IFR.
So I spent a few days grumbling about this and trying to figure out what things to check that weren't working, and what wiring to look at. I was looking at the wiring in an installation manual for the Apollo, when I noticed something I hadn't thought of. The Garmin has its own CDI, made by Garmin. The Garmin product is advanced enough that it assumes that it's the primary guidance in the airplane. However, the Apollo is enough older that it probably doesn't. The CDI wiring for the Apollo goes through an annuciator/selector panel, so that you can have the same CDI reading navigation signals from a radio navigation device (but not at the same time; you have to select one). It occured to me in the middle of the night as I was stewing about this that it was JUST possible that somehow the selector switch had been set to "NAV" rather than "GPS", and so the CDI was fine but since there's no navigation radio in the airplane, the CDI appeared dead because IT HAD NO INPUT.
So the miracle of being an obsessive documentation photographer, I
pulled out my phone and looked at the photos I took of the panel on
the trip home and found this:
Yay! The CDI was broken because I didn't engage it! (Duh.)
So--I spent a good amount of time over the following two weeks, including some ground simulator time and actual flight time getting used to the Apollo GPS. I read the manual voraciously and I'm pretty comfortable with it now. There are definitely things about it that are better than the Garmin interface. But now that I'm confortable with it, I'm fully confident (and legal) in flying IFR with the Apollo GPS as my primary guidance and flying GPS approaches with it.
My wife and I were taking our first big trip in the airplane. We were flying from Illinois back home, but the last quarter of the trip had dodgy weather, so we stopped to check the weather in Lexington. The weather was still dodgy, so we stayed the night and got ready to leave the next morning. And when I went to do the run-up check, the right magneto was dead. Poopy (and other four-letter words).
So...we rented a car and drove home, and then I went up the next week
so that I could talk to the mechanic. It took a couple of days to get
scheduled. Here's a look down inside the engine at the back of the
magneto:
The black thing on the right is the magneto. The lead going to it is
the shielded wire that comes from the ignition switch. It looks to me
like it's almost twisted off, but apparently it's fine. You can see
hte bracket in the center of the picture that in a Mooney makes it
hard to get the magneto out easily. The bracket holds the propellor
speed governor in place; that's the thing sort of to the left.
They looked at it and did a bunch of diagnostics Tuesday and
yesterday. It turns out that the magneto cable is fine, the magneto
itself failed, so they're sending it off for overhaul. So the plane
is sitting in tiedown at Lexington. :-(
Waiting for the mechanic to fix it happens to every airplane owner; I was just hoping it would be later rather than sooner.
My wife and I took our first big trip in the airplane over the last couple of weeks. Yay! During the trip, the plane manifested two major problems. Boo!
I'll go into the plane problems some other time. Because of the delays, we ended up renting many more cars to get around than we expected to. Since I'm sitting a hotel room waiting for the airplane to be fixed, I might as well put phots of groovy instrument panels up.
The first car we rented was a Hundai Elantra. I seem to remember that I've rented these before, and this one was no exception. A nice driving, small, elegant car. I really really like the instrument panel. Partially "analog" gauges, partially digital bar meters. A nice mix; it's clear that they thought about it.
Here's the instrument cluster:
Top center has the gear indicator. To its left, temperature gauge as
a bar graph, fuel on the right. Tach and speedo are "analog"
needles. (I put analog in quotes because I'm sure they're being
processed by the car computer and output as a digital-to-analog signal
rather than being physically attached to anything in the car).
Here are some of the warning lights. Standard analog lights. This
shows the temperature gauge just as the car is started. I took a
photo with absolutely all of the engine lights, but my phone camera
has been flaking out recently and apparently I lost a bunch of those
photos.
And again, with the engine cold. The "ECO" light here puzzles me;
there's a corresponding button on the left of the panel to turn it on
and off. I'm not sure what it is for.
This panel was by far the best of the three cars that we drove. Renting the Hundai was on purpose; we wanted a car to get around the city. Flying home, weather was dodgy so we stopped at the big Lexington airport to check weather. It hadn't improved, so we rented a car and stayed the night. That car was a Honda Civic. It had weird space-age 3D things in the front of the main panel that didn't actually do anything. It was very weird.
The last one, which I'm still driving now, is odd. It's a Chrysler 200. It's as if the design hadn't been updated since about 1985. There are LEDs on the panel, but only a couple. There is an analog clock top center of the center stack. It's an odd car. It's nice to drive, but it's a very odd feature set.
This post is actually a draft. The real post will probably also be a separate page on the site. I'm putting it here to check to make sure the formatting is right and stuff.
How big is data? Human minds often have trouble encompassing more than a few orders of magnitude at a time. When the amount of somethings goes from thousands to millions to billions, it's difficult to really keep the whole picture in your grasp. This is particularly true of the sizes of quantities of data, which vary by many orders of magnitude even within a single computing device.
I sat down one time a few years ago and tried to come up with an analogy for the difference between the largest and smallest useful units of data to the largest and smallest animals on the earth. Unfortunately, animals don't have nearly enough range of sizes to accomplish this. More recently, I realized I could use length as a proxy for data size and give a fairly good idea of how big things are.
The bottom of the scale is 8 kilobytes, or 8000 bytes. This picked for both historical and practical reasons. ENIAC, the first electronic digital computer, only had registers but didn't have anything like a "memory". However, its immediate successor did; the EDVAC had a memory of about 8k in the 1950s (it was a room-sized computer). The Apollo Guidance Computer (super-miniaturized computer built in the mid-1960s about the size of a small suitcase) had an erasable memory of 2000 15-bit words or about 4k. The level-1 cache of a single integer core of an AMD Interlagos processor is 16k. So across the 3, which covers, 50 years of computing hardware, 8k is a reasonable starting point for the smallest amount of data storage worth bothering with. In our length analogy, we'll make 8k equal to 1 inch, 2.5 cm, or about the width of a person's thumb at the first knuckle.
A high-density 3.5-inch floppy disk is 1.44 MB. The analagous length is 180 inches, or about the length of a mid-sized sedan.
A digital photo is usually 2 to 3 MB; the analogous length is about that of a bus or a truck.
A CD is 650 MB. The analogous length is about 81,000 inches, 6700 feet, or just over a mile.
A DVD is 4.7 GB, analagous length is 588,000 inches, or 9.3 miles. It's about the length of Manhattan island.
A reasonable hard drive in a new laptop is 250 GB. The analogous length is 31 million inches which is about 500 miles. That's the distance between Boston and Washington DC, or between London and Edinburgh.
Around the biggest single hard drive that you can buy in a retail store today (as of early 2013) is 4 Terabytes. The analagous length is 500 million inches or 7900 miles, or about 1/3 of the circumference of the earth.
So the ratio of the sizes of the smallest useful amount of data and the largest reasonable single chunk of data is the same as the difference between the width of someone's thumb and 1/3 of the way around the earth. There you have it.