Turtles all the way down

This is a re-printed and slightly re-edited version of a post I made 9 months ago on Avant Music News.

Often in my late teens I would drive an hour or more, from Eastern Long Island to record stores close to New York City, in order to take advantage of their superior selections. This was before I was aware of mail order, before online stores such as Amazon, Wayside and the like, and far before the advent of the MP3. I would typically purchase 5-7 CDs, which was all I could afford, and then head home. If I had the luxury of being the passenger rather than the driver, friends would note how I fondled the CDs in anticipation, reading liner notes and studying cover art. At home I’d add my new acquisitions to a growing collection, which was meticulously sorted. I fantasized of having a much larger collection, perhaps several hundred or thousand CDs, as if that somehow would bring happiness, prestige, bragging rights or just enlightened contentment.

Twenty years later, I’m heading in the opposite direction faster than I thought possible.

With collecting comes the overhead that I will call “collection management”. This refers to the tasks associated with maintaining a collection, such as sorting, organizing, categorizing, and filing. Not to mention finding bigger and better methods of filing and storage, as well as possibly cataloging the collection in order to determine gaps and missing albums, or to create and update sale and want lists. None of the above includes listening, though in fairness these activities do not preclude listening while taking part in them. Note that none of these activities are absolutely necessary, though the undesirable alternative usually takes the form of dozens of random stacks of unsorted CDs. I’ve tried the latter and it just doesn’t work for me, as it seems to make me not want to listen to music, defeating the whole purpose of a collection.

Collection management requires large amounts of time, and this time commitment scales roughly with the size of your collection. If you have the time, collection management can be a reasonable and even enjoyable activity. But if you don’t have the time, it becomes an unbearable annoyance. Combine the relatively mundane tasks of modern life, such as holding a job and spending time with your family, with the hectic pace of our era and you find that you need to make a sacrifice somewhere. You can either have a well-managed but static collection, or a poorly managed but vibrant collection. Add a little work-related travel and other hobbies, and the situation rapidly degrades.

I’m looking for another option.

I have not bought a CD in over two years and do not plan on ever buying one again. I stopped reading liner notes or putting any sort of importance on cover art better than 10 years ago. Between online stores such as eMusic, net labels and other free sources, as well as borrowing from friends, I can fill all of my listing time with an ample amount of new music, and then some. Digital media can be stored effectively in a very small footprint. My storage system, including a full backup, will hold over 4500 “CDs” in the room required for two medium sized books. Collection management consists of acquiring the MP3, OGG or FLAC files, putting them in an appropriately named directory, and filing them on a giant external hard drive. Done. End of story. Thousands of albums ready at the click of a mouse. Compared to the three $500 CD cabinets that I bought to store my CD collection of around 2400 items, this is a major convenience and a bargain as well.

But it can get even better.

Why “own” media at all? The digital world does not require ownership. Rather than download, listen, categorize, store and back up a digital music collection, let’s un-ask the question of how to manage a collection and let’s not even have a collection.

The alternative, which does not exist currently but is becoming technically feasible, is to have a purely on-line model of music listening. This requires that all music is easily available digitally over the Internet. Nothing gets stored on your PC, laptop or MP3 player for any significant period of time. If you want to listen to an album or a track, you type identifying information into a search engine, follow the reference to a download page, download the music and listen. If you’re going to be in a place without Internet access for a period of time, you copy it to your portable player. With recent advances in wireless networking, a few years from now there may not be too many areas left without sufficient high-speed coverage to even require the latter.

The advantages to the consumer are many-fold. Easy access to new music from anywhere with Internet connectivity, a broadening of one’s listening horizons, no need to back up music, and no need for collection management, period.

So rather than own media, you download it on demand from your on-line library of the world’s total recorded output. What needs to be done to make this a reality? The technical barriers are rapidly evaporating, as high-speed wired and wireless Internet access is readily available in most countries. The low cost of storage would not inhibit the massive amounts of space required for redundant copies of high-fidelity digital recordings in data centers. The business barriers, however, may make my personal utopia unachievable.

The music industry loves DRM. In order for this model to work, music would have to be DRM-free, so that customers are not locked into certain players or hardware (sorry, Apple). The pricing for such a service would have to be flat rate, so that it would appeal to both serious and casual listeners. I’ve not done a financial analysis, but as a user I would happily fork over $50-$100 a month for such a service. The pricing would have to be high enough so that the service provider makes a nice profit while low enough that piracy is deterred. The payout to artists would have to be based on popularity (i.e., downloads), in order to encourage competition, so such usage information would need to be tracked. These payouts would actually have to happen as well, which is not always the case in the business today. And all players, labels, artists, and distributors, would have to shake hands and agree to make a collection-less world a reality.

In the mean time, I wait and do my minimal collection management. I try not to think of double-fault scenarios where both drives in my redundant storage system fail, or of house fires, or similar disasters that would wipe out my collection. Whether or not having an un-collection would bring prestige or bragging rights, I will brag if I manage to achieve it, and it will help me achieve happiness and contentment, enlightened or otherwise.

I don’t like solving hard problems.

They can be fun to solve, and appeal to the my geeky side. But trying to solve the really hard problems can be risky for your business. This counters the de facto stance of higher education and academia, where hard problems are solved, in many cases, because they are hard problems. Simple, elegant solutions to practical problems are dismissed as lacking academic rigor.

A couple of examples why solving hard problems can be costly.

The first involves a program that is developed to back up customer records from host A to host B. After an initial full backup, only the deltas (changes) are sent, however the protocol used to send the changes is not guaranteed to send the changes in the same order in which the changes were made. This means that situations where a record was edited and then deleted on host A could result in the record being deleted on host B and then a stale edit being applied. This problem isn’t so bad as long as host B can robustly handle attempts to access invalid records. However a more pathological case to consider is where a record on host A is deleted and then the exact same record is re-added. On host B, this could manifest as an addition of a duplicate record, which would be ignored, and then a deletion of a record. The result is that a record on host A would not appear on host B when the backup was run. Assume that the cost of changing the protocol is very high.

Before attempting the complicated step of re-designing the communication protocol or the change logic to ensure proper ordering of events, one should consider how often this particular case will occur in common usage. If the case is common, then a more robust solution may be required. If the case occurs or is expected to occur very infrequently, then a more practical approach would be to clearly document the scenario and describe its impact. In this case in question, if a second backup is run before any further edits are done to the record on host A, then the data on host A and host B will be synchronized. If the cost of running two sequential backups is small, that might be the better solution.

The second example is to do with two communications systems that need to allow users of each to have access to users from the other. They are currently incompatible, but serve a similar function. An engineer notes that a proxy method could be used to translate the protocol used by the first system into equivalent primitives on the protocol used by the second system. While this solution might have the lowest cost at first blush, it is likely to be the most complicated. Proxying two protocols could exponentially increase the number of error conditions in the combined system. This will make design, development and debugging more time consuming and costly. The total cost of ownership will increase. Even more insidiously, this solution appears very simple even though it is not. Business types, executives and even some engineers, will dramatically under-estimate the timeframe and cost of deployment.

An alternative approach would be to give the clients of one of the communications system direct access to the other. While this solution might appear less attractive, because end users will have to use one interface for each system, the ultimate recurring cost may be lower. In order to evaluate this approach, one would have to consider the cost of deploying the solution and training the user base.

Both of the scenarios above were based on real-world business decisions I was involved in over the last year, and I am happy to say that reason won out both times. The key point to drive home is that complicated technical solutions often have hidden costs, so do your homework before jumping to a conclusion.

An article discusses why outsourcing is not used by the majority of high tech companies. I’ve got a lot to say about outsourcing and it will be a topic of multiple posts in the future. For now I’ll be brief and list the three reasons why companies outsource.

• Cost savings. The typical cost of a software engineer in India or China is one-third to one-fourth that of the average engineer in a US city. This is the main reason for outsourcing. Plain and simple: it’s the cost, stupid.
• Access to markets. Getting traction in many markets requires an engineering presence in those markets. This is certainly true of Korea and China, and is helpful in any market. Localization, customization and working closely with customers and end-users is difficult and not cost effective if the engineering team is remote.
• Access to talent. If the talent pool around your headquarters and other on-shore centers has dried up, opening a center off-shore will enable access to talent. However caveats apply, the main one being that you need to have a project that fits the talent and that can work in an outsourced fashion.

Someone wiser than I taught me this rule, but it bears repeating…

As a general rule, I try to follow the approach that I should be able to immediately answer at least 80% of any questions related to my business. For the remaining 20% it is acceptable to say, “I’ll get back to you.”

Having 80% of the answers off the top of your head indicates that you know what your organization is doing. You know what they are supposed to do. You know the issues and the status. You are engaged in the business.

If you have to do research on too many issues, not only does it take time, it also suggests that you are not on the ball. Being on top of what is going on is critical.

There seems to be an inherent belief that parents should take credit for their children’s behavior and achievements. This belief manifests itself in praise for parents of “good” children, as well as society holding parents responsible for the behavior of “bad” children. I think this belief is somewhat shallow and relative.

On a good day, when my children and polite and listening, I will happily accept praise for my apparently superior fathering skills. On a not so good day, when my children and fussing, pouting and driving myself and everyone else in the vicinity insane, I will explain to you with exacerbation, that my children are independent individuals and not entirely under my control.

Even between my two boys, behavior varies. One is more prone to sit quietly, pay attention and respond appropriately when threats of punishment become necessary. The other will typically march to his own drummer, regardless of how many times I explain or apply the appropriate consequence.

This is not say that good parents do not influence their children’s behaviors. It is fairly clear that there is an influence. As an aside, what isn’t clear is not much of this influence is genetic and how much is environmental. The Nurture Assumption by Judith Rich Harris does a great job of explaining the last 20 years of research into this topic. I’ve influenced my boys positively and watched their behaviors change on many occasions.

On the other hand, there is a core aspect of behavior (let’s call it personality) that seems to be much more hard-coded and difficult to influence. Children are autonomous individuals, and short of using draconian methods that stretch or violate morals or laws, cannot be completely controlled. Some children will be born more likely than others to toss a rock through a window. Good parenting will decrease the likelihood of this happening for any given child, but may not be able to prevent it in all cases.

Recently, laws have been enacted holding parents financially responsible for the behavior of their children. According to some reports, juvenile crime has dropped as a result. Even if it has, should the parents who have raised their children to the best of their ability should be punished for what is essentially their genetic makeup?

One thing is clear. Parental influence is not about absolutes, it is about probabilities. A good parent is more likely to have good children. A bad parent is more likely to have bad children. But this is not written in stone and our society’s expectations and laws need to realize it.

Being a runner and living in a busy area means that I do most of my outdoor running on sidewalks. This is fine until you pass through areas where either the sidewalks are not well maintained, or even worse, obstructed. Most obstructions take the form of cars blocking the sidewalks or low-hanging trees turning a neighborhood jog into an obstacle course. Being tall, I have to duck under branches most people might be able to pass beneath with reasonable clearance, while dodging (hopefully) stationary vehicles.

I’ve always found this aspect of neighborhood jogging annoying but not worth complaining about except under my breath. This week I learned that, at least in my town, all of these sidewalk obstructions are illegal or against code. My first reaction is to wonder why, especially given their apparent lack of enforcement. My subsequent thoughts center on how nice it would be if people would just apply common sense, and not park their cars over sidewalks, trim their branches and bushes appropriately, etc. I know that this sort of common sense cannot be assumed to be present in some people. Instead we get laws that are nice on paper but not very meaningful in practice.

One notable sidewalk obstruction…sprinklers. If you water your lawn, you are not supposed to make it so that passersby on the sidewalk will get wet. This ordinance makes it very difficult to water your parkway (the strip of grass between the sidewalk and road). I’ll raise my hand and admit that my parkway is well-watered. Rationale? I live in a cul-de-sac and few people walk our sidewalk. And as a runner, I can confirm that nothing is more refreshing on a hot run than a brief cold shower from a nearby sprinkler.

You know crisis mode…you’ve got a deadline in 2 days and there is 5 days of work yet to do…a major customer is experiencing a field issue and you’re trying to find out anything about what could be the problem…you’re juggling several projects at the same time and task switching from one to the other. Long hours, busy days, chaos hopefully under control.

I usually hit a down on the day after crisis mode over. Not an emotional down, but a productivity down. I’m left with no tasks that have to be done right now, but a number of tasks that should be done soon. The difference is huge.

I find it hard to get back in the groove on these medium priority tasks. After crisis mode they don’t seem all that important and in many cases they are not. I usually use my post-crisis perspective to weed a few items out of my to do list completely - the don’t-do-it-and-see-what-happens-approach.

I have to wonder about people who circle around the parking lot of a large gym, trying to get one of the best spots. Unless it is very cold, very hot, you’re nursing an injury, or you’ve got small children to lug, I don’t see the point. How many of these people trying to cut 30 feet off their walk from their cars to the door will get on a treadmill and run 3 miles?

Reminds me of a jogging partner I had 15 years ago. He’d always want to take the elevator from the locker room level to the running track level. I didn’t get it then and I don’t get it now.

For years I’ve noticed that I’d be the most sore the second day after a heavy or hard workout. What I didn’t know is that this phenomenon has a name: Delayed Onset Muscle Soreness or DOMS.

In summary, it is normal, lots of people get it, working out in a DOMS state doesn’t inhibit the benefit of the training though you might not be up to full strength, and stretching probably won’t help. Makes sense.

Time to stop worrying about it.

Come to think of it, I’ve had some pretty good workouts while with DOMS. Maybe the hair of the dog is the trick?

Despite being a technology geek, I’m not an early adopter.

I like things, whether gadgets or software, to just work. An early adopter is usually a de facto member of QA team for the new product or service, which itself is usually a “stable beta”. Additionally, technological stuff tends to be relatively expensive at first, then it will decrease in price pretty quickly. I don’t want to spend a lot of time or money on an unproven offering. Once I’ve read enough reviews of the product or service and the price has dropped into the realm of the reasonable, I’ll consider buying.

In the bell curve of technology adoption, I’ll stick around the “early majority” phase. The irony, of course, is that for better than 15 years I’ve done research on and build products that have had a fair amount of appeal to early adopters and risk takers. One of my tests of the usefulness of the product is for me to use and decide it I would spend money on it. If the answer is no, then I have more work to do.

PS: The chart has a typo - its “early adopter” not “early adaptor”.