Ignorance and Misinformation in the Age of Information

A week or so ago, I read Dan Lyons’ article on the iPad, information access, and the effects of information overload on our society: http://www.newsweek.com/id/237809. As sad as his message is, Dan Lyons is unfortunately spot on.

Especially working in the valley here, I see the paradox of information at work everyday. With devices that give us a window to the internet in our pockets, on our couches, at work, and in our leisure spaces, we have a tendency to become fixated on–obsessively informed of, and rabidly concerned with–the minutiae of our narrow scope and interests. Despite the cheap and easy access to information, the only media we consume ends up being that which falls into our individual niche interests. In so doing, we sacrifice the breadth and worldliness which came more easily to a prior generation when information was not nearly as abundant. As Lyons points out, we lose sight of the meaning of information in this day and age; along this same trend, I also see a contradictory loneliness spreading through people, however well-connected we may be.

This trend toward meaninglessness and loneliness came about from two complementary forces. The rapid and cheap dissemination of information the internet facilitates has happened on one hand, and on the other, we ourselves are becoming more specialized. A generation or two ago, we did not have the concept of search engine marketing or social media marketing; there was simply marketing. Likewise, we could not distinguish between social gaming and console gaming when the more global phenomenon of video gaming was still in its infancy. In conjunction with the spread of information, ever and always available, our specializations have allowed us to indulge in our interests, nearly without limit. Our society even encourages this sort of specialization. Technical universities make no apology in falling short of a liberal arts education, and new, more specific jobs cater to the trend.

This has allowed us to pursue our interests at the expense of finding meaning in what we do. There is no enforced “core curriculum” or balance to the information we consume. Like a child eating only dessert, we explore the infinite depth of our narrow specialties, occasionally with an almost fetishistic focus. We are rewarded for this not only in our jobs and at school but also in our virtual social networks, with friends piling on to “like” or retweet our every impulse with the click of a button or the tap of a screen. There is no need to step back to think about anything. Spontaneity wins over thought. Yelling triumphs over reason. Suddenly, our interactions lose the sophistication that comes with understanding meaning.

It is at this point that I see the deepest tragedy that begins to afflict us. Especially in Silicon Valley, we are growing echo chambers around us, and like perfect elastic and inelastic collisions, we react to others in a binary love-or-hate way. Tech blogs like TechCrunch wield considerable power over their readers–and have largely already decided what companies and products they love and hate. So many gatherings, conferences, and “tech summits” are glorified lectures and presentations too frequently designed to stroke the ego of the speakers than to inform their listeners. Why bother finding others who offer reasoned, meaningful dissent and discussion when yes-man agreement and fanboyism is far easier to stomach and so much more comforting?

Information and the way we consume it, ironically, have isolated us. We come to value low-quality interaction and information, and we lose out on deeper meaning. Most everyone I know is barely acquainted with each of his Facebook “friends.” There is no meaning to the relationship. We have even made romance consumable through a monitor with online dating sites, yet contrary to the promise of technology, it remains incredibly difficult to find lasting relationships. As Lyons points out, this is what we deserve; it is what we created. Unfortunately, this is increasingly what we seem to want as well.

Pay, Promotion, and Women in Engineering

A friend of mine shared a journal article with me on why women leave engineering. In the article, Hunt concludes that women generally leave engineering to enter non-related fields (as opposed to leaving to non-employment) and that pay and promotion are the strongest motivating factors pushing women out of engineering (as opposed to childcare, as the stereotype suggests).

I find this conclusion incredibly interesting. Jack Welch and one of his VPs, a woman who sacrificed her family for her work, declared in a conference that it was largely the concerns of childcare which prevented women from advancing in their careers, so Hunt’s conclusion directly contradicts Welch’s. Of course, nothing save anecdotal experiences and gut instinct actually support Welch’s conclusion, but given Welch’s experiences at the helm of GE, a huge, global, and diverse company in dozens of different businesses, I am reluctant to dismiss Welch too quickly. I have a certain amount of personal experience that also contradicts some of Hunt’s sub-conclusions, though again, this is certainly no evidence that her conclusions are incorrect at large.

Hunt’s analysis certainly begs a number of interesting questions. First and foremost, given these surveys are using self-reported data, it very well could be an issue with the manner in which the survey was conducted whether “leaving for pay and promotion” reasons is strictly conveyed as independent of childcare or family reasons. Any self-reported survey is susceptible to this sort of skew or bias, but I would be especially careful of surveys regarding employment–especially leaving jobs–as this topic tends to be deeply personal and sensitive, and questions on this topic are difficult to answer honestly. Unlike taking polls in a product focus group, for instance, here, the polltaker’s ego is intimately connected to the answers he or she is providing.

Another interesting question is why women trained in engineering tend to have a higher percentage in non-engineering fields. The article itself cites that workers (men and women in aggregate here) face an 11.6% wage penalty when leaving engineering. If pay and promotion are the strongest motivating factors pushing women out of engineering, why would women settle for less? One possible answer is that they don’t. Hunt does not segment wage penalty by gender when leaving engineering, but it is possible women may actually, in a majority of cases, earn more by leaving engineering than by staying in it.

Yet another point that seems contradictory is that women trained in engineering tend to have an employment advantage when entering into their careers in engineering versus entering into other fields. Taken in conjunction with the earlier point, this means that women trained in engineering have a greater affinity, statistically, for the field over other fields. Furthermore, pay and promotion are the main reasons they wish to leave engineering. Somehow, other fields for which women show less affinity for and which pay less tend to present a stronger draw to women than to men, despite women citing pay and promotion as the main reasons for their leaving. It could be the case that women, more than men, somehow discover and develop a greater affinity for other fields while in engineering, though this possibility seems far-fetched at best.

I would be very curious if further analysis could reveal answers to the questions above and to other questions the article does not try to tackle. For instance, I notice the longer one stays in engineering in particular the more “stuck” one becomes, and this effect could be investigated and further analyzed. In comparison to many other fields, my experience suggests engineering has substantially less sideways mobility. As a result, many folks I know, in hopes to avoid getting stuck, simply desire a wide breadth of experiences in a diverse range of industries. Many of these individuals pursued engineering in the university to acquire a technical and quantitative background, for the broad merits such a background supposedly offers, often with the explicit intention to avoid getting stuck. Career “stickiness,” and the positive or negative implications thereof, could also be an additional area for Hunt’s regression.

Lastly, another area I would find interesting is the distributions, by university and university tier, of the reasons women leave engineering. Again, I have only my own anecdotal evidence, but in my graduating class of electrical engineering students, out of my friends who entered the workplace out of college, I estimate almost 25% have left engineering, much more than the 10% for men and 13% for women that Hunt cites. Even worse, practically no one in my graduating class is an actual electrical engineer. Higher-tiered universities tend to train their engineers more broadly, with a base in arts and humanities, and place them into more prestigious and competitive environments. In other words, such institutions may actually be training and enabling engineers to leave their field. It would be very interesting to investigate this effect on the engineering-exit gender gap by institution.

Productive Paralysis

Having an unproductive, unhappy organization likely ranks as any general manager’s worst fear. Yet, this terrible circumstance seems to strike at the best of us, even at those with the best of intentions.

What I find especially interesting is the form of paralysis that comes about from overproduction. Eliyah Goldratt covers examples from manufacturing plants in The Goal, but it also happens in software/development shops in very interesting ways.

It goes something like this:

1. A team builds a lot of stuff very quickly, so the architecture becomes increasingly fragile and tricky to negotiate.
2. The pressure to continuously build (and release) new stuff continues and even grows louder and more oppressive.
3. If we compare features and customer satisfaction as a function of engineering effort, the product has seemingly hit an asymptote of diminishing returns.
4. As a result of both the difficulty of engineering on the platform and the pressure to ship, features increasingly generate larger and more difficult bugs, requiring constant fire-fighting. Yet, the team can do nothing about this because they are short on the staff, the time, and the energy to create a better architecture in parallel to swap out with the unmaintainable status quo.
5. Fire-fighting interferes with the team’s ability to engineer new features, requiring more and more effort for less and less return.

The team is eventually rendered almost wholly unproductive yet extremely hard working, an odd and contradictory dilemma.

Software releases–especially web releases–tend to be motivated by big promises and the momentum of “rolling thunder” iterations and updates. The solution to the situation above isn’t especially difficult to see; in any compromised release (and all releases are compromised to some extent), time, quality, and/or features must suffer. One answer to making something more maintainable and workable is to expend more time to build it right. Time pressures, however, often conspire against good but time-consuming design in favor of dirty but fast hacks and tricks that will have to be paid for in maintenance later.

The answer is widely known but counterintuitive. To speed up, sometimes, it’s necessary first to “slow down.”

Chaos works both ways

In the famous scene from The Dark Knight, the Joker tells the now deformed Harvey Dent, “The mob has plans. The cops have plans. Gordon’s got plans. They’re schemers, schemers trying to control their little worlds.”

He later goes on to note, “Nobody panics when things go ‘according to plan,’ even if the plan is horrifying. If tomorrow I tell the press that, like, a gangbanger will get shot or a truckload of soldiers will be blown up, nobody panics. Because, it’s all part of the plan. But when I say that one little old mayor will die, well then everyone loses their minds!”

To twist the Joker’s words just a little, we can apply it, with surprising fidelity, to events which similarly cause chaos in a different light:

“If tomorrow I tell the press that, like, an effective employee gets promoted or a CEO and founder will make millions, nobody rejoices. Because, it’s all part of the plan. But when I say that one little old lady wins a talent show, well then everyone loses their minds!”

The point is that it is never something expected which attracts the most attention, even if what is expected required the most effort or work. It is always the chaos, good or bad, which causes media frenzies, spectacles, and ultimately public consciousness.

Lincoln: the Video Game and Other Bad Ideas

I recently finished Goodwin’s awesome Team of Rivals, a multi-faceted biography of Lincoln and the people surrounding him. A lot has been said about the book already, including comments from President Obama. I’ll only add that I think the work could have been improved had it added more on the political dealings surrounding the war strategy and perhaps omitted the somewhat capricious details around Washington social life dealing with parties, soirees, and celebrations.

In the Wikipedia entry to the book, I discovered to my horror that it may be slated to become a movie directed by Spielberg working with–oh, no, not him–the horrendously overrated and verbose Peter Jackson. I strongly believe that the force of a message is very much inseparable from the medium in which it is presented. Books will almost always have the capacity for greater nuance and subtlety than a mass-market, digestible film; in turn, film generally will have greater capacity for said nuance and subtlety than video games. Some things should be left alone in their original medium.

So in the vein of disregarding authenticity, I can only imagine the types of taglines that would fly for Lincoln: the Video Game. Assuming the game tried to depict as much of the subtlety and not just be a military strategy game…

[CUE DEEP-VOICED ANNOUNCER AND IMAGES OF EXPLOSIONS]

ANNOUNCER: From the studios that brought you The Sims and Madden 2009 comes…LINCOLN!

[IMAGE OF LINCOLN SITTING IN A CHAIR THINKING]

ANNOUNCER: Balance the conservative and radical elements of your party…

[IMAGE OF LINCOLN WITH HAND IN THE AIR SURROUNDED BY CROWDS]

ANNOUNCER: Placate the masses with your oratorical skill…

[IMAGE OF SALMON CHASE, PLOTTING AGAINST YOU AS SECRETARY OF THE TREASURY]

ANNOUNCER: Navigate the dangerous waters of political intrigue within your own Cabinet!

I can already picture the crowds of enthusiastic gamers lining up or preordering weeks in advance. For some reason, real history just isn’t as interesting as video games.

Likewise, I sometimes wonder if the makers of games like Syphon Filter, Metal Gear Solid, or other such “espionage” games actually consulted history. Not like they should, lest they create a game like this:

[DEEP-VOICED ANNOUNCER AGAIN, WITH EXPLOSIONS]

ANNOUNCER: From the studios that brought you Lincoln: the Video Game and the stealthy espionage action of Metal Gear Solid comes…Competitive Intelligence: Corporate Espionage Action!

[GAMEPLAY OF YOU SNEAKING AROUND MICROSOFT OR GOOGLE'S CAFETERIA, SNEAKING UP BEHIND A RANDOM GUY GETTING HIS LUNCH, THEN CHOKING HIM AND INTERROGATING HIM TO GET VALUABLE TRADE SECRETS]

ANNOUNCER: You are the top spy of the world’s greater private espionage contractor (PEC). Infiltrate the ranks of the greatest minds in the search industry…and deliver illegal counterintelligence in this dangerous war of escalation in information retrieval.

[GAMEPLAY OF YOU FIGHTING A POWERFUL, CYBERDEMON-LIKE CREATURE]

ANNOUNCER: Evade the corporate park security and cross blades with high-ranking executives like Qi Lu, Marissa Mayer, Satya Nadella, and Hal Varian!

Once again, the realities of actual espionage fall rather short of depictions in successful video games. Always ask questions, but some things should just be left alone and appreciated for what they are.

The Public, Office Rumor Mill

Earlier tonight, I had dinner with a friend of mine who works at a large, very political company. His role as a sales engineering manager is more forward-facing than mine and involves dealing with customers, partner companies, and internal engineering and marketing teams.

There was a rumor that his organization would be cut in half, and he was visibly distraught through most of dinner.

As much as I would have liked to comfort him, a long time ago, I had read the pointer “90% of the time the office rumor is true,” which my experiences so far seem to confirm. This was in the pre-internet era, when word spread even more slowly, though still incredibly quickly in comparison to, say, executive communication that actually pertained to business. I could not muster much sincere reassurance to my friend now faced with the prospect of unemployment.

Nowadays, especially at large companies, rumors are basically instantaneous, and the office rumor mills are basically public domain. It’s quite frightening to learn from friends that their first knowledge of many big corporate changes happens from the media. Information first leaks out, and then the media disperse the information widely. Ironically, it even finds its way back into the organization from whence it came and sometimes even unpleasantly surprises insiders.

Repost: The Modern Myth of Corporate Loyalty

I had originally blogged the following post in September 2006. Back at that point, Google where I was working was still the king of the valley and Facebook was not yet the emerging new guy everyone was so enamored with. In light of the defections of Google executives like Sheryl Sandberg, Tim Armstorng, and most recently Priti Choksi, I think my original post has become especially relevant now that a new king of the valley has very apparently emerged.

A question I don’t ponder in this post is the broader theme of how to keep a company enduringly great. If great people consistently move on according to trends, how does a company attract, develop, and fill its ranks with great talent? That is a broad question I have never seen a full, comprehensive answer to. Anyways, here’s the post:

At career fairs, talks, guest lectures, communications, email, etc. people make it sound like everyone there has this incredible, strong sense of loyalty to one’s employer.

Sure, people like certain companies more than others. The idea that employees are loyal to their company is, in general, a complete and utter myth perpetuated by HR departments, recruiting, and the self-serving yes-men out to improve their reputation inside their firm and use it to leverage their way into bigger and better things later in life.

As a relic who believes in my work, my industry, and certain companies, I find this actually quite disillusioning. I would like to believe that there exists a sentiment that draws people together into a common, albeit corporate, cause that transcends mere individual ambition. In journeying through the insides of technology companies on the West Coast, I have not yet found this “transcendent union” anywhere.

I question whether what I hope for is necessarily what is best, though. Perhaps, through the overwhelming victory of personal ambition, companies as a whole become stronger. I have a hard time believing this, as the case of working in Japan was entirely different. Even if corporate loyalty were a modern-day myth, though, I would like to believe otherwise.

Great Quotes about Working at VMware

A dialogue:

Me: “How are your options doing?”

Friend: “Yeah…they went up to $120, thought I was going to retire early. Then they dropped to around $20 and were underwater. It’s now kind of above water. I’ve been through the emotional roller coaster watching.”

“And Mendel [Rosenblum] and his wife aren’t with the company anymore?”

“Yeah, you heard the story?”

Dr. Rosenblum was the respected VMware founder and CS professor who taught the viciously difficult CS140 operating systems course to the kids at Stanford. For a tad while, it was the hottest stock on the market.

Interview Class

In a week or so, I’ll take the “interview training” class for Microsoft. Having gone through the interview loop dozens of times, I find it an extremely intriguing, if not entertaining, subject.

At companies I’ve worked at, the interview policy spans the entire gamut. There are some companies who pride themselves on their ruthlessness in interviewing. Making candidates cry, humiliating them, causing convulsions, or pushing them to the edge is the type of practice which earns interviewers praise. On the other spectrum is the warm, fuzzy, “I made up my mind to hire you already so I’m here to convince you to work for us” sort of place.

The ideal, as was explained to me, is somewhere in between where it is a test of competence but also a corporate sell. It makes a lot of sense, but I admit, I find the alternative of grilling the candidate so much more entertaining.

In any case, I am very curious what will happen with PM interviews, as my experience before, both as an interviewer and as an interviewee, was as an engineer. Exactly how does one determine if a candidate would make a good PM?

Again, I’ve seen a span of questions in attempts to determine this, but my experience is nowhere near as strong as for the arguably more challenging but more straightforward realm of reversing linked lists, graph problems, and trivial programming exercises. I’ve seen folks ask areas of the resume for expertise, some design questions, or some questions pertinent to a specific market. How does one determine how disciplined a candidate is with his communication or his time and project management? And in the case of Microsoft, how does one determine this mythical idea of potential during the course of something like an interview?

The Book Problem

Starting at the beginning of 2008 immediately after I graduated, I began a reading spree. School had not given me the time I wanted to read the “modern-day classics.” I’m not talking about good fiction (though I’ve had time to read that too) but rather the sort non-fiction (biographies, business books, topic guides) that everyone and their mother seems to talk about and proselytize.

Initially, buying and reading these books didn’t seem so much a hassle. I had an abundance of space on my bookshelf, and I was largely buying books that caught my eye at bookstores (way overpriced and sometimes I made bad purchases…don’t do it!). The sustained financial impact of doing this on a weekly basis wasn’t immediately apparent.

Then I noticed as I began doing this through the year, books were just plain accumulating in my place without adequate place to put them. My car soon filled up with stacks of books, some completely read, the others skimmed or practically finished, the value of which I had already sucked out from the words in the tree carcass.

I got a little smarter and have started using the Mountain View and Microsoft libraries as much as possible. Yet, as my job picked up steam and I got busier, I noticed books consume a lot of time, in addition to money and space. Technically, most of this problem is, of course, the fact that to get something from the book, I have to read the book which obviously takes time. Yet, the more frustrating aspect about how books take up my time is how hard it is to find (1) a library that carries copies of books I want to read without having to wait months in line behind 50 people or get out of work early to pick up a book from and (2) a nice, comfortable location outside my home that offers a place to spend time reading at any hour of day or night.

Now, I like books. I like the way they feel, the weight they have, the simplicity of writing in them, and the easiness of referencing them later. But the problems of money, space, and time are really starting to be a drag on me. A library solves the problem of money. A Kindle solves the problem of space, but textbooks and certain field-specific works I read still come at full price. This is besides the point that it’s not a physical book I can annotate with stuff that is hard to input like tables, graphs, or mathematical symbols. Finally, folks like PaperSpine, BookSwim, or BooksFree are great and probably the most promising solution I’ve seen so far to all three problems. Still, they do not offer the complete end-to-end package with the ability a physical location has–things like a comfortable environment to read or the ability to preview the book.

What I’m looking for would be a business that did the following:

• It’s a 24-hour “library” that is minimally staffed but keeps great electronic records and catalogs and has an electronic checkout system.
• The “library” is a large, comfortable place with lots of seating, chairs, and Internet. Also, there is a coffee shop inside that makes espresso-based drinks.
• The collection is huge and extensive, especially focusing on textbooks and new releases. Books are still checked out and returned as in a normal library.
• If, for whatever reason, the book is unavailable, a DRM’d digital copy is available for your computer, phone, or e-reader, and your name is added to the queue for the physical book.
• If a customer demands a physical book that is either not yet available or completely checked out, the “library” would have a special relationship with Amazon or whomever is the cheapest seller to rush the book on 1 or 2-day air shipment. The “library” would split the cost of the book 50/50 with the customer, amortizing eating this cost with other proceeds from the customer over time.
• To keep the business running, the “library” would not technically be free but charge a monthly subscription service to customers. There could be varying levels of service, e.g. charging more for things like Internet access and digital book copies, and adjusting the cost sharing percentages.

Such a place, if the fees were nominal, say, $20-30 a month, would definitely get my business.