The Future That Was Never Predicted!

Interesting table below:
.
.
.

Year -------- Number of Computers on the Internet

1988 --------------- 60,000
2006 -------- 439,000,000
2011 ------ 2,000,000,000
.
.
.
.

(Source: The Future of the Internet and How to Stop It by Zittrain)

Industry Data Warehouses

So what is an industry data warehouse? And why do we need them? Are they a threat to our confidential information and core competitive advantage? In very simple terms, industry data warehouses are created jointly by the firms in an industry group. They enable a better competitive environment by providing a common “score card” for participating firms. Now you know the rules, you can find out who is winning and who is losing and the game statistics. The competitive game gets even more riveting. Given the right "fit", while corporate data warehouses have the potential of providing a competitive advantage (please note the emphasis on "fit") to the owner, industry data warehouses create cooperative advantage (What about coopetition instead of competition?) for the users in the industry. Therefore, it becomes necessary that industry data warehouses provide common industry definitions of performance and market measurements, enable comparison of relevant parameters between a firm and the industry, ensure privacy of the data contributed by member firms, and have sufficient number of firms from the industry to guarantee reliability as well as confidentiality of the information. Industry data warehouses will have users who are data contributors as well as those who are non-data contributors. The data warehouse must fulfill the needs of both these groups of users by providing firm-only, comparison between the firm and rest-of-the-industry, benchmarking, and aggregate industry-only type of queries.

Industry data warehouses for benchmarking have become common place. More than a decade back, these were just emerging. If you are interested in learning more about industry data warehouses and how to build them, below is a link to an article that I had written in 1999. Enjoy in your copious free time!!

Link to the Article on Industry Data Warehouses

Note: This is a scanned copy of my article in PDF format available from Google docs.

An SQL Journey: From Non-Procedural to Procedural

The simplicity of non-procedural languages is really enticing. Non-procedural languages relate to describing the result instead of defining the process. For example, when I visit a restaurant I ask what I want instead of describing how my meal should be cooked. Most of the time, we are asking what we need, instead of describing how we want a particular result to be delivered. We leave the how to experts. So we let the cook decide how to cook the food, allow Google to figure out how to do our Internet search and leave it to our car mechanic to figure out how to fix the car. This allows development of deep expertise in each area. This is the reason we have financial experts, who don't know much about cars, doctors who don't know much about computers and computer engineers who don't know much about the pedagogy of the oppressed. To a certain extent object-oriented languages have been able to mimic this real-world paradigm by defining operations and hiding the "know-how" as encapsulated modules. But all object-oriented languages are still procedural languages, since the programmers use flow controls to define the events and actions.

A non-procedural language defines what is being expected, therefore non-procedural languages don't have flow control.

Let me give you an example:

SELECT FirstName, LastName
FROM STUDENTS_TABLE
WHERE TEACHER = "Peter Drucker"

Here I have defined the result that I'm expecting. Give me the first name and last name of students whose teacher is Peter Drucker. I can get more granular and add course too. For instance,

SELECT FirstName, LastName
FROM STUDENTS_TABLE
WHERE TEACHER = "Peter Drucker"
AND COURSE = "Management 301"

This is a purely non-procedural statement, since nowhere in the statement have I defined how the result should be obtained. Both these statements are valid SQL statements.

When SQL started out, it was based on solid mathematical underpinnings of relational algebra and it was truly non-procedural. However, there has been an unfortunate evolution of SQL. Slowly but surely, more and more procedural constructs have been added to SQL. For example, addition of a CASE statement in SQL-92 was definitely procedural. Vendors have added their own flow control. Sybase added "stored procedures". Rest of the vendors including Oracle (Of course, Microsoft had bought the Sybase source code) quickly followed the lead by adding their own procedures. I think Teradata tried to maintain the purity and elegance of non-procedural SQL for a long time. Teradata had its own vested interest. It was very hard to build a procedural language on a "massively parallel processing" (MPP) platform. Therefore, Teradata stayed true to non-procedural operations of relational algebra for quite some time.

Procedural additions to non-procedural SQL have caused a havoc in programming. Initially, it was impossible to debug procedural SQL due to lack of breakpoints. Companies wasted millions of hours of programming time trying to debug procedural SQL. Of course, I must say that debugging of original non-procedural SQL with multiple sub-queries and correlated sub-queries was equally painful. But technology is always seductive. Technology is the ultimate silver bullet to kill the corporate vampires. Therefore, procedural SQL or stored procedures were widely adopted. Under a newly emerging client/server computing architecture, you could write all your business rules in procedural SQL, store them on the server with the database and minimize the network latency when "client" computers made a request for data. This was the end of non-procedural SQL.

To a certain extent non-procedural SQL was doomed due to its one of its major weaknesses: Purity of logic. Its elegance was its failure. It was just too perfect. Good logic is neither common sense, nor intuitive. I can give you some examples, which you can test in your copious free time:

Example: There are a bunch of car manufacturers, who make multiple brands of cars. Each manufacturer can make one or more brands. Car customers purchase one or more cars each year. Pretty simple, eh! Now, can you write a single (not multi-pass) SQL statement to find all the customers who have purchased all the cars made by a certain manufacturer?

Since I'm having trouble going to sleep and I can't think of anything better, I've setup a simple Microsoft Access database for you.

Here is the download link:

Click here to download Car Database as a Zip file from Google docs

or

Click here to download Car Database as a Zip file from Box.net

If you don't have Microsoft Access and want to use another database, I've added an Excel Spreadsheet with multiple worksheets corresponding to database tables. The database has five tables as follows: CAR, CUSTOMER, MANUFACTURER, CUSTOMER_CAR, MANUFACTURER_CAR.

This is a database in 3rd normal form, which means that data redundancy has been reduced to downright minimum. CAR table has CarName and CarID, CUSTOMER table has CustomerID, cFirstName and cLastName, MANUFACTURER table has ManufacturerID and ManufacturerName. Such tables are sometimes called lookup tables. Now CUSTOMER_CAR table establishes a relationship between customers and cars. It shows which customer purchased which brand of car in which year. CUSTOMER_CAR table has CustomerID, CarID and BuyYear. Similarly, MANUFACTURER_CAR table establishes relationship between manufacturers and car brands. It shows which manufacturer produces which brand of car. Therefore, MANUFACTURER_CAR table has ManufacturerID, CarID and ReleaseYear.

In order to understand why I called non-procedural SQL as too perfect, try to write a non-procedural single-pass SQL statement that will return

(a) first and last names of all customers who have purchased all the cars released in 1974

(b) first and last names of all customers who have purchased all cars manufactured by Chrysler

(c) first and last names of all customers who have purchased all cars manufactured by Ford

(d) first and last names of all customers who have purchased all cars manufactured either by Chrysler or by Ford - If you got to this point using Microsoft Access, you will know exactly why it has a weak SQL engine.

You can use sub-querries and correlated sub-queries but no procedural or multi-pass SQL.

I guess I'm about to fall asleep now. Therefore, I'll write the solutions in my blog next week.

Until then enjoy these brain teasers!

Management Meta-Functions and Technology

Note: Demand Management in this blog refers to management of demand for resources within a firm. I want to clarify it at the outset, since all business organizations are on the supply-side from a macroeconomic point of view.

No business organization has a demand management function as a department entrusted with the responsibility of managing internal demand for resources. But this function is critical and strategic. In fact, it is a meta-function of all functional areas and it requires active involvement of executive management and the board.

Within any business organization, demand is the sum total of all current and outstanding requests for resources. Resources could be dollars or labor or technology and generally these resources are fungible. There are countless instances: We need to make two more acquisitions, spend more to enhance customer service, upgrade technology, roll out new products, grab a bigger market share, improve marketing of our new products, spend on improving product quality, give salary increases to hard working staff and so on.

Demand management is mostly the responsibility of senior management and this responsibility extends to the executive level and board room too. Demand is controlled and directed by the mission, strategy, goals, policies, governance processes, capital budgeting, spending limits, return on investment (ROI) targets, et cetera.

On the IT side, demand is controlled and aligned with strategic objectives through a technology governance process that includes, inter alia, "tough love" (sorry, Mr. Cruz Bustamante) prioritization, chargeback mechanisms, and reviews of business cases.

Since demand always exceeds capacity, management of demand is about making choices. Sometimes there is a fear about making choices and at other times macho managers think that they can do everything: expand the target customer segment and meet all their demands, reduce costs and improve quality. Making a choice involves giving up. Choice involves trade-offs. Choice means deciding what we won't do. Choice means saying no. Doing everything and keeping everyone happy seems so much easier. Demand management is left to the winds of organizational weather system - whether it is functional or dysfunctional.

At the strategic level weaknesses of demand management show up in compromises and insidious erosion of competitive advantage over a period of time. But in the short run middle management has to figure out how to bring the resources to meet the demand coming from various corporate projects and day-to-day operational priorities.

Supply management that requires provisioning and allocation of resources is broadly a responsibility of middle management. Middle management has to make sure that their staff are not drinking beer with their feet on the table. This last sentence is a joke but what I mean is that middle management needs to make sure that available staff and skills are effectively and efficiently allocated to critical activities. If demand management meta-function is not effective, then either middle management has to step up to control the demand or it will end up dropping one of the balls - no pun intended.

Finally, we come to the meta-function of capacity management. What is capacity? Capacity is the upper limit of the aggregate output of the firm. It is the maximum ability of a firm to get the work done. There are three key determinants of capacity:

1. Staffing
2. Best practices
3. Technology

Business organizations can expand their capacity by improving capacity utilization or by adding new capacity. Addition of new capacity is expensive, particularly when existing capacity has not been driven to achieve its maximum. All business organizations try to expand their capacity. Often this meta-function is recognized as doing more with less. Some of the firms do this by having their managers scream harder and louder. Others do this by burning the midnight and weekend oil. None of these methods are sustainable. Therefore, these are non-strategic. Strategic expansion of capacity involves improving overall fit among various functions and activities of the firm to optimize effort and minimize waste. Southwest Airlines and Wal-Mart are good examples. Strategic expansion of capacity involves recognizing one's strengths and uniqueness. Increases in productivity are invariably part of improvements in the "fit" among various activities of the firm.

You will notice that technology is only one of the variables in the expansion of capacity. Therefore, acquisition of technology to drive capacity expansion across the enterprise involves a lot more than mere implementation.

Technologists have to understand how they help organizations do more with less. At a given level of technology, when maximum capacity utilization and productivity have been achieved through the use of best practices, demand has to match supply. Most of the IT organizations have begun to understand it and they are establishing control processes to manage demand and supply.

In fact, very large organizations have to build internal economy of the firm in such a way that the resources are routed towards the most profitable strategic opportunities.

Each organization has to evolve its own method of building an internal economy for management of meta-functions of demand, supply, capacity, productivity and competition. For example, several advertising firms have multiple teams competing internally for the same contract. Some of the advertising firms, in fact, launch several competing projects for the same contract. On the other hand, within technology firms, internal competition takes the form of how to skin a cat. IT departments use project governance, activity-based costing, project management and various software development methodologies to manage these meta-functions to match demand and supply. When they don't, they drop the ball. Some commitments are left unfulfilled. Project time lines are extended. At the end of they day demand matches supply but it happens mostly in an unpredictable and ad hoc manner.

Don't Enhance My Shopping Cart Now!

Muckety Muck Electronics, Inc. (MME) was running a discount Internet shopping website for electronic components. MME had a policy of not publishing the prices of their merchandise until the merchandise were added to the shopping cart. The marketing department of MME had found that the rate of impulse buy on their website was quite low. Several of their online customers on their first visit added components of interest to their cart to check the prices and then left the website. However, the return rate of such first time visitors was above 37% and almost 70% of them were making a purchase on their return visit.

The shopping cart of MME was initially designed in such a way that it automatically emptied all its contents at the end of the day. Since search for electronic components with close tolerances was time consuming, the marketing department estimated that if shopping carts were able to retain contents between visits, then sales were likely to increase by at least 5%.

A project was immediately initiated to retain merchandise in the shopping carts. Since MME had to meet challenging financial targets, a tight time-line was implemented. The programmers modified the shopping cart program. Half-way through the project, the procurement department communicated that the inventory and the cost of procurement was changing on a daily basis, therefore the availability and prices of shopping cart contents had to be updated too. This was included as a new requirement and all the testing scripts had to be revised. The impact on the time-line was avoided by having the project team work through the weekend.

Finally, updated shopping carts were released in production. Two months later, sales from return visits were down by 7%. A root cause analysis revealed that the website “listing” module was originally programmed to display only those items that were best bargains based on daily comparative pricing feed. Any item that was not a bargain was not listed. This strategy was decided by the founders. However, when the items were retained in the shopping cart, they often lost their bargain status. They were not listed but still retained in the shopping carts with updated prices. This disappointed the returning visitors, who left disgusting comments on the discussion boards.

I have a list of two dozen questions related to the issues and problems in this business case. This is a real business case but I've changed the industry and the situation to protect business privacy.

No Survival without Operating Efficiency

There is no way a business can survive without internal operating efficiency. For example, the sweet spot of most of the consumer electronics goods prices is about $300. This sweet spot price point has stayed at $300 level for almost two decades. The sweet spot for X'mas gifts in the consumer electronics segment in the US is in the range of $100 - $200.

Whether it is iPod, iPhone, PlayStation or a regular Denon receiver, it is hard to achieve a sufficiently large market without hitting this price point. Therefore, you have to cut down the cost of inputs to a point where you can hit this sweet spot without making a loss. If you can't, then you have to accept a loss on the main unit and attempt to make that up when you sell complimentary products. Sony and several electronic gaming system manufacturers have adopted this strategy for the initial roll-out of their PlayStation units. The search of lower costs and operating efficiency in the area of software development and manufacturing very quickly extends the supply chain to India and China. Several software development firms in India and assembling firms in China, for example, Foxconn Technology, have the sole strategy of maintaining high efficiency of labor and doing everything to lower the cost of labor in assembling of products.

While there is some differentiation based on the reliability of supplies and quality of output among the firms in India and China, their raison detre for being part of, for instance, iPhone supply chain is that they can provide labor-intensive software and assembly services at a lower cost than anyone else. The current state of competition among the software development firms in India and manufacturing/assembly firms in China is mostly cost-driven that relies heavily on tiny profit margins, lower cost of labor, economies of scale and internal operating efficiencies. Is this situation going to be permanent? I doubt. Competitive position based on operating efficiencies is not sustainable in the long run.

There are several other examples of firms that are heavily dependent on internal operating efficiencies to meet a low price threshold and achieve competitive advantage. Is this competitive advantage based solely on internal efficiencies sustainable in the long run? I agree that while the sustainability of competitive strengths based solely on internal efficiencies is highly doubtful, it is a necessary condition for staying in this market.

Does outsourcing really mean achievement of operating efficiencies? This is another question. You can certainly outsource your internal inefficiencies and achieve a lower cost. However, this hasn't worked very often.

For a successful business operational efficiency is necessary but not sufficient. Here is another thought on operational efficiency and strategy: http://techenbiz.blogspot.com/2010/06/efficiency-is-not-business-strategy.html.

Efficiency is not Business Strategy

Companies spend hundreds of millions of dollars on IT to improve internal efficiencies. The expense includes enhancement of internal core technology systems, design of internal workflows, integration between systems and decision processes, simplification of processes, lean manufacturing methods, et cetera. In the software world, rapid software development methodologies, like Scrum, with overlapping stages of concept, design, development and testing help in reducing time. All this need for speed is excellent and an absolute must to stay in business. But this need for speed is certainly not business strategy.

If you don't agree with this line of thinking, then let's assume for a moment that I'm wrong. This means that operational efficiency is your business strategy, which is how you are going to compete with your business rivals. Let's examine this topic.

At constant quality, operational efficiency means that you can keep your cost lower than your competitors. How do you communicate your operational efficiency as a "strategy" to your customers to gain market share? If efficiency is your strategy, then the only answer is low price.

If the only thing that is different between your product and your rivals' products is price, then how would you stop your rivals from achieving the same internal efficiencies and matching your prices? So you run and you run to catch up with the ...

Congratulations! You have just figured out a great way to commoditize your product with price as the only basis of competition.

Similarly, you can use internal efficiencies to deliver higher quality to your customers. Such gains in quality are again not beyond imitation by your rivals.

Operational efficiency is NOT business strategy.

Postscript: This issue is quite debatable. Therefore, here are my thoughts on some of the firms, like Foxconn Technology, that are heavily dependent on internal efficiencies and lower costs to stay in business: http://techenbiz.blogspot.com/2010/07/no-survival-without-operating.html.

A Myth Buster Anthology of Data Warehousing

One of the strongest arguments in favor of data warehousing, which I have often heard from specialists, is that data warehousing can generate golden nuggets of information that can create competitive advantage for the firm. And I've watched several uncomfortable discussions on how to measure return on investment on data warehousing. And I've seen a lot of frustration that several large data warehouses failed to generate return on investment after tens of millions of dollars. Like any other technology, data warehousing has its own share of myths intentionally and unintentionally created by vendors as well as practitioners.

Myth #1 - A data Warehouse can create competitive advantage
Myth #2 - A data Warehouse is required for business intelligence
Myth #3 - Data Warehousing starting point is an enterprise data model
Myth #4 - You need both an Operational Data Store and a Data Warehouse to cover the entire spectrum of business reporting
Myth #5 - Data Warehousing requires an engineering approach
Myth #6 - Data Warehousing fails due to problems with transaction-processing systems
Myth #7 - We can't predict what questions will be asked from a data warehouse
Myth #8 - Data Warehousing improves decision-making
Myth #9 - Data Warehousing empowers front-line and business staff to do their own analysis
Myth #10 - Data Warehousing reduces overall cost of reporting on business performance and opportunities

Myth #1 - A data Warehouse can create competitive advantage
The most often cited example of competitive advantage is the way Wal-Mart created a data warehouse and linked the data warehouse-driven decision support systems with store inventory and transportation logistics to match the demand at various stores with the delivery supply chain to minimize out-of-stock, cost of delivery, excess inventory and maximize sales volume. I don't doubt this story.

There is another often repeated but probably apocryphal story about beers and diapers at Wal-Mart. It is claimed that a serendipitous market-basket analysis showed that on Fridays six-pack beers and diapers were being bought at the same time. Further analysis showed that when young fathers bought diapers on Fridays, they picked a six-pack beer too. Wal-Mart was able to increase sales by moving diapers and six-pack beers in close proximity. I've never seen any data or evidence that would validate this story.

First of all, operational efficiency is not competitive advantage. Wal-Mart's low cost positioning strategy has a large number of components that include marketing, vendor management, contract negotiation, labor and human resources policies, location selection, store management, etc. Inventory management, delivery, transportation and operational logistics are certainly an important part of this overall strategy. In and of itself, Wal-Mart's data warehouse does not create any competitive advantage. In fact, similar integration between decisions and activities can be achieved through other technological means, like, enterprise service bus. The important point to note is that Wal-Mart's strong strategic focus on "low cost" has created a nice "fit" among all its activities that enhance Wal-Mart's competitive advantage. Wal-Mart's data warehouse is a critical part of this overall fit. But to attribute Wal-Mart's competitive advantage to their data warehouse would be no different than saying that Wal-Mart's competitive advantage comes from their vendor negotiation policies.

Sans integration with business activities, a data warehouse is no different than an ATM. An ATM is a cost of entry into consumer banking business. It does not bring any competitive advantage. Similarly, anyone can hire the consultants who designed Wal-Mart's data warehouse and build a data warehouse. But would they get the competitive advantage that comes from Wal-Mart's low cost positioning that permeates millions of activities that Wal-Mart does every day? A competitive advantage is a competitive advantage because it can't be imitated.

A single insight leading to creation of sustainable and long-term competitive advantage is a fiction, though it makes fascinating business stories. In real world, a single insight has to be supported by tons of hard work in analyzing and aligning thousands of activities to fit into an overall coherent business strategy to create competitive advantage. Target has carved its market position in spite of Wal-Mart. Wal-Mart would have a tough time defeating Target's competitive advantage without compromising its position.

Finally, as everyone knows, competitive advantage is not absolute. It is relative to industry competition. It depends upon how you position your business vis-a-vis your competitors.

Myth #2 - A data Warehouse is required for business intelligence
Let's first clarify the vocabulary.

Among senior executives business intelligence means information about markets, customers, suppliers, competitors and business drivers in the industry. For a business process outsourcing firm, an example is a new customer silently searching around to outsource some of the internal processes. Majority of this information is not from inside the firm. This information comes from OUTSIDE. This is an important distinction. Most of the time, business executives rely on their own rolodex to gather such business intelligence. At other times, they engage with industry analysts and review publications to gather this outside information. Astute marketing departments can frequently supplement this business intelligence with frequent surveys of customer tastes and industry trends.

In the vocabulary of data warehousing, business intelligence means consolidation of internal transactional data from multiple systems followed by dissection of data across multiple dimensions of analysis to test hypotheses and derive some high-level conclusions about the state of the business. For example, point-of-sales data could be consolidated across multiple sales territories or lines of service and then analyzed across zip codes, categories, population demographics, etc. In fact, marketing activity tends to be highly data-driven and good marketing departments are all the time measuring the impact of marketing campaigns on sales volumes. I've seen that occasionally this data is supplemented by consolidated industry data but majority of data used in the data warehouse is internally generated. Therefore, business intelligence from data warehouses is predominantly internally focused on business operations, operational efficiencies and internal optimization.

Data warehouses can not provide EXTERNAL business intelligence that senior executives need and I question the premise that a data warehouse is a sine qua non for INTERNAL business intelligence. Most cost effective method of internal business intelligence starts with business analysis and hypothesis development. Data gathering is the next step. Once hypothesis has been defined, data can be gathered from the subject area within the parameters of the hypothesis. You really don't need a data warehouse for that as long as you can enable access to the data required for such analysis. The important point to remember is that you need to enable accessibility to data instead of building a data warehouse and technology presents several options, a data warehouse is one of them.

Myth #3 - Data Warehousing starting point is an enterprise data model
The starting point of any data warehousing effort is not an enterprise data model but a definition of business operations strategy that the data warehouse will support. The data warehouse will be a component of this strategy. Therefore, you can't find ROI on the data warehouse. Instead, you have to work with the business teams to find out ROI on the entire business operations strategy that the data warehouse will be supporting. Not only you have to have a good definition of business operations strategy, but also you have to define the requirements in detail. Data warehousing starts from the top with a clear definition of business issues. I have seen tens of data warehousing efforts fail because enterprise data model was treated as the first step towards data warehousing. At the same time I have seen several data warehousing efforts succeed because business goals, expectations and requirements of data warehousing were defined in clear terms with a business value proposition attached to each requirement. The technical team understood clearly what they were supposed to accomplish.

Generally, enterprise data model is an expensive, time consuming and meandering effort. You start with a conceptual business model, follow-up with a logical data model and then map that to a physical data model. All this is done under a vague premise that somehow once you have done all this, you will be able to get all the data that you need to test any hypothesis across the enterprise. Most of the time, enterprise data models generate a pretty good piece of documentation that is understood by a couple of people in the IT. While it is validated by business analysts, most of the people are unable to make head or tail out of enterprise data models.

Mark Hammond is an Orange County entrepreneur, who founded Risk Data Corporation and later sold it to FICO. He built a data warehouse with a clear definition of business goals, strategy and what was needed for performance benchmarking of insurance and provider claims. The focus was clear and the product was successful. Similarly, Sean Downs founded Enclarity in California with a clear focus on cleaning up provider data in the health care industry. They are some of the entrepreneurs, who have created hundreds of million dollars value by focusing on business strategy and leveraging their understanding of the potential of data warehousing to deliver a business solution.

Harvard Pilgrim was less successful in its enterprise data model driven data warehousing approach. Kaiser Permanente has several data warehouses. Some of them are business-problem driven and some of them are enterprise data model driven. Initiatives driven by enterprise data model have been more expensive and less successful than business-problem driven initiatives.

Myth #4 - You need both an Operational Data Store and a Data Warehouse to cover the entire spectrum of business reporting
In the vocabulary of data warehousing an ODS is defined as a copy of transactional data. Once this data been cleaned, normalized and combined with data from past several years and other business systems, this becomes a data warehouse. Creation of an ODS followed by build-out of a data warehouse leads to huge duplication of effort with feeble business justification. If we accept the premise that we need a clearly defined set of business requirements for driving a data warehousing initiative, then it is hard to prove why an existing ODS cannot justify that data collection effort. Most of the time, if you have an existing ODS, the justification for a data warehouse tends to be pretty weak from a business point of view. Therefore, the right strategy for data warehousing must smartly eliminate technological need for an ODS and a data warehouse.

Myth #5 - Data Warehousing requires an engineering approach
If you are a technology company building an industry-specific data warehouse or a technology consulting firm with a specialization in data warehousing, then you will need to build an engineering approach to data warehousing. For most of non-tech organizations, an engineering approach to data warehousing tends to be an overkill.

An engineering approach to a data warehousing starts with conceptual data mode, logical data model and then a physical data model with lots of arguments among the technical folks about 3NF versus star schema, fully normalized versus fully denormalized, etc. At the same time, meta data management and master data management infrastructure is built. At the end of the day what you get is an overengineered data warehouse that takes too much staffing for support and maintenance, in short a white elephant. A limited approach achieves cheaper, faster and better results by focusing on ETL and reporting tools and allowing the reporting tools to drive the data organization.

Myth #6 - Data Warehousing fails due to problems with transaction-processing systems
Most of the time limitations of business transaction processing systems are already known before a data warehousing effort is launched. Therefore, it beats the logic when a failure of data warehousing is attributed to data integrity issues created by transaction processing systems. The problem lies in overengineered data warehouses. It is a sharp business analyst who provides data analysis, not a data warehouse. The most effective and economical approach to data warehousing is to keep it simple and let it show all the blemishes and warts of the transaction processing systems. The business analysts should have the tools, skills and knowledge to decide which warts and blemishes they want to remove for their analyses. Often success means straight copy of data from transactional systems to a bunch of denormalized tables. I'm sure this will sound as blasphemy to specialists in data warehousing.

Myth #7 - We can't predict what questions will be asked from a data warehouse
If you can't predict what you will do from a data warehouse, then don't build it. This is exactly the reason why so many data warehousing efforts fail. They try to answer "unknown" questions, a very tall order under any circumstances. It becomes hard to calculate ROI or link the data warehouse with a business initiative. Such efforts are often IT-centric and waste of precious dollars.

Myth #8 - Data Warehousing improves decision-making
I have seen that data and analysis driven decision making tends to be cultural in organizations. Possibly, at the top of the hierarchy are McKinsey & Co., who can take measurement to ridiculous levels and at the other end are several small and mid-sized service organizations, who have no concept of measurement-driven decision making. You can't improve decision making in organizations without a cultural shift. This is an issue that is generally bigger than the mother of all business issues. Without a clear cultural awareness of decision-making processes within an organization, a data warehouse will be like an impulse buy that gathers dust after initial excitement.

Myth #9 - Data Warehousing empowers front-line and business staff to do their own analysis
You can bring water to a horse but you can't make it drink. Empowerment used in data warehousing means as follows:
(i) Business users will be able to generate their own reports instead of sending a request to IT

(ii) IT will have less work
(iii) Business users will be able to do more analysis
(iv) Business users will be able to generate more insights into business operations

Here are the assumptions behind the empowerment:
(i) Business users are sitting idle waiting for IT to provide reports
(ii) Business users have the skills and knowledge to do extensive hypothesis testing and data analysis
(iii) IT takes more time in generating reports than the effort that business users will have to make once they have a data warehouse
(iv) Necessary organization alignment (euphemism for laying off IT data analysts and replacing them with business analysts) will occur after data warehouse is ready

Empowerment won't produce the expected results unless all assumptions are clarified and necessary organization changes are made. A data warehouse won't do it ipso facto.

Myth #10 - Data Warehousing reduces overall cost of reporting on business performance and opportunities
A careful measurement of the expenses involved in making data accessible to business analysts and building a data warehouse shows that data accessibility is a much better option than a data warehouse, unless data warehouse is a carefully designed top-down as part of an overall business strategy with clearly defined business requirements.

In fact, analysis of firms who have built "overengineered, enterprise data model-driven, data warehouses to provide answers to unknown questions" shows that they had to hire many more staff to maintain and support such data warehouses.

Michael Porter and I

I've read a majority of Michael Porter's books and articles. Several of his articles I've internalized by reading them over and over again. In fact, if Michael Porter has quoted anyone in his article or book, then more often than not, I've read them too. I think Michael Porter is the first real author of business strategy. Therefore, if you find Michael Porter paraphrased in my blogs, then you are certainly right. My blogs are not original, when I talk about his theory. As far as I understand Michael Porter, I also understand that neither I can imitate him, nor I want to. In fact, that is exactly what Porter would tell me.

Porter developed this model for analysis of competition within an industry. I had read Michael Porter several times but I did not really understand him until I attended Vijay Sathe's course on Business Strategy and Corporate Revitalization at Claremont Graduate University. Vijay brought Porter to me. There are very few professors like Vijay Sathe. In fact, the previous sentence is a direct quote from Peter Drucker, who thought very highly of Vijay.

Vijay makes a very simple grid in his class to evaluate competitive advantage of firms within an industry. A firm can get a +1, 0 or -1 depending upon whether the competitive situation is in its favor or not on the five forces model of Michael Porter. On this five point scale, a firm with solid competitive advantage can score +5 and a firm with negligible competitive advantage will score -5. Of course, a firm with a score of -5 will be doomed, unless they can change something pretty drastically, which is tough. Of course, operational viability is a necessary condition for the survival of the firm but it is not sufficient. Competitive advantage in combination with operational viability provide the necessary and sufficient condition for long-time survival and profitability of the firm.

This was a simple and elegant interpretation of Porter's model. However, you have to apply it to a large number of firms in different industries before you can understand its real value. I spent a lot of time reading Porter's books and articles on the elements of corporate strategy, particularly, positioning, strategic consistency, fit, competitive trade-offs, consistency and fit among activities. Therefore, you can blame me for copying Porter's ideas but that would fill me up with pride in the same way as a value investor would feel when accused of copying Warren Buffett.

Finally, a quick word about my blogs. My blogs are in permanent beta, which means that they will change and evolve as I think through the topic. When I write my blogs, I don't prepare rough drafts for editing and finalization before publishing. I publish my blogs as rough drafts and then I may work on them in full public view to edit and finalize. Often, I just leave them the way they are. I believe most of the bloggers work this way. However, that does not mean that I'm not inviting comments on my initial postings. I appreciate your e-mails and comments, which help me in formulating my thoughts and making improvements for the benefit of those who care to read.

What is FIT?

Make ROI secondary to "Fit" when analyzing investment opportunities and strategic initiatives.

• Fit over ROI
• What "fit" is not?
• How "fit" fits in?
• Do Software Methodologies "fit" with Programming Languages?
• Can't be half-hearted about strategy?
• What's a million dollar baby?
• Search for an optimum "fit"
• Coordinate like a chicken with head cut-off
• BHAG better than thousands of tiny little optimizations in functional areas
• Synergy, supermodularity and fit
• Look for fit, don't cry over ROI
• A very contrived example for techies

I have yet to meet a business executive who does not intuitively understand the idea of fit as a critical component of business strategy. In and around boardrooms, one can always hear incessant discussions about "fit". We can't hold this acquisition for too long, since it does not have a good fit with the rest of our business ... It will be a good idea to buy this other business, since there is a great cultural fit ... We have decided to join hands and merge our businesses due to extensive synergies between us, which will save us over $100 million annually.

While business people have a pretty good understanding of "fit", they become quite intrigued, when asked if they could measure "fit". Some of them define "fit" as synergy that will create savings, others define it as future increases in revenue at relatively lower investments. All these answers are partially correct.

Fit over ROI
My purpose is to show that "fit" and related concepts are not entirely unmeasurable. Of course, it is hard to measure the effect of "fit" due to complex interactions among its various components. But if we can work so well with an imperfectly measurable ROI, then what stops us from using a better metric of "fit". It is no secret that ROI on strategic initiatives, investment opportunities and sundry projects is hard to measure even if we exclude the impact of executive agendas on ROI. At the same time, it is possible that a perfectly accurate measurement of "fit" is NP-hard class of problem and measurement may become too technical. But if we can circumscribe the domain of our strategic decision-making, measurement of "fit" is achievable.

What "fit" is not?
But first, let's discuss what "fit" is not. "Fit" is not economy of scale, though returns to scale is one of the sources of "fit". "Fit" is not positive feedback effect either, since "fit" can exist independently of positive feedback effects and "fit" is not improvements in bargaining position. Though, enhancement of bargaining position may be an important goal during acquisitions.

Here are two examples of what "fit" is not and what it is. Two firms that offer white label "software as a service" solution in the banking industry merge together. As a result of the merger, the combined newco can now get larger discounts from telecom, hardware and power suppliers. This saves newco about $5 million annually. As you have guessed, this is not really "fit". In a seminal article on strategy, Michael Porter has described how different activities of Southwest Airlines fit together to form a strategy. That case study is a great example of "fit" but Michael Porter did not describe any way of measuring the value of that "fit".

How "fit" fits in?
So what is "fit"? "Fit" is complementarity. Complementarity means that doing more of a subset of activities increases returns to doing more of the rest of the activities. For instance--to use a very basic and hackneyed example--there is a high degree of complementarity or "fit" between assembly lines, low product variety, highly specialized workforce and hierarchical management. Similarly, there is a high degree of fit among flexible, programmable equipment, broad product line, knowledge workers, broadly trained workforce, flat management structure and higher quality. These complementarities are so strong that you can't improve profits by having flexible programmable equipment and narrow product line or by putting knowledge workers on assembly lines. You will be wasting investment in machinery in one case and talent in the other. Michael Porter found during his research that mutually complementary activities tended to make a coherent pattern. This gives us another conclusion, which is that you cannot take a subset of a coherent pattern of mutually complementary activities and combine that with another subset of mutually complementary activities. What this means is that you can't take quarter of a "fit" from one company and then try to insert that into another "fit" to improve profits. It just won't work. Cost will go up and the "fit" will be really poor. This issue of "fit" will manifest itself in low staff morale, low productivity, conflicts, wasted resources, problems in the design of HR policies and hiring practices. Management will blame each other for problems without realizing the fundamental problems inherent in the strategy. In short, a poor "fit" can lead to a really dysfunctional organization, an organizational equivalent of American Beauty movie. This is what makes it so hard to imitate successful companies. By the way, this proves another point, which is that you can't pick and choose parts of a strategy. Strategy is indivisible. It comes in one full whole lump with its strengths and weaknesses, with its beauty and warts and if you don't like the warts, its better to give up the whole strategy. Think of Freudian psychoanalysis, which is one full highly consistent set of premises and you can't logically accept one part, without accepting the rest of the theory.

Do Software Methodologies "fit" with Programming Languages?
Let's discuss the issue of software methodologies from the point of view of "fit". Traditional Waterfall methodologies were developed for a factory type, assembly line environment with distinct stages of software development. Modern agile methodologies, such as, Scrum are based more on overlapping phases of software design, programming and deployment to the extent of having a permanent beta in production. Waterfall methodologies had a good fit with programming languages like COBOL, where you could measure productivity by KLOC (kilo lines of code), separate requirements gathering, design, programming, testing and deployment activities and hire people with high degree of specialization in one of these activities. Structured programming methodology was developed to improve quality of software using old programming languages and tools. These mutually complementary activities made a nice coherent pattern. Newer object-oriented languages, service-oriented architecture did not fit in this mold. Programming of objects required programmers to have a good understanding of design. Test-driven development required software developers to become testers. As public knowledge about these languages increased, Agile methodologies emerged to leverage productivity using overlapping product development phases and a new coherent pattern of activities emerged. This coherent pattern comprised languages, such as, Java frameworks, Jtest, broadly trained software developers, short product development cycles and faster time-to-market.

Can't be half-hearted about strategy?
From the point of view of "fit", my question is: Would it be optimal for a subset of "Waterfall" pattern of mutually complementary activities to coexist with a subset of "Agile" pattern of mutually complementary activities? The simple answer is no. Due to lack of "fit", you can't combine parts of these two coherent patterns. In fact, if you don't find my idea ludicrous, you can try using Agile methodologies with Cobol and structured programming and then drop me an e-mail about how far it went. Even at a high-level you can't optimize on business goals by using "Waterfall" in one half of the organization and "Agile" in the other half of your product development organization. Change will have to be more radical that this pussyfooting approach. This is exactly the reason that newer start-ups are more successful in adopting Agile methodologies and achieving a fit with the rest of their activities to drive competitive advantage over established firms. An example is GorillaNation.com.

What's a million dollar baby?
Interestingly, the "fit" within successful organizations extends far beyond these simple concepts. Successful organization create far-reaching coherent patterns of product design, development, marketing, human resources policies, internal communications, internal controls, financial decisions, supplier relations, distribution, etc. Such strong fit among various activities not only leads to improvements in profits but also improves competitive position of the firm. I'm sure that by now you have guessed a problem. Exploiting such extensive systems of complementarites requires substantial high-octane coordination among different functional areas of the firm. This strong fit also causes huge resistance to change. You can't just make a change at the margins and expect to enhance profitability or competitive position. At one of the large organizations, due to high-level of coordination, several projects were called million dollar babies. Even smaller projects took nine months and a million dollar to deliver. If you heard your senior executive team talking about "radical restructuring" or "nothing short of an overhaul", or "wipe the slate to start from scratch", then they were certainly not kidding. You cannot effect a long-term sustainable change in these extensive systems of complementarities, unless you wipe the slate clean and start from scratch.

Search for an optimum "fit"
Let's again take a software example. If you are using traditional "Waterfall" with Cobol and you introduce a new platform with modern languages and Agile methodology, then profitability will decline, expenses will go up and depending upon how much your actions are valued, your failure will be characterized as full or partial. Please note that my objective for optimization is purely financial. In order to maximize profits, such a change will have to be coordinated at the highest level with finance, marketing, human resources and product distribution to create a whole new extensive system of complementarities or a new "fit". Mathematicians will say that in order to improve profitability through such a change, you will have to coordinate across the whole organization due to "non-convexities" created by complementarities or "fit" among all the different activities of the firm. Mathematicians tell me that a non-convex objective function cannot be optimized in time exponential. So does that mean that non-convexity is NP-complete. I don't know but it just emphasizes the need for high-octane turbo supercharged coordination among all the activities of a firm to search for an optimum solution. Exactly the reason why managers have to be reflective practitioners. However, a search for an optimum business strategy does not have to be a wild one. In fact, it possible to show that that you can limit your search to the adjacent domains, which provide improvements over the current optimum. You will still be able to achieve at least half the gains that may be possible from an unrestricted optimization of business strategy.

Coordinate like a chicken with head cut-off
Therefore, if you put your firm ahead of you, you are going to be running around--may I say like a chicken with head cut off in a positive sense--to ensure that all the activities across the organization are strongly coordinated for such a change to become a success. In fact, this was exactly what John Nash (remember The Beautiful Mind) proved. Every coordination game involving two or more players has at least one Nash equilibrium. Once Nash equilibrium is achieved, no unilateral action by a single member can increase the team's payoff. If you recall prisoner's dilemma, you will notice that it's impossible for a single member to unilaterally improve the team's payoff. The condition of co-existence of "Waterfall" and "Agile" methodologies may be no different than prisoner's dilemma. Rational decision-making ends up reducing payoff. This is exactly why big, hairy, audacious (BHAG) goals are considered more valuable, since such goals have the ability to drive a higher level coordination across the firm. Managers know this intuitively. There are complex mathematical theorems developed by Paul Milgrom that show that without coordination at the top, you won't go too far. To get real benefits, you will have to totally redesign the way you do business.

BHAG better than thousands of tiny little optimizations in functional areas
If managers are unable to coordinate their choices and while attempting to optimize within their functional areas they believe that the choices of other managers are fixed based on the past behavior of their colleagues, then they will positively and systematically under-respond to changes. John Nash proved that under such circumstances, if team members were individually allowed to search for improvements, no further improvement will be possible for the team as a whole unless changes were coordinated among the team members. This solves another puzzle for us. Several times I've been asked whether it is better for each manager to optimize within their functional areas instead of trying to do big stuff, since big stuff is frustrating and choices are painful. The answer is no. You will invariably end up suboptimizing your business strategy unless coordination occurs across all the functional areas. There are no short cuts to achieving fit.

Synergy, supermodularity and fit
Another mathematical concept that corresponds to "fit" is the concept of supermodularity. It was Paul Milgrom, who applied this concept to Economics and Michael Porter brought it to analyze business strategy. In a very rough manner of speaking, supermodularity is similar to 2+2 = 5 or synergy effects. Purists would like to hang me for saying that. What supermodularity means in Mathematics and Economics is that the sum of changes, when all components of a system are increased together will always be more than the sum of changes, if components are increased in smaller groups. Whole is more than the sum of its parts. Does it remind you of Kurt Lewin and Berlin School of Gestalt psychologists? Well, here we have another proof that if "fit" is extensive, then nothing will improve substantially unless changes are coordinated across all the functions. Are we talking about radical change again?

Therefore, "fit" is a concept that can be defined, analyzed and measured by looking at the complementarity of activities, supermodularity, coherent patterns of activities, theory of games analysis and returns to scale. But this measurement is not going to be a cakewalk. Allow me to use some circular logic to say that "fit" is important, since most of the firms value complementarity over similarity and "fit" over economies of scale.

Look for fit, don't cry over ROI
So what does this mean from a practical point of view. All measurements of ROI (Return on Investment) will be incomplete unless you can take the effect of "fit" into consideration. Fit is real but hard-to-measure. If the "fit" is going to be improved by the project, the ROI of strategic initiatives will be underestimated. On the other hand, if the strategic initiative won't enhance the "fit" or lead to friction and conflicts, then ROI will be an overestimate. Which proves that sometimes there may be good mathematical reasons for consensus-building within organizations. I'm not saying how this consensus building should occur. Such a consensus-building could be authoritarian or collegial. It doesn't matter how it is achieved as long as it exists. Anyway, that is a digression from our topic. My recommendation is that analysis of strategic initiatives should be more in terms of "fit" than in terms of ROI. In fact, ROI should be secondary to "fit". But as we have seen above, every "fit" comes with its own trade-offs. Alas, there are no free lunches!

A very contrived example for techies
As I had propositioned earlier, software development methodology must "fit" the programming language that is being used. For example, it would be foolish to use COBOL class of programming languages under an agile methodology. While programming languages get the job done or produce the functionality, there is plenty of evidence that shows that it is the software development methodology that ensures the quality of the output. How effectively business requirements are met is a direct function of effectiveness of a software development methodology. Therefore, programming creates part of the output and methodology generates part of the value of the output and then there is a subtle interaction between the two, like, yin and yang or whatever.

Here is a matrix (not really!) based on the theory of games. Assume that the best attainable output using aggregation of all available best practices is 10. The figure shows in top left cell that the output of COBOL class of programming languages in an agile environment is 6 and enhancements to the quality of the output as a result of plan-driven methodology is 6 too. When we use COBOL class of languages with agile methodology, the output of language declines to 5 but the contribution of methodology to the output declines substantially to a level of 4 due to "misfit". This is shown in top right cell. We notice a slightly improved productivity of object-oriented (Java type) and frameworks under a plan-driven methodology with the output of language at 7 and no improvement in the value created by the methodology, which is still at 6 under bottom left cell. Under agile methodology, the value created by language as well as methodology increases. This is shown in bottom right cell. It is easy to notice that Plan-driven methodology is dominant over Agile and object-oriented languages are dominant over COBOL-types. Under mixed environment, it is clear that Plan-driven will be a preferred strategy.

Now let's check out some mixed strategies. What happens when we use a 50/50 mix of COBOL and Java programming languages in an Agile environment. We get an average output from programming languages and methodology at (6, 5.5). This is worse that using COBOL under plan-driven. This is exactly what happens in many traditional IT shops, when "savvy" management introduces new languages and new methodology to achieve higher productivity, notices a decline and finds it hard to accept it. Similarly, let's check out 50/50 mix of COBOL and Java under Plan-driven. we get (6.5, 6), which is a slight improvement.

Now let's make a slight change to the strategy payoffs. In the game that I have shown below, I have slightly reduced the payoff of object-oriented languages in a plan-driven methodology environment. Similarly, I've reduced the payoff of plan-driven methodology in an object-oriented environment. Now we have set the stage that could lead to deep suspicion between the Engineering or Enterprise Architecture department that owns the technology for programming and the Program Management Office (PMO) that owns the methodology. Let's refer to the picture on our left for a review of strategies and "fit" from the point of view of cooperation between functional areas. The firm is using COBOL-type procedural languages for developing its product using plan-driven methodology. Analysis shows that productivity+quality will be raised by about 17% by moving to object-oriented framework technologies for product development using agile methodology. If the Engineering Department and the PMO are able to coordinate switching to a new programming technology and a new methodology, we won't have any problem. However, if PMO is worried that due to shortage of funding the Engineering Department won't be able to keep its words to switch to a new programming technology, then they will pay lip service to moving to a new methodology but won't take any action. The PMO will be afraid that the output of methodology will drop to a meager level 4, if the Engineering Department does not switch to a new programming technology. Similarly, the Engineering Department may be suspicious that due to lack of skills, the PMO will not be able to switch to a new agile methodology. The result of this suspicion will be that the Engineering Department will be highly reluctant to move to a new programming technology due to their fear of reducing their productivity. Now we have this paradoxical situation, where good intentions of functional departments to not allow their productivity to decline under any circumstance will hold them off from improving the firm's payoff. Does it sound like prisoner's dilemma?

As I mentioned earlier, this is a contrived and hypothetical example. But it does present a framework for analysis of "fit". Now you can change the values and run some tests to validate or invalidate your hypotheses. Drop me a comment if you like it. Have fun!!

Bring me a rock!

There is a recurring story about "bring me a rock" that is deeply embedded in the collective consciousness of all failed and un-failed IT professionals. The story is about a boss asking an IT professional to bring him/her a rock and each time the IT professional delivered a rock, the boss said that that was not the rock that he or she had asked for. This back and forth goes on ad infinitum until the IT professional is fired due to his or her utter inability to deliver the right software product. The story ends with the IT professional blaming the boss for not accurately defining the specifications of the rock that was originally requested.

I have issues with this story. A product definition with detailed and proven specifications is an absolute requirement for a functional product. However, that does not absolve IT professionals from the responsibility of understanding the business issues and concerns and incorporating them in the product design. Often I come across IT professionals, who define tight boundaries between IT and business. Well, if IT is not business then IT better not be part of the business. If I could define my specifications accurately, then I'd rather outsource all my technology to vendors with larger pool of technology skills, better specialization and economies of scale. The only reason that IT continues to be part of several businesses is because of its deep understanding of business and business needs. Successful IT organizations are very good at analyzing business issues and designing solutions that directly deal with those business issues. They engage effectively and aggressively with business teams to solve operational and strategic business issues thus blurring the boundary between IT and business. If they can't do it, then it may be more cost-effective to outsource and close down the internal IT department. Not having requirements is a pretty bad excuse for not designing a product right. In the same vein explaining project delays on requirements gathering does not make much sense. The onus of creating business value is on IT.

Google’s Approach to Privacy Issues

"If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place."

— Google CEO Eric Schmidt

Do we have to make the need for privacy a filthy thing?

Eric Schmidt is one of the sharpest technology CEOs. He brings a calm and measured approach to managing the turbulence in technology industry. But Mr. Schmidt is not my pastor and I don't expect a moral sermon from him on my need for privacy and confidentiality. Recently, I was watching Maria Bartiromo interview Eric Schmidt on CNBC, when I heard Eric Schmidt make this comment after explaining Google's approach to privacy.

Let's do some thought experiments:

1. An employee is diagnosed with cancer and her search pattern reveals her identify, likely diagnosis and related concerns. Should her employer and colleagues know about it before she is ready to announce it?
   
2. One Mr. Eric does not want anyone to know about the church he attends. However, his search for directions reveals the church he attends. Should his search pattern related his church remain confidential as he desires?
   
3. A drug-addict decides to transform her life. Her search pattern reveals her identify and her addiction. Do we want this information to remain private to her?
   

There are some complex issues here:

1. Do people realize that when they do a Google-search, they are sharing their private information with a very large corporation?
   
2. Do people realize that Google mines this data and the pattern of their search not only reveals their identity but also their personal matters and concerns, which normally they would not share with anyone else?
   
3. Do people retain their right to the ownership of their private information after they have shared it with Google? US courts say no!
   
4. Do people realize that no search warrant to access their private information is required by a Government agency, if such information is obtained through a third-party like Google?
   

Eric Schmidt realizes that the strategic issues that Google is facing are intertwined with significant moral issues related to people's privacy. But do we have to conveniently make the need for privacy a filthy thing?

Ten Patterns of Failure

Success must not depend on heroics and failure must not depend on lack of talent. But both these situations are recurring themes in many organizations, which shows that organizational learning is not automatic.

Organizational learning is deliberate, non-intuitive and requires a substantial and coordinated expenditure of effort throughout the organization. In IT organizations, there are several patterns of success and failure. In fact, some of the patterns of failure will often look like successes.

Failure Pattern #1: This pattern is quite common. A major problem is encountered and success is achieved through the heroics of a few people. Often this heroics involves working non-stop for long hours to get the job done.

Failure Pattern #2: Success is achieved by timely intervention of a couple of talented individuals. As the target date nears and the output does not look like what was expected, our talented troubleshooter, who is often working on something else, is called in to solve the problem. Mr. or Ms. Talented comes in and solves the problem. Management feels pleased that they identified the problem and took timely action to achieve success. The organization stays exactly where it was with zero learning.

Failure Pattern #3: A solution is found before the problem is understood. The solution is developed. When it does not fix the problem, more effort is made to fix the solution instead of defining the problem. Programmers work twice as hard in a trial and error manner.

Failure Pattern #4: This new technology is so exciting! A solution is purchased. It sits waiting for a suitable problem. A team is assigned to find out the problem that would be fixed by the solution.

Failure Pattern #5: A commitment is made with zero understanding of what is required to deliver.

Failure Pattern #6: This is a technology problem. Our business analysts don't understand technical design. Our computer programmers don't understand our business.

Failure Pattern #7: We are action-oriented. Let's get the job done. But the end state is never defined or a consensus about the end state is not established among the key stakeholders.

Failure Pattern #8: A solution and a system are proposed from the top. The rest of the pattern is all too common.

Failure Pattern #9: I like this technology vendor. We've good relationship. This approach becomes the fountainhead of several other patterns mentioned above.

Failure Pattern #10: Senior management says that we don't have resources. What this means is that we don't know what we are doing.

Google's Made in China Hack

Recently, Google announced that their security was compromised by a "sophisticated" and "organized" cyber attack that originated in China. In addition, Google found that Gmail accounts of Chinese dissidents were accessed frequently by third parties and there was an attempt to access the contents of e-mail accounts of Tibet human rights activists as well. As a result of this, Google is now unwilling to be a partner of the Chinese government in self-censorship of their Chinese search engine Google.cn.

Well, the purpose of this blog is to raise some questions instead of trying to find answers. But if you have answers, please feel free to leave a comment. On the other hand, whatever the motivations of dramatis personae, viz., Google, Chinese government, US Government and yet unknown hackers, one thing that stands out in the clear is that Google's computers were hacked into and sensitive information was accessed. Therefore the information stored on Google's computers is not beyond compromise. Douglas Rushkoff advances an interesting hypothesis on this point of view.

I read about this story in the news and I noticed that the story had many gaps. Here are some of the gaps:

(A) Symantec corporation does not consider an attack using Trojan Hydraq and an e-mail attachment as a method of infection to be particularly "sophisticated".

(B) Google has been carefully avoiding directly blaming Chinese government for this incident, on the other hand Google wants to use this incident to start negotiations with the Chinese government for lowering or possible removal of search engine censorship. The cause and effect link between these two factors is rather tenuous without invoking the already compromised moral principles of free speech and "Don't be evil."

(C) Is it a belated realization on the part of Google that they cannot compete with Baidu in the Chinese market unless search engine censorship is substantially reduced or removed?

(D) Cyber spying is a growth industry. FBI reports indicate that attempted cyber attacks on the departments of defense and state have been rising. Gerald Posner's article on China's Secret Cyberterrorism examines this topic in more detail. I was heartened to know that Google did launch a counterattack to find out the source of the attack. But what was new about this issue, other than the fact that this was perhaps the first time that Google's computers were attacked and compromised?

Repression of dissidents and human rights activists in China has been going on since communists came to power more than half a century back. There has been little change in government-sponsored repression in China in spite of the economic progress of the last three decades. With as many as 470 executions per year, China continues to have the highest number of executions per year. In recent years, Chinese Communist Party (CCP) has brutally suppressed uprisings in Tiannenmen, Tibet and Xinjiang. In China, you are free to express any opinion as long as your opinion is in agreement with CCP. Recent removal of the blockbuster movie Avatar from the theaters in China was another example of this control. Chinese government has not shown particular sensitivity to international opinion in this regard.

Interestingly, during this spat Chinese government accused United States of information hegemony. Recently, Tom Mulvey, my colleague at ABPA, gifted a beautiful globe to me. Looking at the map of China I noticed that the map included more than 80,000 square miles of disputed territory as part of China. Taiwan had the same color as mainland China and Sakhalin was not mentioned at all. Of course, when I turned it upside down I noticed "Made in China" printed in a corner. Whose "information hegemony" it is?

If my reading of history is correct then restoration of natural rights is a political process, not a commercial activity. Therefore, Google's sudden reawakening of their moral commitment to free speech in China seems a little incongruous. Google is a respected and ethical business not unlike Microsoft, Oracle, Intel or IBM, who are rightfully guided by long-term economic interests and competitive advantage. Therefore, I'm afraid that Google's "holier than thou" approach to advancing of commercial interests and covering of their computer vulnerabilities with a righteous indignation will just create more political friction instead of achieving business goals.

A Bend in the River

Only dead people have a past. Rest of us living have a future. I find it more worthwhile to talk about the future, which we can change, than the past. At least until we have figured out how to make a time machine. But as I'm about to reach a bend in the river towards the end of this month when I take the oath of citizenship of the country of my dreams, I want my readers to allow me to indulge myself by reminiscing about the journey of these past twenty years. My plan is to try not to choke when I take my oath of citizenship. I'm proud of earning my citizenship.

Rule #1: Language does not belong to anyone. If you know it, it is your language.
I must have been eleven or twelve years old when I told my dad that I did not want to learn English, since it belonged to British, who treated India as a colony. My dad said spontaneously that a language only belonged to one who knew it. If you learned it, it would be yours.

Rule #2: You can't get something without giving up something.
Twenty years back I had an excellent job in the Reserve Bank of India at a middle management position but I wanted to be somewhere else. I was married with a new-born and I was about to resign from my job to join a masters program in information systems at the Arizona State University. All my good friends and family thought that I was going nuts, nevertheless they all supported me. It was hard to give up a nice job and liquidate the household, it was harder to leave the family behind but the hardest part was when I sold-off my Yamaha motorcycle. That did hurt pretty deep at that time.

Rule #3: You can't plan for everything but if you follow your heart you will find someone to help you.
In my twenties I had thought that I had planned every aspect of my life pretty well. Almost like a Prufrock, I had measured out my life in coffee spoons ... I had achieved what I had set out to do. But very soon I realized that I was just not enjoying what I was doing and I was not ready to do what I was not enjoying for the rest of my life. So I landed at LAX with about a grand in my pocket, about the same in my bank and a scholarship for doing masters in Computer Information Systems from Arizona State University. Very quickly, I learned that that was close to being penniless and I had to find a job at Arizona State University in Tempe. Richard at the Disability Resources Center of ASU was kind enough to hire me as a Tutor. I worked for Richard for the entire duration, while I was studying at Arizona State University. When I was leaving, Richard and his assistant Phyllis gave me a very nice memento that said, "You have opened the doors that will never close ..." I keep that with me.

Rule #4: It is important to focus on your strengths but it is hard to know about your strengths unless others tell you.
I found a job as a Systems Architect in Los Angeles a couple of weeks before I had completed my masters. Professor Michael Goul agreed to have me give the final test from home in LA. I completed the final test in the night after coming from work but I was hallucinating through the test due to sleeplessness. Somehow I completed the test without messing up and got an A in Artificial Intelligence logic. Professor Goul called my paper a "tour de force of AI."

Rule #5: What you can do for others is more important than who you are.
My wife and son joined me in LA after almost a year and a half. My son was excited to see me but I quickly came to a realization that we did not have much of a relationship. Of course, I should have known that there was nothing like a dad-in-absentia. Whenever my wife stepped out for anything, he used to start running around screaming mom. One day my wife was in the bathroom, when our son woke up and started screaming for mom. My wife was quite irritated and she kept quiet. I tapped my son on the shoulder and both of us walked to the bathroom door, where I bent down and showed him how to peek through the slit at the bottom of the door to see mom's feet. He saw mom's feet, felt comforted and then looked happily at me. We had bonded.

Rule #6: Procrastination will kill you.
I joined UCLA Medical Center as a Database and Unix Systems Administrator but I had to complete my immigration paper work. I was like a kid in a candy store. Finally, I had all the technology that I wanted to play with. There were symmetric multiprocessors, mainframes, gateways, token ring and ethernet networks. Any technology that you could ask for, UCLA had it. I loved it and I must have been working at least 80 hours a week on an average. But I procrastinated on my immigration paper work and almost got thrown out of the country of my dreams!

Rule #7: Be thankful when you get what you ask for.
I was working long hours, when my daughter was born. It would be a gross understatement to say that I was excited about her. I was loving every moment of it but I was not willing to cut down on my work hours. One day we had several system outages and I had to work until late night on problem resolution. I came home exhausted and fell asleep. Shortly after I heard my daughter crying in her crib and woke up with frustration and anger. But the very next moment I was filled with guilt and shame when I realized that this was exactly what I had wanted in my life. That was one moment of epiphany for me.

Rule #8: You have to be fair.
I joined HNC Software, Inc. in Irvine. This is now called FICO, yes they do FICO scores. HNC merged with Fair Isaac Co (F. I. Co) and became FICO. This was an exciting time since I was leading the development of one of the largest data warehouses. Vince Bianco was the Chief Operations Officer. When I was promoted to the position of a Manager, he called me to his office and said that one of the most important things he thought was that a manager must be fair. You can hear a sentence a thousand times without it making an impact but there are certain occasions when someone says the same thing and it just stays with you forever. I still vividly remember Vince telling me that a manager must be fair!

Rule #9: Its not about money.
At the peak of Internet boom, I got a job offer from DoubleClick, which was about to go public. DoubleCick was giving me 10,000 stock options priced at a strike price of a penny each. I went to DoubleClick's Irvine office, met with people and joined Perot Systems instead. Somehow I felt that everyone at DoubleClick was working only for money. Within a month Doubleclick went public with stock price reaching over $100. But soon afterwards, there was a bust with several rounds of layoffs. On the other hand, I was having a ball working for Perot.

Rule #10: You just can't connect the dots beforehand but you can make a difference at every point.
After my dad, Peter Drucker made the most influence on my life. However, it was a pure coincidence that I was passing through Clarement when I remembered that that was where Peter was teaching. There are too many coincidences in life. I left a nice job at the Reserve Bank of India, joined Arizona State University, worked at UCLA, FICO, Perot, studied under Peter Drucker, joined Kaiser Permanente and then came to ABPA. There is no way in hell that I would have been able to connect all these dots twenty years back. Sometimes, things just happened because I was at the right place at the right time. The only constant through all this has been that I have tried my best to make a difference and I've tried to be persistent in my efforts.

Rule #11: Sometimes the best moments of life come wrapped in a piss-soaked newspaper.
When I was working for Perot Systems, I had to work for two and a half months from India. I love trips to India but this was the worst time. I did not want to do it, since I had little kids and it was hard for me to stay away from them. But reluctantly I went to India. While I missed my kids and wife, I had a great time with my parents, brother, sister, in-laws and their kids. Those moments became incredibly invaluable for me, when my dad passed away a few years later.

Rule #12: Don't believe anything until you see firm evidence.
My dad passed away when I needed him most. Sometimes I see him in my dreams. I still miss his hugs. When all the drugs had stopped working on my dad, then he was treated with Trisenox. There is a more about my dad's battle with cancer here. Regarding his treatment with Trisenox, I came to know after his death that off-label use of Trisenox for multiple myeloma was really an Internet hype created through unethical and illegal marketing efforts of Cell Therapeutics, Inc. (CTI). In 2008 a federal judge awarded whistleblower James Marchese $1.6 million for tipping off federal prosecutors to a scheme by CTI to illegally promote unapproved uses of Trisenox.

Rule #13: Do the right thing one at a time and this too shall pass.
When I came to this country, I was alone, I was short of money, I had no job, I did not know how I was going to complete my masters or find a decent place to live or what I was going to do after doing masters. Those were some highly stressful moments. Similarly, when I was about to be thrown out of this country for delaying the paper work, I was completely stressed out. This was my country, my choice, my dream. But I had brought it upon myself by procrastinating. There were times when I had to wait in line outside INS offices for five to six hours. There was a time, when my wife was unable to leave for India to see her sick brother due to delay in processing of her paperwork by INS and I was scared. The law is that if your green card is approved pending status adjustment and you leave the country without seeking advance parole from INS, then your green card is considered abandoned and you can't re-enter the country. There are so many scary what-if scenarios that you can only pray and hope that everything will be alright. So next time when you have to sit on the Magic Mountain roller coaster with your kids, relax, take a deep breadth and remember that if you do one right thing at a time, this too shall pass...