Offshoring web development to India (or not) in the new age

August 8th, 2014

The new generation of web architectures and development methodologies, notably client-side frameworks such as Angular or Ember and the agile/scrum approach, represent an existential threat to the Indian offshore software development model.

Recall that the whole offshore model is predicated on reselling the time of Indian developers working for rock-bottom salaries, at markups ranging from 2x to 10x or more. A brand-new graduate being paid $2/hour may be re-billed at $50K/year. A “senior” resource, whose salary is $4/hour, if placed on-site in the US could be billed at well over $100K.

Typical Indian developers are poorly trained and marginally competent. Mentors, architects, and reviewers are close to non-existent and hardly more competent. Overall productivity can be lower than the typical Western engineer by up to an order of magnitude. Part of the problem is that many Indians go into the computer field not because they have the aptitude or interest, but because they heard it is high-paying. Another major factor contributing to the low competency levels is the abysmal quality of CS education at mid-tier and lower-tier schools. It is not unheard of for college graduates in CS to never have written a functioning program running on a computer; they write programs on paper and pass them in for grading. Many graduates can barely write a for loop or access an array element.

Another key problem with the offshore model is the low level of morale and high attrition in the workforce. Twelve-hour days are not uncommon. Turnover can reach 30% per year or higher. As a result, there is little of the institutional memory that is required when working on projects that require higher-level skills. Much of the time paid for by US clients is for onboarding and training. Staff who do stay with the company and project are burdened by the need to bring the constant stream of newcomers up to speed. Of any 10 new members brought on to a project, 3-4 will soon prove to be completely unproductive, and must be off-boarded, their time and that of the people who tried to train them completely wasted. To deal with the attrition problem, Indian services companies have introduced extreme notice period policies of up to three months, with the uniquely Indian twist that salary is withheld until the notice period is served out; if it is not, the employee is actually made to reimburse the company for the unserved portion. This has had predictable consequences: employees are quicker to resign, because they know they will have plenty of time to find their next job. A specialized job site has even popped up targeting those stuck serving their notice. And of course they produce little if any work of value during this extended period when they are halfway out the door, although of course the client continues to be billed for their hours.

Nevertheless, the economics of offshoring can still work, just barely, if the tasks Indian developers are asked to perform are highly repetitive or predictable, such as maintaining existing pieces of software, writing a hundred input screens, or making minor enhancements. Indian developers have reached a moderate level of competence in the previous generation of web architectures, based on PHP, JSP, ASP, and bits of jQuery. They specialize in rote, formulaic tasks.

Of course, I am painting with a broad brush here. What I say is less applicable to the top-tier companies, who have access to a better candidate pool, and more history and process. It is also not applicable to the captive offshore development centers in India run by Western software companies, who tend to hire and groom carefully. Nor do I intend any disrespect to any of the many great Indian developers I have had the pleasure of knowing.

Be that as it may, now the world is changing. We have new web architectures and methodologies. How well will the Indian offshoring model work in the new age? Should Western companies look to Indian service companies to develop their next generation of web presences and customer touchpoints?

A major component of the new web technology stack is the client-side framework, such as Angular and Ember. These are not libraries, they are frameworks. They are foundations for building a company’s client-side infrastructure for the next five years or longer. They replace masses of boilerplate code found in systems based on older technologies–reductions of 10x in lines of code are not unheard of. The key development task becomes how to build on these frameworks to create a high-level structure of reusable, interoperable components. The new generation of CSS frameworks, notably Bootstrap, falls into the same category. Developing the look and feel of your website is no longer a matter of writing hundreds or thousands of CSS rules, but rather designing how to customize and extend and build on your CSS framework, be it Bootstrap or Foundation.

bmw-jugaad-india

In other words, the new frameworks are a starting point for building infrastructure. They are platforms for infrastructure. However, infrastructure is traditionally something India has had a hard time wrapping its head around. Witness their electricity or transportation infrastructure. Rather than building infrastructure, India has come to specialize in “jugaad” (image), a form of small-scale innovation which is basically designed to work around the absence of infrastructure. Much of the code written by the mid-tier services companies is a monument to jugaad. Building infrastructure is not just something Indian companies have a hard time doing; they have a hard time understanding why they should do it in the first place. In any case, building infrastructure requires high-level design skills not typically available, not to mention knowledge of the frameworks which is also absent.

The move to agile methodologies is another trend that poses major challenges for the traditional offshoring model, which evolved in a waterfall context. A part of a project was hacked off, specified completely, assigned milestones, a deadline, and acceptance criteria, then thrown over the wall. In contrast, agile envisions tight feedback loops and rapid changes in course; these are nearly impossible to realize over distances of thousands of miles. If the scrum includes both on-shore and off-shore members, it is impossible to schedule daily stand-ups at a time of day when one side or the other is not half asleep. The poor English skills of many Indian developers also makes nearly impossible the fine-grained communication among team members assumed the the agile process.

User experience, creative design, and messaging are taking on an increasingly prominent role in modern web development. This new paradigm of agile collaboration between these designers and developers is another area which makes the offshore model work even more poorly than in the past. India has an extreme scarcity of competent UX and interaction designers, especially those who understand modern UX paradigms. Therefore, these designers are mainly in the US, whether in-house or outsourced. In either case, if the developers are in India, this represents yet another interaction to try to manage across the Pacific.

None of this, however, has seemed to stop the the executives of these 2nd and 3rd-tier service companies from jetting around the world peddling capabilities they don’t have, based on architectures they don’t understand, using methodologies which exist only on paper at best.

So what is a Western company to do? Completely swearing off offshoring to India, at least for new projects, certainly looks like a promising alternative. Labor mobility in the US workforce is now such that it is easier than ever to find in-country resources at reasonable rates. If one does still want to offshore, the trick is to limit the work to maintenance and upgrades to the previous generation of products based on the previous generation of technologies, where the structure is in place, the platform well understood, and the tasks to be performed well defined.

Whither, then, the Indian services company? There is no doubt that the immense overhang of demand for keeping existing systems running and fixing them when they break can keep plenty of food on the table for them. However, if they really want to tap into the sweet spot of all the new development being done on mobile apps and responsive websites, they need a radical strategic overhaul. They must hire top-notch architects and developers, conversant with the latest stacks, on both sides of the ocean, and pay them what they’re worth. They must reject the human-wave approach of trying to stuff bloated, unqualified teams down their clients’ throats. They must ensure that their employees are fully functional in English. They need to consider bringing skilled foreign resources to India to rapidly upgrade the skill set of their workers. They need to respect their employees and clients. Above all, they need to stop basking in past glories and adapt to the new circumstances.

The Meaning of Meaning (3)

February 1st, 2014

Of course, Zen therapy does not pinpoint only existential angst by mitigating cravings for universal meaning. It also influences the pattern-making and pattern-matching we use day-to-day. Now, our pattern machinery basically works quite well, look how far it’s gotten us. But at the same time there are many ways in which it’s broken, or should I say problems in how we use it. The main issue is how we try to apply patterns when they don’t really fit, how we apply them with no flexibility, how we fail to tweak the patterns as we are applying them. Of course there has to be an essence of the pattern, which is the point of its existence, but at the same time we have to keep evolving the patterns, choose the right patterns to apply, adapt and refine them. Our built-in reward system for finding and matching patterns can lead us to use patterns which aren’t really relevant, or to try to stuff a pattern down some situation’s throat.

Dogen’s Zen therapy not only relieves our cravings for patterns, and leaves us comfortable even in pattern-less situations, but also acts as a kind of lubricant for our patterning systems. It gradually rewires our brains so we more easily find new patterns, modify existing patterns, and choose the right pattern.

But let’s go back and parse a bit more of what Dogen was saying. He refers to “enlightened ones”. This, obviously, in terms of the patterns we know and love about sainthood and rarified states of being, must mean someone who is at a completely different level in terms of grasping deep, intricate patterns about existence, right? No, it just means a person who through intensive therapy has ceased addictively looking for non-existent patterns. He refers to “ultimate awareness”. This, obviously, must mean some exalted state of being achievable only by some sainted person in an altogether different league from ourselves, right? No, it just means someone who has stopped spending his energy seeking unseekable patterns and who therefore is able to focus on the reality in front of him.

Deciding to Start Zen

January 29th, 2013

The decision to start Zen could be treated more like a decision to start taking a painting class at the local adult education center. You could go to one class and not go back, or go for one quarter, or keep going for ten years, or keep going until you die. I guess I also disagree with the people who say this is a really big deal and is going to be real hard work and you have to really make a commitment or else the whole thing is a complete waste of time. Wrong! The thing about Zen is that, and this fits in with my theory that’s basically about rewiring your neurons, or neuroplasticity if you prefer fancy words, you do a little bit of it and experience a little effect or do more of it and experience more or do a whole lot and become an enlightened Zen master. It’s linearly scalable, in other words. If you started off by going to a dojo once a week and sitting with them, and maybe meditating 30 minutes per day, then after a month or two no, you would not have some kind of life-changing realization, but you would notice changes in the way you look at things which would be refreshing and helpful. A little bit of rewiring.

There is another thing about Zen which is sort of like the way that when you work out at the gym, your body changes in a way which helps you be more fit and healthy even when you don’t work out, you consume calories more efficiently, etc. As you keep doing Zen, by which I mean meditation, not reading books or listening to talks, you develop the ability over time to use every moment to enhance the way you view the world, not just when you’re meditating. Or to put it another way, you sort of learn to meditate as you’re going about your daily life, or conversely, you begin to see that your daily life is a kind of meditation, or perhaps that meditation is nothing more than a sort of concentrated form of daily life. As you get there, you might find yourself wanting to meditate more, and kick it up to two 30-minute sessions per day, as opposed to a thought process that says hey, I’m supposed to be doing this Zen thing, gotta work harder at it, I told myself I was really going to get into this, so I gotta buckle down.

The Meaning of Meaning (2)

January 28th, 2013

It would be one thing if our unmet need for existential closure meant only that we had occasional unsettling attacks of meaninglessness, or a vague cloud of doubt hanging over our heads as we went about our daily lives. But unfortunately this nagging psychic unease also interferes with us just getting done our day-to-day business as normal human beings, as if we were lugging around a 20-pound sack of potatoes everywhere we went. For all the design flaws in the whole homo sapiens mind-body system that the messy evolutionary process has bequeathed to us, we basically work remarkably well. We’re endowed with well-functioning instincts and behaviors. We’re lithe. But the bag of potatoes bogs us down, gets in our way, disrupts our rhythm, saps our mojo, puts us off our game.

Then there’s a second order effect which is that our inability to find the big pattern we crave saps our confidence. We can’t find it–so maybe there’s something wrong with us! Maybe there’s other patterns we’re missing! Maybe other people have found the pattern and we’re inferior to them! This fear, this complex is yet more sand in the gears. If the basic existential pain is not sufficient motivation for us to try to do something about the problem, this negative effect on our basic functioning certainly is.

Read the rest of this entry »

The Meaning of Meaning (1)

January 26th, 2013

Here is the theory I have arrived at. Humans are successful animals because we have developed large brains with the capability of identifying and matching patterns of many varieties. These patterns could be called concepts, or theories, or hypotheses, or just rules of thumb. The patterns can range from the highly concrete (“a chair”) to the social (“a family”) to the philosophical (“happiness”).

Evolution equipped us not only with the ability to handle patterns but also a reward mechanism for exercising it, which like other reward systems takes neurochemical form. You will know this as the “Eureka” moment when something suddenly “makes sense” (meaning you found a new pattern it fits into, or an old one, or a modification to an old one), or, less dramatically, the little satisfying mental click when something falls together.

Although it is not directly related to my overall topic here, the human inclination, or one might say compulsion, to believe in God can be tied to this neurotheoretical notion of pattern-finding and pattern-matching (and thus the somewhat unfortunate term “neurotheology”), in the sense that a belief in God again activates our reward systems for finding patterns, in this case a “pattern” or framework that ties together disparate and otherwise unexplainable things. Of course, this fundamental biological motivation that results in a notion of the supernatural is then embroidered and modulated by any number of social and political factors, and people have written entire books on the subject, but underlying it all is the human thirst to understand (in other words, find patterns), which, again, has a neurobiological basis.

Thirst in the very real sense, because reward is the flip side of desire, be it for water or anything else, and desire merely one stage on the path to addiction. Those neurochemicals sloshing around in our brains making us feel great when we have sex are also capable, by their absence, of making us sad, horny, jittery, or desperate. It is when this deprivation becomes so serious that we seek to satisfy it, even at great cost to ourselves, that we call it addiction.

Evolution made a wise enough choice in developing these mechanisms; they mostly serve us well when it comes to nourishment, or reproduction, or social relationships, or even intellectual pursuits, such as in the urge Einstein felt to discover the pattern which resulted in the theory of relativity. But what purpose is served by the deep emotional unease, pain, or even agony, that we feel when we are unable to satisfy our cravings for finding patterns, especially big patterns, or the biggest pattern of all, which is how all these other patterns fit together–the “meaning of life” (yes, patterns are essentially identical to what we call “meaning”)–even in the case where there is no pattern to be found? (The millions of people who are desperately seeking the big pattern by pursuing whatever form of religion or practice or devotion or philosophy are, at one level, hardly different from the crack addict furiously looking for his next fix. Of course, just as not everyone develops an addition to drugs, not everyone develops an addiction to finding patterns. In both cases, I suppose it is, like most other things, a matter of so-called epigenetics.

Crime, Punishment, and the Singularity

May 28th, 2008

Another interesting social issue related to the is its effect on our penal system especially the millions of folks we’ve got locked up right now. Take a prisoner with a 50-year sentence. If he has to serve his entire sentence he wouldn’t be out until 2050, but by that time we expect inconceivable advances in genetics, nanotechnology, and artificial intelligence/robotics, all of which could have implications for his case. Read the rest of this entry »

The Singularity is Near

May 26th, 2008

Ray Kurzweil’s The Singularity is Near ( ) is one of those books that really changes the way you look at things.

For instance, when the Supreme Court rules on virtual child porn as it did recently ( ), you see that the real issue goes far beyond Photoshopping some kidpix. Read the rest of this entry »

Sakiko's new blog

May 22nd, 2008

Sakiko has started a blog at . Expect lots of cat pictures.

Nearman’s new translation of Shobogenzo

May 21st, 2008

Rev. Hubert Nearman, O.B.C. has put out an ambitious new translation of Shobogenzo, a 14-year labor of love, now available from the Shasta Abbey website.

Getting guidance for your life from the web

May 21st, 2008

A recent college graduate I know recently found himself most unhappy in his new job. But was there something really wrong with the company he had had such high hopes for, or was it merely a case of the freshman blues? Where to turn for advice? Friends? Parents? Professors?

Read the rest of this entry »