actuals.ca
actuals.ca
February 3, 2016
There are many "how-to" methodologies available for successfully completing large-scale software projects. "Agile", "spiral", "cleanroom", "kanban", etc., all have advantages and disadvantages, and are all used by extraordinarily successful and woefully unsuccessful teams throughout the industry. The successful teams reliably repeat their success over and over, and this success is based on several foundational principles. Not on a "method". Not on "how". Why they succeed is more important than the how.
Monetate is a maturing SaaS company, established in 2008. The Monetate platform influences more than a third of total U.S. ecommerce sales for the world's leading retail brands. Our software is fast, operates at scale, and constantly evolves. The engineering challenges are vast, but our teams solve them, one after the other, every day, over the long haul. Of course there have been many lessons learned, many war stories, and many failures. But we know how to lead a large group of engineers on the path to success more often than not. The fundamental reasons why are worth noting.
Who am I to proclaim these principles? I am a software engineer with decades of experience on many platforms, with many languages and environments, and with many talented people and teams. I have dealt with success and with failure, each time learning why success happens. The principles underpinning success are clear and simple. I learned that winning requires focus on a goal and the steadfastness to achieve it. These principles apply to C-level executives, to department managers, and to the front-line developers who are modifying and deploying the code. I worked my way through many circumstances and with many different development systems. And the common element underlying all the successful development I have been a part of has been knowing and discovering the component parts and supporting attitudes which achieve goals, both big and small, coupled with an unrelenting will to reach them.
Near the dawn of the internet era I worked in a traditional office
as a general software developer within a small company whose focus
was insurance, not software. Many critical pieces of
infrastructure only ran under DOS, which was a precursor (and
early foundation) to modern multitasking computing environments. I
began the effort to update our code to run under more modern
systems and spent many late nights learning and coding my way
forward. I knew exactly where I wanted the code to go, and I
pushed to get it there.
Meanwhile, half a dozen other engineers within the company
continued maintaining and tweaking the legacy system, unaware of
the progress being made on my workstation. I began to roll out
newer code, replacing older systems where appropriate. This took
forever. I did not know enough about the old systems to
confidently replace them and I was not adept enough in the newer
environments to successfully launch them reliably. I made progress
through force of will, and eventually succeeded after many steps
backward and reversing course out of many dead ends. I had ignored
to a large degree the other engineers, all of whom had superior
insight into the legacy system and many of whom already had
practical knowledge of the newer environments. In my drive to
complete my project, I failed to understand it was actually our
project. I failed to harness the power of our group.
The development groups within Monetate are called 'teams'. The work of multiple developers working in concert far exceeds the work of a lone hero or heroine. This is why we have teams. Team leaders are part of these groups who are all leaders within their sphere. Department leads are focused on the goals of their functional area (software engineering, sales, marketing, etc.). And executives are focused on the strategic goals of the whole company. From our toolchain on up, Monetate developers leverage each other and each other's output. It is perhaps the greatest advantage we have, therefore the team and the work it accomplishes are the most important aspects of working life at Monetate. A single developer in isolation cannot command every intricate piece of machinery which must be tweaked, recreated or removed. A team will do it better.
At Monetate, individual developers are encouraged to join teams which are focused on their particular area of interest. Those who specialize in deployments and "back end" systems become the infrastructure team. Anyone interested in "front end" work (for example, implementing user-facing web applications) will form teams around projects with large demands for those skills. Those interested in machine learning and related areas also form teams which leverage these skills.
I work with a number of teams within Monetate. Each member of each team is talented. But talent alone guarantees nothing when building quality commercial systems which operate at scale. We communicate, we plan, we execute, and we debrief. And at each step, we are solely focused on the ultimate goal. Our goal is the only thing which matters and attaining it the only effective measurement of success. Anything short of the goal becomes a lesson learned on the path to ultimately making it happen. Harnessing the power of our team gives us an incredible advantage when faced with complex problems, so we exploit it.
After the rollout of our new system at the insurance firm, we
recognized how the knowledge of the group could work in our
favor. We teamed up in pairs, or slightly larger groups, in order
to continue working on the new infrastructure. Additional projects
came down from management and we were excited to implement them on
the new platform which provided vast new capabilities and
speed. We realized how we could solve several problems at once,
and created new offshoots of the system to exploit capabilities we
did not even know we had. It grew in size and complexity. New
features appeared, not all of which mattered to anything but our
curiosity. We dove into new areas of technology and played with
new toys. After many months, it became clear that our group was a
bit aimless and worked toward our own ends rather than toward the
mission of the company. We had talent and experience and a
prolific output, but no goal. Our failure to keep the mission of
the company in mind had allowed us to make progress, but in the
wrong directions. Our effectiveness was therefore not as great as
it otherwise should have been.
What is the goal? At the highest level, the goal is the strategic mission of the entire company. It is usually a grand vision, such as 'become the predominant personalization engine for leading retail brands online'. No single individual will accomplish this goal, and likely no single team will either. So the goal is broken down into component parts (or subgoals) which groups of teams can tackle together. For Monetate engineering, this involves subgoals such as '100% uptime' and 'general availability of feature X in Q3′. Each team becomes focused solely upon their subgoal, knowing that achieving it ultimately moves the company closer to winning the larger strategic mission.
There are many heroic events and many exceptional individuals. But building a track record of success is not simply an accumulation of brave feats by ultra-motivated and talented people. The best way to win, repeatably, is by focusing on the goal (at whatever level is appropriate for your job) and fighting together to achieve it. This sounds obvious and simple. But it is not easy. There are many situations and problems, many specific to software development and many general to any job, which conspire to make us fail. A focus on the goal, coupled with the perseverence to get there, provides the foundation for getting it done regardless. Teams are not always a collection of all-stars. In the software world, a development team is likely to be comprised of average engineers with varying sets of skills, experience, talent and speed. A shared focus on the goal creates effective teams built from the average and exceptional alike.
The goal is everything. There may be other mile posts, or supporting tasks or subgoals, which support the team's overall mission. Individual developers on the front line who are writing the code will prioritize and execute based upon these smaller goals. But we always move forward, together, with the ultimate goal in sight. Getting to the next subgoal means we are all moving closer to the big goal. And that is what matters.
In the middle of my career I changed roles and jobs, and I found
myself writing low level library code for a popular open source
software project. Years of practical programming experience with
databases in the general business world prepared me to tackle a
new problem: how could our project provide general database access
to its users? Can we create a common interface which could talk to
the most popular database engines and then expose that data to the
rest of the project in a usable way? The answer was yes, and I
spent a solid year working as part of a team with this specific
goal in mind. The goal was achieved. Our team knew exactly where
we had to go and had mapped out a specific series of steps to get
there.
After my departure from the project another decision was made to
alter the manner in which all parts of the project communicated
with each other (known as MVC design, which means
"model-view-controller"). This required gutting major portions of
the code we worked on for the database drivers. This was a
collosal effort. But the teams now working on the project knew the
exact sequence of steps necessary to upgrade their entire library
to this new and better paradigm. With this roadmap, the new goal
was achieved with the next major release. The programming teams
knew the best way to achieve their goal was to draw up and execute
a clear plan.
Today at Monetate our teams show up every day to take the next steps necessary to get us to our goals. We've learned that we cannot do this blindly, with everyone separately pumping out code and scripts. Leveraging each other's knowledge and talents lets us move further and faster, therefore we communicate constantly. We over-communicate when necessary. We plan. The path to the goal is not always straight or easy or even completely visible. It is a series of steps. Getting to the next step is planned for, so we all know what the steps are and can take them in concert. Unless everyone knows the goal, and knows the plan to get there, no real progress can be made.
A plan might materialize on the back of a napkin. Or it might be carefully laid out in project tracking software as a set of detailed stories or tasks. Or something in between. At Monetate, teams use the most effective way to plan for our level of difficulty and scope of work. But regardless of the form it takes, we plan. Every time. We are steadfast in our method of strategically outlining the steps neccessary to move us forward.
Why? Having a plan to tackle each problem is the most effective way to get us to the goal. It avoids duplicate work. It coordinates the talents of different developers who will mutually amplify the work of the other. It communicates intent and progress to other teams who are themselves embroiled in their own plans for their specific area of concern. It guides us on our effort to make progress on any task. In order for the team to magnify the output of each of its members, a plan is necessary. Since our only concern is making steadfast progress to our goal, we plan.
Fast forward to the current century, and I was now smack in the
middle of a new position with a huge financial services
firm. Software was at the heart of this company, so they had teams
and goals and plans already in place. I was placed with the team
responsible for industry classifications, tasked with tagging
millions of database records and displaying them on screen in an
organized manner. All of the software was low-level, all the
databases large scale and all of the internal systems were custom
built. Developers at this firm had to have their wits about them
in order to make any progress at all.
My team was small, and our task large. We immediately reached out
for help from other teams, attempting to leverage their tools and
knowledge in ways which filled the gaps we knew we had. Our work
proceeded apace, but we stumbled in several critical areas. We
continued to reach out and meetings were scheduled. As usually
happens, goals shifted, and plans were altered, but we continued
our work. The feedback cycle from other developers and teams was
missing. Knowledge sharing was absent, replaced by a set of formal
classes meant to instruct new developers upon arrival. Set adrift
in a sea of tools and chunks of code, we were destined to crash on
the rocks without being able to leverage the experience of the
other teams. Knowing exactly why we had to develop in this
situation hurt our progress even more. Without the motivation of
knowing why, our success was jeopardized.
Without understanding the goal, teams cannot move toward it. In order to understand, teams must know why they are undertaking the task. Many organizations go to great lengths to keep everyone informed and this is good. Monetate specifically lays out the fundamental reasons why a particular project is started. During most of the developer meetings for planning or debriefing we typically open the discussion with the same question: does everyone know why we are here? And if anyone around the room is unclear we first take a step back to review.
We communicate to avoid wasting energy and time and duplicate work. We communicate in order to harness our collective powers properly. Good teams plan, communicate, simplify, and execute. But effective teams — those teams which reach their intended goals — also review their work in some sort of debriefing session at each milestone. Debriefing reveals solutions to problems already encountered, and allows teams to solve them going forward. It prevents repeating mistakes and removes obstacles or revises plans in the face of direct experience. Setting aside time to contemplate the effectiveness of achieving the last goal improves the chances of achieving the next goal.
Completing a goal, as planned and assessed beforehand, provides reliable benchmarks for other teams to base their own plans upon. This in turn feeds on to yet more teams, even in far flung reaches of the company. Eventually, it comes full circle back to us as we embark on our next planning session. Within any team each individual acts as their own unit or building block for the plan. Teams act as the sum of their members, who plan and adjust and estimate among themselves, just as within the company all the teams plan/adjust/estimate between teams. In this way, the overall focus of the company spirals down to every employee, and also spirals up to the echelons of senior management.
Monetate handles web traffic at enormous scales. Hundreds of
millions of transactions occur daily, and when the holiday
shopping season begins late in the year those numbers only go up,
usually quite a bit. As a general platform Monetate has the
capability to perform an unreasonable amount of operations and
complex decisioning on each web transaction. Working on a team
responsible for generating image content, I learned quickly how
much impact my code has on the world of ecommerce. It was
sobering. Simplifying the code, along with the interactions it had
with other sections of the system, became more than an important
principle. It became the only way to effectively operate at
scale. Complexity would cripple not only our ability to understand
the very code we are trying to deploy but could bring down entire
web applications. If we cannot achieve our goals with simplicity
we cannot achieve our goals, period.
It is crucial for teams tackling complex problems to simplify.
Early in my tenure at Monetate I was part of a team which worked
on software deployments. We wrote code which deployed our
applications to the outside world and since we have a variety of
systems and requirements, this was a tricky task. But the goal was
clear, and we had a plan in place to stepwise tackle it. Since
there were so many layers of systems involved we purposely
attempted to keep our deployments as simple as possible. I created
an initial design which utilized a standard file format and
allowed us to quickly and cleanly create the deployments we
needed. I took the notion of "keep it simple" to heart and thought
my approach was dead simple, and highly effective.
Almost immediately my team lead pointed out that my approach was
far from simple. Although a standard file format appeared to
enforce a simple approach to solving our deployment problem, what
part of the system would be responsible for interpreting this
file? Another program would have to be created and maintained to
do this. And in the face of a constantly shifting technical
landscape we now had two areas to update: the standard file format
along with the program which interpretted it. This is the opposite
of our goal of simplicity.
I had worked hard on my design and started butting
heads. Technical arguments became heated, and progress toward our
goal became interrupted and partially side-tracked while the issue
was debated. I allowed my ego to dictate the quality of my
participation with the team. My goal became winning the argument,
and not the successful deployment of our application code. Once I
realized this, once I could see that the only thing which mattered
was creating a simple code deployment tool (our goal), I
reconsidered the entire argument based on the merits of a new and
better approach (which was not mine). Without being blinded by
ego, it was clear how the team should proceed and we went on to
complete our project together.
Once a plan is in place, it is executed it as a team. No one can take command over every detail, every line of code, and every exception handling routine or corner case. Developers are trusted to implement the right solutions since that is what they are hired to do. They are trusted because they know the plan. They also know how the plan achieves the interim goal and by extension the ultimate goal. If you cannot trust your developers to execute, you've hired the wrong developers.
Since we are in constant communication, Monetate developers are always updating, revising and whittling the plan so that it more closely hews toward the goal. And at every step, ego gets in the way. Humlity in the face of a better idea, a better solution, a better technical implementation or a better plan is the only effective way to stay on track and make the most progress possible in the time we have. Every developer wants to contribute, and pride of ownership is important. There must be a balance between developer's self-actualization and the team's goal. Every pet solution or fantastic technical achievement or interesting sideline does not always move the team forward. Realizing that — and actually changing course when appropriate — is a requirement to succeed and is the most effective way to work together. When teams work together, they more effectively tackle the plan.
At Monetate, this attitude has grown into a weekly ceremony called "Thanks". It emerges organically, resting on a real foundation of respect and courtesy. During larger department meetings, teams and team members make specific mention of others who lent a hand, or an ear, or a method, to help them attain part of their interim goal. Recognizing this and giving genuine thanks in a public forum makes us a more cohesive unit, builds trust, and increases our chance of success going forward.
Humility has a broad and deep impact within teams, and among teams. When everyone is treated with respect and everyone is open to better ideas — even when those ideas invalidate or sideline any single individual's contribution — the stage is set for making real progress. Execution — the careful and studied implementation of computer code, along with its deployment — is the materialization of the plan and is the realization of the goal. My code is not important. The team's goal is important. Agility and adaptibility are necessary in order to win. Ego gets in the way.
Being flexible and agile in response to a changing landscape leads to successful execution only when teams know the steps they must take in a certain direction (the plan) and for what specific reasons (the goal, and why it is important). This is the foundation. How this is realized – kanban, waterfall, agile, or any other useful method of organizing a team's activities — is not the critical consideration (though it can amplify efforts). Having a vision of the ultimate goal, communicating that vision so that everyone understands it, trusting each team to deliver, and carefully planning each step to get there will set the stage. A singular focus on the goal and an unrelenting effort to get there is why software teams succeed repeatedly over the long term.
Though developers have a singular focus — toward completing the goal — the path has multiple beneficial offshoots. User groups, online communities, blog posts, talks and seminars — all of these things lend tangential support to the mission. Where possible and beneficial, those offshoots are fully supported by Monetate and other companies seeking to broaden mindshare and to train and develop their teams.
Attaining the goal is why teams are assembled, and achieving it requires attention, diligence and talent. Development discipline harnesses the power of the team, frees individuals to learn and grow according to their whim, and ensures success. Monetate fosters educational training and informal "brown bag" discussions on a wide variety of technical topics which directly impact our progress and likelihood of success. Every initiative of leadership has collaboration, communication, and simplification in mind at all times since these are the foundations of winning teams.
We do not develop in a "clean room". Existing code is often brittle. Current architectures dictate viable options. Real life development has all sort of twists and turns so teams train and educate in order to be better prepared for them. Developers have the job no one wants, with direct control over the front-line code which is in the hands of users. Responsibility is mandatory. Training sharpens our knives. Training opens our mind to other creative solutions and pathways toward attaining our goal.
Repeating exceptional performances becomes a habit. Not just individually, but collectively. It becomes the culture. It is your culture. There is no need to preach about performance standards. There is no need to rely on motivational sayings. Teams simply perform, at the highest level possible, and they pull each other along so that they can effectively amplify each other's talents and achieve the next goal. Monetate is a great place to get things accomplished, and by extension it is a place which fosters collaboration, learning, mutual support, etc. — but it does so all in service of reaching the goal. Monetate is an effective SaaS company, and if you are a developer who is motivated by being involved with that, it is a great place to be.
To an outside observer, software projects may seem to be successful because of "agile" development practices. Or to the magnanimous affect of a new CEO or of a few superstar developers. But the important bit is not the 'how' but the 'why': software teams win because they are focused on the goal, and they do everything — correctly and with discipline — to achieve it. None of it materializes by happy accidents of time or place or circumstance. And certainly not by superstars alone, who come and go. With the team of professionals you have in place, you win on purpose.
Special thanks to Shaun Gallagher, Ashley Sheppard and Kate Barrett for feedback and suggestions.
Further reading on software engineering, design, teams and succeeding: