Skip to main content

How Silos Grow

In my previous post, I suggested several topics to blog about, based on my experiences of 2017. How Silos Grow was the topic most asked for:
This was my initial shock after joining the company. Only a few years after taking off as a startup, the hedges began growing, seemingly almost by themselves, and against the will of the founders.
I've worked in silos, and in companies without them, but I haven't been there to witness them coming into existence before. It is a fascinating phenomenon, yet oh, so common. And a killer of great companies.
This posts digs into why and how silos happen. It's a bit of a long one, so here's the TL;DR:
  • It is about human nature.
  • Departments are a good thing (for some things).
  • People in different departments are inherently different.
  • The project method is still problematic (and reinforces the silos).
  • Cross-departmental teams should be the first-class organizational unit instead.
Although the idea to write this came from my first year at eyeo, this piece is about the sum of experiences I made in my entire career. My employer (still eyeo) went through some of the experiences that I write about below, but not all of them - and they are actively improving the weak spot.

Human nature

If I had some more time, I'd love to dig into the human social and biological reasons for this, but I'll just jump ahead and say there is something primal going on since prehistoric days, and in modern times it is still observable as loyalty to closer/similar people, and distrust for those distant/different. We group ourselves and bind with those of similar means and motives. It makes sense on a very basic level, but like many of our cognitive biases, it is counter-productive in a many modern scenarios, where nomad tribes and saber-tooth tigers are not involved.

My colleague at eyeo, Aaron, describes the mentality of it as "Us vs Them". It seems at times that humans just need that common foe to unite a given group, driving the plot for Wag the Dog, war on drugs, terror, and so on. But let us try to dig a bit deeper on why this happens in companies, especially small-medium sized ones that are growing past the 20 person mark.

Strength in departments

At a certain point in any fast-growing startup, the average count of person-to-stranger relationships will grow much faster than the rate at which people get to know each other.

What people naturally do in such environments is to gravitate towards those they have the most in common with, or simply stick with the ones involved in their hiring and perhaps after hiring through some mentorship program. This tends to be with people of the same profession, or related ones, and the social bonds formed there will stay the strongest ones in the new-hire’s work life for some time.
I think this is a good thing! Having this circle of closer peers provides a safe haven for new-hires while they learn to navigate the larger company. Furthermore, it provides a platform for learning, fostering knowledge and craftsmanship between those who share professions.

Obviously, the kind of department I’m describing above is not a silo. It’s a catalyst for growth and development of people in a company.

Burgers, fries and coke

The typical line-up of departments are an odd bunch. Some of them are knowledge groups like I describe above, while others are defined by some function they provide to the rest of the company. Most of them are a mix of both.

Some of the departments are half-function and half-product: Sales, marketing and support functions (IT, legal, accounting, etc) are often not considered within what I call the "hamburger" of product/service delivery:

  • Product (research and design) 
  • Development 
  • QA 
  • Operations
If you want anything from a particular product or service, these are the entities you have to bite through in order to get anything.

(If you know your Agile history, you'll remember that most of the industry managed to interweave the first three, and with the coming of DevOps, the fourth joined in as well.)
The other less interdependent functions, like HR, sales or marketing (let's call them "fries" and "coke") are still left out of product/service in some of the most agile shops around to this very day. And because they are operationally separate, they get defined by separate purposes. Now, that's probably not a good way to split up your business/work, but hey, that’s just how it’s done in a lot of places. My point being: until all departments are part of the burger, you’re going to have some degree of silos.

Departments’ natural differences

As each employee can only be member of a single department, it draws in some odd borders throughout the company, like the programming designer - is she in Development or Product department? Infrastructure engineer is another one. Dev or Ops?

Two colleagues independent of each other suggested us being divided by those who think fast and those who think slow. This made me think of something I wrote in one of our internal methodology guides (the context was balance in the nature of work we do, but I'll come back to that another post, I hope):
QA, operations and developers tend to focus on stability, while marketing and business people, designers and product managers focus on new features.
Of course, those things are results or symptoms of how we have (or have not) defined purpose within the company. Of course the latter groups will focus on novel things that drive interest from media and customers. The former groups know they'll get swamped with technical debt if they go chasing new features for too long.

Another difference I think is gender-culture. The constellation of technical departments (the same one I mentioned tends to focus on stability) tends to consist of men, and they attract (or select) male new-hires, while the remaining ones have a more healthy mix. I don’t know enough about the psychology in play here, but in my experience, “male departments” tend to have rougher communication, somehow colder and very “evidence” oriented. The mixed departments have more empathy, more respectful tone and tolerance for intuition.

Some departments may be elitist due to generally longer education paths or having more experienced members. Some departments may be made up of dominantly extrovert people while others consist of introverts. Some may be distributed while others are co-located.

There are probably more reasons for departments being different from each other than I've listed above. Point is, these differences pose the departments in tension with each other, and may lead to avoidance, lack of trust or aggression, furthering silofication.

When do departments turn to silos

The very name-sake of the DevOps movement is the constellation of the Developers versus the Operations department. Already during the Agile revolution, we realized that velocity will hit a certain ceiling less we include the customer (often represented by product owners), QA and Operations in the development teams. Yet, we continue to see growing companies split the work by departments once they reach a certain size.

(Side-note: I believe the wiser companies break out sub- or separate companies before this happens, like Fog-Creek splitting out Trello, or 37signals splitting out everything but Basecamp, or going further back: Semco’s satellite companies.)

When I joined eyeo a year ago, around the 80 person mark, I saw the manifestations of these departments, but the divisions had already begun years before. I'm re-telling a simplified version here, but some time during those early years:
  • Business developers were hired to drive sales and partnerships. 
  • IT administrators were hired to make sure everyone had the equipment they needed. 
  • Infrastructure engineers were hired to take care of the infrastructure. 
  • There was a steady hiring of software developers to develop and maintain the products. 
  • A QA manager and testers were hired to take care of processes and well, QA obviously. 
  • A product manager was hired to build an awesome "product team". 
Maybe you already suspect the culprit here: Already at the hiring stage, new people are brought into the company with role-specific purposes, rather than product-specific ones. This does not alone explain why these new-hires organize themselves by their departments instead of the products/services they work on. Obviously, they’re still supposed to join up with people from different departments on demand, kick-off projects together, drive them to completion, then rinse and repeat.

Now, what happens during the early years of a successful startup, is that it goes past that magic number of employees, somewhere between 20 and 40 perhaps, and things just get difficult:
  • Number of products goes up (and platforms per product goes up) 
  • Number of projects goes up (new product launches, new features in existing products, re-designs of existing services) 
  • Number of services and users on these go up 
  • More money, more media-attention, more responsibility to customers 
As the stakes grow ever larger, people want to perform their best, and it seems intuitive for them to focus in on their own specialty and let others practice theirs. The work gravitates towards being department-oriented, because people have defined their purpose that way.

Along with this, the lines of communication between individuals grow factorially. The essence being that any QA-person can do QA for any project, and the same for every other department for every project. Of course, everyone realizes person-project relationships are more sticky than this, but in a department-oriented company, the person-project relationship is a second-class citizen.

Projects and Departments

The cross-department project methodology works most of the time. Up to a certain size. Then you start getting the occasional misunderstanding: "I thought you were on that project. Who is? Oh, I don't know that person. Can I trust them with getting that right?"

Then comes the differences in priorities: project A is more important than project B now, because project A is in the [department x] phase.

Along with it come the hand-offs: [department x] cannot complete their work in project A before [department y] has finished their part. Lead-time goes down. Quality goes down. Rework goes up. Now you automatically get the blame-games.
And at this point, we’ve pretty much re-invented waterfall. The silos are all around, and you’ll notice that project completion takes forever. The rare successes are overly celebrated by the managers that were in charge of them. Things just don’t get done any more. Toxic relationships develop across departments as tension grows.

How not to fix it

Companies that see projects slowing down into waterfall processes between blame-gaming departments may attempt to combat this in various ways.

They may try putting systems in place to drive improvement. The danger here is that they will look for organizational units where they can latch on goal-processes, incentives, processes or responsibilities (think contractual obligations between departments) and hierarchies. But these measures will only strengthen the silofication and possibly lead to a culture of faking success (known from pretty much any large corporation out there), more blaming, less trust and even slower delivery.

They may introduce an entity responsible for making the problem go away: Cross-departmental project managers, agile or DevOps coaches. This is probably a step in the right direction, but it will only do so much - at least the people in this unit should be calling for a new structure, where purpose can be placed where it should be.

How to start fixing it

This is probably where I should make a cut for a future blog post, but here’s the short version:
  • Re-purpose the departments to be what they are best at: knowledge centrals/catalysts, and home-ground for new-hires. 
  • Replace temporary projects with sustainable, product/service-oriented, fixed teams. Make these teams the first-class citizens of the organization. 
  • Redefine people’s purpose in the organization: Ask them to commit to the success of their team, not the success of their department. 
  • Long term (maybe): Spin off autonomous team-clusters into separate companies. 
Department-oriented is a very old-fashioned way to work these days: We should have teams, each of them with a clear purpose focusing on the end-needs of the business or its customers.

This means business-oriented, but also cross-functional and autonomous teams (because anything else will slow stuff down). The natural departmental differences will still occur, but already within the team on a daily basis, where they can be much better handled than between separate departments at irregular, painful intervals. In the sum of a cross-department team's actions we get a healthy balance of maintenance and innovation with shared purpose.

Comments

Popular posts from this blog

Open source CMS evaluations

I have now seen three more or less serious open source CMS reviews. First guy to hit the field was Matt Raible ( 1 2 3 4 ), ending up with Drupal , Joomla , Magnolia , OpenCms and MeshCMS being runner-ups. Then there is OpenAdvantage that tries out a handful ( Drupal , Exponent CMS , Lenya , Mambo , and Silva ), including Plone which they use for their own site (funny/annoying that the entire site has no RSS-feeds, nor is it possible to comment on the articles), following Matt's approach by exluding many CMS that seem not to fit the criteria. It is somewhat strange that OpenAdvantage cuts away Magnolia because it "Requires J2EE server; difficult to install and configure; more of a framework than CMS", and proceed to include Apache Lenya in the full evaluation. Magnolia does not require a J2EE server. It runs on Tomcat just like Lenya does (maybe it's an idea to bundle Magnolia with Jetty to make it seem more lightweight). I'm still sure that OpenAdvant

Git Stash Blooper (Could not restore untracked files from stash)

The other day I accidentally did a git stash -a , which means it stashes *everything*, including ignored output files (target, build, classes, etc). Ooooops.. What I meant to do was git stash -u , meaning stash modifications plus untracked new files. Anyhows, I ended up with a big fat stash I couldn't get back out. Each time I tried, I got something like this: .../target/temp/dozer.jar already exists, no checkout .../target/temp/core.jar already exists, no checkout .../target/temp/joda-time.jar already exists, no checkout .../target/foo.war already exists, no checkout Could not restore untracked files from stash No matter how I tried checking out different revisions (like the one where I actually made the stash), or using --force, I got the same error. Now these were one of those "keep cool for a second, there's a git way to fix this"situation. I figured: A stash is basically a commit. If we look at my recent commits using   git log --graph --

Leaving eyeo

Thirteen blog posts later, this one notes my departure from eyeo after 4 years and 3 months. I joined eyeo around the headcount of 80 employees, and now I think there's just over 250 people there. My role coming in was as operations manager, doing a mix of infrastructure engineering and technical project management. I later on took on organizational development to help the company deal with its growing pains . We introduced cross-functional teams, departments (kind of like guilds), new leadership structures, goal-setting frameworks, onboarding processes and career frameworks.  And all of this in a rapidly growing distributed company. I'm proud and happy that for a long time I knew every employee by name and got to meet every single new-hire through training them on company structure and processes.  At some point, we had enough experienced leaders and organizational developers that I could zoom back in on working in one team, consulting them on  Git and continuous integration

Managing dot-files with vcsh and myrepos

Say I want to get my dot-files out on a new computer. Here's what I do: # install vcsh & myrepos via apt/brew/etc vcsh clone https://github.com/tfnico/config-mr.git mr mr update Done! All dot-files are ready to use and in place. No deploy command, no linking up symlinks to the files . No checking/out in my entire home directory as a Git repository. Yet, all my dot-files are neatly kept in fine-grained repositories, and any changes I make are immediately ready to be committed: config-atom.git     -> ~/.atom/* config-mr.git     -> ~/.mrconfig     -> ~/.config/mr/* config-tmuxinator.git       -> ~/.tmuxinator/* config-vim.git     -> ~/.vimrc     -> ~/.vim/* config-bin.git        -> ~/bin/* config-git.git               -> ~/.gitconfig config-tmux.git       -> ~/.tmux.conf     config-zsh.git     -> ~/.zshrc How can this be? The key here is to use vcsh to keep track of your dot-files, and its partner myrepos/mr for o

Git tools for keeping patches on top of moving upstreams

At work, we maintain patches for some pretty large open source repositories that regularly release new versions, forcing us to update our patches to match. So far, we've been using basic Git operations to transplant our modifications from one major version of the upstream to the next. Every time we make such a transplant, we simply squash together the modifications we made in the previous version, and land it as one big commit into the next version. Those who are used to very stringent keeping of Git history may wrinkle their nose at this, but it is a pragmatic choice. Maintaining modifications on top of the rapidly changing upstream is a lot of work, and so far we haven't had the opportunity to figure out a more clever way to do it. Nor have we really suffered any consequences of not having an easy to read history of our modifications - it's a relatively small amount of patches, after all. With a recent boost in team size, we may have that opportunity. Also the need for be