Skip to main content

How Silos Grow

In my previous post, I suggested several topics to blog about, based on my experiences of 2017. How Silos Grow was the topic most asked for:
This was my initial shock after joining the company. Only a few years after taking off as a startup, the hedges began growing, seemingly almost by themselves, and against the will of the founders.
I've worked in silos, and in companies without them, but I haven't been there to witness them coming into existence before. It is a fascinating phenomenon, yet oh, so common. And a killer of great companies.
This posts digs into why and how silos happen. It's a bit of a long one, so here's the TL;DR:
  • It is about human nature.
  • Departments are a good thing (for some things).
  • People in different departments are inherently different.
  • The project method is still problematic (and reinforces the silos).
  • Cross-departmental teams should be the first-class organizational unit instead.
Although the idea to write this came from my first year at eyeo, this piece is about the sum of experiences I made in my entire career. My employer (still eyeo) went through some of the experiences that I write about below, but not all of them - and they are actively improving the weak spot.

Human nature

If I had some more time, I'd love to dig into the human social and biological reasons for this, but I'll just jump ahead and say there is something primal going on since prehistoric days, and in modern times it is still observable as loyalty to closer/similar people, and distrust for those distant/different. We group ourselves and bind with those of similar means and motives. It makes sense on a very basic level, but like many of our cognitive biases, it is counter-productive in a many modern scenarios, where nomad tribes and saber-tooth tigers are not involved.

My colleague at eyeo, Aaron, describes the mentality of it as "Us vs Them". It seems at times that humans just need that common foe to unite a given group, driving the plot for Wag the Dog, war on drugs, terror, and so on. But let us try to dig a bit deeper on why this happens in companies, especially small-medium sized ones that are growing past the 20 person mark.

Strength in departments

At a certain point in any fast-growing startup, the average count of person-to-stranger relationships will grow much faster than the rate at which people get to know each other.

What people naturally do in such environments is to gravitate towards those they have the most in common with, or simply stick with the ones involved in their hiring and perhaps after hiring through some mentorship program. This tends to be with people of the same profession, or related ones, and the social bonds formed there will stay the strongest ones in the new-hire’s work life for some time.
I think this is a good thing! Having this circle of closer peers provides a safe haven for new-hires while they learn to navigate the larger company. Furthermore, it provides a platform for learning, fostering knowledge and craftsmanship between those who share professions.

Obviously, the kind of department I’m describing above is not a silo. It’s a catalyst for growth and development of people in a company.

Burgers, fries and coke

The typical line-up of departments are an odd bunch. Some of them are knowledge groups like I describe above, while others are defined by some function they provide to the rest of the company. Most of them are a mix of both.

Some of the departments are half-function and half-product: Sales, marketing and support functions (IT, legal, accounting, etc) are often not considered within what I call the "hamburger" of product/service delivery:

  • Product (research and design) 
  • Development 
  • QA 
  • Operations
If you want anything from a particular product or service, these are the entities you have to bite through in order to get anything.

(If you know your Agile history, you'll remember that most of the industry managed to interweave the first three, and with the coming of DevOps, the fourth joined in as well.)
The other less interdependent functions, like HR, sales or marketing (let's call them "fries" and "coke") are still left out of product/service in some of the most agile shops around to this very day. And because they are operationally separate, they get defined by separate purposes. Now, that's probably not a good way to split up your business/work, but hey, that’s just how it’s done in a lot of places. My point being: until all departments are part of the burger, you’re going to have some degree of silos.

Departments’ natural differences

As each employee can only be member of a single department, it draws in some odd borders throughout the company, like the programming designer - is she in Development or Product department? Infrastructure engineer is another one. Dev or Ops?

Two colleagues independent of each other suggested us being divided by those who think fast and those who think slow. This made me think of something I wrote in one of our internal methodology guides (the context was balance in the nature of work we do, but I'll come back to that another post, I hope):
QA, operations and developers tend to focus on stability, while marketing and business people, designers and product managers focus on new features.
Of course, those things are results or symptoms of how we have (or have not) defined purpose within the company. Of course the latter groups will focus on novel things that drive interest from media and customers. The former groups know they'll get swamped with technical debt if they go chasing new features for too long.

Another difference I think is gender-culture. The constellation of technical departments (the same one I mentioned tends to focus on stability) tends to consist of men, and they attract (or select) male new-hires, while the remaining ones have a more healthy mix. I don’t know enough about the psychology in play here, but in my experience, “male departments” tend to have rougher communication, somehow colder and very “evidence” oriented. The mixed departments have more empathy, more respectful tone and tolerance for intuition.

Some departments may be elitist due to generally longer education paths or having more experienced members. Some departments may be made up of dominantly extrovert people while others consist of introverts. Some may be distributed while others are co-located.

There are probably more reasons for departments being different from each other than I've listed above. Point is, these differences pose the departments in tension with each other, and may lead to avoidance, lack of trust or aggression, furthering silofication.

When do departments turn to silos

The very name-sake of the DevOps movement is the constellation of the Developers versus the Operations department. Already during the Agile revolution, we realized that velocity will hit a certain ceiling less we include the customer (often represented by product owners), QA and Operations in the development teams. Yet, we continue to see growing companies split the work by departments once they reach a certain size.

(Side-note: I believe the wiser companies break out sub- or separate companies before this happens, like Fog-Creek splitting out Trello, or 37signals splitting out everything but Basecamp, or going further back: Semco’s satellite companies.)

When I joined eyeo a year ago, around the 80 person mark, I saw the manifestations of these departments, but the divisions had already begun years before. I'm re-telling a simplified version here, but some time during those early years:
  • Business developers were hired to drive sales and partnerships. 
  • IT administrators were hired to make sure everyone had the equipment they needed. 
  • Infrastructure engineers were hired to take care of the infrastructure. 
  • There was a steady hiring of software developers to develop and maintain the products. 
  • A QA manager and testers were hired to take care of processes and well, QA obviously. 
  • A product manager was hired to build an awesome "product team". 
Maybe you already suspect the culprit here: Already at the hiring stage, new people are brought into the company with role-specific purposes, rather than product-specific ones. This does not alone explain why these new-hires organize themselves by their departments instead of the products/services they work on. Obviously, they’re still supposed to join up with people from different departments on demand, kick-off projects together, drive them to completion, then rinse and repeat.

Now, what happens during the early years of a successful startup, is that it goes past that magic number of employees, somewhere between 20 and 40 perhaps, and things just get difficult:
  • Number of products goes up (and platforms per product goes up) 
  • Number of projects goes up (new product launches, new features in existing products, re-designs of existing services) 
  • Number of services and users on these go up 
  • More money, more media-attention, more responsibility to customers 
As the stakes grow ever larger, people want to perform their best, and it seems intuitive for them to focus in on their own specialty and let others practice theirs. The work gravitates towards being department-oriented, because people have defined their purpose that way.

Along with this, the lines of communication between individuals grow factorially. The essence being that any QA-person can do QA for any project, and the same for every other department for every project. Of course, everyone realizes person-project relationships are more sticky than this, but in a department-oriented company, the person-project relationship is a second-class citizen.

Projects and Departments

The cross-department project methodology works most of the time. Up to a certain size. Then you start getting the occasional misunderstanding: "I thought you were on that project. Who is? Oh, I don't know that person. Can I trust them with getting that right?"

Then comes the differences in priorities: project A is more important than project B now, because project A is in the [department x] phase.

Along with it come the hand-offs: [department x] cannot complete their work in project A before [department y] has finished their part. Lead-time goes down. Quality goes down. Rework goes up. Now you automatically get the blame-games.
And at this point, we’ve pretty much re-invented waterfall. The silos are all around, and you’ll notice that project completion takes forever. The rare successes are overly celebrated by the managers that were in charge of them. Things just don’t get done any more. Toxic relationships develop across departments as tension grows.

How not to fix it

Companies that see projects slowing down into waterfall processes between blame-gaming departments may attempt to combat this in various ways.

They may try putting systems in place to drive improvement. The danger here is that they will look for organizational units where they can latch on goal-processes, incentives, processes or responsibilities (think contractual obligations between departments) and hierarchies. But these measures will only strengthen the silofication and possibly lead to a culture of faking success (known from pretty much any large corporation out there), more blaming, less trust and even slower delivery.

They may introduce an entity responsible for making the problem go away: Cross-departmental project managers, agile or DevOps coaches. This is probably a step in the right direction, but it will only do so much - at least the people in this unit should be calling for a new structure, where purpose can be placed where it should be.

How to start fixing it

This is probably where I should make a cut for a future blog post, but here’s the short version:
  • Re-purpose the departments to be what they are best at: knowledge centrals/catalysts, and home-ground for new-hires. 
  • Replace temporary projects with sustainable, product/service-oriented, fixed teams. Make these teams the first-class citizens of the organization. 
  • Redefine people’s purpose in the organization: Ask them to commit to the success of their team, not the success of their department. 
  • Long term (maybe): Spin off autonomous team-clusters into separate companies. 
Department-oriented is a very old-fashioned way to work these days: We should have teams, each of them with a clear purpose focusing on the end-needs of the business or its customers.

This means business-oriented, but also cross-functional and autonomous teams (because anything else will slow stuff down). The natural departmental differences will still occur, but already within the team on a daily basis, where they can be much better handled than between separate departments at irregular, painful intervals. In the sum of a cross-department team's actions we get a healthy balance of maintenance and innovation with shared purpose.

Popular posts from this blog

Encrypting and Decrypting with Spring

I was recently working with protecting some sensitive data in a typical Java application with a database underneath. We convert the data on its way out of the application using Spring Security Crypto Utilities. It "was decided" that we'd be doing AES with a key-length of 256, and this just happens to be the kind of encryption Spring crypto does out of the box. Sweet!

The big aber is that whatever JRE is running the application has to be patched with Oracle's JCE in order to do 256 bits. It's a fascinating story, the short version being that U.S. companies are restricted from exporting various encryption algorithms to certain countries, and some countries are restricted from importing them.

Once I had patched my JRE with the JCE, I found it fascinating how straight forward it was to encrypt and decrypt using the Spring Encryptors. So just for fun at the weekend, I threw together a little desktop app that will encrypt and decrypt stuff for the given password and sa…

Managing dot-files with vcsh and myrepos

Say I want to get my dot-files out on a new computer. Here's what I do:

# install vcsh & myrepos via apt/brew/etc
vcsh clone https://github.com/tfnico/config-mr.git mr
mr update

Done! All dot-files are ready to use and in place. No deploy command, no linking up symlinks to the files. No checking/out in my entire home directory as a Git repository. Yet, all my dot-files are neatly kept in fine-grained repositories, and any changes I make are immediately ready to be committed:

config-atom.git
    -> ~/.atom/*

config-mr.git
    -> ~/.mrconfig
    -> ~/.config/mr/*

config-tmuxinator.git  
    -> ~/.tmuxinator/*

config-vim.git
    -> ~/.vimrc
    -> ~/.vim/*

config-bin.git   
    -> ~/bin/*

config-git.git          
    -> ~/.gitconfig

config-tmux.git  
    -> ~/.tmux.conf    

config-zsh.git
    -> ~/.zshrc

How can this be? The key here is to use vcsh to keep track of your dot-files, and its partner myrepos/mr for operating on many repositories at the same time.

I discovere…

Always use git-svn with --prefix

TLDR: I've recently been forced back into using git-svn, and while I was at it, I noticed that git-svn generally behaves a lot better when it is initialized using the --prefix option.

Frankly, I can't see any reason why you would ever want to use git-svn without --prefix. It even added some major simplifications to my old git-svn mirror setup.

Update: Some of the advantages of this solution will disappear in newer versions of Git.

For example, make a standard-layout svn clone:

$ git svn clone -s https://svn.company.com/repos/project-foo/

You'll get this .git/config:

[svn-remote "svn"]
        url = https://svn.company.com/repos/
        fetch = project-foo/trunk:refs/remotes/trunk
        branches = project-foo/branches/*:refs/remotes/*
        tags = project-foo/tags/*:refs/remotes/tags/*

And the remote branches looks like this (git branch -a):
    remotes/trunk
    remotes/feat-bar

(Compared to regular remote branches, they look very odd because there is no remote name i…

Android Voice Commands for Cyclists Listening to Podcasts or Music

Disclaimer: I do not recommend using earphones while on your bike, but there are times or roads where I think it's OK. Pull out your earphones when nearing potentially dangerous situations (like intersections). At least pause the audio.
These tips also apply to anyone unable to look at and touch their device, leaving voice commands their only option (useful for visually impaired people, people wearing thick gloves, etc). First of all, you need an Android with a fairly new version of Google Now installed, like Lollipop. You'll need a headset with a microphone button. I’ve got an iphone headset that works great with my old Moto G, excluding the volume control. You need to make sure that a connected headset can bypass the device’s lock mechanism. It’s in:         Settings -> Language & input -> Google voice typing -> Hands-free Your audio playback software has to work with the Google Now commands. I’ve tried Google Music and BeyondPod successfully. So, off we go! You’…

Microsoft ups their Git efforts another notch

This week Microsoft announced first class Git support embedded in the coming version of Visual Studio.

Now, it's not completely shocking. We could have seen it coming since Microsoft started offering Git repos on CodePlex, and more recently offering a Git client for TFS. In any case, these are some big news. Scott Hanselman weighs on some features and some more background here.

For those who are a bit unaware of what the Git situation on Windows looks like these days, I've dotted down these notes:
Some explanation on these:

msysGit has long been The Way to use Git on Windows. It's basically a port of Git itself, so it's a command-line tool.GitExtensions (includes Visual Studio integration), TortoiseGit, Git Shell, posh-git and most other tools are powered by msysGit.libgit2 is a native library for doing Git stuff. It is developed completely separate from Git itself. The above tools could (and should) probably use libgit2 instead of hooking onto and around msysGit.Github…