Why SRE and Netflix saved my Saturday!

System Reliability Engineers, I’ve placed a few in my time and this is really becoming a “thing”, and I’m enjoying it being there. SRE’s, blending DevOps with System Administration and elements of development, seems to be something anyone who has high availability systems need to have in their teams, usually working as small teams they are a crack team evolving the way we (as consumers) are able to enjoy services all day, every day.

The first time I came across the term SRE was when I was asked to help build up a team in Berlin for a US based Games giant. Taking on the spec (with little thought for what they actually do), was a really interesting jaunt into a new world of everything being available to everyone all the time and the complexities of the work behind the scenes that makes this happen.

As a recruiter, I’m blessed to speak with thousands of candidates and clients very year, understanding their business models, challenges and movements. Other than using the web for online purchases, you rarely see the work the DevOps and SRE’s first hand, but they are the unsung heroes behind every important thing we do on the internet.

At a meeting last week I was introduced to the Chaos “Apes” Netfilx had released into the wild, using true DevOps principles to test, re test and try and break every network they had to see how the system would cope with redundancies from the server, availability centre and region (check out the Netflix blog for more info).

Recently, on a Saturday evening, I found out the impact of what Netflix had done. Having set myself up for an evening of online gaming, I was horrified to find out I wasn’t able to access any of my online services through my console, gaming was off and streaming was off, in fact anything fun was off! I was staring down the barrel of watching non HD terrestrial TV! Netflix however was still online and functioning when everything else had failed.

After being introduced to the DevOps principles that had been in place, it all made sense as to why this was the only thing working that had saved me from a fate worse than HD!

I have a new found appreciation for what these guys do, all the way back behind the systems I use every day, but appreciate the hard work and innovation behind it all to allow the consumer to have a seamless and constant availability to box sets, games and other entertainment online.