Dev Ops

What is devops?

Toss over the fence mentality

What does devops mean to me?

• Repeatable Infrastructure
  • Any box, at any time, should be easily rebuildable
• Versioned configs
  • Configs shouldn’t be manually edited on a box
• Unit tests
  • Confidence in code being deployed
• Automated Deploys
  • Ex: Capistrano

Why?

• Consistency
  • Confidence in infrastructure
  • Confidence that code will work between environments
• Auditable
  • When did this change?
  • Why did it change?
• Repetable

The Old Days

• "Put this Code Live, here’s a tarball"
• "Run these queries"
• "What rollback?"
• Security?
• High Availability?
• Scalability?
• "Worked on my box"

• Measure Everything
• No manual changes
• Test everything
• Version everything
• Automate everything

Code Wise

More day to day documentation in code repo

• What server(s) is it on?
• What database does it talk to?

Automating Deployments

• Lower learning curve
  • Each app launches the same way, no matter the language or requirements
  • Server lists are committed in configs, so always up to date
• Consistency
  • One persons launch will be identical to another persons launch

How is this accomplished?

Continuous Integration Builds

• Build packaged up code the same way every time.
• Replayable (can build older versions)

Always Deploy?

• Just because you are confident you can deploy, shouldn’t mean you should do so constantly.
• No longer scary to deploy
• Small changes are less scary than monolitic changes

Story Time

• Xbox Live
• Capistrano with alex

Question

If you needed to find out how valdes was doing how would you?

• No Ticket, No problem?
• Ping it?
• SSH in, use uptime/top?
• Check nginx access/error logs?

Questions?

What we are missing

Staging and Prod are not super different, but are different.

• Go isn’t load balanced in staging
• Hard to rebalance apps (bookmarks are evil)

Ideal Infrastructure

• Monitored (up/down)
  • Sensu? Nagios
• Monitored (Statistics)
  • Statsd? Graphite?
  • # of logins? # of uploads? Etc
• Centralized Logging
  • Less of a need to go “what server is this on?”
  • Fluentd?

Lofty Ideal Infrastructure

• Scalable Cloud design
  • Servers are all identical/pools of servers
  • Each server can run any app.
  • Docker? (Make it heroku like)
• Load balancer we control in front of all apps
  • HAProxy?

Puppet

Monitoring - Sensu / Uchiwa

Monitoring - Sensu / Uchiwa

• Sensu scales well, pushes not pulls
• Can collect metrics for graphite/others

Graphite / Graphana

Graphite / Graphana

• Designed to scale really easily
• Any recorded metric can be graphed
• Libraries for every language (very simple protocol)
• statsd outputs graphite easily
• Lots of math support, eg: holt winters forecasting, time shifting, derivative

ElasticSearch + Fluentd/logstash + Kibana

ElasticSearch + Fluentd/logstash + Kibana

• Centralized Logging
  • No longer care what server is running what app
  • Searchable
  • Filterable
  • Queryable
• New to me - last job was just looking into it as I was leaving

Links / More Info

• http://devopsreactions.tumblr.com/
• http://www.slideshare.net/KrisBuytaert/devops-the-future-is-here-its-not-evenly-distributed-yet
• http://theagileadmin.com/what-is-devops/