What is DevOps?
The DevOps methodology, a software and team management approach defined by the portmanteau of Development and Operations, was first coined in 2009 and has since become a buzzword concept in the IT field.
DevOps has come to mean many things to each individual who uses the term as DevOps is not a singularly defined standard, software, or process but more of a culture. Gartner defines DevOps as:
“DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture), and seeks to improve collaboration between operations and development teams. DevOps implementations utilize technology—especially automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective.”
As you can see from the above definition, DevOps is a multi-faceted approach to the Software Development Life Cycle (SDLC), but its main underlying strength is how it leverages technology and software to streamline this process. So with the right approach to DevOps, notably adopting its philosophies of co-operation and implementing the right tools, your business can increase deployment frequency by a factor of 30 and lead times by a factor of 8000 over traditional methods, according to a CapGemini survey.
The Right Tools for the Job
This list is designed to be as comprehensive as possible. The article comprises both very well established tools for those who are new to the DevOps methodology and those tools that are more recent releases to the market—either way there is bound to be a tool on here that can be an asset for you and your business. For those who already live and breathe DevOps, we hope you find something that will assist you in your growing enterprise.
With such a litany of tools to choose from, there is no “right” answer to what tools you should adopt. No single tool will cover all your needs, and will be deployed across a variety of development and Operational teams, so let’s break down what you need to consider before choosing what tool might work for you.
- Plan and collaborate: Before you even begin the SDLC, your business needs to have a cohesive idea of what tools they’ll need to implement across your teams. There are even DevOps tools that can assist you with this first crucial step.
- Build: Here you want tools that create identically provisioned environments. The last you need is to hear “But it works for me on my computer”
- Automation: This has quickly become a given in DevOps, but automation will always drastically increase production over manual methods.
- Continuous Integration: Tools need to provide constant and immediate feedback, several times a day but not all integrations are implemented equally, will the tool you select be right for the job?
- Deployment: Deployments need to be kept predictable, smooth and reliable with minimal risks, automation will also play a big part in this process.
With all that in mind, I hope this selection of tools will aid you as your business continues to expand into the DevOps lifestyle.
Tools Categories List:
Infrastructure As Code
AWS CloudFormation is an absolute must if you are currently working, or planning to work, in the AWS Cloud. CloudFormation allows you to model your AWS infrastructure and provision all your AWS resources swiftly and easily. All of this is done within a JSON or YAML template file and the service comes with a variety of automation features ensuring your deployments will be predictable, reliable, and manageable.
Azure Resource Manager (ARM) is Microsoft’s answer to an all encompassing IAC tool. With its ARM templates, described within JSON files, Azure Resource Manager will provision your infrastructure, handle dependencies, and declare multiple resources via a single template.
Much like the tools mentioned above, Google Cloud Deployment Manager is Google’s IAC tool for the Google Cloud Platform. This tool utilizes YAML for its config files, and JINJA2 or PYTHON for its templates. Some of its notable features are synchronistic deployment and ‘preview’, allowing you an overhead view of changes before they are committed.
Terraform is brought to you by HashiCorp, the makers of Vault and Nomad. Terraform is vastly different from the above mentioned tools in that it is not restricted to a specific cloud environment, this comes with increased benefits for tackling complex distributed applications without being tied to a single platform. And much like Google Cloud Deployment Manager, Terraform also has a preview feature.
Chef is an ideal choice for those who favor CI/CD. At its heart, Chef utilizes self-described recipes, templates, and cookbooks; a collection of ready-made templates. Cookbooks allow for consistent configuration even as your infrastructure rapidly scales. All of this is wrapped up in a beautiful Ruby-based DSL pie.
Ansible is your friend when it comes to automating repetitive IT tasks such as configuration management, application deployment, and intra-service orchestration. With no additional custom security infrastructure and no agents, Ansible is easy to deploy and runs on YAML, allowing you to describe your automation in a way which approaches basic English text.
Puppet may be the oldest IAC tool on this list, and with that comes a lot of experience and maturity in its field, and a bustling community behind it. What sets Puppet apart is its approach to IAC setup and automation, as with Puppet, you’ll need to define a declarative state and Puppet will figure out how best to achieve said state.
Continuous Integration and Delivery
CircleCI is software tool for comprehensive deployment processes that provides a state of the art platform for integration and delivery to release their code around the globe through building and testing automation.
One of the first Continuous Delivery-as-a-Service platforms, Harness helps deployment teams automate its entire continuous delivery process and provide security when deployments fail.
With a simple UI/UX, Buddy is an intelligent CI/CD tool that greatly lowers the threshold of entry for DevOps.
Probot provides bot framework for app creation that is optimized by GitHub. Probot apps are easy to write, deploy, and share.
12. AWS Opsworks
AWS Opworks is for those who utilize Chef Automate and Puppet Enterprise in AWS. With AWS Opworks, you can easily automate how your servers are deployed, configured, and managed.
Event-driven architectures are certainly not a new idea, but Relay was designed specifically with DevOps in mind. Relay has a staggering amount of integrations and workflows to use out of the box, and will be vital in automating low-value tasks so you can focus on what is more important for your team.
Here is a comprehensive guide on everything CA Automic offers when it comes to your plan on automating workloads.
Provides a real-time, block-level replication, which ensures better data integrity for sub-second Recovery Point Objective for databases and all applications. The replication algorithm does not affect system performance and does not even require a system reboot.
Database and Big Data
MySQL is a relatively easy to use database and is used to store large amounts of information that is a stable, reliable, and powerful solution with advanced features. It has been used by industry titans like Facebook, Paypal, and Google.
MariaDB is an open-source database server which is created by the developers of MySQL. Some of its notable users are Wikipedia, WordPress.com and Google. It is a good choice for a fast, scalable, and robust server.
Liquibase is an open-source tool for databases which handles changes and deployment management. It also helps teams track database versioning, database schema deployment, and logical changes.
Datadog lets you collect all the unused metadata that makes your programs slow such as slow database queries, thrown exceptions, unmanaged error logs and cache misses, and growing upstream services. With Datadog, all these events, service states, and metrics are collected in one place and a handy visual graphic representation is created.
DigitalOcean is a rapidly growing cloud hosting provider. In seconds, it can deploy a Linux based Virtual Machine—known as a ‘Droplet’—for you. Digital Ocean boasts high reliability with an uptime percentage of 99.99% and average load times of 219 ms.
Brought to you by Uber Engineering, M3 houses large scale metrics and has been in use by Uber for several years after they realized that existing open source ideas did not meet their criteria. M3 was designed from the ground up to maximize every aspect of the metrics pipeline whilst minimizing its hardware footprint.
Looker, now part of Google Cloud, is a highly adaptable Business Intelligence and data analytics platform that seamlessly integrates with Redshift, Snowflake, BigQuery, and over 50 SQL dialects. With Looker, you’ll be granted unprecedented insight into your data.
23. Apache Hadoop
Hadoop is designed from the ground up to be easily scalable, its framework allows large data sets to be distributed across a single server or thousands of computers. It also boasts a library designed to implement computation and storage on a local level.
24. HPCC Systems
HPCC Systems leverages two decades of experience in the data industry to bring you its free and open-source end-to-end data lake platform.
Brought to you by Google, BigQuery is the search engine’s answer to bringing serverless, cost-effective, and scalable data warehouses to the masses.
26. Apache Cassandra
Cassandra is the go-to tool when it comes to critical data, with its proven fault tolerance and linear scalability, Cassandra ensures your database will always maintain a high level of scalability and availability.
MongoDB takes a unique approach to storing data by storing it in JSON-like documents, this creates a system that is incredibly flexible, scalable, and dynamic. MongoDB believes this is the ideal way to store data, have a look for yourself, and see if you agree.
With Qlik’s QlikSense and QlikView, your raw data will be highly actionable, Qlik utilizes its own end-to-end approach to data integration and analytics to maximize turning your data into insights in which your business can grow from.
Sisense is a driving force behind the building and deployment of analytic apps. The Sisense data and analytics platform offers an agile Business Intelligence system aimed at turning simple data into powerful analytic tools.
First coming onto the scene in 2005, Talend was the first open-source commercial software vendor of data integration software and is still a leading contender in its field.
Sensu offers comprehensive multi-cloud monitoring, from a single server to the cloud. Sensu integrates with the platform you’re already using and offer strong security with Sensu SSO providers.
Solarwinds introduces its own server and application monitor, a comprehensive tool with an impressive list of features.
An adaptable HTTP load testing tool that can be used as a command-line utility or a library.
the smart Data Testing solution
QuerySurge’s main appeal is its focus on automating the validation and testing of your data. Some notable features are its cross-platform support, built in integration with most Data Integration solutions, and BI testing.
Krypton is a U2F authenticator mobile app with un-phishable encrypted architecture. All data encryption keys are stored in the local device storage to avoid unwanted external access.
Mysterium is a network of nodes that provides privacy and security to Mysterium end users. This node image is created for embedded system hardware such as Raspberry Pi.
Okta is an identity management service. With Okta, you can manage all employees access in one place, Okta features include: Single Sign-On (SSO), AD and LDAP integration, Multi-Factor Authentication (MFA). All this is brought together in the Okta Integration Network which provides multiple integration options.
A company which offers cloud security services. These services include access security for mobile users from the branch offices. Data protection, governance, compliance, security monitoring, compliance validation and cloud storage security.
39. Smallstep SSH
Smallstep introduces Single Sign-On SSH (SSO SSH), An easy to use system that prevents you from having to gather, ship, and rotate SSH keys every day. All SSH and sudo access can be managed via a single admin panel.
Helpful CLI Tools
A command-line Interface (CLI) that aims to help developers manage Amazon Web Services. Awless syncs transparently to a representational local graph of a cloud resource and their relationship.
Snyl CLI will help you find, and more importantly fix, vulnerabilities in your dependencies—both in an ad hoc network or your CI system.
Daytona assists you by being a more streamlined version of the Vault client CLI with special focus on automating authentication and secrets fetching.
Bitbucket is built for professional teams where the planning of projects, collaboration on code, testing, and deployment can happen in one place with both free to paid subscription options.
Confluence is a software tool perfect for project planning, meeting notes, marketing planning and blog posting. Create an open and accessible workspace for your business—perfect for teams and companies.
Frame takes a unique stance on your customer relations and helps identify the ‘why?’ behind your customer outcomes through its continuous monitoring features.
Git helps programmers/developers to store, transfer, share, and copy commits to destination repositories from source repositories.
Jira is a software development tool which helps developers in capturing, assigning and setting priorities in an intended task. It allows the developer to manage the whole system development process making sure that every task is completed.
EditorConfig helps developers who are working in large groups having different text editors or IDE in maintaining consistent coding styles.
Tilix is a terminal emulator for Linux that follows gnome human interface guidelines. It has the ability to create multiple panes that can be synchronized.
Jsonnet is an easy to understand extension of JSON. The tool introduces a variety of features such as duplication elimination, integration with custom/existing applications, and can generate other formats including JSON, INI, and YAML among others.
Hazelcast provides an in-memory caching solution that delivers innovative, low-latency, data-centric applications. It can accommodate real-time processing of any application with parallel processing multi-threaded architecture.
Cloudcraft is a visual designer that helps developers to create architecture diagrams in minutes with smart components optimized for AWS. The tool eliminates the need to develop manual static documentation and specs.
With support for SQL and NoSQL databases, DbSchema offers a great and easy to use visual database designer for you to manage your schema.
Caylent provides a critical DevOps-as-a-Service function to high growth companies looking for expert support with Kubernetes, cloud security, cloud infrastructure, and CI/CD pipelines. Our managed and consulting services are a more cost-effective option than hiring in-house, and we scale as your team and company grow. Check out some of the use cases, learn how we work with clients, and read more about our DevOps-as-a-Service offering.
Capgemini.com. 2020. [online] Available at: <https://www.capgemini.com/de-de/wp-content/uploads/sites/5/2016/03/devops-the-future-of-application-lifecycle-automation.pdf> [Accessed 16 July 2020].
Gartner. 2020. Devops. [online] Available at: <https://www.gartner.com/en/information-technology/glossary/devops> [Accessed 16 July 2020].