Building a DevOps infrastructure for apps and web sites starts with having the right DevOps engineering tools. In this post, we’ll cover the essential software our DevOps engineering teams depend on. Let’s start by putting our heads in the clouds.
Cloud platforms
There are many benefits to leveraging cloud services for your server infrastructure. Most important is the ability to add computing power and storage with just a few clicks, and the flexibility to scale up on-demand as your product grows. Developing in the cloud also means applications aren’t dependent on your on-premise resources or subject to data loss if your internal systems fail. The three leading cloud providers are Amazon’s AWS, Microsoft’s Azure, and Google Cloud. We include all three in our DevOps engineering tool stack, and typically choose which to use based on the project needs and our client’s preference.
Amazon AWS
AWS
is firmly entrenched as the leading cloud provider. And in part because of its size and scale, AWS’s greatest strength may be reliability. It promises 99.99% availability, meaning that over the course of a year, its services would be offline at the most for 52 minutes. AWS is the preferred choice by some of the world’s most recognizable brands: Netflix, Twitch, Coca Cola, Facebook, Nasdaq. Netflix has more than 100,000 server instances on AWS. ArcTouch has been using AWS on a daily basis for more than eight years — including for clients such as Hawaiian Airlines. Our DevOps engineers are certified AWS developers. More of our clients use AWS than the other cloud service providers.
Microsoft Azure
Azure
is typically the most cost-efficient of the big three cloud providers and the second most used option with our customers. We like the Azure Active Directory, a powerful tool to manage users and resources. Active Directory is fully integrated with more than 2,800 SaaS applications. Azure is a great choice for developers building cross-platform applications in Microsoft’s Xamarin (like our Xamarin developers do) — as keeping all development tools under one roof has a variety of benefits. Our DevOps engineers are certified Azure architects.
Google Cloud
We don’t use Google Cloud as often as AWS or Azure, because our clients are often already using Azure or AWS. But Google Cloud is on the rise — in part because of its compelling big data and analytics tools. Google strives to innovate more with data (and has access to more data) than any other company in the world. Google’s Faster Cable System offers an important advantage, providing network infrastructure outside of the public internet for premium users, ensuring premium speeds and reduced latency.
Infrastructure as code (IaC)
Infrastructure as code (IaC) is one of the most important concepts in a DevOps process with continuous delivery and improvement. IAC essentially means managing and provisioning your entire infrastructure so that it’s automated and versioned to create entire environments quickly. Once the infrastructure code is finished, maintenance is minimal, freeing up time for the team to focus on other tasks.
Recently we helped a telemedicine client automate the infrastructure so that any member of the team could create an entire new environment, all in compliance with HIPAA. Previously, it would take days to configure a new server. Now with IaC in place, it takes us just minutes. Here are some of the top IaC tools in our DevOps stack:
Ansible
Ansible uses playbooks for infrastructure (YAML files) definition. We’ve found that Ansible is among the easiest IaC tools for teams to learn.
Chef
We use Chef almost every day, including for our own website. Developers create recipes for infrastructure using Ruby. Since Ruby is a high-level programming language, it offers more flexibility than some of our other tools. Chef has been around for more than a decade and has great documentation and community resources.
CloudFormation
CloudFormation is one of the most powerful tools on AWS. Based on fully customizable templates, CloudFormation can help you manage and provision every AWS resource.
Terraform
Terraform is similar to CloudFormation but works across different cloud providers. The same script can be deployed on Azure, AWS, or Google Cloud.
OpsWorks
OpsWorks is similar to Chef and can help you configure AWS resources using a template. OpsWorks is great for monitoring application health and efficiently deploying new versions on those environments. It only works on AWS.
Monitoring and log management
Monitoring and log management tools constantly analyze performance and gather information about your environment to understand how your application is performing. We use these DevOps engineering tools to understand resource bottlenecks, identify bugs, and optimize performance.
CloudWatch
CloudWatch is an AWS performance and security resource. It provides dashboards that expose data and visualize performance. You can create trigger notifications and enable auto-scaling of your application. For example, you can set a trigger that if you reach 70% of your computing capacity, an alarm can notify the ops team and a lambda function can autoscale to dynamically increase your capacity.
Grafana
Grafana is an on-premise service that monitors internet speed and VMS status. With Grafana, you can query any system metrics and create some nice dashboards to see your network performance in real-time.
Sumo Logic
Sumo Logic helps you monitor your network security, both on-premise and in the cloud. Similar to CloudWatch for AWS, Sumo Logic provides real-time stats about your network infrastructure, but across your entire network (not just the cloud).
Elasticsearch and Kibana
Elasticsearch is a log management and a database that captures lots of great information. Kibana is a dashboard that can help visualize that data.
Automation and continuous integration
An important part of any DevOps tool stack is automation and continuous integration (CI). These tools are used for compiling source code, analyzing it, testing it, and deploying it.
Jenkins (and Jenkins Pipeline)
Jenkins is our most commonly used CI tool, including the bulk of the mobile apps and sites we develop.. One of the key strengths is that Jenkins allows developers full control over the build process. Our team has been using Jenkins since ArcTouch was founded, shortly after Apple opened the App Store. We also use Jenkins to manage our own website.
Azure DevOps
Increasingly, we have been using Azure DevOps for cloud-based development, especially in combination with Microsoft’s Xamarin. The toolset makes it fairly straightforward to code-sign and deploy apps on both iOS and Android.
Bitbucket Pipelines
We often use Bitbucket Pipelines to build web applications and banners for advertising.
ANT, Rake, and Gradle
ANT (short for Another Neat Tool), Rake, and Gradle are all build automation tools that we use to code-sign mobile apps as we publish in the app stores. They are all similar in functionality — although ANT is XML-based, Rake uses Ruby, and Gradle is rooted in Java.
Fastlane
Fastlane helps developers automate publishing new or updated apps. It can handle the code signing, although on Android you need to use a different tool (such as Gradle). Fastlane also automates the process of uploading screenshots of apps to the app stores.
SonarQube
SonarQube is a useful open-source tool that can scan and analyze your source code. For on-premise development, SonarQube looks for security vulnerabilities and can detect bugs.
Automated testing
Web and app test automation can scan your code and detect bugs automatically before you push features to a production environment. For this, we most often use Selenium and Appium.
Selenium
Selenium is used mostly for test automation of web services and related APIs.
Appium
Appium is best for mobile app test automation, and it can help you evaluate both native apps and cross-platform apps (including those built-in Xamarin).
Containerization and microservices
Containers and microservices help us make our projects more modular and easier to update. Containerized applications have a single file, called an image, with all the dependencies, configuration, and frameworks embedded with it. The most important benefit of using containers is portability — meaning that it can be run anywhere you need it. Containers isolate software from its environment and ensure that it works uniformly. Our DevOps app development team typically uses both Kubernetes and Docker together for containerized apps.
Docker
Docker is open-source and features the Docker Engine. It allows developers to build and run containers on any development machine. Developers can then store or share container images through a container registry. Most cloud providers support Docker — especially those offering infrastructure-as-a-service (IaaS) and serverless products. These providers run the Docker images and offer the underlying infrastructure for connected applications, such as web servers.
Kubernetes
Kubernetes is the platform that manages the containers your software needs to be stable and healthy. Using Kubernetes YAML files, developers turn infrastructure into code to describe the application and how containers should work. Kubernetes automates management tasks. It will deploy the containers, run the replicas to be executed, monitor container health, and provide the scale needed as demand increases.
High availability and rapid scalability
These DevOps engineering tools are crucial to establishing and maintaining a robust production environment. High availability means your systems are capable of running for long periods of time, without downtimes. All cloud providers use the concept of availability zones in their data centers. If there’s a disaster in one of those zones, your application can be affected. To make sure that your system is always online, you should run it in more than one zone. Rapid scalability is one of the major advantages of using a cloud provider. You can quickly increase or decrease your resources based on changing demands.
ASG
ASG (auto-scaling group) on AWS allows you to manage your infrastructure. It monitors your application health using customized health checks on the different instances, and it can alert you to increase or decrease the number of running instances.
Elastic Beanstalk
Elastic Beanstalk on AWS helps you build and deploy web applications. It can create load balancers and autoscale your application by using ASG. Elastic Beanstalk also helps you direct your application logs to CloudWatch and provides application versioning, making it easier to do rollbacks.
Need help with your DevOps?
Want to scale your app development or web projects with DevOps engineering? Contact us to set up a time to talk about our development services or any of the tools in our toolbox.