xenmaster's devops systems
This list is inspired by the Kamran Ahmed's "2020 Developer Roadmap," now at roadmap.sh
Without further ado, my take on the most important technologies to learn for dev-ops in 2020!
Basics for SysAdmin
These are the basic tools you will need to start your path to being a sysadmin. BASH is the scripting language used in the command terminal while vim will be your go-to text editor and PuTTY your primary solution for remote server access.
Bash is the shell, or command language interpreter, that will appear in the GNU operating system. Bash is an sh-compatible shell that incorporates useful features from the Korn shell (ksh) and C shell (csh). It is intended to conform to the IEEE POSIX P1003.2/ISO 9945.
Xfce Terminal is a lightweight and easy to use terminal emulator application with many advanced features including drop down, tabs, unlimited scrolling, full colors, fonts, transparent backgrounds, and more.
tmux is a terminal multiplexer: it enables a number of terminals (or windows), each running a separate program, to be created, accessed, and controlled from a single screen. tmux may be detached from a screen and continue running in the background, then later reattached.
Vim ("Vi IMproved") is an advanced text editor that allows syntax highlighting, word completion and has a huge amount of contributed content. It also has a GUI version called GVim.
License model
- Free • Open Source
Application types
Platforms
- Mac
- Windows
- Linux
- BSD
- Haiku
- AmigaOS
- OpenSolaris
- MorphOS
- Flathub
- Flatpak
- Snapcraft
Free and open-source terminal emulator supporting SSH, Telnet, and SCP. Portable, lightweight, and works offline to connect to remote systems securely.
License model
- Free • Open Source
Application types
Platforms
- Windows
- Linux
- BSD
- PortableApps.com
- ReactOS
Linux Servers
Redhat is the most common server seen in the field. CentOS is the freeware equivalent.
Red Hat is the leader in development, deployment, and management of Linux and open source solutions for Internet infrastructure - ranging from embedded devices to secure Web servers. Red Hat was founded in 1994 by visionary entrepreneurs Bob Young and Marc Ewing.
CentOS is a community-supported, free and open source operating system based on Red Hat Enterprise Linux. It exists to provide a free enterprise class computing platform and strives to maintain 100% binary compatibility with its upstream distribution.
+2
The leading platform for scale-out computing, Ubuntu Server helps you make the most of your infrastructure. Whether you want to deploy an OpenStack cloud, a Hadoop cluster or a 50,000-node render farm, Ubuntu Server delivers the best value scale-out performance available.
Compiling Tools
For compiling apps from source.
Make is a tool which controls the generation of executables and other non-source files of a program from the program's source files.
License model
- Free • Open Source
Platforms
- Mac
- Windows
- Linux
- BSD
- Cygwin
The GNU Compiler Collection (GCC) is a compiler system produced by the GNU Project supporting various programming languages. GCC is a key component of the GNU toolchain. As well as being the official compiler of the unfinished GNU operating system, GCC has been adopted as the...
Virtualization
I don't see virtualization as much as I used to, but these are the best free options I've come across. Note that in the Corporate environment, VMWare dominates from what I've seen.
Xen.org, home of the Xen hypervisor, the powerful open source industry standard for virtualization. It is a native (bare-metal) hypervisor providing services that allow multiple computer operating systems to execute on the same computer hardware concurrently.
Open source virtualization tool supporting x86/AMD64, runs on Windows, Linux, Mac, Solaris. Offers portable, lightweight, ad-free use.
License model
- Free • Open Source
Application type
Platforms
- Mac
- Windows
- Linux
- BSD
- Oracle Solaris
+5
Create and manage virtualized development environments. Vagrant is a tool for building complete development environments. With an easy-to-use workflow and focus on automation, Vagrant lowers development environment setup time, increases development/production parity, and makes...
License model
- Free • Open Source
Application type
Platforms
- Mac
- Windows
- Linux
- BSD
- VirtualBox
- VMware Workstation Pro
- VMware Fusion
- FreeBSD
Networking + Security
This is the longest section and some of these solutions overlap. Some additional notes:
- Wireguard is looking to become the faster, more secure version of OpenVPN, but is still in the experimental stage at this point.
- Snort is an alternative to Suricata I've seen a lot in the field as well.
Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF and its supporting vendors.
Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages. Squid has extensive access controls and makes a great server accelerator.
License model
- Free • Open Source
Application type
Platforms
- Mac
- Windows
- Linux
- BSD
- Self-Hosted
- FreeBSD
- NetBSD
- OpenBSD
HAProxy is free, open source software that provides a high availability load balancer and proxy server for TCP and HTTP-based applications that spreads requests across multiple servers.
Netfilter software comprises iptables, ipset, conntrack-tools, libnetfilter and more. It is the core of Linux firewalls and NAT. It consists of a set of hooks inside the Linux kernel and a number of utilities to manage callback functions.
Dnsmasq is a lightweight, easy to configure DNS forwarder and DHCP server. It is designed to provide DNS and, optionally, DHCP, to a small network. It can serve the names of local machines which are not in the global DNS.
License model
- Free • Open Source
Platforms
- Linux
Unbound is a validating, recursive, and caching DNS resolver.
The C implementation of Unbound is developed and maintained by NLnet Labs. It is based on ideas and algorithms taken from a java prototype developed by Verisign labs, Nominet, Kirei and ep.net.
Unbound is designed as.
License model
- Free • Open Source
Platforms
- Mac
- Windows
- Linux
Automates HTTPS setup using Let's Encrypt certificates and supports multiple servers. Open source tool by EFF, prioritizing internet security.
License model
- Free • Open Source
Platforms
- Mac
- Linux
- BSD
- Self-Hosted
- Arch Linux
- Gentoo
- CentOS
- Fedora
- Debian
- FreeBSD
- Ubuntu Server
- OpenBSD
OpenVPN Connect provides flexible VPN solutions to secure your data communications, whether it's for Internet privacy, remote access for employees, securing IoT, or for networking Cloud data centers.
License model
- Freemium • Open Source
Application types
Platforms
- Mac
- Windows
- Linux
- Android
- iPhone
- Android Tablet
- iPad
- Self-Hosted
- Cloudron
WireGuard is an extremely simple yet fast and modern VPN that utilizes state-of-the-art cryptography. It aims to be faster, simpler, leaner, and more useful than IPSec, while avoiding the massive headache. It intends to be considerably more performant than OpenVPN.
License model
- Free • Open Source
Application types
Platforms
- Mac
- Windows
- Linux
- Android
- iPhone
- BSD
- iPad
- OpenWrt
- Termux
- Self-Hosted
+1
Web Servers
Two common webservers, each with its own user case and differences.
Apache, is a web server notable for playing a key role in the initial growth of the World Wide Web and in 2009 became the first web server to surpass the 100 million web site milestone. Apache was the first viable alternative to the Netscape Communications Corporation web...
nginx [engine x] is a HTTP and reverse proxy server, as well as a mail proxy server written by Igor Sysoev. It has been running for more than five years on many heavily loaded Russian sites including Rambler (RamblerMedia.com).
Code Management + Deployment
Jenkins is the Continuous Integration / Deployment tool I've seen the most, both the free and enterprise versions. I should note that github is more popular than Gitlab right now, but I see a lot companies making the switch.
Git is a free & open source, distributed version control system designed to handle everything from small to very large projects with speed and efficiency.
License model
- Free • Open Source
Application type
Platforms
- Mac
- Windows
- Linux
- Android
- iPhone
- Chrome OS
- Android Tablet
- BSD
- Linux Mobile
- Haiku
Offers on-premise or hosted Git repository management, including issue tracking, wikis, code reviews, and continuous integration and deployment.
License model
- Freemium • Open Source
Application types
Platforms
- Linux
- Online
- BSD
- Self-Hosted
- Ruby
- Cloudron
+4
Infrastructure as Code
Docker will be your container software whereas kubernetes is the management platform for orchestrating multiple containers in production at once. Ansible is an up and coming configuration manager used to manage setups across many servers and is gaining popularity fast. Terraform is used for infrastructure provisioning where istio is used for service mesh monitoring (microservices).
Docker is an open-source engine that automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere.
License model
- Freemium • Open Source
Application types
Platforms
- Mac
- Windows
- Linux
- Online
- Self-Hosted
- TrueNAS
+3
Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs.
License model
- Freemium • Open Source
Application type
Platforms
- Mac
- Windows
- Linux
- BSD
- Python
Tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.
License model
- Free • Proprietary
Platforms
- Mac
- Windows
- Linux
+1
Istio is an open platform for providing a uniform way to integrate microservices, manage traffic flow across microservices, enforce policies and aggregate telemetry data. Istio's control plane provides an abstraction layer over the underlying cluster management platform...
License model
- Free • Open Source
Platforms
- Linux
Makes it easy to administer Linux servers via a web browser. It allows you to easily perform simple tasks like storage administration, inspecting journals, starting & stopping services, monitoring & administering several servers at once.
Monitoring Tools
Netdata is a free NewRelic / AppDynamic alternative, but it is important to note that both of these solutions are pretty common in the field. Same goes for the ElasticSearch - Logstash - Kibana (ELK) stack, which I've seen is being replaced by Splunk in many places, especially those looking to deploy their production infrastructure to the cloud.
Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community.
License model
- Free • Open Source
Application types
Platforms
- Mac
- Windows
- Linux
- Cloudron
+1
Grafana provides a powerful and elegant way to create, explore, and share dashboards and data with your team and the world.
License model
- Free • Open Source
Application types
Platforms
- Mac
- Windows
- Linux
- BSD
- Self-Hosted
- Cloudron
- Node.JS
+1
Netdata collects metrics per second and presents them in beautiful low-latency dashboards. It is designed to run on all of your physical and virtual servers, cloud deployments, Kubernetes clusters, and edge/IoT devices, to monitor your systems, containers, and applications.
License model
- Freemium • Proprietary
Application types
Platforms
- Mac
- Linux
- Online
- BSD
- Self-Hosted
- C (programming language)
- Python
- GNU Bourne Again SHell
Unmatched relevance. Real-time insights. Anything is possible — when the precision of search meets the intelligence of AI.
License model
- Freemium • Open Source
Application type
Platforms
- Online
- Self-Hosted
- Software as a Service (SaaS)
- Linux
- Mac
- Windows
Logstash is an open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite "stash.".
Easily visualize data pushed into Elasticsearch from Logstash, es-hadoop or 3rd party technologies like Apache Flume, Fluentd, and many others.
License model
- Freemium • Open Source
Platforms
- Mac
- Windows
- Linux
- Online
+1
Cloud Infrastructure
Two major players. There are other options out there, but these are the two I've seen the most in production based heavy use companies.
A pay-as-you-go cloud computing platform with services in networking, compute, storage, and IoT, accessible via global server farms.
License model
- Freemium • Proprietary
Application type
Platforms
- Software as a Service (SaaS)
The purpose of this list is to provide a listing of solutions available and commonly used by system administrators in the field in 2019 for managing production systems and support. I encourage everyone and anyone reading this list to also check out the site "https://github.com/kamranahmedse/developer-roadmap" as it has additional information on a logical order to run and use these apps as well as additional topics that are not software solutions, but concepts that are important to learn.
great list, it helped me