Virtual machines are mainstream in cloud computing. The newest development on the this arena are fast and lightweight process virtualization. Linux-based container infrastructure is an emerging cloud technology that provides its users an environment as close as possible to a standard Linux distribution.
Linux Containers and the Future Cloud article tells that as opposed to para-virtualization solutions (Xen) and hardware virtualization solutions (KVM), which provide virtual machines (VMs), containers do not create other instances of the operating system kernel. This brings advantage of containers over VMs is that starting and shutting down a container is much faster than starting and shutting down a VM. The idea of process-level virtualization in itself is not new (remember Solaris Zones and BSD jails).
All containers under a host are running under the same kernel. Basically, a container is a Linux process (or several processes) that has special features and that runs in an isolated environment, configured on the host. Containerization is a way of packaging up applications so that they share the same underlying OS but are otherwise fully isolated from one another with their own CPU, memory, disk and network allocations to work within – going a few steps further than the usual process separation in Unix-y OSes, but not completely down the per-app virtual machine route. The underlying infrastructure of modern Linux-based containers consists mainly of two kernel features: namespaces and cgroups. Well known Linux container technologies are Docker, OpenVZ, Google containers, Linux-VServer and LXC (LinuX Containers).
Docker is an open-source project that automates the creation and deployment of containers. Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications. Consisting of Docker Engine, a portable, lightweight runtime and packaging tool, and Docker Hub, a cloud service for sharing applications and automating workflows.
Docker started as an internal project by a Platform-as-a-Service (PaaS) company called dotCloud at the time, and now called Docker Inc. Docker is currently available only for Linux (Linux kernel 3.8 or above). It utilizes the LXC toolkit. It runs on distributions like Ubuntu 12.04, 13.04; Fedora 19 and 20; RHEL 6.5 and above; and on cloud platforms like Amazon EC2, Google Compute Engine and Rackspace.
Linux containers are turning to a way of packaging up applications and related software for movement over the network or Internet. You can create images by running commands manually and committing the resulting container, but you also can describe them with a Dockerfile. Docker images can be stored on a public repository. Docker is able to create a snapshot. Docker, the company that sponsors the Docker.org open source project, is gaining allies in making its commercially supported Linux container format a de facto standard. Red Hat has woken up to the growth of Linux containers and has begun certifying applications running in the sandboxing tech.
Docker was last week a lot in IT news because Docker 1.0 has been released. Here are links to several articles on Docker:
Docker opens online port for packaging and shipping Linux containers
Docker, Open Source Application Container Platform, Has 1.0 Coming Out Party At Dockercon14
Google Embraces Docker, the Next Big Thing in Cloud Computing
Docker blasts into 1.0, throwing dust onto traditional hypervisors
Automated Testing of Hardware Appliances with Docker
Continuous Integration Using Docker, Maven and Jenkins
The best way to understand Docker is to try it!
This Docker thing looks interesting. Maybe I should spend some time testing it.
341 Comments
Tomi Engdahl says:
Creating small containers with Buildah
https://opensource.com/article/18/5/containers-buildah?sc_cid=7016000000127ECAAY
A technical problem drives home the power of open source collaboration.
Tomi Engdahl says:
Introducing conu – Scripting Containers Made Easier
https://developers.redhat.com/blog/2018/03/07/introducing-conu-scripting-containers-made-easier/?sc_cid=7016000000127ECAAY
Tomi Engdahl says:
10 layers of Linux container security
https://opensource.com/article/17/10/10-layers-container-security?sc_cid=7016000000127ECAAY
Employ these strategies to secure different layers of the container solution stack and different stages of the container lifecycle.
Tomi Engdahl says:
Just say no to root (in containers)
Even smart admins can make bad decisions.
https://opensource.com/article/18/3/just-say-no-root-containers?sc_cid=7016000000127ECAAY
User namespace is all about running privileged processes in containers, such that if they break out, they would no-longer be privileged outside the container.
OpenShift is Red Hat’s container platform, built on Kubernetes, Red Hat Enterprise Linux, and OCI containers, and it has a great security feature: By default, no containers are allowed to run as root. An admin can override this, otherwise all user containers run without ever being root. This is particularly important in multi-tenant OpenShift Kubernetes cluster
Sadly one of the biggest complaints about OpenShift is that users can not easily run all of the community container images available at docker.io. This is because the vast majority of container images in the world today require root.
Why do all these images require root?
If you actually examine the reasons to be root, on a system, they are quite limited.
Container runtimes can start applications as non-root to begin with. Truth be known, so can systemd, but most software that has been built over the past 20 years assumes it is starting as root and dropping privileges.
the main reason apache or nginx start out running as root is so that they can bind to port 80 and then become non-root.
Container runtimes, using port forwarding, can solve this problem. You can set up a container to listen on any network port, and then have the container runtime map that port to port 80 on the host.
Bottom line
Almost all software you are running in your containers does not require root. Your web applications, databases, load balancers, number crunchers, etc. do not need to be run as root ever. When we get people to start building container images that do not require root at all, and others to base their images off of non-privileged container images, we would see a giant leap in container security.
There is still a lot of educating needing to be done around running root inside of containers. Without education, smart administrators can make bad decisions.
Tomi Engdahl says:
10 tasks for running containers on Atomic Host
https://opensource.com/article/18/5/10-tasks-running-containers-atomic-hosts?sc_cid=7016000000127ECAAY
Learn what you can do with Atomic Host, a container operating system that provides a secure foundation for running containers.
Tomi Engdahl says:
Tom Krazit / GeekWire:
AWS now fully supports Kubernetes with the general release of Amazon EKS, following other cloud vendors, after announcing the service last year at AWS re:Invent — Amazon Web Services was the last of the major cloud vendors to go all-in with the Kubernetes container-orchestration project …
Amazon Web Services now fully supports Kubernetes with the general release of Amazon EKS
https://www.geekwire.com/2018/amazon-web-services-now-fully-supports-kubernetes-general-release-amazon-eks/
Tomi Engdahl says:
Getting started with Buildah
https://opensource.com/article/18/6/getting-started-buildah?sc_cid=7016000000127ECAAY
Buildah offers a flexible, scriptable way to create lean, efficient container images using your favorite tools.
Tomi Engdahl says:
Four years after its release, Kubernetes has come a long way
https://techcrunch.com/2018/06/06/four-years-after-release-of-kubernetes-1-0-it-has-come-long-way/?utm_source=tcfbpage&sr_share=facebook
Tomi Engdahl says:
Introducing conu – Scripting Containers Made Easier
https://developers.redhat.com/blog/2018/03/07/introducing-conu-scripting-containers-made-easier/?sc_cid=7016000000127ECAAY
There has been a need for a simple, easy-to-use handler for writing tests and other code around containers that would implement helpful methods and utilities. For this we introduce conu, a low-level Python library.
This project has been driven from the start by the requirements of container maintainers and testers. In addition to basic image and container management methods, it provides other often used functions, such as container mount, shortcut methods for getting an IP address, exposed ports, logs, name, image extending using source-to-image, and many others.
Tomi Engdahl says:
Running integration tests in Kubernetes
https://opensource.com/article/18/6/running-integration-tests-kubernetes?sc_cid=7016000000127ECAAY
Verify how your application behaves with your full solution stack by using multiple containers to provide a whole test environment.
Linux containers have changed the way we run, build, and manage applications. As more and more platforms become cloud-native, containers are playing a more important role in every enterprise’s infrastructure. Kubernetes (K8s) is currently the most well-known solution for managing containers, whether they run in a private, public, or hybrid cloud.
With a container application platform, we can dynamically create a whole environment to run a task and discard it afterward. In an earlier post, we covered how to use Jenkins to run builds and unit tests in containers.
Tomi Engdahl says:
How to create a cron job with Kubernetes on a Raspberry Pi
https://opensource.com/article/18/3/kubernetes-cron-job-tasks?sc_cid=7016000000127ECAAY
Find a better way to run your scheduled tasks efficiently and reliably.
Tomi Engdahl says:
Why Kubernetes is The New Application Server
https://developers.redhat.com/blog/2018/06/28/why-kubernetes-is-the-new-application-server/?sc_cid=7016000000127ECAAY
Have you ever wondered why you are deploying your multi-platform applications using containers? Is it just a matter of “following the hype”? In this article, I’m going to ask some provocative questions to make my case for Why Kubernetes is the new application server.
Tomi Engdahl says:
https://www.nebula.fi/blogi/kubernetes-ja-docker-mita-miksi-ja-muuttamisen-sietamaton-raskaus?utm_campaign=07%2F2018%20Kubernetes%20ja%20Docker%20blogi&utm_source=facebook&utm_medium=paidsocial&utm_content=kubernetes_ja_docker_blog
Tomi Engdahl says:
Container storage for dummies
https://www.redhat.com/en/resources/container-storage-dummies?sc_cid=7016000000127ECAAY
Although many organizations still use traditional storage appliances, they don’t offer the agility needed by containerized environments. Containers are highly flexible and bring incredible scale to how apps and storage are delivered; traditional storage can be the bottleneck that stops this progress. The underlying storage should be highly elastic, easily provisioned by developers and admins, and, ideally, managed using the same orchestration framework (like Kubernetes).
Tomi Engdahl says:
Building tiny container images
https://opensource.com/article/18/7/building-container-images?sc_cid=7016000000127ECAAY
Here are 5 ways to optimize Linux container size and build small images.
Tomi Engdahl says:
Unleash powerful Linux container-building capabilities with Buildah
Posted by Ben Breard
https://rhelblog.redhat.com/2018/07/11/unleash-powerful-linux-container-building-capabilities-with-buildah/?sc_cid=7016000000127ECAAY
Balancing size and features is a universal challenge when building software. So, it’s unsurprising that this holds true when building container images. If you don’t include enough packages in your base image, you end up with images which are difficult to troubleshoot, missing something you need, or just cause different development teams to add the exact same package to layered images (causing duplication). If you build it too big, people complain because it takes too long to download – especially for quick and dirty projects or demos. This is where Buildah comes in.
In the currently available ecosystem of build tools, there are two main kinds of build tools:
Ones which build container images from scratch.
Those that build layered images.
Tomi Engdahl says:
Linux containers, virtualization, and services: “Three’s a party” in modern IT
https://www.redhat.com/en/blog/linux-containers-virtualization-and-services-threes-party-modern-it?sc_cid=7016000000127ECAAY
Linux containers are often positioned as disruptive to traditional virtualization, frequently culminating in the question: Will containers kill virtualization? It’s a fair question, given the shared similarities in workload isolation, resource utilization, and so on, but the answer is a hard “no.” They’re complementary, each solving a unique challenge for the enterprise; that said, historically they don’t actually integrate or work well together. This means separate application stacks, separate developer workflows, and so on.
Tomi Engdahl says:
Learn Kubernetes using Interactive Browser-Based Scenarios
https://www.katacoda.com/courses/kubernetes
Solve real problems and enhance your skills with browser based hands on labs without any downloads or configuration
Tomi Engdahl says:
From 0 to Kubernetes
https://opensource.com/article/17/11/kubernetes-lightning-talk?sc_cid=7016000000127ECAAY
Clear up your confusion about containers and Kubernetes in this Lightning Talk from All Things Open 2017.
Tomi Engdahl says:
https://www.redhat.com/en/resources/best-practices-container-migration-ebook?sc_cid=7016000000127ECAAY
Tomi Engdahl says:
https://developers.redhat.com/blog/2017/07/27/6-reasons-why-i-started-using-containers/?sc_cid=7016000000127ECAAY
Tomi Engdahl says:
Why Kubernetes is The New Application Server
https://developers.redhat.com/blog/2018/06/28/why-kubernetes-is-the-new-application-server/?sc_cid=7016000000127ECAAY
Have you ever wondered why you are deploying your multi-platform applications using containers? Is it just a matter of “following the hype”? In this article, I’m going to ask some provocative questions to make my case for Why Kubernetes is the new application server.
You might have noticed that the majority of languages are interpreted and use “runtimes” to execute your source code. In theory, most Node.js, Python, and Ruby code can be easily moved from one platform (Windows, Mac, Linux) to another platform. Java applications go even further
Tomi Engdahl says:
Conclusion
Container images have become the standard packaging format to distribute cloud-native applications. While containers “per se” don’t provide real business advantages to applications, Kubernetes and its related projects, such as OpenShift and Istio, provide the non-functional requirements that used to be part of an application server.
https://developers.redhat.com/blog/2018/06/28/why-kubernetes-is-the-new-application-server/?sc_cid=7016000000127ECAAY
Tomi Engdahl says:
Kubernetes Services By Example
https://blog.openshift.com/kubernetes-services-by-example/?sc_cid=7016000000127ECAAY
When explaining Kubernetes to people new in the space I noticed that the concept of services is often not well understood.
In a nutshell, Kubernetes services are an abstraction for pods, providing a stable, virtual IP (VIP) address. As pods may come and go, for example in the process of a rolling upgrade, services allow clients to reliably connect to the containers running in the pods, using the VIP. The virtual in VIP means it’s not an actual IP address connected to a network interface but its purpose is purely to forward traffic to one or more pods. Keeping the mapping between the VIP and the pods up-to-date is the job of kube-proxy, a process that runs on every node, which queries the API server to learn about new services in the cluster.
But what makes the VIP 172.30.40.155 forward the traffic to the pod? The answer is: IPTables.
Tomi Engdahl says:
History of containers on unix like system:
1. chroot 1982
2. Freebsd jails 2000
3. Linux vserver 2001
4. Solaris zones 2004
5. OpenVZ 2005
6. LXC 2008
7. Systemd-nspawn 2010
8. Docker 2013
#sysadmin #linux #unix #macos #devops
Tomi Engdahl says:
A Practical Introduction to Container Terminology
https://developers.redhat.com/blog/2018/02/22/container-terminology-practical-introduction/?sc_cid=7016000000127ECAAY
You might think containers seem like a pretty straightforward concept, so why do I need to read about container terminology?
I’ve encountered misuse of container terminology that causes people to stumble on the road to mastering containers. Terms like containers and images are used interchangeably, but there are important conceptual differences. In the world of containers, repository has a different meaning than what you’d expect. Additionally, the landscape for container technologies is larger than just docker. Without a good handle on the terminology, It can be difficult to grasp the key differences between docker and (pick your favorites, CRI-O, rkt, lxc/lxd) or understand what the Open Container Initiative is doing to standardize container technology.
Tomi Engdahl says:
William Denniss / Google Cloud Blog:
Google to hand over operational control of the Kubernetes project to its community and donates $9M in Google Cloud credits to Cloud Native Computing Foundation — Kubernetes, the container orchestrator created and open-sourced here at Google, has experienced incredible development and adoption since it was introduced in 2014.
Google Cloud grants $9M in credits for the operation of the Kubernetes project
https://cloud.google.com/blog/products/gcp/google-cloud-grants-9m-in-credits-for-the-operation-of-the-kubernetes-project
Tomi Engdahl says:
Intro to Docker: Why and How to Use Containers on Any System
https://hackaday.com/2018/09/05/intro-to-docker-why-and-how-to-use-containers-on-any-system/
Tomi Engdahl says:
Code Execution in Alpine Linux Impacts Containers
https://www.securityweek.com/code-execution-alpine-linux-impacts-containers
A security researcher discovered several vulnerabilities in Alpine Linux, a distribution commonly used with Docker, including one that could allow for arbitrary code execution.
Based on musl and BusyBox, the Alpine Linux distribution has a small size and is heavily used in containers, including Docker, as it provides fast boot times.
APK, the default package manager in Alpine, is impacted by several bugs, security researcher Max Justicz has discovered. The most important of them, the researcher says, could allow a network man-in-the-middle (or a malicious package mirror) to execute arbitrary code on the user’s machine.
Tomi Engdahl says:
A sysadmin’s guide to containers
https://opensource.com/article/18/8/sysadmins-guide-containers?sc_cid=7016000000127ECAAY
What you need to know to understand how containers work.
Tomi Engdahl says:
Enforce Docker Image CIS Policy Compliance with Tripwire for DevOps
https://www.tripwire.com/state-of-security/devops/cis-policy-compliance-tripwire-devops/
We are working hard adding features to our new Tripwire for DevOps service, initially announced at BlackHat 2018. If you are a loyal State of Security follower, last you read we added Auditing for Amazon Machine Images (aka AMIs). Today, we are introducing CIS policy compliance auditing for Docker images.
Tripwire for DevOps allows you to evaluate your Docker Images to check for policy compliance at build time. Doing so ensures those images are compliant with CIS policies before they are put into production.
https://www.tripwire.com/products/tripwire-for-devops/
Tomi Engdahl says:
6 container trends IT leaders should watch
What container trends should CIOs and their teams have on their radar? Let’s dig in
https://enterprisersproject.com/article/2018/10/container-trends-6-watch-it-leaders
Tomi Engdahl says:
Misconfigured Docker Services Actively Exploited in Cryptojacking Operation
https://www.bleepingcomputer.com/news/security/misconfigured-docker-services-actively-exploited-in-cryptojacking-operation/
Adversaries looking for an easy way to mine for cryptocurrency are actively targeting publicly exposed Docker services. They use a malicious script capable to scan the network in search of vulnerable hosts and compromise them.
The point of entry is TCP port 2375 or 2376, the default for reaching the Docker service remotely via REST management APIs, which allow creating, starting and stopping containers. Unless configured otherwise, both ports provide unencrypted and unauthenticated communication.
Tomi Engdahl says:
Misconfiguration a Top Security Concern for Containers
https://www.securityweek.com/misconfiguration-top-security-concern-containers
Report Demonstrates that Security Needs to be Included in Containerization
Although the acceptance and adoption of containers within DevOps is growing, concern over their security remains strong. Thirty-five percent of respondents to a new survey believe their company does not adequately invest in container security, while a further 15% don’t think their company takes the threat to containers seriously.
The survey (PDF) was undertaken by StackRox among 230 IT staff — almost half of whom identify IT security as their primary role. More than 45% are employed in companies with more than 10,000 employees, while 58% are employed in either the fintech or technology sectors. The StackRox inaugural report, ‘The State of Container Security’, found that most organizations feel unprepared to adequately secure cloud-native applications, despite the surging adoption of containers and Kubernetes.
Docker is the most popular container runtime, used by 189 of the respondents. Kubernetes, originally developed by Google, is the most popular container orchestrator, used by 122 of the respondents. Docker Swarm is the second most popular orchestrator, used by 93 of the respondents primarily from the larger organizations with 5,000 or more employees.
https://security.stackrox.com/rs/219-UEH-533/images/StackRox-Report-State_of_Container_Security.pdf
Tomi Engdahl says:
10 things to avoid in docker containers
https://developers.redhat.com/blog/2016/02/24/10-things-to-avoid-in-docker-containers/?sc_cid=7016000000127ECAAY
Tomi Engdahl says:
Frederic Lardinois / TechCrunch:
Microsoft and Docker announce a new joint open-source project, the Cloud Native Application Bundle, that makes packaging and running cloud-native apps easier
Microsoft and Docker team up to make packaging and running cloud-native applications easier
https://techcrunch.com/2018/12/04/microsoft-and-docker-team-up-to-make-packaging-and-running-cloud-native-applications-easier/
Tomi Engdahl says:
Podman and user namespaces: A marriage made in heaven
https://opensource.com/article/18/12/podman-and-user-namespaces?sc_cid=7016000000127ECAAY
Learn how to use Podman to run containers in separate user namespaces.
Tomi Engdahl says:
sed ‘s/docker/containers/g’
https://opensource.com/article/18/12/sed-sdockercontainersg?sc_cid=7016000000127ECAAY
Learn about alternatives to Docker in this All Things Open 2018 Lightning Talk.
Tomi Engdahl says:
What we learned about containers last year
https://opensource.com/article/19/1/best-of-containers?sc_cid=7016000000127ECAAY
The most-read articles about containers from 2018 show what you need to know to be successful in 2019.
Tomi Engdahl says:
Aging like milk, not wine: The realities of container security
https://www.redhat.com/en/blog/aging-milk-not-wine-realities-container-security?sc_cid=7016000000127ECAAY
Tomi Engdahl says:
THREAT RESEARCH BLOG POST
How I Hacked Play-with-Docker and Remotely Ran Code on the Host
https://www.cyberark.com/threat-research-blog/how-i-hacked-play-with-docker-and-remotely-ran-code-on-the-host/
Play-with-Docker (PWD), Docker’s playground website, allows beginners to run Docker commands in a matter of seconds. Built on a number of hosts with each running multiple student’s containers, it’s a great place to learn Docker. PWD provides the experience of having a free Alpine Linux virtual machine in a web browser where students can build and run Docker containers and experience Docker firsthand without having to first install and configure it.
This unique offering was warmly welcomed by DevOps practitioners with more than 100,000 total monthly site visits, where Docker tutorials, workshops and training are also available. The initiative was an effort originated by Marcos Nils and Jonathan Leibiusky, aided by the Docker community and sponsored by Docker.
CyberArk Labs set out to try and escape the mock container in an effort to run code on the Docker host.
The impact of container escape is similar to escape from a virtual machine, as both allow access to the underlying server.
Tomi Engdahl says:
Dockter: A container image builder for researchers
https://opensource.com/article/19/1/dockter-image-builder-researchers?sc_cid=7016000000127ECAAY
Dockter supports the specific requirements of researchers doing data analysis, including those using R.
Tomi Engdahl says:
Red Hat Advances Container Technology With Podman 1.0
https://www.eweek.com/cloud/red-hat-advances-container-technology-with-podman-1.0
Red Hat’s competitive Docker container effort hits a major milestone with the release of Podman 1.0, which looks to provide improved performance and security for containers.
Tomi Engdahl says:
Do Linux distributions still matter with containers?
https://opensource.com/article/19/2/linux-distributions-still-matter-containers?sc_cid=7016000000127ECAAY
There are two major trends in container builds: using a base image and building from scratch. Each has engineering tradeoffs.
Tomi Engdahl says:
How rootless Buildah works: Building containers in unprivileged environments
https://opensource.com/article/19/3/tips-tricks-rootless-buildah?sc_cid=7016000000127ECAAY
Buildah is a tool and library for building Open Container Initiative (OCI) container images.
Tomi Engdahl says:
How to prepare and use Docker for web pentest by Júnior Carreiro
https://pentestmag.com/prepare-use-docker-web-pentest-junior-carreiro/
Docker is the world’s leading software containerization platform. Using Docker we can create different environments for each Pentest type. With the use of containers, you can save each environment on a USB stick or leave it in the cloud. For exemple, you can use the environment in the cloud or copy to any computer or laptop, regardless of distribution. You need only install Docker, if it is not installed.
Tomi Engdahl says:
Kubernetes security: 4 tips to manage risks
https://enterprisersproject.com/article/2019/1/kubernetes-security-4-tips-manage-risks
As you bear down on Kubernetes security, use these strategies to avoid missteps in work with containers and orchestration
Tomi Engdahl says:
8 Docker Steps That Make Everyone Love It. [Guide]
https://www.cect.co/8-docker-steps-that-make-everyone-love-it/
Tomi Engdahl says:
Frederic Lardinois / TechCrunch:
Docker and Arm announce a new partnership to help Docker developers more easily build apps for the Arm platform — Docker and Arm today announced a major new partnership that will see the two companies collaborate in bringing improved support for the Arm platform to Docker’s tools.
Docker developers can now build Arm containers on their desktops
https://techcrunch.com/2019/04/24/docker-partners-with-arm/
Docker and Arm today announced a major new partnership that will see the two companies collaborate in bringing improved support for the Arm platform to Docker’s tools.
The main idea here is to make it easy for Docker developers to build their applications for the Arm platform right from their x86 desktops and then deploy them to the cloud (including the Arm-based AWS EC2 A1 instances), edge and IoT devices. Developers will be able to build their containers for Arm just like they do today, without the need for any cross-compilation.
This new capability, which will work for applications written in JavaScript/Node.js, Python, Java, C++, Ruby, .NET core, Go, Rust and PHP, will become available as a tech preview next week, when Docker hosts its annual North American developer conference in San Francisco.
Tomi Engdahl says:
CVE-2018-15664: docker (all versions) is vulnerable to a symlink-race attack
https://seclists.org/oss-sec/2019/q2/131
There is no released Docker version with a fix for this issue at the
time of writing.