Week of June 5th

Another dimension, new galaxy Intergalactic, planetary”

Happy National Donut Emoji Day!

“There is an inexorable force in the cosmos, a place where time and space converge. A place beyond man’s vision…but not beyond his reach. Man has reached the most mysterious and awesome corner of the universe…a point where the here and now become forever…. A journey that takes you where no man has been before Experience the power⚡️! A journey that begins where everything nothing ends! You can’t escape the most powerful force in the ‘DevOps’ universe.”

Mission #7419I 

So once again, we boarded the USS Palomino 🚀 and continued our exploration to the far depths of the DevOps Universe.  Just to pick up where we last left off,👨‍✈️ Captain Bret Fisher had taken us through the Microservices galaxy 🌌  and straight to Docker🐳 and Containers.  But.. “with so many light years to go.. And things to be found” we continued through the courseware  Docker Mastery: with Kubernetes +Swarm from a Docker Captain  and reconnoitered Docker🐳 Compose, Docker🐳 Swam, Docker🐳 Registries, and the infamous Kubernetes☸️.

Again, we leveraged the portability of HashiCorp’s Vagrant for both Docker🐳 with Docker🐳 Compose, our 3 Node Docker 🐳 Swarm , and the K8s☸️ environments. We were grateful that we had our previous experiences with Vagrant in earlier learnings as it made standing up these environments quite seamless.

We started off with Docker 🐳 Compose which can be quite a useful tool in development for defining and running multi container Docker🐳 applications.  Next, we headed right over to Docker🐳 Swam to get our initiation into Container Orchestration. You might ask why not just go straight Kubernetes☸️ as they are the clear winner🏆 from famous Container Orchestration wars? Well, Orchestration is great for solving complex problems but Orchestrators themselves can be complex solutions to try to learn. From what we witnessed this week we were glad we started there.  We also learned that the “combination of Docker 🐳 Swarm, Stacks, and Secrets are kind of like a trilogy of awesome features” that can really make things easier if we went this route in production.

“Resistance is Futile“

If you not familiar with the story of Kubernetes☸️ or affectionately known as k8s☸️.. It came out of Google by the original developers who worked on the infamous Google “Borg” project.. In fact, here is a little bit of trivia, the code name project for K8s☸️ was called Project Seven of Nine, a reference to the Star Trek🖖 character of the same name who was a “friendlier” Borg. K8s☸️ was certainly uncharted territories for us and bit out of my purview but it was a good learning experience to to get a high level overview of another important component of the infrastruture ecosystem.

Captains’ log  Star date 73894.9 These are missions covered in earnest this week:

  • Created a 3-node Swarm cluster in the cloud
  • Installed Kubernetes and and learn the leading server cluster tools
  • Used Virtual IP’s for built-in load balancing in your cluster
  • Optimized Dockerfiles for faster building and tiny deploys
  • Built/Published custom application images
  • Learned the differences between Kubernetes and Swarm
  • Created an image registry
  • Used Swarm Secrets to encrypt your environment configs
  • Created the config utopia of a single set of YAML files for local dev, CI testing, and prod cluster deploys
  • Deployed apps to Kubernetes
  • Made Dockerfiles and Compose files
  • Built multi-node Swarm clusters and deploying H/A containers
  • Made Kubernetes YAML manifests and deploy using infrastructure-as-code methods
  • Built a workflow of using Docker in dev, then test/CI, then production with YAML

For more details see the complete Log

This turned out to be quite the intensive undertaking this week but we accomplished our mission and here is certificate to prove it

Below are some topics I am considering for my exploration next week:

  • Google Big Query
  • More with with Data Pipelines
  • Google Cloud Data Fusion (ETL/ELT)
  • More with with Data Pipelines
  • NoSQL – MongoDB, Cosmos DB
  • Working JSON Files
  • Working with Parquet files 
  • JDBC Drivers
  • More on Machine Learning
  • ONTAP Cluster Fundamentals
  • Data Visualization Tools (i.e. Looker)
  • ETL Solutions (Stitch, FiveTran) 
  • Process and Transforming data/Explore data through ML (i.e. Databricks)

Stay safe and Be well –


Week of May 29th

“This is the return of the space cowboy…Interplanetary, good vibe zone”

Happy Friday!

A short time ago in a galaxy not that far, far away….” 

Actually it was only just a few weeks ago that we traveled 🚀 through time 🕰 and space💫 and into the DevOps Universe. For those who recall, we just merely scratched the surface as we dipped our toe🦶in the Lunar 🌚 mare 🌊 with HashiCorp’s Vagrant. At the time, we played around with combination of Vagrant and Oracle’s Virtualbox. This time we wanted to go full tilt into the Microservices galaxy 🌌 with the infamous “batteries🔋 included but removable approach” a.k.a. Docker.

We had explored the rings🪐 of Docker🐳 once before or I should formally say images and containers but it had been about 4 years since we last journeyed there. At time, we were lead by the charismatic and often enthusiastic Docker🐳 Captain 👨‍✈️ Nigel Poulton 

Since so much time has past we were overdue for a refresher and who better to lead us through this exploration than equally ebullient👨‍✈️ Captain Bret Fisher as he would take us through his brilliantly put together courseware Docker Mastery: with Kubernetes +Swarm from a Docker Captain

Bret’s courseware contains a chock full of DevOps awesomeness 😎 including full lessons on Docker🐳 , Docker🐳 Compose, Docker🐳 Swam, Docker🐳  Registries, and Kubernetes. There were even quizzes after most lessons to see how much you were able to absorb. Unfortunately, time would only allow for us time to explore just the Docker🐳  portion. Leaving us with Docker🐳 Compose, Docker🐳  Swam, Docker🐳 Registries, and Kubernetes for future missions. Our initial goal was to use to utilize Vagrant with Docker 🐳 (similarly to how we used Vagrant with VBox a few weeks back). However, to get into the spirt of how ubiquitous Docker🐳  is we decided we would try to install Docker🐳  on multiple platforms. All was well, until we decided to install Docker CE on Windows 10. 

Since we are primarily running on the Mac book💻 we would need to create a Windows 10 Pro Image on VirtualBox.. Sure no problem.. After several minutes we had our Windows Desktop up and running. For the finishing touches we just need to install Docker🐳 before we would head over to the next lesson. After Docker 🐳 completed it’s installation on Windows, we tried to fire🔥 up Docker 🐳…  Bam💥, Slam, Crash ☄️! Uh, oh what now?? Well, actually this was a familiar scenario we had seen a few weeks back when we tried to install Vagrant & VirtualBox under an Ubuntu VM that was already running on VirtualBox for the Mac book💻.

At the time we just chalked this up to an overly convoluted solution because it was virtualization inside of virtualization. But apparently, there is an actual technical term for this which is called “nested virtualizationThis configuration is not generally recommended but it is supported in certain scenarios. The way to make this solution fly Under VBox is to enable “Enable Nested VT -x/AMD-V” Settings -> Processor in Virtualbox.
Of course on the Mac book 💻 this is inconveniently greyed out. Not to fret, just like the hidden trick to increase the VDI disk size . Their is hidden trick to enable Nested Virtualization on VBox on the Mac book 💻

Ok, so now we enabled the Nested Virtualization setting and made some additional recommended local setting changes on the Windows VM. After rebooting the VM, we would try to install Docker 🐳 but again no luck. It’s turns out that Docker 🐳 will only work with nested virtualization with VMware Fusion and we weren’t going to go there. So after spending enough time on this we decided to cut or loses and just work with Vagrant with Docker 🐳 and Ubuntu which in reality is the best way to run docker (Under a native Linux OS)..

Now with our a Docker🐳 environment ready to go and fired 🔥 up it was time to have 👨‍✈️ Captain Fisher take us to warp drive 💨. As we charted our course, we hit a minor bug when building the popular Docker🐳 container nginx image on our environment.  We would execute the below syntax which worked fine without any issues:

docker container run –publish 8080:80 nginx:latest

However, after we killed the container with control-c and ran docker info we noticed the following error message

WARNING: No swap limit support

Although, the nginx container still ran fine, we are not big fans of those pesky little error messages.  So we resolved the issue by running the below syntax in the Docker 🐳 container:

vagrant@node-1:~$ sudo nano /etc/default/grub

GRUB_CMDLINE_LINUX=”cgroup_enable=memory swapaccount=1 

$ sudo update-grub

$ reboot -iThen From: Mac book 💻

vagrant halt

vagrant up

For good measure we filed a bug with the developer on Github. Fortunately we know the developer, so this minor bug was fixed almost immediately. So it can be used with future builds. Next, we moved on as our journey touched on all the essential topics for Docker 🐳:

  • Manage Multiple Containers
  • Management of Docker Networking
    • DNS and How Containers Find Each Other
    • Using Containers for CLI Testing
    • DNS Round Robin Test
  • Images and Their Layers
  • Image Tagging and Pushing to Docker Hub
  • Dockerfiles and Run Containers From it
  • Persistent Data: Data Volumes, Bind Mounting
  • Database Upgrades with Named Volumes
  • Running In Containers with Bind Mounts

For more details on syntax see log

“DevOps is the union of people, process, and products to enable continuous delivery of value to our end users.”

Below are some topics I am considering for my Journey next week:

  • More with Captain Brett on  Compose, Swam, Registries, and Kubernetes
  • More with with Data Pipelines
  • Google Cloud Data Fusion (ETL/ELT)
  • More on Machine Learning
  • ONTAP Cluster Fundamentals
  • Google Big Query
  • Data Visualization Tools (i.e. Looker)
  • ETL Solutions (Stitch, FiveTran) 
  • Process and Transforming data/Explore data through ML (i.e. Databricks) .
  • Getting Started with Kubernetes with an old buddy (Nigel)

Stay safe and Be well –