Standardize your Stack with Buildpacks

This year has been chaotic to say the least. The pandemic, presidential election, and a myriad of other events have left most of us feeling somewhat reeling and exhausted. Typically, the best way to move forward is to focus on the small things that you can control and not to fret about what you cannot control.

Recently I was asked by an intern, “What is the most valuable programming language that I should learn now so that it will be easier for me to get a job after I graduate from college?”

What is the most valuable programming language that I should learn now so that it will be easier for me to get a job after I graduate from college?

My initial response was the same as you would probably get from any engineer, “There is no standard language that you should focus on. Focus instead on how to correctly map languages to use cases in the industries that excite you the most. When you find that industry, or that segment in technology that is interesting to you, then the language you’ll need to learn will be apparent with research.”

When you find that industry, or that segment in technology that is interesting to you, then the language you’ll need to learn will be apparent with research.

I could have been even more abstract. I could have just responded, “Individual programming languages don’t matter. Just make sure you learn some OOP, functional, and procedural languages.” But this would only confuse the matter more. Further, a simple Google search results in hundreds if not thousands of different blogs and websites that claim to have the Latest Top 20 Programming Languages.

It’s only natural that someone just starting out in engineering would be confused.

Let’s now kick it up a notch. Enfuse.io does a lot of work with enterprise clients. We’re lucky in that most of these enterprise clients we’ve been working with have been with us for years. We’ve been around so long now that we’ve witnessed scores of engineers come and go through client sites, and we’ve done a fairly good job of staying in contact with departed engineers. I, personally, am fascinated in how departing engineers feel about their new employers vs their previous employers after the grass is always greener honeymoon period has faded.

Bar none, the number one complaint I hear from engineers that have moved on to new pastures is that they love the freedom of writing applications using the language of their choice but the process is nonexistent and management of these applications in production is deplorable. Each of these companies have DevOps teams, however more and more we’ve seen DevOps turn into a facade for kanban IT departments.

[…] they love the freedom of writing applications using the language of their choice, but the process is nonexistent and management of these applications in production is deplorable.

We should define process here before we get any deeper. Process, within our context, is a set of standardized guidelines, workflows, and/or gates for developing and deploying applications within the given organization.

Enfuse.io engineers have been working with a PaaS called Cloud Foundry (IBM Bluemix, Pivotal Cloud Foundry, Pivotal Platform) for a long time. It’s an excellent choice for a PaaS that has been battle hardened after a number of years. The platform is simple enough to use. Build your application, test it locally, creat your application manifest, and push it to the platform whether it be CF in the cloud or on-prem.

Now enter Kubernetes, or k8’s rather. Kubernetes is the future and is the single most followed and developed PaaS at this moment in the tech industry. Could that change as quickly as a developer’s javascript web framework preference? (lol) Maybe, but that’s pretty unlikely since Google has been the largest proponent for k8’s for a number of years now. With Google pushing and client adoption rapidly escalating with the help from mesh frameworks like Istio, we doubt that the k8’s train is going to suddenly stop or derail anytime soon.

DevOps love Cloud Foundry because it incorporates process at a number of different levels. The most basic is that of buildpacks.

Buildpacks are essentially wrappers for an applications. They envelop the application artifact into a runnable container within the platform. Sometimes they’ll add optimizations to the artifact before deployment. This may seem like a small feature, so why do we at Enfuse.io think it’s such a big deal?

Imagine you’re a Project Manager or a Technical Product Manager. A substantial part of your job is to translate product value from stakeholder requests into features so that your team of engineers can build and ship the logic in software to create or return the aforementioned value within some sort of agreed upon timeline. Typically that is where the communication stops with the team (save daily standups, retros, and IPMs). Software Engineers and developers usually resort to whatever the best programming language is that fully satisfies the needs of the application or feature. Sometimes though, they’ll use the opportunity to write an application in an entirely new or up-and-coming programming language.

As a stakeholder you may ask, “Why should I care? As long as the value I requested is present after development, why should I care what language it was written in?”

As a stakeholder you may ask, “Why should I care? As long as the value I requested is present after development, why should I care what language it was written in?”

This is a valid question, and it deserves a valid thoughtful response. An enterprise organization (or a startup) benefits on having a standardized streamlined stack in many ways.

  • Hiring and creating JD’s for new positions is much quicker and easier.
  • DevOps quicker to diagnose issues and provide quality feedback.
  • Employee onboarding time significantly reduces.
  • Depth of granular programming language knowledge increases.
  • Code quality increases substantially along with code coverage.
  • Stability and health of apps in production increases.

The above are just a sample of benefits an organization receives when they standardize their stack.

If your organization isn’t talking about Kubernetes yet, then it will be sooner rather than later.

Engineers love k8’s. Why? For years now engineers have been using Docker to containerize applications locally. Think of it this way: prior to Docker typically engineers would using virtual machine images, which essentially were snapshots of entire operating systems to develop and test their applications. The reason for this is simple — at all cost the engineer wants to mitigate the “Well it ran fine on my computer” issue when an app doesn’t behave as expected in production. If they can develop and test an application within a virtual machine image that mirrors production, then you’ll probably mitigate a lot of bugs.

Virtual machine images are resource heavy. They’re big, they’re clunky, and they’re very difficult to keep up to date and distributed properly to different teams. Compare that to a standard Dockerfile which is more or less a blue print for an image. This Dockerfile is just a normal text file that can be versioned in git and made accessible to your entire team, where you can update it when you see fit.

Kubernetes containers are built using Dockerfiles. This offers a ton of freedom to the engineer or developer because they can craft a Dockerfile to support their application with any language they choose. This is certainly needed in some cases where essential functions or services don’t line up to your standardized stack. All to often however, this can backfire. It can turn out that your organization standardizes on a container platform for infrastructure, but it can spiral out of control at the app level where you have apps and services deployed into production using any number of different libraries or programming languages. This leads to complexity, difficulty in on-boarding new engineers, difficulty in adding features and maintaining existing applications, and slower response times from DevOps with the bare minimum of feedback. Essentially everyone involved in each layer now needs to be a polyglot with respect to programming languages. I don’t know about you, but it’s difficult enough to scale with even a few programming languages, let along the kitchen sink.

Buildpacks are essentially versioned blueprints that help envelope an engineers application, mitigating the need for the engineer to create their own Dockerfile. But as of recently buildpacks were only available in PaaS systems like Cloud Foundry and Heroku.

Google recently announced that they’re incorporating buildpacks into their cloud platform. This is a huge development that shouldn’t be understated. This throws GCP (Google Cloud Platform) directly into the app space, where they’ll be competing with the likes of IBM and VMWare. Here is what Google is saying — why pay the middleman when we can provide the services you expect intrinsically within our cloud platform?

[…] why pay the middleman when we can provide the services you expect intrinsically within our cloud platform?

Buildpacks are not the end-all be-all when considering how to standardize your stack. They simply help at one level. However that level is critical when deciding how you’re going to standardize. You can enable and disable any buildpack you choose, which means that you can funnel your engineers into your preferred stack. If there are outlier cases where a language is needed that isn’t in your stack, you can accommodate those anomalies easily.

In the next post we’ll be demoing how to deploy an application using a buildpack in GCP, so stay tuned.