Monday, March 23, 2015

Bringing in the herd

Everybody is doing microservices at the time of writing. They promise to solve the problems we had with monolithic architectures: They are easy to deploy, scale, understand, and throw away, they are resilient and may be implemented using different technologies. That’s hell a lot of promises, but there are also downsides: Microservices come in herds, and herds are hard to handle ;-) In our current project we use the Jenkins CI server to implement a continuous integration pipeline. For every microservice we have a couple of jobs:
  • Build: Compile the classes, build a jar, run the JUnit tests
  • ITest: Run the integration tests against the built jar
  • Deploy: Deploy the microservice to the environment
These steps are run one after another using the Build Pipeline Plugin. But when it comes to getting an overview to the state of the jobs, you have few choices: The All-View is quite inadequate for that. Even if you have only a couple of dozens of services, there are three jobs for every service, so the all view is quite crowded:

all

Jenkins provides views that let you filter the job list either by selecting each job, or by using regex. So we could easily create a view providing a nice overview of all build jobs:

all-build

But what about the integration-test and deploy jobs? Well, we could create corresponding views for that in the same manner. But that’s also not very appropriate, since we are interested in the pipeline. The Build Pipeline Plugin brings a special view for visualizing the state of the pipeline, so you are able to see the build-health of your microservice in a single:

one-pipeline

That’s fine for the developers of that microservice: they have all steps that matters to them in one view. But if we create a build pipeline view for every microservice, that’s still a confusing lot of views. In this example its only a couple of microservices, think what you will experience with dozens or hundreds of services:

all-pipelines-marked

If you are a team leader, or if you are developing multiple services, it would be perfect, if you had an overview to all build pipelines.  That’s where the Nested View Plugin comes to the rescue: It allows grouping job views into multiple levels instead of one big list of tabs:

pipeline-overview

You can still get down to the pipeline view by selecting the corresponding link:

pipeline-overview-one

That’s already quite nice, but the really neat thing is: you can aggregate the state of the complete pipeline. Let’s see what happens if one step of the pipeline fails:

pipeline-overview-failed

That’s what we want, you can see the state of the complete pipeline(s) at one glance. And if step down to customer pipeline subfolder, you will see which step failed:

pipeline-overview-one-failed

Currently only the state and weather columns are supported in the nested view plugin, but there is already an open issue requesting other columns.

That’s it for today
Ralf

Update 24.03.2015

Just to make that point clear: you can also nest nested views. If you have just a couple of microservices, it is ok to have all build pipelines on one overview. But if they don’t fit on one view, you can use nested views to create groups:
nested-overview
Here we use three (imaginary) groups backend, base and shop. You still have the state aggregation feature, and if you step down into the next level, you’ll see pipelines contained in that group:
nested-overview-2
Regards
Ralf

No comments:

Post a Comment