Blog

Blog

Culture as Code

Welcome to culture as code! Now: obviously, this is a lie - culture is about social behaviour, about norms found among humans. Culture is more than what happens at your workplace. It is found in music, art, religion. You can't possibly put that into code. And you're right, you can't.

But hear me out. 

Let me tell a story showing that code can indeed be the source of influence on human behaviour - and help to improve on your team's culture. And by adding that human to the mix, make this work.


Suspect a required movement

The situation in software development if often diverse. It all started with good intentions, enthusiasm, even huge ambitions. Then it all drifted into some arrangement of progress and staying alive. A drift that companies often try to come by with organizing work into departments, increasing delivery cycles. And people get used to those cycles. Rely on them, maybe even demand them.

A culture driven by processes and rules emerges. Not too bad, not great either.

So the cycle continues: That application needs to be done in a couple of months?! We will do testing when the software is more complete. Does the CI server show a broken build? We can fix that once we need it. It is more important right now to be able to work locally. Do the integration tests do whatever they want? We can take care of those once all services are integrated.

This mental model of postponing important things in favour of urgent work will only change when the culture changes! We need to change our culture!! This is what developers are born for. This is what developers are given time for!!! Culture changes!!! Psychological work, social competence. Well, maybe not every developer is up for the task. Still, this situation needs to improve. 


Starting a movement

Copper plate with laws used by RomansBut what are developers good at? We declare our manifestos to follow, search for patterns to implement and re-use, define principles and write guidelines. Come up with practices to repeat the next time, since they worked. With rules to follow to not stumble over the same thing again. 

Rules - we can implement those - can't we?


Investigating the situation

Some rules are surely harder to implement than others. 

Like: Be humble to each other. Nice rule. Sounds good. Sounds important. The implementation may require some Commander Data brain. No one got time for that. 

New rules, more concrete, more useful! 

So here we go: The CI build is broken? Drop all your pencils and go fix it!

The code analyzer brought up a crime to light? Drop all your pencils and go fix it! The security scanner found a new vulnerability? Drop the pencils and update that library some insane person added! The deployment failed? Drop all your pencils and fix the deployment. The service crashed miserably? Drop all your pencils and investigate!


These are all important activities, and the quality in which we perform them affects our efficiency and how much time we can spend on other things.


We don't do all this because we would need to check the CI server, the code analyzer, the security scanner, the deployment tools, the service logs and monitoring tools - just to be able to know what is going on, if there is something to do at all and if the problem that came up is actually ours or caused by some other poor fellow.


So back to our plan: what are developers good at? Well, I do hope at writing code! And all of the above can be expressed in code and be shown to your team in an instant. Is that a culture change? It sure is not. But we ain't there just yet. Let's keep trucking.


This overview will help us to identify what task is in the wild and should be done - or - honestly - should have always been done - but we didn't - because it was too fiddly to check everything. So we can go from dropping the pen directly to addressing the issue, taking a huge jump over all the forensics.


Once we have a collection of this information and can show a summary, we make transparent what is hidden somewhere. No more constant searching for the same information. For us and others. The situation of our software is transparent. And obvious state in software? Isn't that beautiful.


Writing a build monitor

Well there is one already, isn't there? Sure there is. But none that shows our tools, our state, our information the way we want it. And - to anticipate a little bit here - we want to do more than just rules on some state. We target culture.


So once we wrote a small application that collects all the information we need (from the CI server, the code analyzer, the security scanner, the deployment information, and the application health state itself) we figure: we have quite some information at hand!


For example: is the production version newer than the version on pre-production? Is the application un-healthy because of some other service that is unavailable - because that may just mean to let them know about it? Does the meta-information of our application suggest any actions? Did we configure the application correctly so all monitoring tools can work? Was there a SNAPSHOT version deployed? Is the version that was created by the CI server the one that was deployed to the test environment? Or is there a version gap? Is the test-coverage anywhere near where it should be?


The pirate codex

All kinds of things can now be checked, beyond just a state in a single tool. The state of a delivery pipeline can be observed and checked against our rules. The rules we currently need.


Some may call this IT governance. But there is one big difference. These are our rules, they represent our current focus and priorities. Ideally these match - but let us not jump into this snakepit today.


Observe the change

Now as every team has a reddish screen, some will take on the challenge to go greenish. And they will notice that once something comes up, it is usually caused by other teams. These teams will now either: be lost and frustrated (because there is no one to lead them out of the mess), or they take on the challenge themselves. Because this is our tool and our rules. And no one wants to come in last. Some claim it - but I don't believe you. So application by application, pipeline by pipeline, team by team the situation improves. Because people care. Because they see the state, they see the effect of their work, they notice the improvements over time.


And when people start taking care where they previously didn't: there is your culture change.


So a simple build screen? A tool that allows implementing some custom rules? Does that work? 

The laws carved into a wall in Gortyn (Creta)

It does. But it does not on its own. Because tools are only tools. If a team does not find a way how to help themselves or to whom to turn to for embarrassing questions, they will ignore this. Delete the transparency and keep living in the mess. You need the humans willing to go through some essentials - application by application, team by team. You need some of them who care and carry this care to others. By making the things they care for transparent - for everyone.


This worked exceptionally well for me. Because you can always find other knights willing to ride along to battle. Because they are sick of living in the mud. And this way - step by step, floor by floor you will reach the roof under the stars. I will not argue it is a fast process. But some processes are more healthy if slow. And even with 40 applications, if you can only heal 2 a month, after only two years you will have healed them all. Looking at some mess today: don't you wish you would have started 24 months ago? Because today: it would be the roof and the stars! 

So if it is not today: let it be tomorrow.


When we think about some mobile apps and how they changed how we meet people, how we connect and get in touch: then why would code not be able to influence and change a culture? Code certainly already did this on other occasions.


Yes! Code can change a culture, together with the people that hold on to it. I've seen it.


The tool that came out of all these steps was called "Mobitor" - because it needed a name at some point. You can give it a run yourself: https://mobiliar.bitbucket.io/mobitor/

The next steps are on www.cultureascodemanifesto.org to collect some guidelines to help you orient yourself on the journey.



The picture of the wall with laws is from wikipedia. The Roman bronze plate picture is from Claude. The Codex of Pirates of the Caribbean is from the fandom wiki (remember: it's just guidelines).


Kotlin all the things

So, after all, it seems JetBrains is very serious with Kotlin. And I have to admit it comes with some handy features and good IDE support. But this is not about the Kotlin language, this is about where it can be used.

As we have seen in Migrating from Gradle to Gradle writing Gradle build scripts using the new Kotlin DSL is supported. So far we have

  • our sources in Kotlin
  • our build configuration in Kotlin
  • but not our Continuous Integration configuration
    (depending on how far you want to push it your build chain or pipeline as well)

Since TeamCity (the CI server) is from JetBrains as well, it supports storing your build configuration not only over the UI. It supports storing the configuration in your VCS in XML format and (since around version 10 and 2017 in a Kotlin format. The current version 2019.1 comes with even more improvements and simplifications in this section. So throw away your build config yaml file! Kotlin you build config too! Although this seems a bit weird at the beginning, there are some big advantages:

  • it real source code
  • it compiles
  • you can share common pipeline definitions via libraries
  • since Kotlin is a typed language, there is a nice support for in your IDE with auto-completion
  • you can compile the code prior to pushing it - compared to the try and error cycle that YAML config files come with, I'll argue it is the better method

There is a very nice Blog from JetBrains on this topic that I can highly recommend to read:

And while you're at it, maybe watch the webinar on "Turbocharging TeamCity with Octopus Deploy" as well. Octopus is an additional commercial service. But distinguishing between continuous integration and deployment seems a good split in responsibilities.

Since I consider you know to be convinced let us test-drive it! We first need a TeamCity Server, a TeamCity Agent, and an example project.

To have it quickly set up to test-drive, I created a docker-compose.yaml file (yes a yaml file, isn't it ironic): https://github.com/brontofundus/kotlin-all-the-things

Clone the repository and fire it up:

$ docker-compose pull
$ docker-compose up -d

Then point a browser to http://localhost:8111/

This will bring you to the TeamCity installer. But don't panic, it will only take a few minutes!




Proceed

  • Select PostgreSQL
  • download the driver
  • and use the same settings as used in the docker-compose file:
    • "postgres" as host
    • "teamcity" as username, password and database name

Proceed

Wait

Scroll down and agree to the license
(well read it of course - but don't tell me you are not used to selling your soul)

Create an admin account

(for simplicity use "admin" and "password" here)


There you are

As you may notice on top there are 0 agents. Which is not entirely true.

But to have the one we have enabled it needs to be authorized first


Go to "Agents" and "Unauthorized" and enable the agent

If you now go to the start page (click on the logo on the top left) you will be able to add a project

Builds run already

If you have a close look at the project you will notice it contains a ".teamcity" directory that contains the build configuration: https://github.com/brontofundus/gradle-groovy-kotlin-dsl/tree/kts/.teamcity

The new 2019.1 format comes in a "portable" variant. So the number of files to earlier TeamCity version is reduced to only the settings.kts file and the pom.xml 

From here on no more clicking in the UI is necessary.

And even more comfort

Since you probably use IntelliJ to develop in Kotlin anyway and you now have a running TeamCity server from the same company, even more, comfort is possible!

Just install the TeamCity plugin in IntelliJ and point it via the new menu entry to your local server.

This will show the build status of your projects, but in addition, also allows you to run your local changes remotely as personal build! This is not a new feature but people tend to forget about it:



Install the plugin in IntelliJ

Restart and point it to the local TeamCity server
(we used "password" as password above)

And you can now remote run builds with local changes!

Even with not yet committed changes

Personal builds have this nice additional icon to mark personal builds. These are only visible to the user that created them

If you create an additional user and re-login you will not see other peoples personal builds


So we have:

  • our software in Kotlin
  • our Gradle build script in Kotlin
  • our CI configuration in Kotlin - able to share and re-use our build chains 
    (read the JetBrains blog above for more details on this)
  • and free of charge with this setup: remote runs


What a Kotlin world to live in!

Gradle build scripts have been written in a Groovy based DSL for a long time. Although flexible, the IDE support was always a bit of a problem. You either knew what to type or you searched the docs or tried to find an answer on stackoverflow. IDEs always struggled to provide help on writing tasks or configuring them.

For some time now, a Kotlin based DSL is in the works and as of Gradle 5 it is available in 1.0. So is it any better compared to what you can do with the Groovy based DSL?

To get started, some reading the documentation (later, after this blog post!) helps:

If you need to learn about Gradle in general, there are free online trainings available that I can highly recommend (from starting with gradle to advanced topics).


The example project created for this comparison is on GitHub and contains a simple spring boot application also written in kotlin that spits out a docker image. The master branch uses the groovy DSL, the kts branch uses the new kotlin DSL but does exactly the same.

Overview of the groovy build

The groovy based build script uses the new plugin syntax:


new plugin syntax
plugins {
  id "com.palantir.docker" version "0.22.1"
}

Instead of the old syntax which would look like this:


old plugin syntax
buildscript {
  repositories {
    maven {
      url "https://plugins.gradle.org/m2/"
    }
  }
  dependencies {
    classpath "gradle.plugin.com.palantir.gradle.docker:gradle-docker:0.22.1"
  }
}

apply plugin: "com.palantir.docker"

This will simplify the kotlin script migration, as the kotlin syntax is very similar to the new one.

Note on the new plugin syntax

There have been some issues with this new syntax when a Maven Repository Proxy (like Nexus or Artifactory) is used. But the Gradle plugin repository is available as maven repository as well and as of Gradle 4.4.x plugins can be loaded via a repository proxy too (previously this would only work without any authentication or with direct internet access - which is unlikely in an enterprise environment). So Gradle 4.4.x comes to the rescue! You can add your repository proxy to an init.d script and use the new plugin syntax.


$HOME/.gradle/init.d/repository.gradle
apply plugin: EnterpriseRepositoryPlugin

import org.gradle.util.GradleVersion

class EnterpriseRepositoryPlugin implements Plugin<Gradle> {

    private static String NEXUS_PUBLIC_URL = "https://<nexushostname.domain>/repository/public"

    void apply(Gradle gradle) {
        gradle.allprojects { project ->
            project.repositories {
                maven {
                    name "NexusPublic"
                    url NEXUS_PUBLIC_URL
                    credentials {
                       def env = System.getenv()
                       username "$env.NEXUS_USERNAME"
                       password "$env.NEXUS_PASSWORD"
                    }
                }
            }

            project.buildscript.repositories {
                maven {
                    name "NexusPublic"
                    url NEXUS_PUBLIC_URL
                    credentials {
                        def env = System.getenv()
                        username "$env.NEXUS_USERNAME"
                        password "$env.NEXUS_PASSWORD"
                    }
                }
            }
        }

        def referenceVersion = GradleVersion.version("4.4.1")
        def currentVersion = GradleVersion.current();
        if(currentVersion >= referenceVersion) {
            gradle.settingsEvaluated { settings ->
                settings.pluginManagement {
                    repositories {
                        maven {
                            url NEXUS_PUBLIC_URL
                            name "NexusPublic"
                            credentials {
                                def env = System.getenv()
                                username "$env.NEXUS_USERNAME"
                                password "$env.NEXUS_PASSWORD"
                            }
                        }
                    }
                }
            }
        } else {
            println "Gradle version is too low! UPGRADE REQUIRED! (below " + referenceVersion + "): " + gradle.gradleVersion
        }
    }
}



Other than that the build does not contain any unusual things. There are task configurations and a custom task. The spring boot dependencies are added, the test task is configured to measure the test coverage using jacoco. The build uses the docker-palantir plugin to create the docker image. And there is a task that prints some log statements to tell a TeamCity server about the coverage results (will not hurt on Jenkins or Bamboo).

The docker plugin uses task rules to create some tasks, so it's configured via the extension class - there are several ways to do this, the variant used in the example is also to have IntelliJ understand what task it is so auto-completion works.

Overview of the kotlin build

The kotlin DSL build script (in build.gradle.kts - the buildscript has a new filename!) uses a plugin syntax very similar to the groovy one. It looks almost the same. The build script looks very similar if you compare them both.

The way tasks are referenced or created changes slightly. Once you get used to it it's fairly easy to use, and the IDE supports what you are doing!

Quirks and conclusion

As the IDE support is currently limited to IntelliJ we can only look at that. But if you were used to compile gradle builds on the command line anyway, the gradle wrapper will automatically recognize the kotlin build script and is by default capable to run these.

An improvement you almost immediately notice: auto-completion in build scripts suddenly makes sense! For some reason it sometimes is very slow to show up but the suggestions made are way better than what you would see using the groovy builds.

Yet IntelliJ will sometimes mark fields of plugins as not accessible - the build will work, it's just the IDE that complains. There are workarounds for some of the warnings.

The expected variant:


tasks.test {
    extensions.configure(JacocoTaskExtension::class.java) {
        destinationFile = file(jacocoExecFileName)
    }
}

But there is some access warning. But you can switch to using the setter:


tasks.test {
    extensions.configure(JacocoTaskExtension::class.java) {
        setDestinationFile(jacocoExecFileName)
    }
}

Not too nice, but not a showstopper - and unclear if Gradle is to blame or if it's some IntelliJ issue. 

In other cases it helped to hint the task type:


tasks.withType(Test::class.java) {
}

To have better auto-completion. The documentation on the kotlin DSL gives some hints how to help yourself.

In every case where IntelliJ complained, a workaround could be found. But these are just to have IntelliJ not complain on the build script. Not ideal. But you can reach a state where IntelliJ does not mark any line with an error or warning! Compared to the random errors and warnings mess in the groovy build scripts: way better.

Comparing the length of both scripts, doesn't really show a clear winner. Both scripts have about the same length and structure. The tasks are often a few lines shorter, but the type declarations will add an import statement on top. Overall this simplifies the migration and keeps the readability one got used to. I wished everything was just shorter and more expressive - but that probably was just a personal wish - actually it is a bit unfair to the groovy DSL, which is already good. The build scripts seem to initialize slower but builds run at the same speed. But the way gradle optimizes task execution or determines if task configuration needs to be loaded at all did improves with gradle 5 - so the speed penalty might not be there for you at all. So the way it looks today: quite good :-)

I have no concerns to use the kotlin DSL in production builds at all and the IntelliJ support is in a good state, so you will not need to flip between the IDE and the command line all the time if you don't like doing that.

Is it a migration?

The title states this was a Gradle to Gradle migration. But the resulting build scripts look very similar. So is it really one? I would say yes. It took me two attempts with a couple of hours of searching around in the documentation and experimenting (as there are not so many examples around yet). Although the result does not look like much of a change, it took some effort to get there. But effort in the meaning of hours to days - surely not weeks (as I'm not the most experienced gradle nor kotlin user). Of course this may fall apart if a lot of plugins are used or they don't properly interact with the Gradle API in this version (as you will probably upgrade to gradle 5.x from a 4.x version).

Hints for the hasty

The linked documentation on top already contains this but just in case you are a very hasty developer, here are some useful gradle tasks in this context:


$ gradle kotlinDslAccessorsReport

prints the Kotlin code necessary to access the model elements contributed by all the applied plugins. The report provides both names and types.

You can then find out the type of a given task by running


$ gradle help --task <taskName>


Another important statement is in the migration guide in the configuring plugins section:


Keeping build scripts declarative

To get the most benefits of the Gradle Kotlin DSL you should strive to keep your build scripts declarative. The main thing to remember here is that in order to get type-safe accessors, plugins must be applied before the body of build scripts.



So if you are programming in kotlin anyway, and you also use TeamCity and its kotlin build DSL you can now also use kotlin in your builds too. Kotlin all the things!

Kotlin has certainly more momentum today compared to Groovy. The typed DSL solves some crucial handling issues in gradle build scripts. I would guess the new DSL may become the default at some point, not that I'm aware of any timeline. Just an assumption. So don't hurry, the groovy DSL will be around for quite some time. If you are starting with gradle, I would try using the kotlin DSL from the beginning.


Give it a try! 


These are some notes that were taken when watching this video: https://www.youtube.com/watch?v=lE6Hxz4yomA


One pattern of the book is “be a Hands-on Modeller” (you have to have some contact to the ground level or you won’t give good advice, stay up to date, stay sharp, keep learning things you can talk about).

Every effective DDD person is a Hands-on Modeller.


A lot of things are not exactly different from the book but there is a little different emphasize.

What is (really) essential in the book?

  1. Creating creative collaboration of domain experts & software experts à ubiquitous language pattern
    (you’re not supposed to create that model for yourselves)
  2. Exploration and experimentation
    the first useful model you encounter is unlikely to be the best one. When there are glitches and you start working around if you’re already frozen. à “blast it wide open”, explore with the domain experts
  3. Emerging models shaping and reshaping ubiquitous language
    (say things crisply an un-ubiquitous, no complicated explanations), explore with the domain expert
  4. explicit context boundaries (sadly it is in chapter 14, would be chapter 2 or 3 today)
    a statement in a language makes no sense when it's floating around, you could only guess the context it is in.
    Draw a context map! Should be done in every project!
  5. focus on the core domain (sadly it is in chapter 15)
    find the differentiator involved in your software: how is your software supposed to change the situation for the business you’re in (we do not mean efficiently, something significant)

These are the things to focus on.

Building Blocks (chapter 5)

Our modelling paradigm is too general, we have objects and relations – this is just too broad. We need something that structures this a little more, puts things into categories, helps communicate the nature of your choices.

Services - Entities - Value objects
Repositories – Factories

  • They are important but overemphasized
  • But let’s add another one anyway, as an important ingredient: Domain Events (interesting for domain expert):
    The level of event (important for the domain expert) you want to record something important happened in your domain. There is a consistent form:
    • Tend to happen at a certain time
    • Tend to have associated a person
    • typically immutable (you record it happened and that’s it)
  • Domain Events give you some architectural options, especially for distributed systems
    (record events from different locations)
  • consistent view on this entity (“runs” in a game reported from different locations) across a distributed system à event oriented view


More options:

  • Decoupling subsystems with event streams (Design Decoupling)
    • Have a core transactional system, send out a stream of domain events
      (you can change the definitions and only need to maintain the stream of events)
  • Such distributed systems are inconsistent but well characterized
  • Have multiple models in a project that are perfect for their purpose say reporting and trading (of course you don’t have to)

Another aspect of domain events are distributed systems:
Enabling high-performance systems (Greg Young)

Aggregates (super important)

  • Do people often ask how to access what’s inside? But that’s not the most important question.
  • Aggregates help to enforce the real rules
  • You have something you think of as a conceptual whole which is also made up of smaller parts and you have rules that apply to the whole thing.
  • Classic example: purchase order, having a limit, an amount, line items that add up, …
    but with thousands of line items object orientation gets a little stuck
  • Beware of mimic objects (that carry data around but not doing anything useful)
    • "Where is the action taking place?”
  • Sometimes it might be useful to give aggregates more privileges so they could execute a count query themselves.
  • Aggregate: we see it as a conceptual whole and we want it to be consistent
    • Consistency boundaries
      • Transactions
      • Distribution
        (you need to define what has to be consistent when crossing the boundaries)
      • Concurrency
    • Properties
    • Invariant rules

Strategic design

  1. Context mapping
  2. Distillation of the core domain
  3. Large scale structure

Large scale structures do not come up that often.


Setting the stage

  • Don’t spread modelling too thin (“you need to know why modelling is done”)
    Modelling is an intensive activity, so the more people understand it the more value you gain
  • Focus on the core domain, find out what it is. Find the need for extreme clarity.
  • Clean, bounded context
  • iterative process
  • access to a domain expert


Context mapping

Context: the setting in which a word or statement appears that determines its meaning.

Bounded context: a description of the conditions under which a particular model applies.

Partners: to teams that are mutually dependent. Forced into a cooperative relationship.


“Big Ball of Mud”: http://www.laputan.org/mud/  (the most successful architecture ever applied)


How to get out? Draw a context map around the big ball of mud. Build a small correct module inside the ball until eventually the ball of mud captures you. But you had that time to do it right. So think about an anti-corruption layer.

If you transfer a model into a different context, use a translation map:
model in context <--> translation map <--> model in context

Explain the meaning of something. Because meaning demands context.

Strategy

  • Draw a context map
  • Define core domain with business leadership
  • Design a platform that supports the core domain