Following on from my trip to Glasgow for DDD Scotland, on Saturday 21st April I took a much shorter trip along the Great Western to Bristol for this years DDD South West. Here after a couple of weeks is my write up of the sessions I attended. There was no need for an overnight stay, but I did get to experience an almost deserted Reading Station very early on a Saturday morning!
First off I went along to Callum Whyte’s session on how his team took a Monolithic Monster and are currently turning it into a Majestic Microservice App.
As Callum told the story of how his company picked up this project, there were lots of knowing laughs from the audience – seems like a good number of us had experienced similar, when our non-technical directors had taken on nightmare projects. In this case it was a favour for a friend who had paid a Lithuanian outsourcing company a grand total of seven thousand pounds to produce an app for the the NHS and had delivered it with a couple of bugs that needed sorting before it went live in a couple of months time. Callum’s team had opened the proverbial Pandora’s Box and discovered a horrific mess, and so far his company have burned through the equivalent of half a million pounds worth of time and resources getting it in a fit state. The app is currently live, and is now being improved.
When they originally took on the project the source code they received appeared to be six months out of date, and wouldn’t even build, even when they got the apparently latest code it was still different from what was on production. There were major security bugs and holes, for example certain bugs would reveal a complete list of all the patients registered on the application. It was also running on the free tier of Amazon Web Services, and was totally unable to scale.
It was quickly decided that there wasn’t the time or money for a complete re-write, so they made a plan for what their ideal architecture would be – a micro service architecture based on Azure – and then looked at how to get there. They also then modelled how the system actually worked, and identified the critical areas that needed to be addressed first.
In opting for a micro service architecture they effectively treated the existing monolith as just another micro service. They then used the Strangler Pattern to start to replace the monolith, creating new micro services to replace parts of the existing application.
The key aspect of the strangler pattern is that as far as possibly you don’t work on the monolith, you’re creating a total replacement alongside and then connecting into that new replacement leaving the old monolith behind. Eventually you will have totally replaced the old code. The only work on the monolith is to reroute to the new micro services.
Whilst the original monolith had no tests, the new code was developed with a full test suite. They made extensive use of Azure Functions, but kept them small – don’t create a new monolith with giant Azure Functions!
They also made use of Azure APU Gateways which allowed them to reroute parts of a single API either to the old monolith or new functions.
Callum particularly recommended Aram Koukia’s series of blogs on A Microservices Implementation Journey as a good introduction to developing micro services, and said this was something they used to onboard new developers onto their team.
For session number two I had a total change of gear, and went along to hear Dan Clarke talk about Developer Productivity.
Dan’s session was a good selection of hints and tips, reinforcing some things I do already, and pointing me towards tools to try. Even better I later found out that for giving feedback on Dan’s talk I won a license for LinqPad a tool I’ve used previously and that Dan demonstrated during the talk.
Dan started off talking about the importance of taking notes and snippets. He stressed the importance, especially for a contractor of keeping the notes in a place that goes with you, not one which stays at your company, as it is a personal goldmine. For a long while I’ve kept a notebook at work where I write down notes during the day (and at DDD events), more recently I’ve taken to keeping a daily work journal of key events during the day, and have already used that to refer back to work I’ve done previously. The key thing is to record anything that you think might be useful in the future.
Dan then talked about focus, recommending the use of headphones to filter out distractions – to be fair I’ve often found using headphones annoying in themselves. He also talked about the Pomodoro Technique as something he has found valuable. Dan has written his own desktop application for keeping track of his time called Tomatoad. He also recommended Music to Flow By a set of Pomodoro length tracks designed to have the right tempo and style to both stimulate the brain to perform their best, but not be too interesting that you end up listening to the music rather than focusing.
Dan then moved on to some tool recommendations.
He kicked off with LinqPad, which I had seen as a result of having bought Joseph Albahari’s excellent C# in a Nutshell book and updated to a number of subsequent revisions. As an aside if you want an excellent book on C# this is my number one recommendation – whilst you could get much the same from the MSDN documentation, this book covers it in a much more readable format, and before I switched largely to e-Books, this was the book I had to hand on my desk. Back to LinqPad, the tool has grown massively from when I had originally looked at it, and it was great to be win the prize of a free license. It is already installed on my PC at work as a handy scratchpad for working with LINQ, and also querying databases.
Dan then moved on to talking about Resharper, a tool that I and lots of other developers have installed, but one which many of us fail to use completely. Whilst Dan showed some parts of the tool I was already aware of, I was still able to pick up some good tips, and realise that I was only using parts of some other features.
Dan also talked about the resurgence of the Command Line. He conceded that the Windows Command Line is still a bit rubbish, but highlighted that Powershell is a lot better. He recommended some useful tools.
Firstly he pointed me towards PoshGit a Powershell extension that helps when working with Git in Powershell. The most obvious improvement is that it changes the Powershell prompt to give a compact summary if the current directory is within a Git repository. This again is now installed on my PC at work.
For working better with multiple console windows he recommended ConEmu a console window that wraps any other console application be that any of the Windows consoles like the regular command line or Powershell, or any of the Unix variants.
For better clipboard handling he recommended Ditto a clipboard extension that significantly extends the functionality of the standard clipboard.
Moving on Dan then recommended the use of mind mapping, a technique I’ve used quite frequently for recording subject areas when modelling systems. He also talked about how to be more organised using to-do lists and GTD techniques. He also talked about techniques for achieving Zero Inbox which were pretty similar to the process I’ve used for many years on my Mac at home using MailActOn and MailTags.
He finished off with a look at keeping your brain healthy, stressing the importance of not multi-tasking, and also eating and drinking to keep your brain healthy – quite the opposite from traditional developer food, but again something that I’ve been trying to stick to for a while already.
Next I moved on to a Git Deep Dive presented by James World. I’ve been using Git for a while, and I certainly agree with James’ opening point that Git is a tool that it takes most people three to six months to start to get their heads around. As he said that underlying concept of Git is beautifully simple, built around the commit. However built around it is a truly hideous user experience, and the documentation really doesn’t help by using multiple terms for the same thing.
James showed something similar to the techniques we are using at work to smooth out the sometimes quite disjointed path developers take to get to a solution into presenting a more straightforward story.
He also highlighted a number of important Git concepts that are sometimes missed.
First off he used the git hash-object command to compute the unique and immutable hash value which is at the heart of how Git operates. A key concept is that the actual filename is separate from that hash.
When looking at merges James highlighted that Git is able to do an octopus merge where it merges from multiple branches. James also said that we really shouldn’t try it!
Another useful tip which explains some behaviour I’ve seen from Git is that Git remembers how you resolved a conflict in the past and will reuse the technique if a similar problem comes up in future. This certainly explains some weird merge problems I’ve come across where Git has automatically done some strange things because it has actually been reusing a merge it had seen me do previously.
James also demonstrated rebasing, and the use of an interactive rebase, something we use extensively at work when working with multiple branches.
He also talked about various different branching strategies and how we can use them with Git.
James finished off with a look at the libgit2sharp package on Nuget which allows us to manipulate and analyse at Git repository from C# – this allows us to do some powerful analysis of commits and the commit messages to take a look at our projects in useful ways.
At lunchtime there were some Grok talks which I went along to. The first that I came into towards the end was a whistle stop tour of server less. Following that we had an interesting demo of the power of the new Linux Subsystem in Windows 10, including demonstrating it running on a pretty low spec Microsoft Surface. It finished off with a talk on the potentially pretty philosophical question of exactly how big a Microservice should be.
My first session after lunch was one which I had wanted to attend at DDD Scotland, but had been unable to as it clashed with another session I was interested in. It was great to be able to get to see it here, Ismail Mayat talking about Teaching and Old Dog New Tricks.
The session had grown out of Ismail having attended training given by Uncle Bob Martin himself, someone whose evangelical zeal for clean code I have been following too.
Ismail made liberal use of quotes throughout his talk, including this gem, known as Weinberg’s Second Law:
If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilisation.
He started off with an exploration of why developers write lousy code, highlighting that awful code is most often written by reasonable developers but in awful circumstances.
Often developers are under time pressures and cut corners planning to go back and sort out the mess, but they never do, and the mess grows and festers. He also highlighted the problems of working with the latest hipster framework or tool, and the problems that can be produced by trying to learn a new framework whilst solving a complex problem, again this can produce lousy code. He also highlighted that often when developers start on something they do not fully understand the problem – more often than not a developers understanding of the problem is growing and forming as we are coding the solution.
He came up with a good analogy of working with lousy code – you’re working on your house and you have to do a simple job like change the doorbell, but in changing that doorbell the oven explodes. This is just the kind of thing we experience with difficult to maintain code.
He then went through a number of techniques that Uncle Bob covers in his books about using descriptive variable names, using nouns to name classes and verbs to name methods, limiting function length, the Single Responsibility Principle for for functions and not using booleans as control parameters on functions.
He also highlighted the importance of keeping functions pure – they shouldn’t mutate state and should have no side effects. It is important that if you repeatedly send the same inputs to a function, you repeatedly get the same outputs.
Inside every large program is a small program trying to get out.
He suggested that the software industry is currently similar to the medicine in the Victorian era – we are making great strides with techniques and are starting to learn about the problems, but we’re not there yet, and patients are still dying on the table!
He suggested that Test Driven Development is one way we are starting to apply engineering rigour to software development, and that Test Driven Development is an important technique because of the well established difficulty adding tests to an existing system after the event.
Alongside the Uncle Bob books he also recommended a couple of others that are on my read list: The Art of Unit Testing and Growing Object Oriented Software Guided by Tests.
I finished off with Joseph Woodward talking about Patterns and Practices for Building a Better Web Architecture.
The basic purpose of the talk was to look at Josephs experience exploring whether it was possible to improve on Web API – not surprisingly given he was presenting on it he was clear that he was able to.
Firstly he talked about the the Command Query Responsibility Separation pattern, more commonly known as CQRS. The pattern was first described by Greg Young and separates the usual design of Web API controllers into two, one consisting of commands that write, and the other of queries that read. The commands mutate the state, queries do not.
There are a number of reasons for doing this, most importantly it allows us to use different models for reading and writing. Trying to construct a single model that works for both reading and writing is complicated, and results in most services violating the single responsibility principle.
Also we have different requirements for when we might use the commands to write, and queries to read, often we will be reading much more than writing, so having the two separate gives us more flexibility over our architecture.
Next he moved onto techniques for loosening the coupling between our services using the Command Dispatcher Pattern, and the Mediator Pattern, in particular making use of MediatR a .NET library that allows us to decouple dependencies between services in a micro service environment.
Joseph also showed a useful extension for VSCode called Rest Client that I’ve taken to using for testing API through REST – although in our system a lot of the API have Swagger, in particular the parts built from Azure Functions do not have that option, and Rest Client is useful for testing those parts.
Joseph gave a number of good tips for how to design our Web API projects, removing business logic from controllers, and decoupling the domain from the UI framework.
He also highlighted that often we are putting validation logic in two places, and are restricted by the way ASP.Net puts validation as attributes on fields. He recommended using Jeremy Skinners Fluent Validation library which allows for more complex validation rules to be constructed, and for us to encapsulate validation in validation objects derived from the AbstractValidator class. This allows us to reuse custom validation across different objects, and offers a much more flexible and reusable way to validate.
Another library he recommended was The Polly Project which again helps us with separating out our system into micro services by implementing important Circuit Breaker, Isolation and other patterns in a fluent and thread safe manner. We’re not currently using these patterns in our systems at work, but we probably should be as the system grows.
Joseph finished off talking about how we structure our projects. The way Visual Studio often encourages us to arrange them is grouped by technical concern, so we put all our controllers together in one folder, all our views in another and so on. Instead Joseph recommended arranging by domain concern, so all the related objects are together, and developers only have to look in one place for everything needed for a particular feature.
Joseph also recommended a couple of videos that helped him with how to construct his controllers better: Slices Not Layers by Jimmy Bogard and Fat Controller CQRS Diet by Derek Comartin.
One final note was that Joseph was the first person I’ve seen at DDD using Jetbrains Rider under MacOS X – I’ve experimented with it, but ultimately I still use VS2017 as that is what we have at work. However on a Mac I can actually get Rider to work, whereas the Mac version of Visual Studio has never worked for me.
So that is an overview of my day at DDD South West. As with all the other DDD events, my thanks to all the volunteers who put it together and spoke, as always I learned loads during the day, and it is great to have people in the community willing to share their expertise for free like this. With so many companies with limited training budgets and resources it is great for us as developers to be able to keep our skills updated and relevant without breaking the bank!
