Developer groups quite often go through ups and downs, so after last holding an event back in May 2018 – the last one I attended was DDD Scotland almost three years ago – I thought Scottish Developers had long since disbanded. However I was really pleased to see a new event pop up on Meetup a week or two back organising a pandemic safe online virtual meetup with two very different speakers.
I have to say that one of the definite plus points amongst the numerous negatives for the lockdown is that many groups are discovering online groups and conferences. As the bigger tech hub in Scotland many of the events are held in Glasgow, so whilst I’ve taken days off to go to events like Code Craft Conf back in 2019, evening meetings are a lot more difficult. Going online allows much wider participation, so this event was hosted by Andrew in Caroline who were in different cities, with one speaker in Seattle in the USA, and the other in Aberdeen.
First up was Stephanie Stimac, a Microsoft employee on the Edge team whose talk was about the absolute minefield of HTML controls on web pages. This is a problem that has been around right from the earliest days of the web, and even almost thirty years later is still an issue – if you use the standard HTML controls in a web page by default they will use operating system components in a page and will look and function differently depending on which browser is being used to view the page, and especially if the browser is running on a different platform. This problem has never been fixed, web developers have merely worked around the problems initially by creating their own controls within pages, or more often than not using one of the now numerous frameworks that wrap up solutions to all these problems.
Thankfully, since I work very much in the middleware space, and with a standard browser for internal websites I don’t have to deal with browser complexities in my working life, however this did serve as a reminder of why I’ve always found doing that sort of stuff frustrating. This really is a problem that should have been solved long ago – Stephanie had some interesting statistics part way through her presentation of quite how many hours of developer time have been spend faffing around trying to deal with problems with the controls. Stephanie did offer some cause for hope, but we’re really not there yet.
The second talk, Gary Ewan Park talking about a Build Orchestration Tool called Cake was maybe more relevant to me, especially given that we’re in the process of moving our product builds from an ageing TeamCity platform to a modern Azure DevOps platform.
Although the build in TeamCity and ADO is doing the same task, how it does it is very different between the two, such that we currently have a specialist build team that is effectively recreating the TeamCity build for each of our products in ADO. Cake gets around that problem by moving the build into code. This means that the build can be run locally, and with one line in either TeamCity or ADO can be run the same way there too. The build orchestration is also written in standard C# – so the language that most .Net developers use, rather than particular configuration files for the build platform in use.
We’ll have to see whether we change how builds at work are done – as they’re the responsibility of a separate team who have been working on scripts in ADO specific YAML for a long while, but Cake does seem like a good way around the problem we’ve been having.
Anyway, it was a great restart for the Scottish Developers, and with another virtual meetup tentatively planned for April, hopefully not nearly so long to wait for another one.
If you want to watch the talks, a recording of the whole event is available on YouTube.
Adding to the list of things that COVID-19 has changed, we can add the regular community Developer Days that run around the country. The last one that took place was DDD North back in February just before the pandemic hit, with every other one unable to run. That didn’t stop the community organisers though, and this weekend they put together an online free developer conference, DDD 2020.
Unlike the in person conferences, DDD 2020 was unencumbered by limitations of venue, so whereas even the biggest DDD events I’ve been to have only had four tracks going, DDD 2020 had seven, and no limit on the number of attendees. All in all there were 50 different sessions to choose from, so plenty to interest everybody.
As with the in person conferences I attended a mix of sessions, some of which piqued my interest, others of which were relevant for work.
First off I attended the session on .Net 5 and C# 9, hosted by Stuart Lang. This was a whistle stop tour of some of the new features in .Net 5, and changes in C# 9. In all honesty I doubt I’ll get to use many of these in anger any time soon, as I’m very firmly in the realms of legacy applications for the moment, but it was certainly interesting. Stuart also showed a pretty interesting graph that highlighted that Microsoft is going to have a much more aggressive attitude towards supporting older .Net frameworks going forward, so legacy apps are clearly better off staying on the .Net 4.8 Framework as this will be supported for many years to come, but even .Net 3.5 has support going out to 2029.
The other interesting aspect is that the lifetimes for the even numbered releases of .Net were significantly longer than the odd numbered releases, so he suggested that unless you can update the version of .Net easily you should only be targeting the even numbered releases.
Next I took a diversion into psychology with Adam Griffiths session on a Geeks Guide to People. This was a lot of interesting insights into how people work together and respond, not surprisingly with some insights on the effect of interacting primarily online.
Also on the subject of interacting online was the great session on speaking and presenting virtually from Dylan Hayes. As you can see from the picture it got quite meta when he got to the point of trying to explain the use of OBS for a presentation during a presentation he was hosting with OBS!
Dylan had lots of good tips that are useful both if you are doing full presentations, or even just regular meetings in Teams or Zoom. In particular he advised against the use of virtual backgrounds – ironically being used by several other speakers during the day, and instead recommended paying attention to what is behind you and curating what people can see. He also had good advice on the use of external cameras, where and where not to place lights, and using and external microphone. A big takeaway is that the cameras and microphones on laptops are junk, and to get a good effect you really need to be investing in external webcams and microphones.
Next up I went along to a session called “Break Up with your Dev Machine” presented by Callum Whyte, looking at whether developers really still need the kind of beefy powerful computers we have at the moment. As a travelling consultant he was looking at whether you could get by with something smaller and less powerful by taking advantage of online service such as Virtual Desktops in Office 365, or GitHub Codespaces, or a hybrid combination.
Whilst new tools like GitHub Codespaces are pretty impressive – it is effectively running VS Code in a browser, I’m not sure many developers will be able to cope without a powerful machine, not least because at times you’re not going to have a reliable internet connection.
My next session was another change of gear with the CTO of Black Marble talking about his Azure DevOps plug in that automatically generates release notes. During the session he went through the way the plugin has developed, and showed off some of the functionality. Given that at work release notes and deployment instructions are very much a manual process, it certainly looked like something we could make use of.
Into the afternoon, next up was Sean Farmar talking about using Domain Driven Design to decompose your monolith. Again with the monolithic legacy applications I’m currently dealing with, this was really relevant for work, although in an hour long session Sean could only barely scratch the surface.
His first piece of advice is probably one of the most important, that the easy temptation to throw your hands in the air and rewrite a monolith is a fantasy, and often produces similar problems after a lot of time and expense. Much better to look at what you currently have, and use patterns such as the strangler pattern to put a facade in front of a monolith and slowly evolve it into something more maintainable.
The session took a look at a simple example, and splitting the domain into a number of bounded contexts where there was high cohesion and coupling within a bounded context, but not outside. Communication between contexts should use messaging or events.
Another important point he made towards the end is that there is not a one to one relationship between bounded contexts and micro-services – logical boundaries in the design and physical structure are two very different things.
Next up was Simon Painter talking about Pride and Prejudice and C# – a bit of a random title, and even more random when he started the session talking about breakfast cereal!
What he was actually talking about was Markhov chains, a concept first described over a century ago by Russian mathematician Andrey Markhov, but which turns up all over the place in computing supporting AI systems, generating test data, or predictive text engines, hence the Pride and Prejudice connection.
Having started off demonstrating the principle with breakfast cereals he moved on to using the technique to analyse Pride and Prejudice, and then to generate text at random that whilst it didn’t make much sense, certainly sounded like nonsense written in the style of Jane Austen. Fascinating stuff.
My last session of the day, was the only time we had a serious technical glitch, with the session starting almost twenty minutes late after they had issues with Teams. Given the hiccups we have with Teams at work at times I’m surprised we got as far as we did during the day before something went wrong.
Having said that it was worth waiting for. Gary Short is a data scientist, and having worked for various companies over the years is now working for himself, currently looking at how people handle unforeseen events on the road network.
In the past the organisations responsible for our roads have often relied on experienced road managers who would set up closures by gut feel, however Gary demonstrated how they can use the large amounts of data gathered from roadside sensors to determine the effects of particular actions on the road network.
As so often with data science solutions it’s as much about gathering large amounts of data and determining patterns, but as ever with Gary’s sessions, really interesting seeing what the kind of data volumes modern computers can handle can now produce.
In summary, as a first attempt at an online DDD I think the day was a great success. Given that DDD Scotland hasn’t run since 2018 leaving Scottish Developers with DDD North as probably the nearest event, it was great for me as a developer based in Scotland to not have to drive for several hours to attend the day. Whilst I have travelled to DDD events previously, most notably to the last DDD Scotland from down south, and also to DDD South West, it was great to have access to such a good variety of content without leaving my home office. Whilst it obviously lacked the interaction and networking that are also parts of the DDD events, the wider inclusion of streaming online brought in many more developers than would usually be able to attend, and for those of us with family commitments allowed us to participate as well. Hopefully the experience with DDD 2020 online will lead into some more hybrid events for 2021 as restrictions are eased.
This last weekend was the tenth annual Global Day(s) of Code Retreat, with over one hundred and fifty Code Retreat events running on either the 15th and 16th of November. Not having been to one before it was great to find there was an event being run by Barney Dellar and Natalia Zon from Canon Medical Research at their office in Edinburgh, so I signed up to head along.
The basic idea of the Code Retreat is to look at how we as software developers code by coding the same basic problem in a series of exercises during the day with various restrictions. The basic problem is based around Conway’s Game of Life devised by British mathematician John Conway back in 1970, although during the course of the day it’s unlikely that anyone will get a working solution as your code is thrown away at the end of each exercise!
Each exercise lasts forty-five minutes, which is followed by a short retrospective session and a break, and then we start again. First off we had a simple attempt to solve the problem, to get used to pairing and the Cyber Dojo environment that we all used, however from there it got more difficult.
Session number two we were heavily restricted in the length of methods we could write, and in my case that was made more difficult by myself and my second coding partner coming at solving the problem from totally different directions! As a result of that in session number two I came up with a better explanation as to my thinking, which then came totally to nothing as session three we weren’t allowed to talk to or communicate with our pairing partner at all, except through the code we were writing, and we took turns to write tests and implement the code to pass the test.
Session four thankfully we were allowed to talk to our pairing partner, but had restrictions of not being allowed to use conditional or loop statements to solve the problem, this was just the kind of challenge that had a lot of the developers at the event still thinking about how to do it as they were leaving at the end of the day!
In session five we were exploring the Kent Beck idea of Test and Commit or Revert so if our tests failed for any reason we had to revert to the last successfully passing code, and delete what we had written since then. This is to encourage you to make small simple changes and always have working code rather than perform big changes that could potentially break.
The final session, Barney who along with Natalia was leading the event told us that since it was the last of the day we could do what we liked, resulting in lots of experimental solutions. However it was a ruse, and after twenty minutes he swapped the pairs around, and then five minutes later swapped development environments handing us all what was effectively legacy code with non-working tests, and in some cases in a language the new developers were not familiar with. The retrospective after that was interesting with all of us trying to explain where we were going with the half finished solutions that were passed on to others.
So was it worth giving up a Saturday for? Absolutely. Usually as developers we’re trying to learn development techniques at the same time as doing our actual work, it’s rare we get a chance to actually just look at how we code, and as such it was a really worthwhile day. Canon Medical Research actually ran the day twice, firstly on the Friday when it was mainly their own developers who attended, and then the public session on the Saturday. Certainly if you have a large number of developers it is the kind of event that could be run internally if you’re wanting to improve the coding skills of your team.
Last Friday was the annual CodeCraftUK conference in Glasgow. As a community organised event it’s much more reasonably priced than some of the bigger conferences, and also has a rather different structure than most of the big conferences – as they say on the conference website they put the confer back in conference, focusing instead on small group guided conversations and workshops. Since we have a daily parliamentary train that uses the Winchburgh Loop to go direct to Glasgow from Kinghorn that could get me to Glasgow in time for the conference I picked up a ticket and headed along.
Before talking about the conference, a little side note about the parliamentary train, if it ran more frequently it would actually be a pretty useful service – there have been a couple of occasions we’ve been going to events in Glasgow when there were events on in Edinburgh, in particular big games at Murrayfield when the normal route of taking a train to Haymarket and changing to the express service just really wasn’t attractive and we opted to drive instead. If there was a regular service direct to Glasgow, we would have used the train instead. Even with the meandering route the train takes, especially with the evening service there is little difference between the time it takes to go direct and going into Edinburgh and changing to a faster train.
Anyway, on to the conference.
First off I opted for a guided conversation entitled “What lies behind a job title?”. We had a mix of people in the group from recent graduates through to people like me who have been in the industry for years, and also from a variety of company sizes.
The conversation certainly confirmed my experiences over the years aren’t unusual. We had examples of companies, often the bigger ones, with rigid hierarchies, where teams had to have certain numbers of senior, mid-level and junior staff. There were also the common frustrations of junior engineers who were told what to do by seniors, seniors who only seemed to have reached that level by way of being in the role for a long period and so on. However there were other examples of more progressive companies that had done away with levels in job titles as they were regarded as unhelpful. There were also so amusing anecdotes such as a participant who is CTO in a small start up and said there was only one other developer, but that he used the CTO title because it opened more doors talking to larger organisations.
I along with quite a few others from the first conversation moved onto a second session called “Being T-Shaped”.
One interesting take away of the T-Shaped session is that there is not a single understanding of what being T-Shaped actually is. One explanation came from Agile Kanban practice where software development is split into a number of stages, the T shape idea coming from team members having most experience in one of the stages, but able to work on tasks in other stages if need be. However other participants in the conversation took T-Shaped as having a breadth of soft skills alongside deep technical skills in a particular area. Again this was a conversation that showed the differences within big and small organisations. In smaller organisations being T-Shaped is almost a requirement of the role, whilst in larger organisations there is sometimes push back against people working in roles that are not their own.
After the next break I opted for one of the workshop sessions, an hour crash course introduction to Machine Learning.
Machine Learning is one of the current buzzwords that sales and marketing people love to use, with really very little comprehension of what the term actually means – certainly they often confuse artificial intelligence in their products with genuine machine learning (check out this Wired article for an explanation of the difference). For what was necessarily a brief introduction we built a Jupiter Notebook that used freely available dataset, python code and TensorFlow to recognise handwritten numbers. This used a neural net that was trained with 60,000 handwritten numbers, and across the participants in the workshop achieved between 98% and 99% accuracy when given a second testing dataset of 10,000 numbers. Although it was a whistle-stop tour, we did get something working at the end of the session to take away, plus some recommendations of where to go to learn more.
For the afternoon sessions I went along to a two part workshop on Mob Programming, given by somebody who has been leading a team working entirely as a mob for the last five years. Alongside giving us experience of working in a mob to accomplish a task, he was also able to give some good advice.
For the first part of the workshop we were in a mob of six, and given the simple task of programming FizzBuzz, but using C++, a language quite deliberately picked because most of us would have little experience of it. We also followed the recommended practice for teams starting mobbing of having a driver who was at the computer and only allowed to do what they were told, a navigator who was doing the thinking, and the rest of the team not allowed to talk until their turn as navigator. We then rotated around the roles in order every two minutes and forty-five seconds. The rest of the mob not talking proved to be the critical practice, in that otherwise the mob has the potential to be dominated by some of the group, having the navigator being the only person allowed to speak ensures even quiet members of the team have a chance to speak and participate.
One of the key criticisms of mob programming, almost always from people who have never tried it, is that having the entire team working on a single task is slower than having everybody working on different parts of the problem. The workshop leader however is working at a company with two similarly sized and experienced teams, working on the same codebase, but one using mob programming, the other team using more traditional methods. Bearing in mind the usual caveats about it not being a good way to compare two different teams, the teams are working at a similar velocity, and certainly the company has not had reason to ask one or other team to change because of any significant difference in the work delivered.
The arguments in favour of it are an extension of the arguments in favour of pair programming in that the team is reviewing code as it is written, rather than the write, review, repeat loop that more traditional methods produce. It is also a good technique for ensuring consistent knowledge sharing amongst team members, and is an excellent way to on board new team members as they are just brought into the mob right from the start.
Certainly as an introduction to mobbing it certainly showed me what a beneficial technique it can be. There were several participants in the second part of the workshop who had been mobbing themselves for a while, and some who had been skeptical when their teams had started. All of them found the technique useful, and many said they would struggle to go back to not working in a mob. It’s not for everybody however, another participant in the workshop mentioned a team member who despite being an excellent part of a mob decided it wasn’t for them and swapped to a team who weren’t using the technique. On the basis of the workshop it’s certainly something I’d like to try in a team.
So was it worth the trip across to Glasgow? Definitely – and I’ll certainly be looking out for the conference next year. I probably learnt more from the workshop sessions, and the mob programming was especially beneficial, but the guided conversations were also good too. When you compare the cost of the bigger conferences, which sometimes veer towards being sales pitches for particular products, the conference is a bit of a bargain, and a great opportunity to learn from each other, not just from whoever is giving a particular presentation.
After a busy July, finally I’ve got a quiet moment to catch up with my notes from the recent Develop Developer Developer event held at Microsoft HQ in Reading.
I attended a real mix of sessions this year. First up was a real mind bending session led by Francess Tibble and Anita Ramanan, two software engineers at Microsoft talking about Quantum Computing and the Q# language. The session was split into two parts, the first a bit of a crash course in the physics involved in Quantum Computing, with quite a bit of maths too. The interesting take away is that present day quantum computers are expensive and unstable as they are particularly sensitive to external factors so can lose state in seconds. As a result we currently have the Quantum Development Kit that simulates how a real quantum computer should behave.
The key difference with a quantum computer is in the bit, in classical computing the bit is ether 0 or 1, but in quantum computing the bit can also be any point in between, taking the usual light bulb analogy for a classic bit, it’s like having a dimmer attached. I really haven’t got the space to cover all their content in detail, but they did do a version of the same talk a few days before DDD which is online on YouTube.
Moving on I then attended Joseph Woodward talking about Web Assembly, Blazor and the Future of Web Development.
Joseph started with a run through of the history of web development, and the perennial problem that whilst there has been a relentless move towards providing applications in a web browser, the tools to create rich applications in a web browser are really pretty limited. JavaScript, the main language of the web has become that largely by historical accident, and is pretty slow. Web Assembly is the latest of a number of attempts to replace JavaScript as the language of the web, in this case providing what is effectively a low-level byte code for the web and then compiling other languages into this byte code. At this stage it’s still very much a minimum viable product, but does seem to show some promise with multiple languages being able to compile into Web Assembly byte code.
For C# and other .Net support, since they also compile into the intermediate language of the .Net platform, Microsoft offers Blazor, which is a full .Net machine written in Web Assembly byte code. This of course does mean that .Net intermediate language is then being interpreted into Web Assembly byte code, so there are plans to compile to avoid this double layer of interpretation.
The actual coding is familiar to any C# programmers with familiar dependency injection, and the ability to pull in code using Nuget. Interop with JavaScript is provided, and is necessary because Web Assembly does not provide access to the DOM.
It was clear from the talk that the platform is still immature, it lacks performance and has no threading or garbage collection. However it does show promise. Even if it doesn’t provide a replacement for JavaScript, it does allow us to mix and match languages picking the language that is best suited for a particular task.
Next was what for many people was one of the big draws for this years DDD, the return of Barry Dorrans, now .NET Security Curmudgeon at Microsoft, but who before joining Microsoft and moving across the pond had been a regular speaker on security at developer events. Barry was presenting his Code Behind the Vulnerability session, variations of which he has presented for a number of years at conferences around the world. The great advantage of presenting it here however is that it allowed developers who don’t work for companies with the budgets to send their developers to paid for conferences to see this important session. Indeed Robert Hogg CEO of Black Marble who organise the DDD event at Microsoft considered the subject matter so important that he said to any of his developers in the room that they’d be fired if they did anything that Barry had spoken about!
The purpose behind the Code Behind the Vulnerability session is basically to go through security issues that Microsoft have found in their code, and the cause so other developers don’t make the same mistakes. Barry updates this session periodically as new exploits and problems come to light, so it is well worth keeping an eye out online for new versions.
Barry covered eight different security advisories, including hash tables that could bring a system down if they received specific user data – the tip here being not to use user supplied data as keys for a hash table, exposed endpoints that allowed users to work out encrypted messages, and a number of occasions where people had turned off or misused features making security holes, for example turning off signing on view state allowing attackers to create .NET objects, or simply writing a GET API call that changes state.
Barry’s summary slide is the basics, but the whole slide deck is worth a read. His summary is:
– Sign your data, even when it is encrypted
– Don’t use regular expressions
– Don’t use BinaryFormatter
– Don’t overbind in MVC
– Use the right HTTP verb
– Validate your inputs
Barry’s session is a critical one for anybody doing .NET development, many of the issues he shows are easy to make, but can have catastrophic consequences.
Dylan starts off with talking about how news of his birth and a first picture made it from where he was born in Africa, back to his grandparents back in Oxfordshire – a process that took weeks. He then looks at technology today where we can get a photo appear on a phone in your pocket and respond immediately. In the space of his lifetime the way we communicate has fundamentally changed. His session goes through the basic technology that underpins these changes, and is absolutely fascinating.
This was probably my favourite session of the day as it covers so many different areas of technology. It was also presented in an easy to digest way, and in a way that I’ve been able to show it to my children and they can start to understand all sorts of technological ideas.
My final session was one of those I picked more because I enjoyed the speaker – Gary Short talking about AI Dev-ops. Gary started looking at how the principles that have brought about dev-ops can be applied to AI and machine learning work, for much the same reasons. There has always been a big disconnect between data scientists and coders. Data scientists have a very niche skillset, so in the past they would do the specialist work, and then hand their carefully designed models to developer to implement. However tools are now being produced that allow data scientists to develop an implement their models, and coders to just connect to these rather than implement them.
Gary also had some useful tips, he highlighted that you can only optimise algorithms for false positives, or false negatives, not both, so it is a business decision as to which costs more, false positives or false negatives. This is a useful tip with regards to our products at FISCAL as we have a continual tension between reducing the number of false positives we produce, whilst not missing results, i.e. a false negative.
In summary DDD 2018 was a good day, and well worth spending a Saturday. For many developers there isn’t the budget to go to paid conferences regularly, so it is particularly good to be able to see sessions from those conferences presented live at a free community conference. Particularly for sessions like Barry’s important information about how to code securely is something all developers should be hearing, not just the ones who work for a company with a good training and conference budget!
Thoughts from, and the lives of a Canadian and a Brit living in the UK.