Sam Starling
7 April 2015

Learnings: January – March

I’ve done a lot in the first few months of this year. I left my job at the BBC, I moved to Berlin, and I started working for SoundCloud.

I recently spotted @KingScooty‘s monthly retrospective article, and thought I’d steal the idea for myself and use it as a way of not forgetting some of the interesting code-related stuff I’ve come across recently.

IDLs

In a world of microservices, your application might do something like this: parse a request, do some work, and send a reply. At either end, you’ll be spending some time and effort (computationally) to convert between objects and the representation of your choice – JSON, for example.

If you have lots of small services in a chain, then you’ll be repeating this cycle of parsing and serializing at each step, which starts to look inefficient.

Interface description languages (IDLs) allow us to model interfaces in a way that is language-independent. For example, Google’s protobuf will allow you to create a .proto file, which you can then compile into class files for the language of your choice. Clients exist for Java, Scala, Go, Ruby, and a variety of other languages.

Objects that are created can be serialized in a binary format, transmitted over the wire, and deserialized at the other end. The reduced size of messages provides an advantage, and the binary format makes the conversion back into an object much quicker. But, the messages themselves then become more difficult to inspect – you can’t inspect them as you would with JSON or XML.

I haven’t sat down and had a proper look at either protobuf or Apache Thrift yet, so I don’t have much of a handle on the benefits and drawbacks, but IDLs are definitely worthy of a separate post.

Microservices

Everyone’s favourite buzzword at the moment. SoundCloud are pretty big on microservices, and the team I’m on are starting to spin out parts of our large Rails monolith into smaller, separate services. Whilst I’m familiar with the concept, I’m starting to learn more about the practicalities.

To me, complexity seems to increase with the number of microservices. Sometimes it’s difficult to see exactly where failures are coming from, or what is causing them. Zipkin can trace individual requests through the network, and is a useful diagnostic tool. Prometheus, an open-source monitoring tool produced by SoundCloud, also gives us a clearer view of which services are doing what.

Further Reading

Here are some other articles and videos I’ve read and watched recently, which don’t quite fit in above: