Soundbyte 150: Cloudy with a chance of data


Recently I found a fun analysis that compared the vocabulary of Shakespeare with modern day hip-hop artists. While my musical taste shifted towards ‘lots of guitars’ and away from hip-hop somewhere during high school (yes that was my thing for a while), the data is quite interesting. It shows that some of the hip-hop artist have a much larger vocabulary than Shakespeare. Not the most useful analysis every published, but it certainly put a smile on my face while reading this over breakfast.

The obvious music choice after this introduction would be something like Wu Tang’s C.R.E.A.M., but I still have an image to keep. Ok probably not, but still, let’s stay with guitars at least ;-)

I stumbled on the Virginmarys recently, which I started to listen to a lot since.

Two weeks ago Google hosted their IO conference. While I was not attending, I did enjoy the keynote and some sessions on the couch with a tablet and beer. Besides having a good laugh at the protestors during the keynote that started shouting “You all work for a totalitarian company that builds machines that kill people!”, there were quite a lot interesting announcements. Of course there is a large Android update coming up that looks really good, and the watches that finally made me actually want one. The main reason I finally start to be interested in smart watches is that during my last travels I was truly impressed by Google Cards. They actually manage to give relevant info at the times you need it. And yes, this is both useful and scary. If we accept the scary part for now (apparently I’m more practical than paranoid) I can see the use of having this information available on your wrist instead of checking a phone all the time.

Anyway, Android and watches are nice but I was more interested in the updates on Google’s cloud offerings. There are two things in particular that I find interesting; Google DataFlow and their move towards Docker. DataFlow is the successor of MapReduce and will be available as a service. You basically stream data from any source into it, and perform analysis and transformations on the data using relatively friendly APIs. On first sight it looks like a hosted version of something similar to Spark or Storm, which could be very useful. Data analysis like the one in the introduction of this post should be easy to implement based on a service like DataFlow.

The other cloud related news is the move towards Docker and open sourcing some of their related tools like Kubernetes. Although they used similar technology internally already, it now becomes available to users outside of Google as well. Technology like this (RedHat is on a roll as well) brings PaaS without the usual downsides of lock-in and all kind of limitations closer and closer. In PulseOn we gained a lot of experience with hosting an application on auto-scalable and auto-recoverable clusters in the past few years, and tools like this will make that sort of setups easier to create, test and move to different environments.

Jigsaw apocalypse

Last week an updated was posted about Jigsaw; the zombie Java specification that keeps returning from death. We seem to be back at square one again, with a proposal for both a modularized JDK (good thing) and a modularity solution focussed at application developers (bad thing). In an ideal situation I would obviously be in favor of having modularity constructs at the language level, but with the current constraints of backwards compatibility etc. we will get a half baked solution at best, which will not really help anyone. Maybe we should just all switch to something hip and cool like Node.js than! Well! Maybe not.

The fall of the NodeJS empire

Recently it seems like more and more developers are moving away from Node.js again. I never really understood the move towards server-side JavaScript in the first place, so this seems to be another good opportunity for a rant about server-side JavaScript. While it’s always good to keep a very open mind for new technology and drastically different ideas; Node seems to be based mostly based on the utopia that everything becomes easier once you just ignore problems. Yes, you can write your web server in a few lines of code, and yes we do so without even compiling once. But does that really prepare us for creating large applications? Of course not, there is a lot more too that, no matter which language or framework you use. The exact same thing happened several years back when everyone jumped on top of Ruby on Rails.

So why are those frameworks so appealing in the first place? The main reason is probably that getting started is easy, and getting the feeling that you’re making progress immediately is very satisfying. There is really no reason this can’t be the case in our familiar (Java) environment however. Slow (Maven) rebuilds and redeploys are unacceptable, but haven’t anything to do with the language. On a side note, I think that we have a very competitive stack with Amdatu when it’s comes to this aspect; easy to get started, hot code reloading for a scripting like development experience, standard components to solve common problems, but also an architecture that scales to large and complex systems.

In that aspect I’m also extremely happy with Java 8, specifically the addition of Lambas. A lot of code just becomes a lot more easy to read when using a more functional style of programming. If flatMap, filter and reduce still make your brain explode there is an excellent interactive tutorial as part of the RxJS. The first half of the tutorial is about basic functional concepts that apply to every (partly) functional language which includes Java 8, JavaScript and of course languages such as Scala. While you’re at it, finish the tutorial to the end, and you’ll have a pretty good idea about the Reactive Extensions as well, which is pretty awesome. And you can use all of this within Java and OSGi as well thanks to RxJava.

Bootstrap API

For the last few weeks we have had many discussions about the API direction that we want to take for Amdatu Bootstrap. For those who forgot; Amdatu Bootstrap is a command line tool to simplify OSGi development by automating common tasks.C heck out a video of Bootstrap in action here.

The most important aspect of Bootstrap is that it is very easy to create your own plugins, so that an ecosystem of re-usable plugins will grow. Although we didn’t reach a 1.0 version yet; the ideas seem to work out nicely. The tool is picked up quite a lot of developers and feedback has been great.

With the growing number of people looking at the, the number of ideas about how to use the tool increase as well. Besides just a command line tool we also want scripting and Bndtools integration. This means we have to come up with APIs that work for all these different scenarios, while the API should still be simple enough to easily pick up plugin development. This is a game of trade-offs, which is even more difficult when it’s about a public API. Prototyping different approaches is an important aspect of these discussions; it turns out that most ideas are great, just until you implement and start using them ;-) This again shows the need for both whiteboard discussions and early implementation. Up-front design is never going to give you the insights that an implementation is giving; while jumping into coding immediately will get your thought process stuck in a particular direction. We need both, and shouldn’t be afraid to throw away work once we learned from an experiment; it’s the only way to improve.

In any case; we’re making good progress but can always use more feedback. If you’re using OSGi, give it a spin.

i18n of Luminis Technologies

We already knew that Luminis Technologies often does things slightly different by, for example, not having an office. This month we hired our first non-Dutch member: Dan Bar-Yakoov, living in Tel Aviv. We work with people from all over the world in the open source projects that we are involved in, and that works great, so why limit ourselves to finding team members in the Netherlands? Of course this introduces some new challenges as well, but at least I’m excited to see how that plays out. And of course: Welcome Dan!

With Amdatu related talks recently at JavaLand, ApacheCon, DevNation, GeeCon, OSGi DevCon, Code Motion and JavaOne and JMaghreb coming up later this year our international visibility keeps growing as well.