Many professions are based on the idea that you do a big amount of learning upfront (university) and then only small updates will be necessary throughout the rest of your career. However, in software development technology advances so quickly that your whole career is really one of continuous learning. To further this
end, MOOCs (Massive Open Online Courses) have been become popular, especially over the past year or two. I don't see them as replacing traditional universities. However, they are a very useful resource, both for reminding yourself what you learnt in university and introducing you to new paradigms as they become popular
in your field of expertise.
A great example is the Functional Programming Principles in Scala course at Coursera. Coursera is based at Stanford University, and this particular course is led by the Swiss professor Martin Odersky, inventor of the Scala language. I am now on the fourth week of the
seven-week course. Each week requires the student to watch a couple of hours of videos interspersed with quizzes, and then take some more serious exercises, which take several hours more and submitted online using the Scala tool sbt. A student can do this work at any time during the week, meaning there doesn't need to be any interruption to one's regular work schedule. At the end of the course, a certificate is available, as well as the total score on the exercises.
In practise the majority of students drop out of online courses soon after starting them. However, I fully intend to pursue this one to the end, as I think the material is both excellently presented and highly relevant to the direction the software world is moving in. Furthermore, completing the exercises feels like solving puzzles in a way that I found quite addictive, and I shall be sorry when the course is complete and there are no more.
For instance this week's lectures, on "Types and Pattern Matching," set the stage for exercises on creating Huffman trees, which was quite interesting. In this course, there is an emphasis on data structures and algorithms that reminds of my days studying Computer Science at university twenty years ago. In those days we learnt Scheme, a LISP variant and a classic functional programming language. These days there are more popular FP languages like Scala and Clojure, which are entering the mainstream for many good reasons including allowing access to the JVM and Java libraries.
Anyway I highly recommend the course. This is the second run of it and I expect there will be another chance for people to take it in a few months.
Sunday, 21 April 2013
Tuesday, 16 October 2012
Load Testing with Gatling
Recently, I've been working with an organization that had a huge increase in traffic due to mobile phones, and needed to determine how its legacy application would handle the millions of users flooding into its servers. They were accustomed to using JMeter, as it's a full-featured mature Java application for load testing and also useful for functional testing.
However, it turns out that when the numbers get really big, you have to figure out how to deal with the load of JMeter itself! You end up distributing the JMeter application across several servers. Surely this shouldn't be necessary? Also, because it's so UI-oriented, the "scripts" you use for testing are big clunky XML files. You might prefer writing them in a modern programming language, giving you the option to integrate them into a general network management system.
Enter Gatling. This tool is based on the modern Akka framework, so it has much less overhead than JMeter. In fact, I was able to run it on a single machine all the time. Furthermore, since the scripts you write are in Scala, you can make it as easy or complex as the needs of the organization warrant. A simple script (or "simulation") looks like this:
Run gatling.sh, and choose the script you want to run. After it finishes Gatling gives you a HTML page filled with graphs. Here is an example. This graph is actually showing a problem with the application under the test. The bumps in the red line mean that connections are being dropped from time to time.
In summary, Gatling is not as fancy as JMeter yet, but I think this is the future of load testing.
However, it turns out that when the numbers get really big, you have to figure out how to deal with the load of JMeter itself! You end up distributing the JMeter application across several servers. Surely this shouldn't be necessary? Also, because it's so UI-oriented, the "scripts" you use for testing are big clunky XML files. You might prefer writing them in a modern programming language, giving you the option to integrate them into a general network management system.
Enter Gatling. This tool is based on the modern Akka framework, so it has much less overhead than JMeter. In fact, I was able to run it on a single machine all the time. Furthermore, since the scripts you write are in Scala, you can make it as easy or complex as the needs of the organization warrant. A simple script (or "simulation") looks like this:
val myScenario = scenario("My Sequence Of Actions")
.feed(csv("feed_of_users.csv"))
.exec(http("Confirm Newspaper Subscription")
.get("/myapp/hasSubscription?userId=" + "${userId}")
.check(status.is(200))
.check(regex("""<response status='ok' """).exists))
Run gatling.sh, and choose the script you want to run. After it finishes Gatling gives you a HTML page filled with graphs. Here is an example. This graph is actually showing a problem with the application under the test. The bumps in the red line mean that connections are being dropped from time to time.
In summary, Gatling is not as fancy as JMeter yet, but I think this is the future of load testing.
Sunday, 30 September 2012
Keywords as Clutter: Doing More with Less
When I first started writing Scala, I found it very easy to code in a Java-like
way. Perhaps a little too easy in fact. The problem is that if you persist in writing
code in an imperative style, you miss a lot of the FP (functional) advantages that
Scala provides.
For example, consider a simple for loop in Java, the unenhanced version where you need the index:
It is easy to transliterate this into Scala like this:
This is OK. However, a more idiomatic way is to use a foreach function:
This is functional, rather than procedural. People will have various reasons for preferring this style. But the interesting thing to me is that foreach is not a Scala keyword like for. It is just another function. Code is easier to compose when you get rid of the magic of keywords. You can do more with less. Another magic keyword is if. Again, Scala allows you to use it, and sometimes you need it. However, if you think about it a little, very often you find that using a Scala idiom is better. For example, it is tempting to translate a Java null check to something like this:
However this check on the functionResult Option can be a bit tedious. The same thing can be written like this:
This may look a bit odd if functionResult is only a single value. However, by writing it in this way you have succeeded in moving the boring check down into the functionResult Option implementation. This is especially useful when you have a lot of such "null checks" and wish to compose them. You don't want the code to become spaghetti-like with if statements. Here is a classic cheat sheet about dealing with Option's in a functional way. A different way of getting rid of if is to use pattern matching. Here is some Scala written in a Java-like way:
Better is the Scala idiom:
Admittedly, match and case are still keywords. But in this case, it's probably worth it. Unlike the Java case label, Scala's case clause can take, not just an int or a String, but anything, including nested structures. That is the power of pattern matching. I am collecting Scala idioms, so if you can think of other good examples, especially ones that do away with keywords, add a comment.
For example, consider a simple for loop in Java, the unenhanced version where you need the index:
for (int i = 1; i <= 10; i++) {
System.out.println("I like the number " + i);
}
It is easy to transliterate this into Scala like this:
for (i <- 1 to 10)
println("I like the number " + i)
This is OK. However, a more idiomatic way is to use a foreach function:
Range(1, 10).foreach(println("I like the number " + i))
This is functional, rather than procedural. People will have various reasons for preferring this style. But the interesting thing to me is that foreach is not a Scala keyword like for. It is just another function. Code is easier to compose when you get rid of the magic of keywords. You can do more with less. Another magic keyword is if. Again, Scala allows you to use it, and sometimes you need it. However, if you think about it a little, very often you find that using a Scala idiom is better. For example, it is tempting to translate a Java null check to something like this:
if (functionResult.isDefined) // dubious style doMyProcessing(functionResult.get)
However this check on the functionResult Option can be a bit tedious. The same thing can be written like this:
functionResult.foreach { doMyProcessing(_) }
This may look a bit odd if functionResult is only a single value. However, by writing it in this way you have succeeded in moving the boring check down into the functionResult Option implementation. This is especially useful when you have a lot of such "null checks" and wish to compose them. You don't want the code to become spaghetti-like with if statements. Here is a classic cheat sheet about dealing with Option's in a functional way. A different way of getting rid of if is to use pattern matching. Here is some Scala written in a Java-like way:
if (actorMessage == "toast_warm")
pop()
else if (actorMessage == "toast_cold")
toastSomeMore()
else
println("Who moved my toast?")
Better is the Scala idiom:
actorMessage match {
case "toast_warm" => pop()
case "toast_cold" => toastSomeMore()
case _ => println("Who moved my toast?")
}
Admittedly, match and case are still keywords. But in this case, it's probably worth it. Unlike the Java case label, Scala's case clause can take, not just an int or a String, but anything, including nested structures. That is the power of pattern matching. I am collecting Scala idioms, so if you can think of other good examples, especially ones that do away with keywords, add a comment.
Subscribe to:
Posts (Atom)