In his (perhaps successful) attempt to sink the BGGA proposal, Captain Bloch send the closuralists packing to Scala island in the Swiss Sea and promised milk(CICE) and honey(ARM) for the rest.

Besides some remaining choices of freedom a ARM block will look somehow like this:

do (InputStream in   = new FileInputStream(src);
    OutputStream out = new FileOutputStream(dest)) {
  byte[] buf = new byte[1024];
  int n;
  while ((n = in.read(buf)) >= 0)
    out.write(buf, 0, n);
}

I thought this should be pretty easy to imitate in Scala as well. Ideally I’d like to have

with (fi<-  new FileInputStream(src);
      fo<- new FileOutputStream(dest)) {
  val buf = new Array[byte](1024);
  def cp() {
    val n = fi.read(buf)
    if (n>0) {
      fo.write(buf,0,n)
      cp
   }
  }
  cp
}

But writing a “with” is not possible - the 2nd idea was to have a with where the resource is “this”:

with (new DbConnection) {
  executeUpdate("delete from scratch")
  commit()
}

As Jamie Webb pointed out a simple wrapper is easy to write and might do the job:

class ManagedResource[T<:{def close()}] (resource:T) {
  def foreach(f : T => Unit) : Unit =
    try {
      f(resource)
    } finally {
      resource.close()
    }
  }
}

and so it is easy to write:

for(pi<-new ManagedResource(new PipedInputStream);
    po<-new ManagedResource(new PipedOutputStream(pi))) {
  po.write('A');
  Console.println((pi.read).asInstanceOf[Char]);
}

So interesting part it: How does this magic work. On Scala island everybody knows swiss-army knives. We usually use “for”. The code above is equivalent to

for(pi<-new ManagedResource(new PipedInputStream)){
    for(po<-new ManagedResource(new PipedOutputStream(pi))) {
  po.write('A');
  Console.println((pi.read).asInstanceOf[Char]);
    }
}

and each block (or closure) is passed to foreach:

new ManagedResource(new PipedInputStream).foreach(pi=>
  new  ManagedResource(new PipedOutputStream(pi)).foreach(po=>{
     po.write('A'); //where is po defined?
     Console.println((pi.read).asInstanceOf[Char]);
   }
  )
)

Here you see the for-magic in action. For defines some nice symbols pi and po for us that make writeing the function much easier than wrapping it all up in nested anonymous functions.

As the expansion of for shows we are getting the nest of resources as we would get if we would write everything by hand. Less visible is the cleanup. Note that the argument to each foreach is a closure. This means th code is excecute where it is writen and not evaluated and then passed to foreach. By this the finally block in foreach executes two times:

1. After the print
2. After po had been closed

So far so simple, but why some might ask can I refer to pi and po as PipedInput/Outputstreams instead of ManagedResource[T]s?

The reason is that “for” is just some compiler magic, what eventually gets executed is the foreach which takes a function T=>Unit (i.e. a funtion with a single argument of type T and return value void). T here is either PipedInputStream or PipedOutputstream and the pi and po are type to accommondate for that.

In practice it is a bit more complicated to define ManagedResource because there are three different possible expansions (via map and flatMap). We investigated as well lazy resource acquisition, but this might open some other wholes, but it is possible to write ManagedResources that aquire the underlying resource iff it is accessed. You opening a ctor to the application developer is too dangerours as inadvertedly an open resource might get passed to the ManagedResource without that it can tell that it got already opened.

Resources with explicit open methods as Josh mentioned in his text would solve this problem easily with no overhead for the application developer. An alternative are Linear Types, but that doesn’t go well together with a language like Scala.

Josh Bloch’s talk at Javapolis caused quite a stir in the blogosphere, why is best summed up in a blogpost and its comments: Is Java a simple, less skill demanding language or is it - better can it be - also a home for programming geeks.There are two sides and a point can be made for both of them. As java had been need it was a geeky language - admittedly fueled by the internet hype - which attracted brillant minds. Thanks to these forerunners Java is now one of the top choices for mainstream programming. The ones of these who still stick around in the Java world now look with envy on Ruby, Scala and sometimes back on Smalltalk and want to use the ideas and techniques prevalent there in Java as well. But in practice - unless you happen to work at Google - most java programmers are not of this class. Big application written in Java are written and maintained by ordinary developers, not übergeeks that think in monads.The typical Java programmer in a large enterprise is more of the type of the VB programmer, perhaps he did some C or COBOL before. These people and their thinking about programming is deeply rooted in imperative style.A point could - and personally I think it should - be made if projects really require such large staff and if it weren’t better to use more advanced stuff to build it with fewer, but appropriately skilled developers (this is the Ruby claim), but face it, the enterprise world changes too slowly to accomondate for that.More than 10 years ago we evaluated some new development environments as a replacement for Oracle Forms 3. The most interesting candidates had been Forms 4.5 (obviously), Smalltalk (I don’t remember the vendor) and Franz Allegro CLOS (Ok, this had been the one I fancied). Java had been 1.1 or so in these days and the first looks at it were disappointing, C++ had been already in use, but there had been so many problems with it that it wasn’t ever really a consideration. Eventually we went for Forms because it would give our staff the easiest transistion. The main problem with Smalltalk and CLOS had been the paradigm shift. Neither the pure OO of Smalltalk nor the functional programming in Lisp were accessible to them.I think that the same could happen to Java if all the fancy functional programming (I love it!) would get into it. There are three problems

• FP is not compatible with the mindset of the majority of enterprise programmers. Also FP programs are harder to debug - this is a problem for the prevalent trail-and-error programmer (aka debugging into existence)
• Java is not Scala, one problem of BGGA is the non-local return. This isn’t an issue in true FP languages which are expression oriented, the statement oriented Java makes it awkward to deal with it
• DSLs are in fashion and a powerful concept for layering a complex application, but they create as well new languages: You can’t hire a Lisp or C++ programmer and expect him to be immediately productive as they have to learn the DSLs of your projects first

I initially found CICE (which Bloch promotes) to come a bit short, but now I am convinced that this is the right way to change Java as it keeps the characteristics of the language and can easily integrate into larger teams in any organization.This is the curse of being a mainstream language, you most not let out you users in the cold.For the other classes of problems languages like Scala that run on the JVM could be chosen. There some research has to be done by the system architect(s) how such code can integrate with Java code and how maintenance can be organized. This isn’t a trivial problem as not everything from another JVM language integrates seemlessly with Java code (i.e. you use this code as if it was written in Java), as well the question has to be answered (in each indiviual organization) if there is enough stability in the project to support an other language throughout the life-cycle - imagine a freelancer writing some core code in a language only known by him.

Original problem: You have a simple hierarchy, class B extends A. Now your simple program needs customization for a client. You create CA extends A. Now should B extend CA or A?

This was the simple part. Now we expand our business to serve martians. Lets extend A by M. But the mars is a region and for customers on mars we’ll have MA. You get the idea. In terms of product line management it gets messy when juggling with these inheritances. Sooner or later it won’t work anymore - hopefully soon enough before millions of lines of code had been duplicated.

OK, it boils down again to prefer composition (by delegation) over inheritance. But it is ugly and hard to maintain. In most case the concept of Traits is sufficient. Scala is using traits extensively and the syntax is vey nice. I looked for a way to achieve something similar in Java. The result is my first code.gogle.com project: Open-Traits.

Here in short how it works. We have some class Cx implementing interface U,V,W,X. We can factor now a traited object by:

  Traits ts = new Traits()
    .with(new CU())
    .with (new CVW())
    .with(new CUX());
  Object traited = t.traited();

The object traited is a Proxy which does all the plumbing for us and we can simply downcast it to:

  U u=(U)traited;
  V v=(V)traited;
  W w=(W) traited;
  X x=(X) traited;

It would be nice to define a type “Vuxw” that extends all these interfaces, but the current implementation doesn’t support that - more a choice on my behalf than some impossible*

Of course our traits CU,CVW,CUX have to obey some rules, if they all inherit Trait the above could be written as

  Traits ts = new Traits();
  new CU(ts);
  new CVW(ts);
  new CUX(ts);

There are many other things that can be done (and things that can’t be done) - check them out in the API doc available on the project site.

Something very special are trait-super and trait-virtual methods. If CU contains a call to a metod defined by U it is possible that our Traits can virtualize it and make CUX’s implementation being called, conversely CUX can call “super.u()” (in Open-Traits overriden().u()) to call the method defined in CU. Pretty nifty, isn’t it?

It gets even better : If we kept a reference to the new CUX instance, cux we could do:

  ts.remove(cux);
  ts.with(new MyCUX());

This means we can change (in some save limits) the traits composition. This is a great tool to create Adapters and Visitors which might come handy when writing interfacing modules.

I am aware that using another language would be more elegant and more efficient, but let us not get too greedy. It took quite a time to convince people that valuable software can be written in other languages than COBOL .

*Fixed Traits

I am planning to write an alternative to Traits, FixedTraits. A fixed trait will be a generic class of type T which is instantiated as

new FixedTraits<U,V,W,X> where the method traited() has the type <T extends U,V,W,X>. This would save the casts to the result of traited and is thus more type-safe. But it is as well much uglier at other parts. As well the dynamic add/removal will pperhaps be lost. It would be much easier if two things existed in Java had Interface compositions. BTW: Scala solved this problem elegantly: You never declare it, you rely on typ-inference

Monads are usually associated with being something highly theoretical that nobody really understands - unless you either worked with category theory or came to Haskell via another route. An elephant in some sense - big grey matter…

James Iry provides a Monad introduction for Scala programmers, he leaves out most of the what a monad is, but explains more pragmatically what the qualities of monads are. So far part 1, 2, 3 and 4 (of a series of 5?) are available on his blog.

Give it a read, you might learn that you had been already using monads without knowing and why this perfectly nice Java code is hard to understand for other programmers. If this is the case give this intro to those programmers, in any other case learn it yourself.

(updated 15.11.2007)

Already a bit late hour, but after a nice couscous some things become clearer.

Currently I feel quite a pressure. After more than 12 years in this monkey business I have learned to ignore it - “people under pressure won’t think faster” (oracle@lister.com)

What was more important is a sudden shift of my manager. He became quite nitty-gritty and micro-manages some developers (we have “team-leaders” for that:-). This means the pressure he feels must be enormous.

I had been completing two important functions in the last days - both had been properly prepared and well - though not formally - specified. Implementing them had been easy. Two issue of highest priority and severeness resolved. Normally something that makes a manager’s face shine.

But a bit to my surprise he had been mildly angry with me. His point was that working on another issue was more important as the testing was more complicated. During the afternoon I grabbed together what was going on. It is true that we don’t even have a plan to test it, three parties are involved in the development, the specification is vague as the functional architect hasn’t barely any knowledge about the underlying technology. To make it worse the three parties work completely disconnected have not even a process to synchronize their work using a common code repository (each part is build and deployed independently!). I have advanced my work to everything that can be tested at the moment and I was reluctant to write anything that is either untestable of would stop me from completing over parts. Instinctively I stopped working on the project and turned to something more fruitful.

I had been chewing all evening long if I had been acting correctly. What had been the alternative? Instead of providing real value I had been writing junk code, perhaps got lost in an architectural beauty that just camouflaged my lack of understanding of the problem - I know myself: When I start to think deeply about a design, despite my experience, it is a warning sign that I actually didn’t understand the problem and try to build something very general that can be turned into the right thing or even anything in the last minute. Useless to mention that these attempt fail sufficiently often and with all the time consumed for these over-engineered solutions such an approach it ridiculously inefficient.

Question is now: Can we savely say that we should focus on the parts that are thoroughly understood and postpone everything else, or at least put the least understood at the back of the queue? This would be a radical approach to priority management, but it falls in line with some “truths” we have learned to accept:

1. Don’t start coding until you have the complete spec (from the waterfall processes)
2. Every function must have a provable business value. (the XP stuff)

Truth 1 is rarely fulfiled, but if you accept it: a problem you don’t understand is perhaps poorly specified. For Truth 2 it is easier: How can you prove business value if you don’t understand what you are talking about.

My stance therefore is: Do what you really can do

From time to time I re-read two books: The Deadline and The Soul of a new Machine. They both are a sanity check to me and to what is going on around me. Both are in a novel form that makes them also a bit fun to read. This night I picked up again The Deadline, went to my favorite brasserie for some dinner an wine, I flipped through the pages and started reading somewhere in the middle.

Fine grained design (sorry I have only the German translation)

The key statement was that you can move quicker if you spend less less time on debugging. This is undoubtedly true I think, but their remedy is something I ever disliked: Fine-grained designed where you try to verify before starting coding . I never liked theses design documents for a simple reason: They spend lots of pages on the obvious to hide the sloppiness of what is poorly (if at all) understood. When verifying it you spend much time on the obvious and happily overlook the parts that aren’t well defined, as they can’t be that important – otherwise the author(s) would have spend more space on it, haven’t they? When implementing it, you’ll get the 90% syndrome: ask the developer after about half of the estimated time how far he got and he will answer “90% completed” - problem is – these ten percent will stay around forever, after 200% of the estimated time the programmer (now with tired eyes and a bad shave) will still tell “about 90% done”.

Short, I believe the only true design is the code (“The system is fully documented – granted you can read C++“- Z.Köntös), but of course not any code, it has to be expressive. In C days a basic design had been made from documented header files, in Java you create interfaces or if someone pays you to do so you’ll sketch endless variations of UML diagrams. UML is believed to be a superset of what you can code – given a smart generator it transforms your design into code where “you just have to fill the gaps” and changes to the design in the code “can be reverse-engineered”, NOT. There are some cases where certain diagrams are really useful to get you an overview of a project, but all examples I know fall in two categories:

1. Reverse engineered and handcrafted for better readability

2. Metacode that is feed into interpreters and MDD tools

If we have to accept that the design and the code have nothing in common, why not simply skipping this stage. 20 years ago this hadn’t been a proposition. But it is not true anymore. Modern languages and IDEs provide browsing and refactoring or mix in mechanisms that make the design malleable even if it it is (already) in code. With C and COBOL it might be possible as well, but it hadn’t been possible in their time. This means that your design process is less a constant than perhaps the design! If we accept this the consequences for the work are huge: If you use a language that is capable of expressing your design, it is idiotic to use a design tool outside to describe it. You’ll break your neck to describe in prose or diagrams what is a common idiom in your language (…API, DSL) – only to end up with something that is weaker than the code you have written. And this is the crucial point: Design tools and generators assume that their representation is complete, as any generator can only loose information – if your code is more expressive the generation and the reverse engineering of code cannot work at all. If you are inclined to follow, beware of the caveat: If your language is expressive enough, but you don’t use it appropriately the representation of you design will stay incomplete and you might need additional documentation - or better (but fewer) programmers.

You can use an expressive language to get around formal design, what do you get:

1. Instead of an intermediate layer, you have something you can work with

2. Your design is as live as the code is – any refactoring updates your documentation

But back to the claim of DeMarco’s book. It is is indeed still that verification of the design saves debugging, but that doesn’t mean that the design has to be made in another medium than the code. I came to this during my recent experiences learning Scala. I wrote some pretty complicated stuff and went naturally a bit wild with some feature of the language, but I noted one thing: I spend very little time on debugging (one thing I wrote was a specialized implementation of nauty). The language forced me to (or perhaps I did myself as I wanted to learn as much as possible from my examples) express concisely what I want to do. And as the language allows you to be extremely type-hard in your code it actually enabled me to do so. I also discovered that I wrote code that tended to be more declarative to avoid debugging into deeply nested list comprehensions. This topped even the positive experience I had years ago starting using Java, which I found pretty crippled as a language coming from C++.

One of my famous war stories is that I once wrote 400 lines of C++ on a single afternoon that compiled with no errors and also worked as I intended. I never managed to do it again. It was pretty convoluted code (technically a small interpreter) and I never managed to the same in another language I know very well: PL/SQL. And I have also Java code that has a complexity below those 400 lines C++ where I think that I’ll never be able to get them to work correctly.

So what is the key difference? Abstractions. Scala makes it easy to build abstractions, also because you get instantly rewarded by it, C++ as a very flexible language allows it also (though it is much harder), Java knows abstractions in form of APIs and frameworks – but this doesn’t help you as soon you are getting into region far from the API and your framework. These abstractions can be called DSLs, you shape your environment to you needs, perhaps even in multiple layers. In C++ operator overloading has been the too for that, in Java frameworks and APIs (quite clumsy in terms of syntax) where in Scala the difference between operators and methods dissolves completely ( as does the difference between a block and a parameter; and the extractors as antonyms to constructors). My C++ example was btw a good example of abstractions, many classes, few methods with am orthogonal interaction – such code either works or can’t be written at all. Making good abstrations in PL/SQL is as in Java on API-level.

Where is the dynamic world?

As much as I share the thoughts of Paul Graham, I can’t relate to his indissented preference for dynamic typing; there he is missing the point. Yes, malleable code is important, by if you have to keep track of each instance you might reinterpret it will likely blow your mind. I think he meant that you are coding DRY (Don’t Repeat Yourself), something that can be done perfectly (not that hacking style that is considered as expert Ruby) in Lisp (if you know macros!). Static languages can offer you the same comfort without resorting too often to macros (hard to implement in any language with syntax – Lisp doesn’t have any that’s why it is the only language having this feature) if you have type inference as well (Scala, C# 3.0). If you need to change a type, do it, the program will follow, but the compiler will shout at you if you crossed the line. This makes it even possible for others to change you program, because they get a quick feedback when they do something your initial design didn’t foresee.

The others

How is most software written today? Unfortunately with the methods of the 70’s and the tools of the 80’s (if at all). Even if they happen to use a technology of the 90’s, the methods are the old ones and the code will not exploit what is possible, but what is in the line what the environment gives you. In summary: You have a spotty design, verbose inexpressive code (“it is simple everyone can write it” – “if you have some dozens of programmers in India”) and exploding budgets and/or disgruntled customers. Poorly understood design leads to missing/faulty features, correcting these in a mess of thousands of lines of indistinguishable rubbish slows you down; the actual development moves to the debugging stage (if you are still lucky with a customer a beta(?) tester) – I call this “debugging into existence”. Even if development is always trail and error - debugging is different: When you discover a misconception during development, you go and update the spec – in debugging, the only trace will be a “resolved” in the bug database, more over debugging should fix issues and mustn’t create new ones, thus most people that correct bugs, correct them as locally as they can, even for the price of copy&paste code and compromised architecture.

A while ago I visited this already. Now I think that it is not that bad, at least as I know now how to fix it. A valid remark is of course, why not doing it right in first place, but actually this is why I changed my mind.

In fact, even if you start out with a proper domain model, you might end up later with an anemic one. Or an over-complicated that creates other issues. Another reason is of course that you have a legacy application that is just reimplemented using pseudo OO (”I have get and set instead of public attributes, thus I did OO”)

So the fix. I have to admit that I forgot it. I started writing this post some months ago and abandoned it, then openend it up again as my mind changed, etc.

Read the rest of this entry »

I have to say that I find the enhanced for loop useful as a chocolate teapot. I cannot count how often I refactored an iteration into a for just to redo it afterwards. Some times you want to delete via calling remove on the Iterator, sometimes you need an index, well it quickly gets ugly.

Triggered by http://kawagner.blogspot.com/2006/08/java-enhancements-i-would-like-to-see.html and learning C# I started playing with the idea to wrap Iterable to provide access to an index, remove and filtering (plus paired iteration). Well it became ugly. Java and the generics are bitchy sometimes, have a look at it, maybe you find it useful (I find it insane:-)
enumerator.zip

I don’t think it is finished and I don’t think that I will finish it ever. The reason why I put this is out was just a side-kick in Javaposse #94 where the index issue was raised with the remark that someone might implement it someday - well I already did.

The point why I stopped working on it and I really developped an aversion against this code (now you have to have a look at it) was that I started with Scala; a really nice language that has enough FP to write expressive code, enough OO to fit into today’s development mainstream and integrates into .NET and Java so that existing code can be leveraged. It is statically typed (I had too much hassle with JavaScript that I could ever consider dynamically typed code in larger scale development) that makes it strict enough to have powerful type inference. Of course it has it rough edges and limitations, but it addresses the problems I find hard to tackle with Java reasonably well - there are problems that can be solved more elegantly with Ruby, but I think I still need some more practice with Scala before I give up on them.
PS: I think the index in a loop is not needed in a pure OO-design, but where do you ever find a pure OO design

A while ago a co-worker implemented an API and he found that there are many situation where he simply has to throw an exception. The exceptions had been all of this kind where you can not do anything about them, but nevertheless you have a lot of information to pass to the client that tell what was considered inconsistent.

He found himself repeating code and he refactored all the throws into a method that did nothing but the construction of the exception and throwed it.

So instead of

public int doSomething(String a){

if (a!=null) { return calc(a);}
else { throw new RuntimeException(”Argument must not be null!”);}

}

we had

public int doSomething(String a){

if (a!=null) { return calc(a);}
else { throwException();}

return -1; //Ugly ,but otherwise it fails to compile

}

But this does not compile without the bogus return, as the reachability analysis does not dive into methods like

final void throwException() {

throw new RuntimeException(”Argument must not be null!”);

}

The solution is simple and gives also easier to read code, but it uses inner-classes:

private class InnerException extends RuntimeException {

InnerException () {

super(”Argument must not be null!”);

}

…

so we get a happy compiler by writing

public void doSomething(String a){

if (a!=null) { return calc(a);}
else { throw new InnerException ();}

}

Now it is clear that an exception is thrown and as the inner classes ctor has the same access rights as a method, even more complex constructions of exceptions can be performed.

Interesting enough, this contradicts my Golden Rule 11

Note that the example is rather simplistic, the original throwException made access to different fields to prepare the error-analysis.

Today I had a job interview - in french, this is what makes this most remarkable; the interview drifted a bit off and I stated the following (I’ll give the english transcription, as my french writing is terrible)

You can reify the internal flow of a software-system by looking at the social network of the people who build it. The opposite is also true.

If you look at a sufficiently large piece of software, you’ll find different components and of course interface between these components. The developers of components are likely to form clusters, while over the interfaces you’ll find (occasional) communication that is more formal than within a component’s clique.

As a corollary to this observation I’d like to state: If you have an interface it will be working if and only if (iff - for mathematicians) the cliques hav a working communication.

I explained this to my girl-friend over diner (rest assured, she tells me about her professional poker experiences in turn) and we agreed that this is the key for the success of open source software.

An open source project can only exist if you have a working network and good communication among the subgroups. The mere fact that you found a group of people capable of forming such a network assures (at least) the necessary condition to create a good piece od software.

The existence proves statistically the need and the compliance with the mainstream of such a solution. Otherwise the likehood that such a group can find itself in the vast space of the internet will be zero.

(we had wine with and some calva after the dinner - so I am not capable to tell if this observation is just trivial or perhaps a real deep insight - anyway I am proud that I’d been able to express it after only three years of french)