Against Monolithic Frameworks

When I saw Apache Cocoon, I loved it. Its view of content generation as a series of XML-XSL transformation is radical. Seeing Apache Struts, made it clear that there are areas in XSLT which cannot match the power of a programming language, and Java could be that language. Who will not enjoy the extreme plug-in architecture of JEdit! And that Netbeans people's IDE. And now this eclipse, with its yet another deceptively lean plug-in framework, and the focus on being a complete IDE, instead of just an editor. How can we miss JBoss from this list, with its revolutionary vision to make its App Server(!) in a plug-in model. These and other such OpenSource projects are exposing what we missed in the hitherto proprietary model of software making. When a company, with its rightful motive of making money, creates a software, it will keep, and has kept its activities closed to outsiders. Not only the source, but also extensibility of the software is kept closed. This, they thought, has to be done in order to keep the revenues flowing in. After all, if they allow users to create their own useful extensions, then how will the original company sell its next version? But this all changed with the coming of OpenSource software, which started just as a philanthropic rebellion to the shooting software prices. When people make software for the sheer fun and beauty of just making it, they don't worry about the next version, but only its usefulness. It is definitely not useful for the users of a software, to wait for the creator to serve all their requests promptly. Moreover why wait for a small tweak, instead of doing it ourselves. This extensibility, and the consequent modularity is the hallmark of the OpenSource software development.

Plugins Everywhere

In C (the language) we modeled a software as a collection of functions. In Java we imagine the same software as a collection of classes, that is, as a bigger grouping of functionality. In both, the idea is the same: to conquer complexity by divisioning. The debate is only on the right form of modularity, and not the usefulness of the modularity itself. Components and Plugins are just terms denoting the extensions of this business of breaking a software into manageable and yet meaningful chunks. And only with Plugins we are squarely dealing with the user based divisioning rather than the usual programmer based functions and classes (though the OO gurus have been pleading for a long time now, to make reality based classes).

Plugins epitomizes the axiomatic approach to software engineering. Plugins are manageable, and simple-to-understand pieces of code, that when arranged suitably in a framework, can provide a complete solution to the task at hand. Plugins depend on contracts. The contract stipulates, how a given plug-in will use the given data, or its output format will be, or else how it can cooperate with other plug-ins. Though many plug-ins are useless when alone (we'll revisit this point soon), their sum can be greater than the parts.

More the rules a plug-in requires, less useful it is. And more 'complete' it is, the less extensible it is. Ideally a plug-in does one thing, and does it well, following the age old golden rule of functional software. The upcoming webservices, and the service based modeling of software as such, are the outcome of this plug-in like modular visions. It is becoming very clear that pouring millions into the maintenance of software is plainly unwarranted. Yes, Microsoft does manage its Windows this way, successfully. But you must be as rich as they, to follow suite. Not many can keep their product inextensible, and still have the wherewithal to maintain it. Nowadays OpenSource is like a waiting hound. Either you open your code and have the initial bite in the market, or leave it to charity. Also remember, that leaving code source open, makes you vulnerable to peer reviews, with whom only extensibility and simplicity wins.

The Role of Frameworks

For plug-ins to be useful, and be well orchestrated, we need a framework of cooperation. It seems, Sun is the first and the best company to use the magnetism of framework to garner more market share. Of course IBM did this long back with its open and free PC model, but soon somehow managed to loose its creativity and dare. Sun took over the baton of being creative by being the author of Java and the now growing Java Community Process (JCP). The idea of keeping the framework as standard and vying in implementation, is daring for sure. This involves the courage of letting go. Microsoft need not do this, as they already have their market, and only have to hold on to it. But for how long?

OpenSource is here to stay, and hence the extensibility of Plugins, and hence the importance of Frameworks. The early bird gets the worm. If Eclipse is any indicator, only the strongest early bird gets it. Strongest can mean anything: Maybe the advertising power, or a complete suite of services, or simplicity, or a mix of these and more. If Microsoft could be envying Sun (look for Palladium) for its Liberty Alliance; If IBM et al's Eclipse can compete with Sun's Netbeans aka Forte; And if every non-Mictosoft company is flocking to build a competing App server adhering to the J2EE standard (and if JBoss can win the race with its plug-in model): This only shows the power of frameworks in both its Openness and the Profitability.

Against Monolithic Frameworks

There is no denying fact that frameworks are central to the present distributed and open model of development. But what sort of frameworks can win? We must realize that our understanding of automation is far from complete. Only few, that too rarely, can claim that they understand a requirement decently. Yes, our challenge of making a good software is not just the question of technology alone, but the far subtler question of comprehension itself. Extensibility is the first causality of poor comprehension of a product. More all-encompassing a framework is, and more it claims tight integrations with its parts, less chance does it have of surviving our varying needs.

Why at all we need a framework? To allow the plug-ins to cooperate smoothly, in spite of each being from different stables. When does a framework cross its limits of usefulness? When it does anything more than orchestrating the plug-ins. By keeping the framework loose, we only put the onus on the plug-ins themselves. The mistake a framework can make, can be made by a plug-in too. But, plug-ins being designed as replaceable ones from the start, are expected to cause less problems. But the danger is still there. The question of, what minimum features a plug-in should have to cooperate, can itself develop into a rigid framework. It seems the maximum constraints of a framework could not be larger than the usual Sockets models over TCP/IP. Anything less could be frivolous. And anything more could be dangerous. Though wire protocol should be the norm, there should be a way to separate even this dependency through more layers of indirection. Yes, performance comes into the picture here, and that's where the compromises start, and that is where we must be careful.

In this light, inverting the starting point might be useful. Where till now we tried to build the most flexible framework, we can now try to build a set of most homogenous plug-ins, and then work with them to arrive upon a suitable thin wrapper framework around. By homogenous, we mean that the number of concerns addressed by a plug-in (or a component, or a simple block of code), must be minimal, and ideally just one. Now, 'after separating what is impossible, whatever remains is however improbable, is the solution'. By no stretch of imagination can we claim that we know what we have to do, we only know what is dangerous. Monolithic frameworks are dangerous. Rigid standards are dangerous. And autonomous and homogenous plug-ins are promising.

How does this translate into few of our day to day uses? If we can see that transaction are totally separate from pooling - J2EE, EJB are unwanted rigidity then (don't even talk about .NET here). If XML generators are seen as sufficiently independent of the framework to generate web pages, then Cocoon is an unwanted rigidity. If XML and XSL can create 90% of the web pages generically and openly, then Struts like OO deliberations are superfluous. Think how independedt a spell checker is, and see how nicely it fits as a plug-in into jEdit. Note that Jython or Beanshell are independent scripting engines, and yet they fit snugly as a plug-in into an IDE. See how Apache Ant, Jalopy like code formatters are useful and homogenous tools in their own right, and hence make a good extensions to anything, as a plug-in.

Plugins can themselves expose a thin extensibility hooks, and act as a mini framework. If a plug-in itself can be made out of smaller plug-ins, or can be extended or configured by a plug-in, we now have a open network of plug-ins. A good ideal to keep as our bearing.