Recurring patterns in applications

I was just doodling around the architecture of my latest project when I suddenly realize that I have done the same thing over and over again for every project. I'm sure anyone of you who has successfully delivered more than ,say ,5 projects would realize that there is a pattern to almost 80-90% of stuff out there.

I started to scribble some of the recurring patterns I have seen again and again in many projects. I'm putting it here and am very greatful if anyone could add more to this. Now to verify what I'm saying is in fact recurring, I'm going to examine the presense of these patterns in three different context a) A social network ala facebook b) A workflow application c) A content management system (CMS)

1) All system must have Users

a) Social network : Yes
b) Workflow app: Yes
c) CMS: Yes

2) Users live in a Context (for example, roles, location etc.)
a) Social network: Yes. Two very obvious context: logged in logged out.
b) Workflow app: Definately yes. Swimlanes (roles) can be considered context
c) CMS: Yes. There are plain old users, there are admins, there uploaders, there are downloaders etc.

3) Given a User and a Context, there are a few Lists of Items.
a) Social network: YES. When you log in to facebook or twitter there is a list of entries/microblog
b) Workflow app: YES, you would have a list of tasks (either done, pending, kiv, delegated etc.
etc.)
c) CMS: YES. You would have a list of contents

4) An Item can contain either basic data: texts, numbers, links, images, videos, file attachments or links to other Items
a) Social network: Yes. You have all the classic types of data but also replies and "retwitts" etc.
b) Workflow app: Yes, you would have tasks descriptions and attachments for example. If the item is delegated, you will also see the delegated items
c) CMS: Yes. You will see the content and if you have "folders", you can have links to other data.

5) An item is aware of whom (Users) and how (Contexts) it should be accessed
a) Social network: Yes, only users and probably their friends could access certain photos for example
b) Workflow app: It is obvious with swimlanes, certain users canonly access certain data. What is more interesting is that certain users can access part of an item while others can only access other parts (think of government forms where you have "For Office Use Only" part)
c) CMS: Yes, folders have rights much like a Unix file system.

6) A User could be in a Relationship with N other Users. We call this relationship Contact.
a) Social network: Yes, obviously
b) Workflow app: Yes. In most workflow applications I've seen, Contacts are amed 'Supervisors', 'Subordinate' or 'Team member'.
c) CMS: You can certainly share content with other people. Not formally a Contact but still uses the same principles.

7) User should be able to log in to the system and eventually log out
a,b,c) Yes, obviously

8) A user can create Item and make it accessible to other Users (either from within his Contact or not)
a,b,c) Yes, obviously. This puts the responsibility of creating an Item on the User's hand.

9) User who creates an Item is responsible for the lifecycle (CRUD) of that Item
a,b,c) Yes, obviously

10) User who has access to an Item should be able to Read and optionally Update an Item
a,b,c) Yes, obviously

11) A system should be able to listen to the lifecycle of Items. User defined filters could be defined at each lifecycle action.
a) Social network: Yes. This is similar to Facebook email alerts when people reply to your post. The reply can be considered an Item, creating a reply is an event listened to by the system.
b) Workflow app: Yes. At every stage of a workflow, a filter can be triggered (I think Adempiere works this way to update its account database).
c) CMS: Yes. Again, email alerts can be sent when items are uploaded/created/deleted

12) There are 2 types of lifecycle action 1) Rules: To validate the lifecycle action
a) Social network: Yes. An email alert should only be sent if the new item created is a response to an existing item.
b) Workflow app: Yes. A rule is always attached to every "decision path" (the exclusive choice pattern) in a workflow.
c) CMS: Yes. A rule for example that states when a non-empty forlder cannot be deleted. Or a content viewed by another user cannot be deleted.

13) Type 2) Events that could trigger lifecycle actions on other objects.
a) Social network: Yes, adding a Contact (I.E. adding another person into your Contact) triggers an event that will add you to the contact of that other person. (I add u as friend implies, you will also add me as a friend).
b) Workflow app: Yes, for example in Adempiere, a workflow that creates an Invoice triggers the modification of Account.
c) CMS: Yes. Recursively deleting a non empty forlder might delete all its contents.

14) Users can do CRUD lifecycle on their Contacts. These actions will also generate Events.
a,b,c) Yes, obviously. You can always delete and add new people to your contact.


Now why are these patterns important? Well imagine now I've created a bare bone application (call it a framework if you will) and imagine that I want to create a Social Network. All I have to do is customised that "framework" and within a short time, tada~ a social network. I have only to define what is Item in my application context (which is social network), what is a User, what Events need to be generated and what Rules need to be triggred and I'm done. Someone else might define Items as Account, Activity, Invoice, Inventory etc. They might define Events as CreditToAccount and DebitFromAccount etc. and define Rules such as "When Inventory is low, start Procument process" and bam!, the framework is now an ERP.

Now let us take this further, what if this framework define not just bare bone stuff but also things like open persistence policy (which allows you to connect your app to any database), a well thought out GUI, a caching mechanism out of the box, a clustering facility , a reporting and BI facility and some security enhancement here and there. Suddenly, I can focus on creating my latest shiny new social network or create my own ERP focusing mainly on what the application should do rather than all the nitty gritty stuff like scalability.

Of course, what is defined here is probably incomplete at best and naive at worst. There are overlaps I'm sure (like Rules and Context could be the same thing). If you do have other stuff to add, please do enlighten me. (Or if you think certain things are just superflous). Maybe creating this framework could be my next big FOSS project :)

Weld DI and Glassfish Example

Last Thursday I gave a presentation to a few Experian staffs on Weld DI. You can find the presentation here:

http://www.scribd.com/doc/24005727/Look-Ma-No-XML

Examples can be found here:

http://www.freedrive.com/member/viewfolder/250059

Message Driven POJO with Java Content and Dependency Injection API (Weld) a.k.a Look ma! No XML!

I am supposed to be presenting 'Something new and exiting in Java' to staff of Experian Malaysia. Looking around, it seems that I am frekin lucky since we have a few new and exciting things in the Java world. Java EE 6 is finally here! (Phew!~), Netbeans 6.8 RC 1 is out! (with support for Java EE 6 - you can create a session bean right in your war file - I kid you not!) and finally Java CDI (I think, JSR-299) is finally here too!.

I decide to play around with CDI implementation by JBoss (called Weld). There are other implementations out there notably CanDI by Caucho and OpenWebBeans by Apache.

Anyway, CDI allows you to do strong typing dependency injection without XML (of course, XML is still there just in case you need it). Weld can be downloaded here [http://seamframework.org/Download]. A full tutorial by JBoss teamcan be found here [http://docs.jboss.org/webbeans/reference/current/en-US/html/index.html]. Note that the tutorial was created about a year ago [on the internet scale, that is the equivalent of the time between the Triasic era and the French Revolution], so some things are not up to date. A better approach is to learn stuff off the examples in weld zip file.

Here's an example of CDI:

With CDI, we can create Java Beans with a particular scope. In this example we have a Singleton scope. This means that there will be a single instance of SomeService for the whole application.

@Singleton
public class SomeService{
public String doSomething(String parameters){
//do something here
return result;
}
}

To access the service, we can just inject it to a class

public class Client{
@Inject SomeService someService;

public void run(){
//use someService
}
}

Note that we do not need to invoke any constructor or factory of SomeService in Client. The lifeCycle of SomeService is managed directly by Weld.

Anyone who uses Spring before knows that Spring can also do much of the same thing. Its just that Weld can do all this while being strongly typed and xml-less (Maybe Spring does make XML optional these days, I have not checked)

Now the big finale, one use case of Spring that I find fascinating is Message Driven POJO (MDP). So, I wanted to create one using Weld.

OK, since I don't have much time, I 'cheated' a bit by using ActiveMQ libraries and by making my MDP works only with ActiveMQ... but hey, it works >:) [Maybe I should call it Message Driven ActiveMQ Producer/Consumer instead ;) ]

I based my implemetation on this [http://developers-blog.org/blog/default/2008/10/28/A-simple-ActiveMQ-example] (Thanks Gisbert) .

After much toiling (my first implemetation uses Spring libraries... a pile of fun that turn out to be), I manage to create an MDP using Weld.

An MDP looks like this:

@Consumer
public class Processor {

  public void processRequest(RunnableMessage runnable){
      runnable.run();
  }

  public void doSomething(String string){
      System.out.println("Received string="+string);
  }
}

The Consumer annotation indicates that this object is ready to receive messages. The messages are in fact POJOs themselves, so when a message arrives, depending on the type of the message (or rather the type of the payload of the message), a method within the Processor object is chosen.

To send a message, I just do this:

RunnableMessage r = new RunnableMessage(); //r here is just another POJO
activeMQService.sendMessage(r);

And processRequest will be invoked.
If I do:

activeMQService.sendMessage("If you type google into google, you will break the internet");

Then I will invoke the method doSomething instead.
Cool eh :)

Anyway, the source code with all its dependencies are here [http://www.freedrive.com/file/1045164,weldmessage.zip]. To run this, you need to download ActiiveMQ
[http://activemq.apache.org/download.html] and run it. (Unzip the file and run the executable in the bin directory). Next, just run the example.

Have fun!

Will JavaFX die?

Will JavaFX die? I really hope so because it will teach the next RIA wannabe to not dump another scripting language to the masses and hope, with the sincerest of hope, that GUI designers will become coders over night. How on God blue Earth does an RIA does not have a visual editor? (And BTW, no matter how good you are,... wishing that a text editor somehow, through the magic of Christmas, becomes a "visual editor" will not make it so .. you hear me Exadel!) . JavaFX need 2 concrete things to survive: 1) An Android runtime 2) A visual (you know, like Flex Builder visual, not XEmacs visual) editor

Upgrading (kids and adults)

It was a fine weekend to do a seminar + upgrading. Kids upgrading was load of fun. Most of them work hard for the upgrading and it shows. It's quite a revelation to see how much these kids grow over the years.

The seminar was a success. So many people attend that we had to work with limited space but it was worth it.

Congratulations to all!

Adempiere references in Plain Old Java

Right now, I'm learning Adempiere and finding ways on how to hack it apart (and put it together again). I'll be "reporting" my findings on this blog on how things are done and how some of these concepts can actually be used in Java in general.

One thing that interests me in Adempiere is the Rapid Application Development feature, especially when building rich CRUD based stuff. Adempiere has this concept of "everything must have metadata". By manipulating this metadata, CRUD based application can easily be done.

In the general Java world, metadata manipulation used to create CRUD based apps is not new. Frameworks such as OpenXava (who claim to have beaten RoR development record time - apparently that article is not accessible anymore) is attracting much attention. One thing Adempiere does differently is by qualifiying relationships between Domain Objects (of course in Adempiere, the Domain Objects are Relational Database Tables since Adempiere uses direct / semi-direct SQL access). Disclaimer - I'm not an OpenXava expert, so if OpenXava does these stuff too, please do tell.

Imagine if we can port this to the Java world in general

[If you are not familiar with the concept of Reference in Adempiere, I would like to redirect you to my good friend, Red1's, tutorial ]

In Adempiere you can qualify a reference (a link from one table to another table). For example you can say that a reference is "Searchable". When the reference is rendered, you would see a "search box" that is linked to data from the reference table. You can type away a search term and the reference table will be searched.

How can we do this in Java in general?

To demonstrate this, we'll look at an example: Imagine that I have 3 entities, User (plain old User - ok, maybe not so old :) ) , PurchaseRecord (a record of a purchase made by the user) and Product (the actual product the user purchase)


Using JPA or Hibernate I could have codes representing the entities as:

@Entity
public class User{
@OneToMany
private Set purchaseRecords;
}

@Entity
public class PurchaseRecord{
private Product product;
private int number;
private Date purchaseDate;
}

@Entity
public class Product{
private String name;
private String unitPrice;
}

If I'm using plain old CRUD generator , I would create a User list. Clicking on a User from the list would give me the details of that particular user and a list of PurchaseRecords. That list is presented as is. If I'm lucky, the CRUD generator will allow me to page or sort the data. One thing I do not have is a quick access to a particular record and the subsequent Product attached to that Record. But, imagine if I can do this:

@Entity
public class User{
@OneToMany
@Display(qualifier="PurchaseRecord.product.name")
@Searchable(by="PurchaseRecord.product.name")
@DefaultFilter(qualifier="PurchaseRecord.purchaseDate=DateQualifier.TODAY")
private Set purchaseRecords;

}

Automatically, when I click on a user, I would still see a list (but this time a list of names of products, which is more palatable) and also a text box, a dropdown list and a button.

The textbox represents a search term box. You can search your PurchaseRecord by its name. Enter your search term and click on the button. The dropdown list represents filters. By default, PurchaseRecord will be filtered by date=TODAY (only today's entry is displayed). The user can always choose any other available filters.

Of course I'm just imagining stuff here and I'm sure there are some gaps that need to be filled in (for example, does having a @Display tag violates MVC) but the concept of Reference in Adempiere is really powerful. Hopefully someone could bring this concept to the JPA/Hibernate world.

What is wrong with Malaysian IT scene

Before reading this, please see the disclaimers at the bottom.


What is wrong wth the Malaysian IT scene can be dully observed from one of the most significant ICT event here in this country called the MSC Malaysia International Advisory Panel (or IAP) meeting. Here's the website of said event: http://iap-portal.msc.com.my/ .Surf the website and you'll see what I mean.

1)It's the the people, stupid

Browsing through the IAP website, we have the impression that it is just a business event. Nothing "techie" about it. In fact, asking around, it seems that the Malaysian tech communities are blissfully unaware of such event is going on right now under our collective hairy noses in PICC this week.

How on earth can the most important part of the tech. industry (i.e. the tech. people) are not even informed, heck, are not even welcomed to the biggest MSC event this year?!

Is it just me or is MSC a place where techies are regarded as second class citizens compared to quote unquote businessmen? How on earth can MDeC justify limiting their allocation for tech. employees under their pre-seed and iconity grant to only RM 1500 per month (irregardless of qualifications or experiences)? BTW, just to put things in perspective, the government of Malaysia recently declared that families earning RM 3000 or less per month and are living in cities are called the urban poor. If two techies, a boy and a gal, working for a preseed or iconity recipient companies decide get married, they superchalafragilisticexpiallodociously become the urban poor. And MDeC lament about the lack of talents in ICT, who wants to work hard during their University days and end up being declared poor!

Have these guys never learn?!

Build a strong community and business will come. That's the lesson of Silicon Valley that has somehow escape the collective brains of our policy makers. Silicon Valley started with the universities in the area. These universities have been building tech communities and when the electronic/transistor revolution came into the picture BAMM!!, the magic happens. But it started with communities. Not businesses.

The MSC brand has been a failure because it was attached to companies. MSC companies are known today as companies that do not pay EPF, do not give salary on time, try to treat their employees as slaves (gee, I wonder where they learn that from) by asking them to do extra hours and not getting paid for it. Companies are soulless money making machines. They would cut corners and take advantage of any loophole just to get another dime on their balance sheet. This can be seen by companies opening up "ghost branches" in MSC status area (such as Cyberjaya) just to escape paying taxes.

Communities on the other hand live and die by the spirit of its people. They rely on people being truthful, honest and willing to sacrifice. And yet, MDeC is blind to these sacrifices. In fact, some of our community leaders are so pissed poor, they can't even afford to buy LRT tickets. Some have to beg for jobs and some got so upset they left the country for good (good for them, bad for Malaysia). Those who bite their lips and stay are riddled with redtapes and unfulfilled promises that its not even funny anymore.

If MDeC would have spent as much on building the soulless MSC companies as they would on building communities, we would have gone further by now!

2) The locals are just as good

Looking at the talks and the programs listed on the website, I was really wondering "Gee, I can do that talk too. Heck, I can even do it better since I know the local context".

I mean gosh!, it says "international" but are the locals too stoopeed to advise MSC and how to go about doing things? Isn't Dr. Mahathir's main message to the world is that we want to be strong, but we will do it OUR WAY! Somehow that message, that twitt, does not reach MDeC.

Are our collective brains still clogged with the imperialist white-men fetish that we can't even see how much our own people have grown over the years?. Somehow, just because you're white, no matter how stupid , incompetent or evil minded you are, you are always correct. "Yes sir Mr. Balmer sir, you can rob my country and kick my butt to boot. It's ok because you're an American and you're the CEO of Microsoft"

Stop this stupid nonsense! I can give more relevant advices to MSC and I'll do it for free (heck, I even said it outloud here and here) and yet, just because my butt is the wrong color, no one gives a crap!

3) So much more

I have so much more to rant but tears are running through my eyes watching how much my beloved country is wasting yet again an oppurtunity to turn things around. I really hope our future leaders would have enough insight to consider my rant. Hopefully, things will get better. We have no other choice, we either get better... or die ...

DISCLAIMERS

If you are an MDeC employee, please do not think of my rant as an attack on you personally. I do think that people in MDeC are also individuals (and thus part of the community) who choose to work with MDeC when they can pour their talent somewhere else (and probably get better pay). Heck, I know a few good people in MDeC who were courted by oversea companies and yet they decline the offer for the love of the country. If anything, they are more into it then us developers. I believe that people in MDeC are heroes but some of the policies are real time-wasters.