After googling around here and there for different things, I came across a number of interesting and useful items such as this one, which got me thinking about different ways to help expedite the process using some generic base tester classes that could be extended as needed. We already have something like that here, which later had to be duplicated here and then again here.

I really don’t like having things in more than one place, and when it looked as if I would have to do the same kind of thing again for any servlet testing tools, I decided that it was time for a new .jar file dedicated exclusively to those items whose sole purpose was to support testing. It may only end up with a few things in it, but these are things that I don’t want to put in the core .jar, as they are only used during the testing phase, and I don’t want to have to duplicate all over, either.

This new project will be a new Maven project called core-test, and will produce the core-test.jar file, which will become a test-scope dependency for those projects that can benefit from any of its contents. Now, if the test coverage report on this project starts to bother me, I’ll have to come up with unit tests to test the unit testing tools, but for now, I think I’ll just try not to look too closely at that report! We’ll just focus on the code, starting with the BaseDaoTestCase, which we can now yank out of all of the places where it is currently found and reference from here instead.

• org.restafarian.approval.dao
• org.restafarian.approval.dao.impl
• org.restafarian.approval.manager
• org.restafarian.approval.manager.impl

We can now delete the original RequestForApprovalManager in the old org.restafarian.approval.data package, and we could even delete the package itself, were it not for the presence of another remaining class, the RestServiceManager. That one, we have no intention of replacing, but as long as we’re cleaning things up, I think at this point I’ll go ahead and rename that package now to org.restafarian.approvalservice.impl, just to bring it in line with the convention used on similar packages in other projects.

That takes care of the Java components in the approval project, but we still have some changes to make to the Java servlets in the approval-web project as a result of these recent modifications before things will work with new approach. Before we do that, though, we need to create some unit tests back in the approval project to complete that entire aspect of the process. Yes, I know, you’re supposed to do the tests first, but it’s better late than never. We’ll jump on that next!

In attempting to put together the stand-alone, ready-to-wear zip file of all of these little services, I have come to the realization that it is high time to bite the bullet and just dive in and make these older services look and operate just like the newer ones. That’s actually a little more work than simply DAO refactoring, but it’s as good an explanation of the endeavor as any at this point.

Anyway, that’s the general idea … we’ll have to wait and see how it all turns out.

Today, though, I’m just going to toss out a little idea without actually doing any real coding, and then hopefully follow this up later on with some tangible parts that will actually work. But first, a little background, just to provide the context.

It’s been a while since I completed the major work on the Notification Service. Since that time, it’s been used for a variety of purposes, and in a number of different ways, which was the intent. The basic approach to sending out a notice is something like this:

notifier.sendNotification(sendTo, templateId, noticeData);

… where sendTo is a list of recipients, tempateId is the id of the stock language for the message to be sent out, and noticeData is the data from which values are extracted to resolve variables in the template. Now, the format of the “data” passed was intentionally left obscure so that you could choose to implement this element in any way that best fit your particular use of the service. This actually worked quite well, as many people chose many different approaches to populating this element before making the call to send out the notifications.

However, in order for all of this work as it should, there must be a clear understanding between the template designer and the individual passing the data so that the person laying out the message understands what data is going to be made available and how it is to be acessed in the template language. In our standard implementation of the Notifier interface, we use Velocity to resolve all of the variable data, so the templates are written using Velocity syntax. The recipient data is loaded into the Velocity context under the name recipient, and the notice data, in whatever form it is passed, is loaded into the context under the name data. In this way, all recipient data is referenced using the Velocity syntax $recipient.whatever and all notice data is similarly referenced using $data.whatever. It’s the format of the “whatever” portion of the field names that changes with each developer’s unique way of bundling up the data to send into the notice process.

So, from the template designer’s perspective, it seems rather confusing that you have to code $data.user.id in one instance and $data.request.remoteUser in another when the result turns out to be the same value. Shouldn’t you guys all get together and come up with a consistent approach or a standard as to how you are going to do this?

Well, yes and no …

Consistency is good, but flexibility is good, too. I don’t really want to force people into doing things the way that I do them. The whole point was to create something that everyone could use in the way that best fit their needs. Still, it would be nice, for a lot of different reasons, if everything were approached in some kind of consistent maner.

So, I had this idea.

What if we created some kind of static utility or method call that included as much “standard” stuff as possible, yet still left room for individual elements to be added into the mix as well. Something like:

noticeData = notifier.getStandardNoticeData(request, response);

This would return a Map with some standard elements already populated in a consistent format, and then you could add other data to the map as well for your own unique purposes; however, the standard stuff would always be the same if you used this method. And if you didn’t want to for some reason, you could go without. That would retain the original flexibility, and yet provide some common framework for common elements such as the current date, the current user, incoming form fields, the request object, the session object, and any number of other elements commonly used in web applications. The best of both worlds: consistency right there for easy picking if you want it, and the flexibility to ignore that for those that don’t.

That’s the theory, anyway. Now I just have to code it!

One of the features of the Look-up Table service is that you can define a property in a look-up table that another look-up table defined as its source. You can read more about this concept here. Now that we have introduced the idea of the context, though, we have a little issue that needs to be resolved regarding the possible sources for a look-up table property.

In the current system, the drop-down list of choices for a property’s source comes from the list of all look-up tables. Now that look-up tables have been separated into specific contexts, we need to decide if the list of potential sources comes from all tables (in which case the “source” will have to be expressed as <context>/<tableName>), or if the potential sources should be limited to just those look-up tables in the specific context of which this particular table is a member.

On top of the philosophical decision, is of course, the technical ramifications. When we modified the LookupTableJsonServlet to accomodate the new context, we assumed that the context would be passed as a parameter, limiting the servlet’s output to just those tables in the specified context. If we elect to allow sourcing from outside the current context, this will obviously need to change. On the other hand, the current design of the example table definition edit page does not include the context as a parameter in the existing script tag that pulls in the JSON object produced by this servlet. Since the context is now passed to this page via a query string parameter, the query string will first have to be parsed and the context value extracted before this JSON object can be retrieved using the context. This will require a change to the way in which this script is brought into the page.

At this point, I think the safest thing to do would be to expand the output of the LookupTableJsonServlet to potentially include all tables from all concepts, with the option of limiting the results by passing in a specific context. This way, this servlet will support both approaches, and we can then elect to go either way on the client side.

We better do that first, before we take on the actual edit page changes.

The first things we’ll need to do is refactor all of the serlvets. To start with, we’ll need to come up with a new strategy for all of the URLs. When we first introduced the idea of the Look-up Tables, we envisioned the following URL structure:

http://<serverName>/<contextRoot>/tbl/<tableId>

http://<serverName>/<contextRoot>/tbl/<tableId>/<entryId>

The eventual structure that we ended up with turned out to be relatively close to that with some minor improvements:

All tables:

http://<serverName>/<contextRoot>/table

One table:

http://<serverName>/<contextRoot>/table/<tableId>

All entries for a table:

http://<serverName>/<contextRoot>/entry/<tableId>

One table entry:

http://<serverName>/<contextRoot>/entry/<tableId>/<entryId>

Now that we’ve added the context, we’re going to have to work that into the URL structure as well:

All tables:

http://<serverName>/<contextRoot>/table

All tables in a specific context:

http://<serverName>/<contextRoot>/table/<context>

One table:

http://<serverName>/<contextRoot>/table/<context>/<tableId>

All entries for a table:

http://<serverName>/<contextRoot>/entry/<context>/<tableId>

One table entry:

http://<serverName>/<contextRoot>/entry/<context>/<tableId>/<entryId>

… which means that we have some changes to make to all of the servlets in the Look-up Table subsystem. Time to get back to work!

Anyway, one of the main things that I want to showcase in the demo system is the Notification Service, mainly because I think it has a lot of potential uses in a variety of different areas. A major component of that service is the capability of creating and maintaining the templates for the notices that go out, but one thing that I never really finished was the template editor. Yes, it works, but there are a number of things that I wanted to add, including the addition of a Rich Text Editor, that I just never went back a completed. One thing that I thought would be pretty simple to go ahead and do was to use the Look-up Table service to replace the open entry text boxes for both the Content Type field and the Delivery Method field with drop-down select/option controls for the specific valid values. That seemed like a relatively easy thing to do, but it got me thinking …

The Look-up Table service is very nice, and it creates an easy way to set up a quick drop-down or pop-up pick list of valid values for all kinds of things, but not all lists of valid values are the same. Some of them, such a list of valid US State postal codes or a list of the days of the week or months of the year, are universal, and are the same for all applications. But there are many others that are very application-specific, and really have no place in a shared repository of look-up tables. The two that I was thinking of creating, Content Type and Delivery Method, were quite specific to the Notification Service, and not really relevant to any other application. Which got me thinking …

I need to add another level of “qualifier” to the whole Look-up Table concept. In the Authorization Service, we utilized the idea of a “context” to separate authorizations for one application, or functional area, from another. We could adopt the same strategy with the Look-up Table service, and possibly even use the same values. In fact, we could use the Look-up Table service to create a Context table of valid contexts, which could then be used by both the Authorization Service and the Look-up Table service. We could create a “shared” context for common tables, and then a “notification” context for the two tables that I wanted to create for the Notification Service. And, of course, all of that could be pre-loaded into the demo system database so it was all there to see when you brought up the demo system.

Of course, to do that would involve refactoring most, if not all, of the Look-up Table software, but I just got through a massive refactoring project of all DAO components, so it’s not like I haven’t done something like that before. It’s a lot of work, but one way to get a lot of work done is to get started doing it.

So, let’s get started!

The obvious benefit of the second alternative is performance, particularly in cases where there are a number of messages to deliver and/or large blocks of text into which variable data must be injected. Depending on how much work you have to go through to set up the process to be launched in its own thread and how much work must be done once the new thread is released, this performance improvement could be significant, or it could be inconsequential. I am pretty sure, though, that there are definitely scenarios in which it would be quite significant.

The down side, of course, is that you have no way of knowing how successful that process turns out once you release it to run on its own. Sure, there are various ways to have it report back to someone, somewhere, but by that time, the original thread may be long gone and the user moved on to something else. For all intents and purposes, once you spin off a thread to do this kind of work, it’s gone from view. So, the process has to account for that, and have some way to recover in the event of some kind of failure.

One other thing that will impact performance is the point at which you resolve variable data in the message. If all recipients are to receive the exact same message, then you can resolve the message once at the onset of the process, and then loop through the recipient list and send each recipient the resolved message. On the other hand, if each recipient is to receive a message customized just for them, with data for variable replacement coming from the individual recipient objects, then the message resolution must occur inside the loop, once for each recipient. With a large list of recipients, this second approach will be considerably more time consuming than the first. However, it is the only way for each recipient to receive personalized messages directed specifically to them.

So, for personalized messages to a large list of recipients, launching a separate thread is about the only way to do this and still respond back the user in any kind of timely manner. I would imagine, though, that the vast majority of the uses of the Notification Service will be for a single recipient, and spinning off a separate thread would probably be more trouble than it would be worth.

Of course, I guess you could set the service up in such a way that either approach would be an option, and then you wouldn’t have to choose one over the other until run time. That approach certainly avoids the decision, but at the cost of additional work to code for both alternatives. I guess that makes the decision one of three choices: do one, do the other, or do both!

The basic premise of the service is simply this: given a boilerplate template for a message, an object containing data to be used to resolve variables in the template, a collection containing recipient data, which could also be used to resolve variables in the template, and few other, minor tidbits of information such as the content type, delivery method, and name of the property containing the recipient’s address within the objects stored in the collection of recipients, the service will loop through the recipients, resolve the variables in the template for each recipient, and deliver the notice via the specified delivery method. Collectively, I call all of these items needed to carry out the work of the service the Notification Request, and I envision the entire process to look something like this:

At some point in the future, I envision the potential for several independent delivery methods such as e-mail, instant message, text paging, Tweets, fax, and maybe even hard-copy to specified printers or sent to some service that prints out the message and drops into an envelope and sends it via snail mail. There are all kinds of possibilities there, but to start with, I am just going to focus on e-mail. I am going to attempt to do it in a way, though, that allows new delivery methods to be added at will without having to do much in the way of overhauling the existing code base.

Right now, I don’t see this particularly as a REST service, although some kind of REST wrapper could cetainly be added at a later time. At this point, though, I am looking more at a simple object that could be instantiated and configured via Spring, upon which you would simply invoke a single method such as notifier.sendNotification(notificationRequest) or notificationRequest.send(). Under the hood, of course, there would be a number of layers of subordinate services (such as the SimpleVelocityService) to perform such functions as retrieving notice templates or delivering messages for a specific delivery method. From the outside, though, I hope to make it a rather simple, basic API to get the job done.

As far as the templates are concerned, I do see the need for the standard REST service to maintain the templates, incorporating all of the standard CRUD operations so that templates can be created and kept up to date. I’d like to set that up in such a way that you can drop in the maintenance widgets onto your own plain HTML pages in much the same as was done for the maintenance of the authorities in the Authorization Service. I would also like to organize the templates in some way so that you could have a central repository of templates that included both shared templates and templates specific to certain applications or departments, with of course, the option to use your own repository as well.

I think I will actually start with the templates themselves, and go ahead and start to build the Notice Template maintenance process as a separate project. The DAOs and the data managers and the associated unit tests will all be pretty much clones of things that we have done before, so that will be a nice, familiar place to get the ball rolling. Once that is in place, then we branch out to the features and functions more specific to the particular goals of this particular project.

That’s the view at 30,000 ft, anyway. Next time, I think we will create a NoticeTemplate bean, and then start tossing together all of the parts to connect that to a back-end database and some front-end data maintenance processess. These things have a way of evolving over time, so when all is said and done, the final result may not look anything like the picture above, but that’s the vision that we will set out with, anyway!

Since we are using the look-up table subsystem to develop a portion of the look-up table subsystem, it gets a little confusing at times, but the resulting “virtual” tables that are created are pretty much equivalent in concept to any other data table — they just happen to reside in a single, generic physical table in the database. The Data Type table is no exception, and like many look-up tables, it has some additional data in it beyond the basic labels and values normally found in a look-up table. Here are the additional properties that I have defined so far:

maximumSize (Decimal)

Eventually, this value will be used to set the size of certain text input fields, and will also be used in user input validation. A “size” is not really appropriate for all data types. For data types where size is not a useful data point, this property should be set to zero (0).

sourceUsed (Boolean)

The idea of “source” refers to having a fixed list of valid values. At this point, the only “source” available for such values would be another look-up table. Eventually, I can see including other, outside sources as well, but I haven’t even gotten the internal look-up table source working at this point, so I’m not quite ready to get that ambitious.

inputControl (Text)

This is the first “code snippet” text block, and unlike all of the others, the language for this one is HTML. The concept here is that you provide the basic HTML template for the input control for this type of data, with some placeholders that will be resolved at runtime. So far, the only placeholder that I have programmed is the tagName, which should be coded as {0} … eventually, I plan to have a few others, particularly the “size” attribute, which will probably be coded as {1}, and then {2} will end being something else and so on … somewhere, there will have to be list of all of these, but for now, it’s just the tagName, so we’re pretty safe.

populationScript (Text)

This is another “code snippet” text block. The language for this one, and all of those that follow, is Javascript. The concept here is that you are provided with a populated field called “value” and you need to provide the code to take that value and place in within your input control (see above) for user editing.

validationScript (Text)

The script that you place here actually has two jobs: 1) validate the user input, and move the data out of your control and back into the field called “value”. Aslo, if you need to do any data conversion because you store the field differently than you display it, then this would be the place for that as well.

formatScript (Text)

This one I am not using yet, but the thought here is that, for viewing purposes (not editing) you might want to provide some code to format the data to make it look pretty for some reason. Again, I have not done anything with this as yet, but that’s the theory on its purpose anyway.

When all is said and done, you should be able to use the look-up table subsystem to list out the contents of the Data Type table and then add any new data types at will and have them start showing up as options for additional properties in any table definition. The Data Type table itself actually used three different types of properties (Decimal, Boolean, and Text), so the Data Type table definition is an example of how data types are used in the system.

Right now, I just have a handful of data types defined, but my thought is that I will just define more as I need them. If all works as it should, that should be a simple as defining a new entry in the table. If I’ve done the job right, there should be no coding or development work involved.