This document seeks to provide the most condensed
possible introduction to SAND. The vast majority of detail has been
relegated to a reference implementation, with only a few of the more basic
points referenced here in the appendices. This introduction represents
an extensive reduction to the bare essentials, and should by no means be
understood to represent a complete description or specification.
SAND (Structs and Nodes Development) is a system architecture and
development platform for building highly flexible and scalable
applications. It is based on structs (which declare data), and nodes
(which receive and produce data), being highly leveraged from within a
fully supportive development environment.
In effect, SAND is a return to the simplicity of a struct declaration for data, and code module to work on it, but at a level that
The intention of SAND is to allow for immediate and continuous
application development, focusing almost exclusively on business logic
rather than underlying technologies. Wherever possible, code is
automatically generated from struct and/or node declarations, and the
business logic is insulated from specific implementations of messaging,
configuration, control, persistence or other technologies. Structs and nodes provide the
conceptual basis, and the development environment makes
the concepts executable.
The term "struct" is borrowed from the C programming language. Like in C, a struct declares a conglomeration of basic data elements. In SAND, structs are standard .java class files that form the meta-source for all data declarations. A struct will
BaseUserStruct.java)
int, long, double, String, Date, or
protected long userID; &
@ref BaseUserStruct)
protected long[] administrators; &
@ref BaseUserStruct)
protected BlackboardStruct workstate;)
protected NodeDeclarationStruct[] nodes; )
Structs form the basis for messages, which define the input and
output for nodes. Each struct definition generates a
corresponding message definition, so for example
BaseUserStruct.java generates the BaseUser.java
message class, plus optional verb
messages used for instance management if declared. A message
extends the struct, and adds both autogenerated and manually created
utility methods to manipulate the data within the java runtime.
Structs may extend other structs. So for example
UserStruct.java might extend BaseUserStruct.java
and add other data elements useful to a particular application.
Inheritance allows for common code, while also allowing for customized
applications.
Struct declarations are leveraged to generate code for configuration,
control, messaging, caching, persistence, serialization etc. The use of
structs allows a data element to exist in a variety of forms (XML,
relational database row, HTML form data, message object etc) without having
to write interfacing and validation code for each format.
The term "node" is borrowed from graph theory, and reflects the graph
formed by the flow of messages in a running SAND system. In SAND, a node
is a java package containing code which is responsible for fulfilling a
single business logic function.
A node is specified by its declaration class. For example an
Authorizer node (in package xxx.xxx.Authorizer) would contain
an AuthorizerNodeDecl.java class which is the node
declaration. From this, the SAND development environment will
automatically create an abstract AuthorizerNodeBase.java class
containing the base implementation code. The developer must then override
AuthorizerNodeBase in AuthorizerNode.java to form
the main processing entry point. All other classes and files in the
package are unrestricted.
A node declaration class must:
int, long, double, String, Date, or
int[], long[], double[], String[], Date[]
A node base class provides:
onReceive
stubs and overloaded send methods)
Code in a node package:
The SAND development environment (sandbox) takes care of the majority of
"plumbing and processes" necessary in a development project. Creating a
sandbox is a significant effort, but is
both highly flexible and highly re-usable.
A sandbox includes (at a minimum) the basics
project, code generators, and the SAND configuration management tool (sandman). It will typically require additional
supporting technologies (version management, build tools, J2EE container,
database, XML serialization, web server or other UI rendering tools etc) to
be fully functional. The total installation of a sandbox is kept as simple
as possible.
An installed sandbox includes all directory structures, automated build processes, documentation, and tools. While a complete list of sandbox services and usage is beyond the scope of this document, some of the more noteworthy points that occur during the product lifecycle include:
SAND/docs/index.html, which
provides links to other documents in the docs directory tree in
addition to project navigation.
The simplicity of the struct declarations, and the verifiable behavior
of the system as a whole, provides enormous leverage at almost all points
in the product lifecycle. SAND provides huge advantages in initial
development, but its primary focus is on the application over time.
The SAND development environment supports at least the following javadoc tags and parameters for structs:
The SAND development environment supports at least the following javadoc tags and parameters in node declarations:
onReceive method is generated in the
node base class.
send method is generated in the node
base class.
query method is generated in the node
base class.
onDelivery method is generated in the
node base class.
Limited types for configuration parameters:
Configuration parameters are limited to the basic types, or arrays of
basic types, in order to provide a consistent configuration interface for
all nodes. Cramming more advanced information into these basic types
through serialized object forms, associated arrays etc. is strongly
discouraged. Where more advanced information is needed by the node, a
struct declaration should be created, with the instances loaded via query
at node initialization time.
All configuration parameters are declared locally to a specific node.
There is no inheritance between groups of nodes, nor is there any concept
of a global configuration attribute. Global values can be specified
through system variables.
If a struct were allowed to declare a public final static
member, or other data members that are not recognized as part of the
canonical form, then the message class would have access to data not
available in other data representations.
If a struct were allowed to declare methods of any kind, then the message class would have access to data manipulation logic that is not specified for other data representations.
While it may seem odd to only allow protected data elements in a java
class, this is necessary to produce a canonical data declaration which can
manifest itself across more representations than just java classes. Note
that additional data members, constants, methods etc can be added to the
resulting message through both automated and manual code generation, so in
practice this limitation is not a significant development issue.
A SAND development environment (sandbox) provides verb messages (generated from struct tag declarations) that have specific processing requirements. At a minimum, this includes:
public class BaseUserUpdate ...
protected long messageID;
protected long messageVersion;
protected int action; // add|update|delete
protected SandAttrVal[] messageData;
...
Notes:
public class BaseUserCollection
protected BaseUser[] baseUsers;
...
Notes:
public class BaseUserQuery ...
protected SandAttrVal[] matchInfo;
...
Notes:
public class BaseUserHistory ...
protected SandAttrVal[] matchInfo;
...
Notes: SAND development environment effort:
There is a tendency to begrudge the time spent up front in setting up a
development environment since it is not tied to a specific deliverable
feature. However this one aspect of software development frequently
results in an order of magnitude more work than initially estimated, and
ends up either getting done or manifesting itself as wasted development
time. In short you pay for the time either way, but if you do it up front
it takes less total time and development is more productive (and fun).
A quality development environment is part of "doing it right". The test
for if something was "done right" is increasing returns for decreasing
development effort over time.
The "basics" project ships as part of the sandbox, and provides the
fundamental declarations and code needed for SAND development. The basics
project is set up like any other SAND project so it serves as an example in
addition to providing the needed structs and nodes.
A partial sampling of the basics project includes:
protected String username;
protected String password;
protected String attr;
protected String val;
protected SandAttrValStruct[] data;
protected String classname;
protected String instanceName;
protected int status; // (INITIALIZING,RUNNING,SUSPENDED,STOPPED)
protected String name;
protected String description;
protected SandStructMessage[] initialData;
protected ServerDeclarationStruct[] servers;
protected String name;
protected String description;
protected NodeDeclarationStruct[] nodes;
...
protected String nodeClassName;
protected String nodeInstanceName;
protected ConfigParam[] configParams;
protected NodeDeclarationStruct[] helperNodes;
protected SandMessage[] messagesToSend;
protected CallResponseStruct[] queriesToMake;
protected CallResponseStruct[] queriesToAccept;
protected SandMessage[] expectedInput;
protected SandMessage query;
protected SandMessage response;
...
public int getStatus() // NORMAL, SYSERROR, SANDERROR, APPERROR
public String getErrorMessage()
public static RefManager getRefManager();
... methods to resolve ID references of all types ...
public void setUpdates(SandUpdateMessage[] updates);
public SandUpdateMessage[] getUpdates();
SAND configuration management tool (sandman):
A sandman implementation is a significant application, which among other things obviates the need for a separate "admin" application under most circumstances. It vastly simplifies many development, deployment, and testing tasks, and under most circumstances can be viewed as a "black box". However there are several aspects which are worth explicitly noting for comparison and implementation purposes:
A SAND application relies on sandman and the generated code, both of
which in turn rely on the supporting technology. This means that the same
application can be configured to run on a single small computer, or
multiple computers in multiple data centers, depending on the underlying
technologies involved.
A typical sandbox will assign specific semantics to reserved directory names. For example:
docs directory holds documentation files (typically
HTML documents) and the generated javadoc directory if applicable
src directory holds source code
build directory holds build automation files
env directory holds .jar files resulting from the
build, and other files needed by the application for configuration
or other purposes
test directory holds test configurations, test scripts
and load test scripts generated from sandman (typically as XML files)
So a subsection of an actual SAND installation might look like:
Pushing the source three levels down is unfortunate, but unfortunately
fairly typical. It is possible to factor the source prefix directories,
but the build processing is usually slowed down far beyond any gain from a
more concise directory structure. The structure may improve as java
compilers continue to evolve.