README.TXT -- ShapeShifter Traffic Shaping Engine ( TSE ) Beta 3 0.093h
~~~~~~~~~~

This file contains last minute additions to the documentation and
lists known problems / fixes in the current version.  Please read
this prior to upgrade or installation.  Start with the original
Beta 1 documentation and then review the Beta 2 Changes Guide.


System Recommendations
~~~~~~~~~~~~~~~~~~~~~~

The SHAPER.NLM requires a '486 or better processor.  A '586 class
CPU is strongly recommended because of its more accurate timing
characteristics.

The TSE works on NetWare 4.10 and higher without too much of a
preference.  NetWare 6.x offers distinct advantages in supporting
SMP for both TCPIP and NSS file system activity.  This frees up
CPU 0 to perform the TSE's time critical work, and allows the TSE
to use additional processors for audit logging and other work.

For simple rate limiting the TSE requires less than 2 MB of free
RAM.   Per-flow auditing and rate limiting require 128 bytes per
flow.  This can result in substantial RAM if you audit or rate limit
down to an individual TCP/UDP connection.   10 MB of RAM per 1000
workstations is a good starting point.

As a general rule, an average of 4 packet receive buffers will
be used to manage a single UDP or TCP flow.   Because not all
workstations have active flows, you can typically get away with
2 to 4 buffers per workstation.  For heavy P2P user populations
this might be as high as 8 to 12 per workstation.  The maximum
packet receive buffers should be set about 20% to 50% higher.
You should plan for the worst case and titrate downwards
untill you see the server growing the number of buffers.


Contents Of The ZIP
~~~~~~~~~~~~~~~~~~~

    \Docs contains the Beta 1 & 2 documentation.
    \Docs\Beta2Changes contains the HTML Beta 2 Guide.
    \Client contains client side files including the config tool.
    \System contains server side NLM and SYM files. 


Installation - New / First Time Install
~~~~~~~~~~~~
Follow the directions in the documentation.

Here is a VERY brief summary:

	1. Copy the contents of System to SYS:SYSTEM
	2. Copy the other folders ( Client and Docs and README )
	   to your management PC, any location is fine.
	3. Create a shortcut in the Send To folder that points
	   to Client\SHAPER.EXE.  This is the config tool.
	4. Use config tool to modify SYS:SHAPER.CFG as needed
	5. Create SYS:SHAPECFG.NCF to hold startup commands
	   which load the SHAPER.NLM, bind, and configure it.

Please read all the documentation first prior to activating the
TSE on your server.  If you have questions, please ask, answers
are an e-mail away! 


Upgrade - If You Already Have the TSE
~~~~~~~
This version, 0.93g, of the TSE is compatible with any configuration file
created for use with 0.51c or greater, including 0.92i.  To update your
servers with the 0.93f build:

	- Backup the existing SHAPER.NLM and SHAPER.SYM files
	- Copy the new SHAPER.NLM and SHAPER.SYM files to SYS:SYSTEM
	- DO NOT copy the SHAPER.CFG provided, you will lose your config.
	- UNLOAD SHAPER from the server console
	- LOAD SHAPER from the server console
	- Perform your normal binding and configurations steps

Due to features added in 0.93c and higher, the configuration files are
larger.  They are, however, backwards compatible with previous versions.
However the configuration options added in 0.93c and later versions will
be ignored by prior versions of the TSE.

Note that the Address Mask configuration dialog has been modified.  The
edit dialog allows you to specify an optional default action to be taken
for any connections created by the mask.   The default action is embedded
in each connections and determines how traffic for that connection is
handled.

If you have existing connection tables, note that two extra REQUIRED fields
have been added which determine the connection action to be taken.  You should
modify your connection tables to include these fields.   The TSE will generate a
warning when you attempt to use the older format tables. 

If you have applications consuming the TSE's audit log, please note that two
additional fields were appended to the end of each record.  The first field
records the duration of the connection, the second is the interval between
the time the last packet was received for the connection, and the current time
as of when the records was added to the audit log.

See below for additional information on changes to auditing configuration
and connection processing.


Problems Fixed ( This Build 0.93g )
~~~~~~~~~~~~~~
The 0.93g build offers fixes for several problems identified by users
and found during quality testing following the release of the 0.93h
code.

Located additional code requiring LOCK# prefixes.   Included code to
dynamically balance the node management multilist.   Affinity should
not exceed about 3%.  For 0.93g, the -NODEBALANCE option enables the
balancing algorithm, this will be the default behaviour by Beta 3's
final release.

Fixed annoying column wackyness in the audit log and in the connection
table export code. 

Fixed bug in the WebStat and JSWebStat code resulting in improper sorting
by Key Group ID.   The bug caused each key group to be fragmented into
many output tables as the subsort struggled to summarize for each change
in key group.

Changed packet logging to show overflow condition with sequence numbers
on separate lines rather than the "mega line."   Also addressed logger
overflow with -LOGBUF option to increase the logger buffer to prevent
overflows ( see New Features below )


Problems Fixed ( Since 0.93h )
~~~~~~~~~~~~~~

In previous builds, the TSE allowed both foreground and interrupt
processes to return memory to the free list directly.  This allowed
a foreground task to temporarily lock the list when its was returning
nodes to the free list.  This task could be interrupted with the
free in progress.  The interrupting task would find the free list locked
and could not allocate nodes.  This caused the "SHAPER - Queuing Failure.
Could not enqueue a frame. It was Forwarded" messages when auditing
is enabled.  Now foreground tasks return nodes to a "to be freed"
list which is periodically merged, during interrupt time, with the
free list, avoiding the whole mess.  Occurences of the error message
indicate a real problem and should happen VERY infrequently.

The TSE now allows the -BUCKETS parameter to be set as high as 22.
The -NODES parameter can be set as high as 16000000.  This allows the
TSE to scale to more than 8 million connection table entries while
maintaining the TSE's low per packet overhead.  

The TSE's link list library should now properly assert LOCK# on the
list semaphore.  The previously library code provided protection
under a uniprocessor system against reentrancy only.

The TSE uses a data structure called a multi-list for management of
memory nodes.  Multi-lists are a collection of lists accessed in a
round-robin fashon.  This allows several concurrent allocation and
free operations to run concurrently or in a blocking fashon, i.e. on a
multi CPU sustem or nested interrupts.  A bug in the underlying link
list libraries would permanently take a sublist's semaphore when
an allocation request made against that sublist found an empty sublist.
This would cause the vast majority of nodes to accumulate in a small
subset of sublists making the multi-list less efficient. This has been
fixed in the current build.

The TSE would not allow half-bind number greater than 7 to be used
from the console.   The TSE can be bound to up to 8 logical
boards, resulting in more than 8 half-binds.  However it was not
possible to modify their state using the SHAPER BIND n ...
commands.  This is fixed.

When more than 200MB of RAM was allocated by the TSE for memory nodes,
as specified by the -NODES=x command line option, the TSE would refuse
to load its config file reporting that "the config file was found, but 
could not be opened."   This has been fixed.   The TSE now handles up
to 1.088 GB of RAM allocated for memory nodes.   A GB of RAM... who
dreamed you'd ever need that!

In previous builds, the TSE allowed the simultaneous Import and Export
of the connection tables.  This is a very bad thing.  Connections
were imported properly, however the exported tables would, obviously,
be incomplete.  In addition, these processes set a flag preventing
the expiration of connections.  This prevents a connections from
being deleted or aged out by background tasks from creating bad
pointers for the foreground tasks ( Import / Export / WebStat ).
The TSE now refuses to start one when the other is already in
progress.

In previous builds, the Connection Audit Process would attempt to
display all audit records to the screen when the audit screen was
enabled.   When in uniprocessor mode, the audit thread could
potentially stall so that a growing backlog of audit records would
build up.  This would exhaust Shaper memory triggering an auto-
unload of the TSE.   The current build will attempt to display only
the last 25 records during a pass.  This should avoid the backlog
condition.  This issue was only seen with enormous numbers of audit
records per second, generally thousands per second.  It is advised
if you have this many audit records / second, that you increase the
TTL so that the data is more summarized.

When using SMP, the number of active connections displayed in the
status screen and other monitoring modalities such as the Web
Status Page, would show in inaccurate or extremely large number.
The number of active connectons is computed from two counters
which count the number of created and deleted connections, the
difference being the number active.  A LOCK prefix was added to
ensure atomicity.  This issue may occur with other counters, such
as queue depth, however this has not been seen in production.

The Import and Export code has been modified to be a bit faster,
typical throughput exceeds 5000 entries per second.  Most of this
time is parsing / generating the formatted connection tables.

A bug was discovered which caused the TimeLastHit for connections
to be displayed / exported incorrectly.  This data is ephemera and
is not reimported, as such this caused no issues on import.  This
issue also impacted the JavaScript WebStat information as well.  The
error would incorrectly calculate the TimeLastHit, which is in
clock ticks ( 18.2ths of a second ), by subtracting from the
current microsecond timer value.  The resulting number is useless.
The TSE now calculates the values properly.

A bug was discovered in a January 2003 build of TSE Beta 2++,
a EAR release of Beta 3, which could abend if the TSE ran out of
memory while trying to add to the connection tables.  This problem
has been resolved and all similar codepaths have been corrected.


Problems Fixed ( Since 0.93f )
~~~~~~~~~~~~~~
The Config Tool and TSE now implement the "Audit Connections" option
in the Address Mask Edit dialog.  In previous builds, this checkbox
would be cleared by the config tool, the TSE would records audit info
for ALL connections.  In this build the TSE will only records auditing
information for those connections generated by Address Masks for which
the Audit Connections checkbox is checked.

The Connection Import, Export, and Audit improperly dealt with the
"IP Port" template causing the high and low order bytes of the two
byte Port field to be transposed.   This caused the TSE to report
the wrong port numbers in export and audit files when using this
template.  For example, addresses with the format  *:port#, such as
*:5190, would have incorrect port numbers.  Only the "IP Port"
template was effected.  Addresses reported by "IP+port" template
were processed correctly, for example 192.168.20.20:5190 would be OK.
This is issue is now fixed. 

   Important Note About Address Masks

This build includes modifications to correct operational problems
using bi-directional Address Masks.   In previous builds, the TSE
used the hexacecimal masks built by the configuration tool even
when a template was specified.  In other words, the configuration
tool implemented templates.  To correct bidirectional address mask
issues, the templates had to be moved into the TSE itself.  This
allows the TSE to understand the "intent" of the mask rather than
have a static mask defined in the configuration.

For example, now the TSE knows both the mask's intent, via the
template selected, and if the template applies to source or
destination address information.   This allows the TSE to select
the appropriate mask depending on the setting of the bi-
directional checkbox.   THIS FIXES ALL KNOWN OPERATIONAL ISSUES
regarding address masks and connection building.  This field test
build has been tested eith IP Address, IP+port, and IP Port templates.


Problems Fixed ( Since 0.93c )
~~~~~~~~~~~~~~
The 0.93c build ( first public release of Beta 2 code ) offers fixes
for several problems identified by users and found during quality
testing following release of the 0.92k code.

The 0.93c build now properly checks for the allocation of all
resource tags. 

The 0.93c build checks to ensure that data required to perform a
comparison operations does not extend beyond the end of the frame.
Because of the way NetWare reuses Rx buffers, it is likely that data
beyond the end of a frame in the buffer is actually the contents of
a previous frame of a longer length.   Comparisons which test data
beyond the end of the frame actually test the previous frames data.
SHAPER generates an error message when this occurs and adds an
entry into the log file showing an offending frame and the rule number.
See Known Issues below for details.  Basically, YOU are responsible
for constructing rule sets which prevent this from happening.

The 0.93c build will properly deal with the unloading of MLIDS ( LAN
drivers ) to which the TSE is bound.  In previous builds, if an MLID
were unloaded, SHAPER would not remove the bindings from its internal
tables, resulting in an inability to rebind the TSE.   Now the TSE
will perform an UNBIND from the MLID when the MLID tells SHAPER it
is going away.  The TSE issues a console message telling the operator
that it may be necessary to rebind the TSE.  Unloading of MLIDS can
result from use of INETCFG or manually using the UNLOAD command.

The 0.93c build fixes a potential abend in the connection table export
process.  If a target file cannot be opened, the export process thread
would abend because portions of the code did not check to see if the
file was actually open.  ( Duh! )  

The 0.93c build of the Beta 1 code Fixes a rare abend condition resulting
from the TSE running low on memory while under extreme load.  The TSE
should gracefully unload under load memory conditions. But under high load
it is possible that the remaining low watermark memory can be exhausted
before the shutdown can complete.  The TSE would completely run out of
memory causing the queuing of packets to fail completely and a ABEND
would possibly result.  

The 0.93c build of the Beta 1 code fixes a problem with DiffServ / TOS
tagging where frames would not be tagged because the TSE didn't think
they were IP frames.  ( This bug was discovered during limited beta for
Beta 2 and confirmed during private testing, so bug does not apply to
previous public releases - none of which has this tagging feature.

The Configuration Tool supplied with the 0.93c build of Beta 2 clears
the Always Hit / Always Miss bits in any rules when the config is loaded.
These bits were introduced in early versions of the TSE, 0.5x and prior,
for debugging.  They force the outcome of the rule, causing the rule to
ALWAYS execute either the Then... or Else... action irrespective of the
actual outcome.  Since these bit were not set by the config tool, their
state may get stuck in the set state causing the rules to malfunction.
Use the SHAPER DEBUG RULES command to see this state.   In the Flags: 
output you will see a "Hit!" or "Miss!" or both keywords.  This means
the bits are set.  Edit the config using the config tool, go into any
item ( such as a Rule or QoS edit screen ) and click OK, this will trigger
the Config Tool to save the configuration when you exit.  The bits
will be cleared on ALL rules ( and all empty rules ) fixing the problem. 

The Configuration Tool supplied with the 0.93c build of Beta 2 fixes
several problems which can cause changes made in the GUI not to be saved
to disk.  An internal flag is set when data is changed to allow the tool
to know when the config is dirty and need to be written.  This was not
being properly set during certain operations such as deletes, and moves. 

It has been discovered that it is safe to use packet discarding on Tx
frames as well as on Rx frames.  There is apparently some spooky thing
happening in the way NetWare tracks ownership of Tx frames as they are
passed to the TSE.  This causes what appears to be a resource leak in
the TSE.  In working with Novell on this, it appears that discard of Tx
frames does not leak the frame's ECB, but merely fails to update the
counters which track the Tx ECB resource.  As a result NetWare will
complain about resources not being freed by the TSE when it is unloaded
after Tx ECB's have been dropped, as is caused by using BIND x DOWN,
or part of optional drop from congestion control and queue overflow.
We are currently figuring out a fix for this, but it appear safe to use
despite the resource counter problems.  A workaround has been proposed
whereby the NLM will manage the resource counters as needed - this is
still in testing.

The "not less than" and "not greater than" comparisons failed to return
true when the packet data exactly matched.  This now works properly. (Bug)

There is a rather rare ABEND condition resulting in an
"Interrupt Service Routine Nested 2" ABEND.  This ABEND causes the server
to spontaneously reboot.  The ABEND log will record the full abend message.
This ABEND has not been seen in production use of the TSE.  It seems to
be triggered by our prolonged testing at very high frame rates.  SHAPER
symbols may be present in the stack dump.  This is expected, as the TSE
processes frames at interrupt time.  The ABEND has also been seen when
the server is NOT executing TSE code ( i.e. is not loaded or not bound )
so it is very likely this is a NetWare issue.  This ABEND has not been
observed with the latest support packs for 4.x and 5.x, nor under NW4SP8a
or NW5SP3 or NW 5.1 without service packs.  If you encounter this ABEND,
please ensure you have the minimum patch level indicated. 

Versions prior to 0.93c use a variety of sources for packet length when
accumulating statistics as well as calculating the cost / delay values
for packets.   This is because for Rx frames there is a single Packet
Length field available to use ( which includes the media header ) as
well as Packet Size field which is the length of the payload only.  For
Tx frames, there is only a collection of packet fragments and no
centralized field for the entire packet length or payload length.
Version 0.93c calculates the payload length for Rx and Tx frames and
uses this in both statistics and traffic management calculations. 

Beta 2 implements SynchronizeStart() to stall NCF file execution until
the TSE is completely loaded and initialized.  To prevent this behavior
use the -NOWAIT command line option described below.  This means it is
safe to LOAD SHAPER and then issue BIND and SHAPER commands in an NCF
file without fear that SHAPER might not be ready for work.  

Beta 2 fixes a problem with priority queuing which was present in private
releases of Beta 2.  The pre-release builds of Beta 2 had a bug where the
TSE would calculate a pointer to an ECB incorrectly when using certain
combinations of metrics.  By some miracle, the pointer always pointed to
a valid piece of the server's address space.  This prevented a page fault,
but, of course, resulted in incorrect processing of the priority queues.
This has been fixed in the public Beta 2 release.

The SHAPER DUMP MEMORY routine was updated to reflect additions to the
TSE's runtime memory tables.  It now produces the proper .MAP file.

Beta 2 now properly deals with command lines which are > 80 characters in
length.  Command lines which are up to 238 characters are now properly
handled.  The TSE's console command processor and processes which deal
with console parameters ( such as Import, Export, WebStat ) can handle
file names up to this length as well.  However the command line is still
uppercased by the TSE.  As a result all file names are passed as upper
case strings rather than whatever mixed case is presented to the console.


Problems Fixed ( Since 0.92k )
~~~~~~~~~~~~~~
The 0.92l build fixes several annoyances and has significantly
restructured connection handling code.  

The 0.92l build of the Beta 1 code Fixed the linker script so the screen
name is no longer shaper.nlm but "ShapeShifter (TSE) Beta 1".


Problems Fixed ( Since 0.92j )
~~~~~~~~~~~~~~
The 0.92k build of the Beta 1 code offers fixes for several problems
identified by users and found during quality testing following release.

The 0.92k build fixes a serious bug with rule processing and
connection oriented rate limiting of outbound frames.  While this
bug does not lead to an ABEND condition, it essentially made rules act
unpredictably when the rule tested data crossing over multiple fragment
boundaries ( TCP port numbers in IP frame using newer builds of
TCPIP.NLM )  This issues does NOT impact inbound rule processing.
Since most sites manage traffic in dual interface situation ( Border
Manager / NIAS Router ) and use the TSE to manage frames inbound on each
interface, the bug was rarely exercised.  The current build contains a
complete re-write of the transmission handler for rule processing and
connection oriented rate limiting.  While the code for these sections
is small, it is important and heavily exercised.  

This build adds code to detect a loop in rule execution.  If you should
create a set of rules which performs an infinite loop, the TSE will detect
this, place the TSE in BYPASS MODE ( Forwards All Frames ) and generates
a console message indicating the situation.  Previously, had a loop of
this type occur, the server would hang. 

The 0.92k build addresses a few functional issues relating to the queuing
of large number of frames.  Four commands were added to allow or prevent
queue overflows, and forward or drop overflowed frames if they occur.


Problems Fixed ( Since 0.92i )
~~~~~~~~~~~~~~
The 0.92j build fixes an ABEND issue in rule processing code.  The bug
can ABEND the server when rules specify a Rule... or Delay... action.


Problems Fixed ( Since 0.92h )
~~~~~~~~~~~~~~
The TSE offers a fix to Queue Preemie errors caused by timer backsteps
when the TSE us run under Netware 4.xx on '386 and '486 servers.  Queue
Preemie messages can only occur with this build when there is actually
a fault in the queuing algorithm or queue data structures.  The timer
backstep errors will be displayed on this older equipment until the
SHAPER OPTION RACHET SHRT command is used to disable them.  This option
does not fix the underling error, only mask it.  Even so, these servers
are still capable of managing T1 data streams.  The frequency of backstep
errors is reduced to 30 minutes.

When the server goes down, and SYS: is dismounted, the TSE sends an error
message to the console indicating a problem with the configuration file.
For various technical reasons this has not been completely eliminated, though
the frequency of the error should be reduced to 1 minute.  Following any
error in accessing, reading, or writing the config file, the TSE will
wait 1 minute before retrying the operation.

A few aesthetic issues have been addressed in this build.



New Features ( This Build 0.93g )
~~~~~~~~~~~~

The TSE now integrates with the www.TrafficShaper.com MAC2USER utility.
MAC2USER queries NDS and other sources to resolve IP or MAC addresses
to the NDS user object using them.  After configuring MAC2USER on the
server, it allows SHAPER to interrogate a in-memory database of address
and NDS object names.   The result is that Audit Logs, WebStat, and
JSWebStat will show the actual NDS user object generating the traffic!
The following commands are related to this new feature:

The SHAPER [NO] MAC2USER command to enable / disable this feature.
When enabled, SHAPER will query the MAC2USER module any time the
SHAPER would normally output an IP address or MAC address.   The TSE
dynamically links to MAC2USER when enabled.   You can unload SHAPER
at any time, you can unload MAC2USER when the TSE is not actively 
using it ( i.e. use ... NO MAC2USER first )

The SHAPER WHOIS [ a.b.c.d | xx:xx:xx:xx:xx:xx ] command attempts to
return the NDS user object ( or "best name" ) that MAC2USER can find
for the address provided.  Ex: SHAPER WHOIS 00:e0:29:11:0B:AD would
link to MAC2USER and perform a search for the MAC address.  It returns:
0:E0:29:11:B:AD --> .Bob.Developers.wwwTScom  which is the same as
would show up in logs when MAC2USER is enabled.   This lookup returns
the "best name" for the address provided.   The "best name" would be
the NDS object or DNS resolution most recently associated with the
address.

The TSE now supports a "Log, Then Next Rule" and "Log, Then Forward"
rule actions, which can also be used as the action code used for
connection table entries.  This action records the pending packet to
the SHAPER.SNP log ( see below ).

The new memory management code introduced in 0.93f has been modified to
reduce the near term ordering phenomenon observed in the initial
implementation.  A separate cursor is used for accessing the multilist
which reduces the impact of patterns of alloc and free.  The current
cursor value is displayed in the SHAPER DEBUG NODES command.  The
affinity percentage represents the percentage of nodes which are
unevenly distributed.

The -NODETEST command line startup option was added to make finding
issues with node memory handling a bit easier ( for me, at least ).
It makes the node management code wipe the contents of nodes as they
are freed.   If the TSE is improperly handling node memory, this
clearing will typically cause abends within a few seconds rather
than a slow creeping death.   This is actually desirable from a
troubleshooting / debugging standpoint.  Please use this option on
test systems, especially hen testing the TSE's -SMP option, as it can
tease out bugs which might otherwise go unnoticed.

The -NODEBALANCE command line startup option was added to enable
a new dynamic multi-list balancing algorithm.  This algorithm will
be the default un future releases and this option will be eliminated.

The Connection Audit Process writes two extra fields to the end of each
audit record.  The first field, Duration, indicates the duration of the
connection from the first packet received to the last.   The second field,
DeltaT, indicates the elapsed time between the last packet received and
the time the audit record was written.  The Audit Process also outputs
a more detailed comment section outlining the format and meaning of the
fields.

The  SHAPER LOAD AQM PROFILE n filename  command was added to allow
for the management of AQM profiles.  The AQM Profile is a sequence of
packet drop probability values from 0 to 65535, 0 being 0 probability
of a packet being dropped, and 65535 being 100%.   The series of 256
PDP's maps out a depth vs drop rate curve.  This curve is used by the
AQM algorithm to determine if a packet should be dropped or not in
light of congestion.  The Maximum Queue Depth, assigned on a per FIFO
basis, corresponds to the 256'th element in the series.   Comments
starting with % or ; can be used as well as blank lines.  The first
256 numeric elements are read, the rest are ignored.  If less than
256 elements are parsed, an error is reported and the AQM Profile is
NOT loaded into memory.

The -LOGBUF=nn command line startup option was added to enable the
system administrator to allocate additional RAM for the logger daemon.
This allows the logger daemon to more reliably cature packet traces and
other debugging info under bursty load.   This will NOT, however, make
up for a lack of disk throughput under high, constant levels of logging
activity.   ( This parameter has no effect on audit logging, which is
unbuffered. )   This value is used to create a buffer of 2^n bytes, where
n ranges from 16 ( 64KB ) to 27 ( 128MB ) defaulting to 16.   For recording
packets, the default is not enough.   Packet recording and rule debugging
may require 1 to 3.5 K per entry, and thousands of entries may happen
a second.   Try using a value in the range of 20 - 22  ( 1, 2, or 4 MB )
to start, and increase by one to titrate up to a value which allows
the logs to be recorded without overflow.

The SHAPER ROLL DEBUG LOG commands was added to roll the debug log
periodically.   Normally nothing will show up in the log except for the
startup / shutdown event.  However when capturing packets or debugging
rule execution, this option will roll the log without loss of entries
provided the -LOGBUF=nn option is set to increase the logging buffer
to accomodate entries while the log is rolled.   

The TSE includes a packet capture facility which will save captured packets
in Sun Snoop format.   You can use Ethereal, an open source packet analyzer,
to view these files.   You can obtain Ethereal from http://www.ethereal.com
for a variety of OSes.  The PCAP rule actions allow you to determine which
packets you want to log.   This allows you to record interesting traffic
based on any arbitrary criteria.   The Snoop Daemon is started by default,
and will create a SYS:SHAPER.SNP fileinto which capture packets are saved.
The following are added command line options and console commands related
to packet capture:

The SHAPER [NO] SNOOP LOG command enables / disabled packet capture
globally.   If you are interested in capturing traffic around a
specific event, you can configure the TSE's ruleset to capture traffic
using PCAP rule actions, then turn on / off logging as required.

The SHAPER ROLL SNOOP LOG command will roll the SHAPER.SNP file
periodically, moving the current file into /var/log/Shaper/SNOOP.nnn
You must have created /var/log/Shaper prior to rolling the logs.
Provided a large enough SNOOPBUF is provided for, the rolling process
is lossless.  Use this command from CRON periodically to age your
capture files.

The -NOSNOOP command line option is used when you load the SHAPER to
disable the Snoop Daemon process.  If you never intend to use the
packet capture facility, or are low on available RAM, this option
will prevent the spawning of the packet capture thread and its use
of the capture buffer.

The -SNOOPBUF=nn command line option determined the size of the
buffer used to hold captured packets prior to the Snoop Daemon
writing them to disk.  See the -LOGBUF=nn option above for the
meaning of the numeric parameter.

The SHAPER DEBUG SNOOP [DAEMON] command displays debug information
on the status of the Snoop queue.

The SHAPER [NO] SNOOP SCREEN will enable / disable a status screen
for the Snoop Daemon which shows the current state of the daemon
as well as the number of packets captured or dropped because of
the capture buffer overflowing.

The import process now recognizes options which changes the way
subsequent connections are imported.   Options are set by including
a line formatted as follows:

	[ option_name = option_value ]

option_value can be a boolean flag: "yes" "no" "true" "false" "on" "off"
or a signed integer value.

The import option [ reset_creation_time = yes ] will request that
the creation time stamp of existing connections updated by the
import process have their creation time stamp reset to current time.
This has the effect of resetting the absolute TTL timer for the
connection.  This does cause a loss of audit data integrity since
it will be unknown how long the connection really lived, and it
prevents the connection from expiring.  It is an appropriate option
for periodically reloaded lookup tables.

New Features ( Since 0.93f )
~~~~~~~~~~~~

The TSE now offers a API ( programming interface ) so that the TSE can be
used as a utility managed by 3rd party developers.  Please see the TSEAPI
documentation for additional information.

The TSE now exposes all information available via the Web Status Page
through a JavaScript library.  This file is a piece of JavaScript which
can be included / refered to by a web page of your own design.  It
includes a superset of state variables as well as connection table
and configuration information.  This serves two purposes: it allows for
a more data dense delivery of the information, and allows you to create
custom / branded status pages exposing the information in any way you like.
The JavaScript status info commands are the same as the Web Status commands
except for the use of the WEBSTATJS keyword.

Beta 3 allows each individual connection to specify an operation similar
tho those used with rules.  This allows each connection to specify if the
traffic is to be dropped, forwarded, rate limited, priority queued, or
tagged.  This powerful capability allows the TSE to use the connection
entry as a "final rule" in determining how the traffic is to be processed
and allows connections to perform any bandwidth management action desired.

This capability is exposed in three ways:  You can specify a default action
in the Address Mask Edit tab of the configuration utility.  This action is
stamped into each new connection created.  ( The default, um, default
action is to perform no action at all and manage the traffic using the
individual QoS node attached to the connection.  You can modify the
action on a per connection basis by importing connection tables with
entries specifying the action.   In addition, a 3rd party developer
can use the TSE APIs to manipulate the connections directly.

If an action is specified, the rule processor will ignore the returned
rate limiting enqueueing request and execute the action specified.  The
action can be ANY action allowed for rules, though not all actions would
make sense.  For example, each connection could specify a desired
priority queue level to send the connections traffic to.  Similarly
a connections could forward or drop traffic.  This allows for greatly
increased programmability, and simplifies large rule sets which
implement if...then bucket brigades.  In all these cases, the QoS node
is updated to reflect traffic usage, allowing statistics to be gathered
on a per connection basis.

Beta 3 allows a connection to be given a symbolic name.  This symbolic
name can be used in connection tables for creating new connections,
or performing connection sharing.  In connection sharing, a dynamically
created QoS node is used by several other connections by simply pointing
their QoS pointer to the shared QoS node.  Since the QoS node acts as the
point where statistics are gathered, the individual traffic stats for the
connections are lost.   This facility is used when connection actions
used in conjunction with static QoS nodes would not provide enough
FIFO rates.

The TSE recognizes a new address type, type 16, which is a length preceeded
ASCII string.  This address type will be used internally in the TSE to
accomodate various "address to name" capabilities being added to the TSE.
The connection table import / export now accepts source and destination 
addresses formatted as quoted text.  Any address presented as quoted
text is interpreted as a type 16 length preceeded ASCII string.  For
example, consider this address key:

	"Bob is Great, he gave me chocolate cake!" --> 192.168.20.5:1234

would be a valid address where the source is a ASCII string, and the
destination is an IP+Port.  The size of the entire address key is limited
by the -NODESIZE selected, and may not exceed 255 bytes.  The individual
strings my not exceed 255 bytes and is similarly limited by the total
size of the address key as dictated by -NODESIZE.

The -NODESIZE=nnn command line parameter allows the selection of
larger sized memory nodes to accommodate larger address keys and
address string literals.  ( i.e. NDS object names and DNS
addresses. )  This value ranges from 64 to 287 bytes and
should be set slightly above the expected maximum ( key size
+ 32 bytes ) or as required be special applications.

The -NODES=nnn command line option now allows up to 16000000
nodes to be specified.  ( This is enough nodes for 8,000,000
connection entries. )   A minimum of 64 bytes is required for
each additionla Node you specify.

The -BUCKETS=nnn command line option now allows values from
7 to 22, allowing for the accomodation of up to 8,000,000
connection entries to be searched efficiently.  Each
increase in the Buckets parameter results in a doubling of
the size of key data structure in the TSE.  For example,
setting Buckets to 16 requires 1M of additional RAM.
Setting to 17 requires 2M of additional RAM.  A million
connections can easily be accomodated by a Buckets setting of
17 or 18.

A MAC Address template is now available in the Address Mask dialog
which allows MAC addresses in connection tables to be represented
in XX:XX:XX:XX:XX:XX format.

The TSE is now SMP enabled.  Using the -SMP command line option will
instruct the TSE to use NetWare's Multi-Processor Kernel ( MPK ) to
schedule the TSE's more intensive tasks, including auditing, logging,
connection import and export, as well as the web status page generation.
On a busy server with SMP support, enabling the use of SMP may provide
significant increases in the speed of these operations, as well as
reducing the workload for CPU 0, where the TSE's packet scheduling
system is forced to run.  The -SMP option should be considered for
testing purposes only on non-production systems untill enough
feedback is gathered, or you are satidfied with the results.  So far
feedback has been encouraging.

When the -SMP option is selected, the scheduling of tasks on additional
processors is at the whim of the MPK.  Unless CPU 0 is highly utilized,
work may not be farmed out to the other processors, making it appear
as though the TSE is not using SMP.  However, on servers with a highly
active CPU 0, significant improvement in import / export / audit speed
should be reallized.  

The SHAPER ROLL AUDIT LOG command will move the audit file to
SYS:VAR\LOG\SHAPER and maintain a series of 20 logs.  You
must create this folder prior to using the command.  The audit
logs are named AUDIT.xxx where xxx is 000 thru 019.  The logs are
rolled, i.e. shifted so that AUDIT.000 is the most recently
rolled log.  When you roll the logs, the audit process will alert
you that the primary audit log was closed, rolled, and opened.
The roll is completely lossless, no audit records are lost.
Use CRON.NLM or another scheduler ( backup software, etc ) to
schedule the roll as needed.   To maintain, for example,
only 5 days of logs, just schedule the roll to happen
4 times a day.   The command uses a unix "mv" style operation
to relocate the file within the same volume, this does NOT
copy the file and delete the original, which would be far too
time consuming, as such the roll operation typically completes
in under a second and should not impact server performance.


The TSE offers the SHAPER [NO] IMPORT SCREEN command.  This console
command enabled a debug screen for the import process.  It can cause
a significant increase in CPU utilization, and should only be used
for diagnostics.  The scrolling of the output is paused by 
pressing any key.

The TSE is now built using Novell's Modular CLIB.  Previous versions
relied on Watcom Static CLIB.  The change from Watcom's static
libraries will improve the speed of various TSE threads including
logging, import, export, and web status page generation.

The Audit Process was redesigned to take advantage of SMP.  This
required a completely different means of supplying data to the
Audit process to avoid contention between the background aging
of connections ( always performed on processor 0 ) and the
recording and freeing of audited connection data.

The SHAPER EXPIRE CONN command forces the expiration of ALL
connections.  This command should only be used to purge the
connection table in situations where it is not possible to
unload and restart the TSE.  This command marks all connection
entries as have an absolute TTL of 1 second, and the IMMORTAL
bit for the associated QoS node is cleared, allowing the
connection to be purged.  ( See Known Problems ) 

The SHAPER [NO] SMP command toggles the TSE's assignment of
any new threads the TSE creates and overrides the -SMP
command line parameter for any threads created after the
state is changed.  This command does NOT impact the SMP
state of existing TSE threads, nor can it be used to
enable SMP when SMP was not available at the time the
SHAPER.NLM was loaded.

The connection search code was modified to more efficiently
handle long keys.  The previous method used a byte by byte
comparison, the new method will compare up to 4 bytes at a
time.

The SHAPER DEBUG NODES command was added to show debug info
for the new, SMP thread and interrupt safe, memory node management
code.  The nodes are managed as a set of free lists each of
which is capable of being used for allocate and free
operations.  The "Affinity" percentage shown indicates
the percentage of nodes which preferentially migrate to
specific sublists of the multilist.  This value is not a
performance metric, but rather shows a curious near
term ordering phenomenon in the multilist which
is currently not well understood.   Nodes become unevenly
distributed between members of the multilist which exhibit
a slow oscillation.

New Features ( Since 0.93f )
~~~~~~~~~~~~

The WebStat output has been expanded to include a list of connections
broken out by Key Group ID and sorted by the amount of traffic in
the last sample period for the connection.   However for the sort to
work properly, all connections for that key group must use the same
sample period and same rate limiting strategy ( By Bytes / By Frames )
indicated in the associated QoS Node.

The SHAPER WEBSTAT MEMORY x command was added to allow the specification
of the amount of memory to use for generating the connection reports.
The default is 68K, enough for 500 entries.  When more connection
entries are in use, the WebStat process will display only those
connections with activity during the last sample period.  When there
are more connections that this, the WebStat process "gives up" and
will not display the connection information.  This setting takes
effect the next time the WebStat process is started.

The SHAPER WEBSTAT TOP x command specifies how much of a sorted
table should be displayed as a percentage of the whole.  For example,
setting to 40 will tell the TSE to display details on the top 40%
of bandwidth usage and a single summary line for the remainder.  This
is useful in reducing the bulk of the .HTM file and for highliting
"top talkers" in connection tables.  This value ranges from 1
to 100.  When set to 100, all entries are displayed.

The SHAPER WEBSTAT ROWS x command specifies the maximum number
of rows from a sorted table to display.  If you were only interested
in the top 20 bandwidth consuming entries, set this to 20...  This
valie ranges from 1 to 999999.  When set arbitrarily high, all entries
are displayed.

The Web Status page now remind you when a connection table is truncated
because of lack of buffer space to sort the table.  The tables also
indicate the total number of rows ( including those supressed by the
above commands. )

The SHAPER [NO] WEBSTAT SCREEN command enabled a debug screen, which
is presently pretty boring.  Don't bother using this unless you have
problems.

The WebStat page is now opened shared deny read & write when the file
is being updated.  This prevents the browser from getting a partially
completed file and forces the WebStat process to back off if it attempts
to overwrite the file while someone is using it.

The SHAPER [NO] AUDIT SCREEN command, which existed in 0.93c, enables
and disables an audit screen.  Previously this screen had no output.
In this build an abbreviated version of the audit log is simultaneously
displayed.  The activation of the audit thread is controlled by the
SHAPER [NO] AUDIT LOG command which spawns and terminates the audit
process.  The screen is unabailable when the process is not running.
I.e. you must be auditing to the log to use the screen.  


New Features ( Since 0.93c )
~~~~~~~~~~~~
Priority Queuing.  The TSE and Config Tool have been updated to support
priority queuing.  The priority queuing system provides 64 priority queue
levels ( buckets )  These buckets can be used to construct priority queues
with as few or as many levels as needed.  For most applications 3 to 7
priority levels will provide the desired result.  This means that you can
typically construct > 10 priority queues.  The new PriQ Rule Action allows
you to use a rule to place traffic into a given priority level.   A priority
queue uses a QoS note to establish its data rate.

Priority is determined on a packet by packet basis.  The weight of the
pending packet from each level is computed.  The statistical chance
of each of these packets is computed relative to the collective
weight of all pending packets for the queue.  So for example, if two
levels have packets queued, and the next packet for each ahs weights
100 and 200 respectively, the packet with a weight of 100 has a 33%
chance of being sent next whereas the other has a 66% chance of
being sent next.  Two packets with the same weight have a statistically
equal chance of being sent next.  A Level has 4 configurable "metrics"
associated with it.  For this version, Metric 1 is the per byte weight,
Metric 2 is the per frame weight. You control the magnitude of each
effect by varying the proportions of the metrics for various levels.

The 0.93c build, in conjunction with a new version of the config tool,
supports a few additional tests which rules can perform.  Rules can
test if a packet was Sent or Received and if the current frame matches
a connection already learned by the TSE provided a Address Mask.  The
Sent / Received test simplifies rule construction and allows you to
construct a single rule set to handle special cases based on packet flow.

The "If Connection Exists" and "If Connection Doesn't Exist" tests are
EXTREMELY useful for a variety of reasons.  Since the TSE supports multiple
instances of the same address key, provided they reside in different "Key
Groups," You can create any number of connection tables which can determine
rule behavior.  Suppose you had 14,000 workstations.  Perhaps some
subset of these, say, 3,000 of them, required special rule processing,
or perhaps you wanted to filter their frames, or whatever.  You create
a connection table with the workstations in question, all in a given key group
used only for this purpose.  Your rules would check the frames against this
table and allow you to determine any desired outcome.

The 0.93c build, in conjunction with a new version of the config tool,
supports a few additional actions which rules can perform.  Rules can
pass frames through an address mask for the purposes of auditing or
building connection table entries for use with "If Connection..."
tests.  The "Audit..." action passes the frame through an address
mask, builds the connection table entries, and maintain connection
use data, but then forwards the frame.  This allows the TSE to
transparently audit / monitor traffic without imposing a data rate.
( Currently this can be done by setting an arbitrarily high data rate
in the referred QoS node used by the address mash. )

The TSE also support an "Audit Then Next Rule..." action where the auditing
steps above are performed, and the frame is presented to the Next rule.  The
next rule in the sequence can send the frame to additional address masks
for auditing OR perform whatever further actions are desired.  For example,
if you wanted statistics by destination port number as well as by destination
IP address, you can use two rules to send the traffic through different
address masks with different Group ID's.  The audit log or the connection
table would have both types of information in them.

The Configuration Tool has an extra Tab which shows problems detected
with your configuration.  The Validation tab shows the results of
several basic checks against the configuration.  All config elements
are checked to ensure that other elements they refer to are in fact
present.  The validity of the priority queue configuration is also
checked.  To reduce the chances of an invalid configuration, check
the status in this tab just before you save your configs!  No attempt
is made to force you to have a valid config before you save.  This
allows you to have work in progress configurations which, by their
very nature, will be incomplete / inconsistent.

The SHAPER SAVE CONN {connfile} and SHAPER LOAD CONN {connfile} commands
were added to start the connection import or export processes.
These command read / write to the SYS:SHAPER.IMP and SYS:SHAPER.EXP files
by default, or {confile} when one is specified.   The export file can
be edited and reloaded as the import file allowing any arbitrary number of
workstations / connections to be assigned individual independent data
rates.  For example SHAPER LOAD CONN SYS:HOGS.IMP   The speed of this
process is generally >> 1000 connections per second on recent hardware.
Utilization may become high during these processes, they perform frequent
task switches to ensure they are as "nice" as possible.

The SHAPER CANCEL IMPORT and SHAPER CANCEL EXPORT commands are used to
signal the import or export process to terminate itself.  For connection
tables which are exceptionally large, of the order of 1,000,000
connections, the import / export process can takes a few minutes.  These
commands allows you to abort the process as needed.  These processes are
automatically aborted when the NLM is unloaded.  

The SHAPER WEBSTAT filename and SHAPER NO WEBSTAT commands start and stop
the generation of an HTML document which contains status information
including the ouput of the status screen as well as DEBUG command
output.   In the future this facility will be developed further to include
more "pretty" output.  This feature is new to the TSE, and so you should
gain confidence in it on a test server prior to using in a production
environment.  if no filename parameter is specified, the default name
SYS:SHAPER.HTM is used.  To make this document available through a
HTTPd ( web server ) running on the server, simply select a location
which is part of the web server document collection.  Use the short
8.3 formatted file name.  For example, Netscape Enterprise Server
running on NetWare 5.1 would use something like
SYS:NOVONYX\SUITES~1\DOCS\SHAPER.HTM   If the file cannot be created,
the Web Status process will wait 60 second and retry.  If you specify
an invalid file location, use the SHAPER NO WEBSTAT command to stop
the process and start it again with the proper file name.

The SHAPER WEBSTAT PERIOD x command allows the default 2 second refresh
rate to be modified.  Using this command without specifying a value
will display current settings for WebStat even when it is not running. 

The -WEBSTAT command line option starts the Web Status process.  The
default location for the output file is SYS:SHAPER.HTM.  To specify
a location which is exposed by your web server, you should NOT use this
option and instead use the SHAPER WEBSTAT filename command to start
the process.  Since the default HTML documents collection will vary
between versions of NetWare and the HTTPd you are running, the default
location in SYS: is used to allow testing.

The -NODES=X command line load option ( e.g. LOAD SHAPER -NODES=30000 )
allows up to 2,000,000 additional 64 byte memory nodes to be allocated
for use by the TSE.   1 node is required for each queued frame, and 2
for each connection.   This allows the TSE to handle enormous numbers
of connection table entries.  The actual amount of memory allocated
by the TSE is about ( X + 10000 ) * 64 bytes.  The actual memory usage of
the TSE is 700 K to 135 MB depending on the number of additional 64 byte
nodes allocated above the default of 10,000.  When set to 2,000,000
the TSE can support > 1,000,000 independently managed connections.

The -BUCKETS=X command line load option ( e.g. LOAD SHAPER -BUCKETS=14 )
allows additional RAM to be allocated for use by the TSE for organizing
the connection tables.   This value ranges from 7 to 17, with each
increment roughly doubling the amount of RAM used up to a total of 2 MB.
BUCKETS should not be set lower than 10, except for stress testing
connection lookup.   

Table - Connections vs. Buckets

       BUCKETS      Connections
       -------      -----------
	 7-11		 2000
	 9-13		10000
	11-15		50000
	13-17	       200000
	15-17	      1000000
		       
The above table indicates the maximum BUCKETS value needed to achieve the
maximum connection lookup speed.   If you have, for example, 10000
connections, it is unnecessary to use a BUCKETS value of more than 13.
A value 2 to 4 less than the ideal value will often be 

The SHAPER OPTION SORT CONN and SHAPER OPTION NO SORT CONN commands
were added to enable / disable a "most frequently used" optimization
in the connection tables.   When a linear search is necessary to find
a connection, that connection is percolated up to the beginning of
the table so as to be found more rapidly.   Enabling this option does
incur a small penalty in overhead, but often greatly reduces the
effort of searching for connections.  This is especially useful when
ENORMOUS connection tables are present and a small fraction of the 
connections are heavily used at any instant ( i.e. the 80 / 20 rule )   

"Key Groups" were added to the Address Mask processing system.  In
prior builds, ALL keys generated by ALL address masks were thrown
into a pile.  Key Groups solve this problem by allowing you to
partition these keys into several search domains.  When enabled,
the scope of a search for an existing connection is limited to
keys created through masks with the same Key Group.  For example,
suppose you want to maintain separate data rates based on IP
source address through several routed interfaces.  You might have
3 address masks created, each applying a different data rate to the
same IP address depending on which interface the traffic enters.
To ensure that each interface creates a distinct rate for the same
IP, you enable Key Groups on all address masks, and assign a unique
Key Group ID to each ( 0 to 255 ).   Conversely, if you wanted
a single data rate to be applied to all three interfaces, you might
not enable Key Groups at all... or better yet, enable them and use
the same key group ID for all three.  A connection table entry
created by any of the address masks would then apply to the others.
This also improves the efficiency of key searches by further
reducing the number of keys searched.   This only applies when
the masks create keys of identical size and type.  

DiffServ / TOS Support For Transmitted IP frames.  This is a first
attempt at supporting tagging of IPv4 frames.  Please test this
capability in a test environment.  The configuration tool now has
additional rule actions: DiffServ and TOS.  When selected, the
parameter is set to the TOS / DiffServ priority to assign to the
packet.  While the tag will be applied to both inbound and outbound
frames, only frames which are tagged and subsequently transmitted
will be prioritized by the PHB of upstream routing equipment.

The SHAPER OPTION EXPIRY and SHAPER OPTION NO EXPIRY commands were added
to enable or disable connection expiration skulking.  When expiry is
disabled, the TSE will not expire a connection whose time to live has
been exceeded.   This can be used to build connection tables for export
with connections that would normally be to transient to see.  Internally
this is used when connection import / export is in progress to prevent
the deletion of connections when were being exported.

Shaper Config Tool Up / Down Buttons.  Most Tabs in the Config Tool
have buttons to move the selected item up or down.  Any other Rules,
Address Masks, or Priority Queues which refer to the moved item are
updated to reflect the new entry numbers of the moved item.  This 
allows for the re-ordering and movement of rules.  When rules are
moved in this way, only the Rule x and Seq x actions are modified.
I.e. if a rule refers to a "Next Rule" and the "Next Rule" is
moved, then the rule points to whatever takes the original rule's
place.   In other words, direct references, by number, to items
is fixed up automatically.  Indirect references, such as "Next..."
are not.  This is actually desirable when you want to resequence
a series of rules which use the Next... action.  Just make sure
the resulting rule set is consistent.  ( See Known Problems... )

Shaper Config Tool Rule Wizard(s).  I have had the MOST complaints
and confusion about having to define rules in terms of hexadecimal
notation.  In addition, defining rules required an understanding
of the layout of the packets involved.   Despite this harsh limitation,
many people have figured it out.   However for the rest of us who DO NOT
think in hexadecimal I have made a small concession.  It is NOT perfect
but may elevate much of the work for IP frames.   There is a button
on the Rule Edit dialog which brings up a simple screen, the "IP
Wizard," in which you can select which items you want to compare,
and fields to place the data to look for.   The hexadecimal salad
is generated automatically.  ( See Known Problems... ) 

Shaper Config Tool Address Mask Tab.  The Address Mask Tab has been
modified to allow the use of standard templates for source and
destination addresses.  These include various combinations of network,
node, and port.  Checking the "Use Templates" checkbox reduces the
GUI down to two dropdown lists from which the desired masks can be
selected.  In addition, the TSE will format address data in the
connection tables to match that selected.   If you are currently
using address masks, re-create them using the templates to let
the TSE generate human readable address info in this way.

The Connection Import / Export facility properly deals with non-
template addresses properly.  Any non-template address is formatted
in dotted hexadecimal notation.  This is also properly handled when
imported.  The key is expressed as a variable sized dotted hex string
rather than a fixed 32 byte expression with trailing zeros.   This
is more space efficient and looks nicer.

Provided CONNGEN Win32 Console Utility to test connection import file
generation.  Creates a file with the specified number of randomly
selected address types and values.  Useful for testing and proof of
concept for large connection tables.

-NOWAIT command line option added to force SynchronizeStart() to be
issued as soon as the NLM is loaded.  This makes SHAPER act as it did
in previous versions with respect to NCF files.  This options will
prevent SHAPER from locking the console during initialization.

-NOAUDIT keeps SHAPER from spawning the Connection Audit thread.  This
thread creates a SYS:SHAPER.ADT file which is filled by expired 
connections' total throughput data.  This data can be used as an audit
trail of who talked to whom when as well as a means of gathering the
total number of frames / bytes the connection moved.  See above.  If
auditing is not desired ( or connection oriented rates will never be
used ) this option saves the small amount of memory and CPU time
consumed by the thread when it is idling.

-NOLOGGER keeps SHAPER from spawning the Logger Daemon thread.  This
thread creates a SYS:SHAPER.LOG file which is used to store debug
and status messages to disk.  Often no messages are stored.  If
logging is not desired ( or connection oriented rates will never be
used ) this option saves the small amount of memory and CPU time
consumed by the thread when it is idling.

-FORCEPAINT will force SHAPER to update its status screen even if the
screen is not visible ( not the current screen nor being viewed by
remote consoles )   This option imposes additional overhead of
painting the screens, though that operates at the lowest possible
priority anyway.   This can be used for troubleshooting the TSE
when it unloads itself  while nobody is watching: the current
information will be sitting on the screen.   In the case of an abend
( related to the TSE or incidental to it ) the core will have the
current info in the screen as well.  Use for problem resolution
only.

-3X will allow the TSE to load on a NetWare 3.12 server.  However
the TSE will abend immediately when run on a 3.XX server.  This is for
future development / backports to 3.XX, which people have sadly been
asking about with increasing regularity.  DO NOT LOAD SHAPER ON ANY
3.XX SERVER - IT WILL ABEND!!!  


New Features ( Since 0.92k )
~~~~~~~~~~~~

The SHAPER DEBUG LOG and SHAPER NO DEBUG LOG commands were added to
enable and disable logging to the SHAPER.LOG file.  The default setting
will place all debug output to the SHAPER.LOG file, which can grow
substantially - 600K a second or more on a busy server.

The SHAPER DEBUG SCREEN and SHAPER NO DEBUG SCREEN commands were added
so that debug and error messages could be viewed in real time from the
server console.  The logger daemon can activate a debug screen on the
server to display the debugging info.  The default is that no debug
screen is displayed and that all messages go to the log file. 

The SHAPER OPTION RECORD FULL and SHAPER OPTION RECORD BREIF commands
determine if, when recording frame contents to the log, the entire
frame is recorded or just the first 80 bytes - which is normally enough
to troubleshoot rule processing issues.  The default is full frame capture
which generates logs that are approximately 4.5 time larger than the actual
raw frame data.  The BREIF invocation allows the headers of the frames to
be recorded in about 512 bytes.

The SHAPER OPTION RECORD HEX DUMP, SHAPER OPTION RECORD ASCII, and
SHAPER OPTION RECORD BOTH, SHAPER OPTION RECORD NONE  commands
determine the format of the recorded frame data.  This records either
a hex dump, readable ASCII dump, both, or neither.  The default value
is to record both.

The SHAPER OPTION RECORD ECB and SHAPER OPTION NO RECORD ECB determine
if the ECB data structure is recorded during frame recording.  The
default is to record the ECB.  Without the ECB and the RECORD ASCII
or HEX options above, only a placeholder line item is put in the log
indicating the frame ID number used to track the frame.

The SHAPER OPTION DEBUG HASH KEYS, SHAPER OPTION NO DEBUG HASH KEYS 
commands were added to allow the logging and debugging of hash keys
used for connection oriented rates.


New Features ( Since 0.92j )
~~~~~~~~~~~~

The SHAPER OPTION NO QUEUE OVERFLOW command was added to allow the TSE
to prevent QoS queues from overflowing by imposing the largest delay
allowed instead, typically 8 seconds.  This will cause the TSE to
operate without loss or desequencing of frames.  However this will cause
larger numbers of frames to be queued during peak.  Queues which are
chronically in an overflow state may be an indicator of am excessively
strict bandwidth limitation impacting large numbers of connections.  It
can also indicate a denial of service attack in progress or a network scan,
situations where imposing delays has no effect on the rate of reciprocal
traffic.

The SHAPER OPTION QUEUE OVERFLOWS command was added as the converse of
SHAPER OPTION NO QUEUE OVERFLOW.  Frames which would be queued for more
than 8 seconds will overflow the queue and either be forwarded or dropped.
The next two commands determine the outcome...

The SHAPER OPTION DROP OVERFLOWS command was added to control the fate
of overflowed frames.  When used, this command will tell the TSE to drop
and frames which overflow the queue, i.e. would be delayed for more than 8
seconds.  This will force the sending end to resend the frame after it goes
unacknowledged.  This will cause "excess" frames to be discarded.

The SHAPER OPTION FORWARD OVERFLOWS command was added to control the fate
of overflowed frames.  When used, the TSE will forward overflowed frames.
The order of frames forwarded in this way are not guaranteed, i.e. frames
can be desequenced if the queue overflows.  However, desirable frames can
be forwarded in the event of a denial of service attach or other catastrophic
event which floods the queue.  This command counteracts the
SHAPER OPTION DROP OVERFLOWS listed above.

This build also implements logic to keep track of frames which overflow
QoS FIFOs.  These stats are maintained for each half-bind.  For example
the SHAPER DEBUG BINDS consists of entries like:

        0: 'NGRPCI_1_EII', ChainID=D2508180, MBN=D0467B80, State=2, Rule=0
           RCV=121101, FWD=210, FLT=231, OVF=88, QD=13


New Features ( Since 0.92h )
~~~~~~~~~~~~
The monitor screen directly displays the actual queue accuracy in
microseconds.  Ideally this value will be 50 to 100 microseconds, the
theoretical limit of the queuing algorithm.  This is accurate
enough to apply data rates of 1MB / sec without significant deviation
from the desired data rate.  Expect this value to be larger on slower
or busy servers.  See below for additional ways to improve performance.

Pressing the spacebar or Enter key from the monitor screen will
cause it to redraw.  The monitor screen supports the use of
 + and - keys to vary the frequency of screen refresh from 1/8th
to 8 seconds.  Forcing frequent screen updates will impact server
performance on very slow servers. 

The TSE allows the use of interrupt driven packet reception to
service the queue.  This can greatly improve queuing accuracy
on busy servers.  The more traffic the TSE processes, the greater
the improvement.  The SHAPER OPTION INT DEQUEUE and
SHAPER OPTION NO INT DEQUEUE commands activate or disable this
feature.  This code was part of the 0.92h build but was unavailable
to Open Beta testers.  This feature has been rigorously tested in
production and is safe for use.

The TSE now allows the SuperHighResTimer to be read while interrupts
are disabled.  This option can be used to support systems which
cannot handle the enabling of interrupts during packet reception.
The SHAPER OPTION STI and SHAPER OPTION NO STI commands enable
or disable interrupts during this function call.  This option
must NOT be used on Pre NetWare 5.x servers running on '486 or
'386 hardware.

The SHAPER OPTIONS command was added to list human readable state
of each option.  Default values are show with an  *  the non-default
values are shown with a  -  next to them.

The SHAPER UP command was added to quickly put all binds into the UP
state...  This complements the SHAPER BYPASS command.  Be sure your
binds are fully configured ( i.e. rules are selected ) prior to using
this command.

The re-write of transmission rule processing and address mask processing
added a bit of overhead to these processes.  In previous builds, the
TSE would recreate the frame header information and specially process
it for each rule and address mask operation.  This build caches this
information for the current frame.  So while more work is performed
initially, it is only done once per frame.  This essentially increases
TSE performance over the 0.92j and previous builds.  Rule performance
is comparable to that of received frames ( i.e. negligible )


Known Problems ( Current Build )
~~~~~~~~~~~~~~
The 0.93f build of the TSE is believed to be the most stable version of
the TSE so far.  ( Of course, if you've been using the TSE for a while
you will note I always say that. )

The MAC2USER NDS integration code is completely new.  It has not been
tested to the extend that the core TSE codebase has.   Every effort
has been made to compartmentallize the changes so that if you are not
using MAC2USER you should not be incurring any risk.   I would advise
caution when using MAC2USER when SHAPER is loaded with the -SMP
option.   In theory any shaper threads using MAC2USER API calls should
end up funneled to CPU0, I cannot gurantee this as not enough testing
has been done.   Please report any issues with MAC2USER when used
with SHAPER -SMP.   Eventually MAC2USER will be MP safe.

The -NODESIZE option should not be used unless directed.  This
option DOES NOT allow the construction of larger address keys in
address masks.  These keys will ALWAYS be limited to two partial keys
of up to 16 bytes each.

Use of -NODESIZE must be tuned to balance SHAPER's memory usage with
the actual key sizes expectd to be supported.  All nodes are pre-
allocated and are of uniform size.  Allocating bigger nodes allows larger
address keys to be accomodated ( i.e. literal address strings, for DS,
NDS, and other human readable names )   So NODESIZE needs to be set so
that it accomodates the largest key expected, but not much more.  For
network managers who have decided in rediculously over long container
names, or have many levels of containers, the TSE will require larger
nodes.  However, no address literal string can be more than 255
characters.  This limit conflicts with the maximum possible object name
length for NDS.  It is expected that a means to represent longer
names using some internal representation can be used.

Currently the size of an address key can be no more than 255 bytes.  The
length of each partial key can be no more than 255 bytes individually.
So it is currently possible, even using the largest NODESIZE to create
keys longer than the structures will accomodate.  However 0.93h does check
this on import.  

On SMP systems, O/I operations to files on traditional file systems
will typically result in lower performance as compared to NSS file
systems.  This is because the TSE's thread will be marshalled to
CPU 0 for such I/O operations.  This effect may not be appreciable,
or it may be very noticable.  If you find the performance unacceptable
for operations such as connection import / export, try storing the
files on NSS volumes instead.

On SMP systems, it is possible for a slow leak of memory nodes during
the auditing and connection import process.  This is caused by contention
between interrupt driven tasks, such as packet reception, and foreground
tasks on additional processors.  It is possible that the TSE will not be
able to return a freed memory node to the free list.  Please understand
that the TSE allocates a single large block of memory which it manages
itself.  This "leak" is NOT a memory leak leading to the depleteion of
server memory or resources.  It is only a leak of individual 64 byte
units of the preallocated memory used by the TSE.

On SMP systems, if is possible the connection import process will fail to
import a small number of connection entries when the server is under
load.  This has been seen with VERY large ( millions of entries )
connection tables.  The TSE reports these errors on the server console
as "CONN_ImpAHN( ... ) returned 0" ... errors.  The TSE will retry
to import the connection entry at least 4 times before giving up.

On SMP systems, when the -SMP option is specified, the TSE will
perform ThreadSwitch( ) rather than ThreadSwitchWithDelay( ).  This
makes TSE threads more agressive in terms of CPU usage.  However this
is necessary to fully utilize the benefits of SMP.

On SMP systems, it is possible for debug logging ( for example,
recording ELN's or rule processing ) to become garbled because
of contention between accessing the Logger process message queue.
If this is a problem, use the TSE without the -SMP load option,
or use Stop / Start Processors to disable / enable additional
processors during logging.  This has only seen to be a problem
on rare occasions and under extreme server loading conditions.
Generally SMP greatly improved the logging facilities ability
to keep up with large volume logging activities.

The -LOGBUF=nn and -SNOOPBUF=nn options increase the size of daemon
buffers to make logging more reliable under bursty conditions.  However,
it is very easy to enable various logging features which make the
logger daemon I/O bound, creating an ever increasing backlog of
unwritten data so that server performance is impacted.   For example,
a 100Mbps aggregate flow could potentially require 35MB of disk
I/O per second.  This is typically far beyond the capabilities of
the average server.  However, the TSE is capable of recording
short bursts of traffic at this speed of the order of a few seconds,
replying on these load option as well as NetWares disk caching.

It is possible that a backlog of unwritten audit information can
deplete Shaper memory nodes causing the TSE to auto-unload.  This
typically can only occur when: (A) the TSE is already low on nodes,
(B) the server is highly utilized and so can only commit a limited
number of audit records per second, (C) there is an extremely high
volume of audit records to process.  This issue has only been seen
on otherwise healthy servers when the rate of audit records being
generated were in excess of 3000 per second.  Combinations of
increasing the server's disk write performance, adding additional
memory nodes using the -NODES=x command line option, or reducing
the number of audit records by increasing TTL will help.

It is possible that the packet capture facility will fail to
work properly on SMP systems.  The packet capture facility is a
new feature and has not been fully tested with SMP systems under
extreme load.  This could lead to corruption in the packet
capture files.  Please report your experience using this
feature on SMP systems so that issues can be addressed.

The connection creation process is subject to an extremely rare,
but possible, scenerio where the incorrect key group ID is
assigned to a connection.   The resulting connection entry, if
created during packet processing, will be used to handle
the pending packet.  Subsequent packets will not match the
spurious key + ket group combination and force the creation
of the proper connection entry.   It is unlikely this will be
observed during production usage of the TSE.

You are responsible for ensuring that any comparisons against frame
data are performed against frames of the proper length to ensure the
comparison does not extend beyond the end of the frame.  

When a comparison does extend beyond the end of the frame,
a console message is generated and the offending frame data and rule
ID are dumped into the log file.  Because of the possibility of numerous
frames failing in this way, only one frame per second is logged.   To
log all frames you should use the SHAPER BIND x START RECORD and 
SHAPER RULE x START RECORD command.  Any comparison operation beyond the
end of the frame will produce unpredictable comparison outcomes.
Future builds may include a means of selecting a default outcome,
on a rule by rule basis, when a comparison cannot be completed.

Priority Queuing...  The next few sections deal with Priority Queuing
issues.  Basically this facility is so new that there is little to prevent
you from telling the TSE to do something stupid... and the TSE following
orders.  The following are a few common problems exposed during testing
internally, as well as by limited Beta testers:

Priority Queue does not drop or overflow packets.   Traffic is dumped
into the priority queue.  To ensure that the priority queing algorithm
work properly, it is necessary to keep any backlog of packets waiting
for egress in the priority system.  This prevents the QoS node from ever
reaching an overflow condition.  As a result, traffic not responding
to the increasing delays will just pile up in the low priority levels.
This can cause the TSE to run low on nodes, or cause the server to run
low on Rx ECBs.    A resolution to this problem is being sought.
If you encounter this issue, it is most likely caused by runaway
illbehaved portscanning apps, such as KaZaA / Morpheus, which should
be blocked anyway.

Priority Queue configurations can now be modified dynamically.  The
TSE maintains a runtime version of the config which is freshened
when a new config is loaded.  It is generally safe to modify
parameters of existing queue buckets, but there may be issues
with the dynamic addition and deletion of buckets or entire queues.
It is safest to modify these configurations with the TSE unloaded.

Each priority queue will refer to a QoS node which actually applies
the data rate.  For example, if you had a T1 line you would set a
QoS Node to rate limit at T1 speeds.   Each priority level in a given
queue SHOULD refer to the same QoS node.  However this is not enforced
by the TSE.  The config tool does attempt to force this, however.
Failure to use the same QoS node consistently throughout all levels
of a given priority queue is NOT supported and will have strange
effects on the actual data rates.  There may be uses for this, but
I have yet to find any.  If you find an application where this is
somehow desirable, please tell me.

It IS possible to have several priority
queues feeding the same QoS node.  However because the priority
queues are serviced in queue ID order, lower numbered queues
MAY gain an advantage because they will have the first shot at
available bandwidth.   Once again, it will work, but the
results may not be what you are looking for.   It is best to
create a single priority queue feeding a single QoS node.

Traffic which enters a QoS node referred to by a Priority Queue
which bypasses the priority queue itself ( such as by Rule actions
which feed the QoS directly ) will be at a greater priority
than the highest priority level in the queue.  This is because
this traffic is enqueued as the traffic is received, whereas
the priority queue feeds traffic to the QoS node later on in
packet reception or during regular queue servicing.  This bandwidth
robs available bandwidth which the priority queues would otherwise
use.  It is best to send such traffic through the highest priority
level, or create additional levels to the queue to encapsulate
the higher priority.

Always maintain separate queues for inbound and outbound traffic.
Several people have run both inbound and outbound traffic through the
same priority queue ( and hence QoS node ) and wondered why they get
half their purchased bandwidth...  If your bandwidth is purchased for
symmetric flows, you need to have a separate Tx and Rx QoS node and
priority queue.   If you have asymmetric connections, you can apply
different rates on the inflow / outflow.  ( Think cable modems or
asymmetric DSL )

The Config Tool and TSE now implement the "Audit Connections" option
in the Address Mask Edit dialog.  The TSE will NOT record
audit information unless this checkbox is checked.  If you have
connection tables built with a config where the checkbox is not
set, these connections will not be audited.  You can modify the 
Address Mask Flags field to enable auditing.

When connection auditing is enabled, and the server is under
high utilization conditions, the audit process may develop
a backlog of audit records.  This can cause the TSE to run low
on memory and ultimately fail triggering a graceful unload.

When the NLM is unloaded, it is possible for a backlog of audit
records to delay the termination of the NLM.   The TSE will wait
up to 30 seconds for the audit process to finish.  If it does not
complete in this time, the remaining unsaved audit records are
abandoned.   Both of these conditions are a result of chronic
high utilization of the server and so can be avoided except under
the most severe of conditions.  If the high utilization is a temporary
or sporadic condition, configuring additional memory using command
line options will usually fix the problem.  Else disable auditing.

When the NLM is unloaded, any connections which do not formally
expire prior to the NLM unloading are NOT audited.  The audit process
looks at expired connections and saves their information to the
audit file.  This could allow some audit information to escape.
If this is critical, you can use the SHAPER SAVE CONN command
to save the current connection table to a file, the connection
table contains a superset of the audit fields.  Future versions
will allow the TSE to expire all connections, forcing the
audit process to record the final info for these connections.

Key Group ID values range from 0 ( default ) to 255.  Do NOT use
KeyGroup ID's above 63 - even though examples may show use of
values above 63 for this field.   ( Note that there are only
64 address masks, so no more than this number of ID's is needed. )

The WebStat status page generator can display sorted lists of
connection table entries.  Since these are formatted in HTML,
they get to be very bulky.  SHAPER.HTM grows by about 400 bytes
for each connection entry included.   The WEBSTAT MEMORY x command
allows you to allocate space for up to 128,000 connections to be
shown in the status page.  This would result in the generation of
a file in excess of 50MB.  Obviously this is NOT desirable.  It is
best to set WEBSTAT MEMORY so that it accomodate the number of active
connections which should be substantially less than the total number
of connection entries.  The WebStat process will display only
active connections if there is not enough for the entire table.
So, if you have, on average, 1000 active connections, allocating
132000 bytes for WebStat would limit the number of entries displayed
to 1000 and so the resulting output HTM file would be about 400 * 1000
bytes.  An "active" connection is one where > 64 bytes were seen
in the last time period ( about .5Kbps ) 

The HTM file generated by the WebStat process may load VERY slowly
in some browsers because of the enormous tables.  The browser has to
figure out formatting and layout for the tables to render them.  This
takes some time.  It may take a few seconds to display the page
if > 1000 connections are displayed.   ( I dare not think of what
100,000 entries would take! )   This is another reason to
limit the number of connections using the WEBSTAT MEMORY x command.

Since it is possible to mix and match rate types in a given Key Group
the WebStat process looks at the first connection for a given key group
and uses the rate type specified.  WebStat will NOT properly deal with
key groups which contain connections of both by bytes and by frames.

Attempting to use LARGE numbers of connections requires a lot of RAM
to be available to the TSE.  At 1,000,000 connections, 130 MB of RAM
is preallocated by the TSE.  Be sure your server has enough available
memory to meet these needs if you plan on testing or using the TSE
under such conditions.   The amount of RAM used is determined by the
-NODES=x command line parameter.  By default the TSE required only
1 MB of RAM to operate.  The -NODESIZE=x parameter further increases
memory utilization.

The TSE will make an orderly shutdown in the event it runs out of
memory because of growing connection tables.  The TSE will not
dynamically allocate additional memory.  This may be a future
enhancement.

The  SHAPER [ SAVE | LOAD ] CONN filename  commands do NOT accept
long file names.  Use short 8.3 formatted file and path names.
The default location for files when no path is specified is 
the root of SYS:   This will most likely be modified in future
versions to be SYS:ETC - at which point all SHAPER related files
will end up there.

When you disable connection expiry for long periods of time,
and then enable it, the connections will only expire at a rate
determined by the -BUCKETS setting and available CPU idle time.
The connection expiry process happens at extremely low priority
and may take several seconds to purge a huge backlog of
expired connections.   Be aware that disabling the expiration
process for long periods is NOT advised.

The Config Tool's IP Wizard, when used against existing,
hand created rules will "attempt" to fit the existing Data and Mask
into the template.  This process will most likely loose any special
Masking you have created.  Any fields which are not exposed by the
template are NOT preserved.  You CAN use the wizard to create
a skeleton from which you can build a more elaborate rule.  The
IP Wizard is a good prototyping tool for advanced users and a
good general tool for simple rules.  For example, it is great
at creating a rule to find TCP packets from source network
192.168.20.0,  or identifying all AOL ( port 5190 ) traffic.
But incapable of checking for SYN / ACK bit or IP checksums...
Future versions will elaborate on this though.

The Config Tool's IP Wizard generates a single comparison which
is up to 16 bytes long.  The IP Wizard will warn you when you have
exceeded this limit.   Selecting multiple items amounts to a 
Boolean AND for each.  Meaning that ALL items must match for the
criteria to be met.  If you need to perform Boolean OR for several
items, you need a separate rule for each.  This was a pain in the
past, but the IP Wizard, and the ability to move Rules around,
make this much easier.

The Config Tool's IP Wizard, when presented with a class A, B, or
C network address ( ending in zeros ) will automatically "assume"
a corresponding mask ending in zeros.   ( Watch the hexadecimal
portion of the Mask field change as you enter in various IP addresses
and you can see what is happening. )   If you prefer a different mask,
or no mask at all, you must manually edit the Mask field just before
you accept the changes in the IP Wizard.

The Config Tool's Up / Dn Buttons, when used with address masks,
DOES NOT properly modify Address Mask number of Connection Exists
and Connection Does Not Exist comparison types.  You MUST manually
re-enter the new Address Mask number into the field.  This will be
fixed in future versions.

When -BUCKETS=x is set high, it may take up to a minute for a
connection to be purged after its has formally expired.  Normally
this delay is only a couple seconds.   ( This is a result of
having to wade through up to 1,000,000 connections to check for
expiration. )   Future versions of the TSE will provide for
more aggressive expiration.

The CONNGEN utility program provided is a WIN32 Console program used
to generate test connection files with random types and addresses.  It
does NOT check for duplicate addresses when building the tables.  It is
possible for the same address to be present several times.  When loading
this test data, be aware that the number of connections created may be
several percent less than the number of entries in the files.  These
will show up as "modified" rather than "created" connections in the
import summary.  DO NOT USE THESE RANDOM CONNECTION TABLES ON A PRODUCTION
SYSTEM. See the CONNGEN.TXT file for more info,

If an attempt to unload the TSE is made when the server is chronically
operating at 100% utilization, various TSE threads will fail to die in
the time allowed ( about 40 seconds ) This will cause the TSE to ABEND.
This failure mode has only been observed under extremely severe, 
artificial test conditions.  It is not expected that this could occur
in production environments.  Do not unload the TSE when the server is
severely CPU bound.  Future builds of the TSE will include extreme
measures to accommodate this failure mode.  ( See Note Below )

Similarly another ABEND condition may occur when the TSE is loaded on
a server that is chronically operating at 100% utilization.  The TSE
may be unable to start various TSE threads in the time allowed
( about 40 seconds ) This will cause the TSE to unload, potentially
exercising the conditions above.  This failure mode has only been
observed under extremely severe, artificial test conditions.
It is not expected that this could occur in production environments.
Do not load the TSE when the server is severely CPU bound.
( See Following Note )

In Beta 2 0.93c when the TSE is requested to terminate, a flag is
set which increases the priority of TSE operations which normally
yield the CPU...  In virtually all cases this eliminates the stall
condition mentioned above. 

The TSE uses the payload length in performing all calculations.  This
means that the TSE ignores the length of the media header when
managing traffic.  Future versions of the TSE will provide a means
use either the packet size or the payload size in these calculations.
( This is an improvement over prior versions, as previously each
portion of the code used a different means to calculate the length. )

The TSE can be forced to load on a NetWare 3.XX server using the -3X
command line option.  This option is for development only, to be used
for a possible backport of the TSE to NetWare 3.XX.  When 0.93c is
loaded on a NetWare 3.XX server, it will ABEND the server.  DO NOT
LOAD SHAPER ON A NetWare 3.XX SERVER, IT WILL ABEND!!!   ABEND I SAY!

There is very little validation of the data entered through the config
tool.  Bad or invalid values may cause unexpected results.  Please
double check your configurations before placing them into effect.
Most often the TSE will simply ignore invalid rules or QoS nodes.
In very rare instances, especially bad values can cause server ABEND.
This has, to our knowledge, not happened in production environments.
The moral to the story is: if you have questions, ask!  Including having
me take a look at proposed configuration files.

NetWare 5.x may perform frequent polling of the CD-ROM drive when no CD is
loaded in the drive.  This causes frequent periods lasting up to 60 ms long
where no interrupts are serviced.  This severely reduces the accuracy of
the TSE's queuing mechanism.  To prevent this from impacting TSE 
performance, do one of the following:

	- Always keep a non bootable CD in the CDROM drive
	- UNLOAD IDECD.CDM, which unload the CDROM device driver

This is NOT a TSE code issue, the periodic 20ms to 60ms pauses impact all
threads and interrupts on the server - essentially halting the server during
the polling of the CD-ROM drive.  The TSE is particularly sensitive to this
huge "work stoppage" as the normal resolution of the queue is about 50 to 100
microseconds, and the pauses caused by the CD-ROM polling are 500 to 1000
times larger than that.   This issue is severe enough as to impact the
interrupt servicing of the NIC.   Novell has a new build of IDEATA.HAM
that may eliminate this problem on NetWare 5.x servers.  Please look for
ideata5a.exe in the file finder or the following TID

  http://support.novell.com/cgi-bin/search/tidfinder.cgi?2956780&header=no

The TSE has many counters which may roll over when they reach 4 billion.
The roll over will cause erroneous calculation of several values
displayed on the monitor screen.  If this is troublesome, please
unload and reload the TSE to reset these counters.  A manual and
automated means for resetting these counters will be provided in
future builds.  The rollover of these counters should no impact the
the TSE's ability to manage traffic. 

Following any error in accessing, reading, or writing the config file, the
TSE will wait 1 minute before retrying the operation.  This accommodates
SYS: dismount conditions that exist while the server goes down.  To
prevent these errors while the server is going down, unload the TSE prior
to downing the server.

The TSLast value maintained at a QoS and Connection level is 
incorrectly set to the microsecond timer value rather than the
clock tick timer.  But on reviewing the code, it appears that the
TSLast value is not used, and that expiration decisions seems to be
completely determined by TSETA ( a microsecond timer ).  Therefor
the TSLast value exported and shown in the JavaScript WebStat info
are essentially menaingless.  I already have an idea how to make use
of this field to enhance bandwidth banking.  But for the moment
please ignore the value.  This value is ephemera and is ignored
by the import process.


Revision History - Milestone Releases
~~~~~~~~~~~~~~~~

 TSE Beta 2 - 0.93h - Pre Beta 3

	- Added per connection action support!  Whoo Hoo!
	- Modified connection table structures for larger keys
	- Modified node management for adjusting node size
	- Cleaned up hideous BeginThreadGroup() mess!
	- Fixed possible abend issue if threads failed to startup
	- Added variable length literal string address type
	- Modified import and export for string partial key
	- Created separate path for nodes feeed by threads
	- Added TSEAPI code for alpha testing by developers 


 TSE Beta 2 - 0.93f - Major Fixes Rev 

	- Fixed many bi-directional address mask issues
	- Added support for the Audit Connections checkbox
	- Added several minor enhancements to Web Status
	- Added comparison to test number of connections
	- Minor fix to priority queuing algorithm
	- Separated out address formatting

 TSE Beta 2 - 0.93c - Second Public Beta Release ( Beta 2 )

	- Tweaks to Import / Export
	- Added comparison overrun reporting
	- Enhanced priority queuing ( more efficient + bug fixes )
	- Added Audit... rule actions
	- Added Web Status featurette
	- Cosmetic changes to status screen

 TSE Beta 2 - 0.93a - Second Private Beta Release

	- Added rudimentary priority queuing
	- Added connection audit logging facility
	- Added connection import / export facility
	- Restructured decision - execution interface for rules 
	- Added various command line switches
	- Added ( working ) connection sorting option
	- Added Key Groups to address keys to be grouped
	- Increased bindings permitted to 16
	- Added Alpha DiffServ / TOS tagging code for TX ( & RX )
	- Added SynchronizeStart( ) To Make SHAPER more NCF friendly
	- Added Connection Expiry Queue API ( Will be open API soon )
	- Added templates for address masks ( in GUI Config Tool )
	- Beautified address output in logs, added tabs "to make perdy"
	- Added additional conditionals for rules

 TSE Beta 1 - 0.92l - Minor Fixes Rev

	- Added several SHAPER OPTION commands to better control debugging
	- Added a debugging screen to mirror debug log contents
	- Restructured connection code to accommodate import / export

 TSE Beta 1 - 0.92k - Minor Fixes Rev

	- Re-Wrote handling of rules and address masks for outbound frames
	- Added watch dog to rule processing to detect endless loops
	- Added console commands to control queue overflows
	- Added display of overflow stats to half-bind debug output
	- Added cache for TX frame reconstruction for improved performance

 TSE Beta 1 - 0.92j - Minor Fixes Rev

	- Fixed ABEND in SHAPER.NLM related to rule processing

 TSE Beta 1 - 0.92i - Minor Fixes Rev

	- Enabled existing OPTION INT DEQUEUE command for Open Beta use
	- Added OPTION RACHET SHRT command to accommodate '386 / '486 CPUs
	- Modified SuperHighResTimer code to prevent timer backsteps
	- Added code to properly handle config file when server goes down
	- Fixed extremely rare startup memory allocation failure ABEND
	- Modified queue monitoring to collect actual delay accuracy values
	- Enabled existing OPTIONS command to list current options settings
	- Modified monitor screen and other cosmetic issues

 TSE Beta 1 - 0.92h - First Open Beta Release

	- Added OPTION commands to manage small and zero delay queuing
	- Vastly increased performance of connection aging process
	- Resolved bandwidth banking roll over bug
	- Added code to report Queue Preemies and log them to the log

TSE Alpha 4 - 0.83f - 4th Private Alpha Release

	- Improved performance of the queuing mechanism
	- Reduced queuing resolution from 4ms to 100us
	- Fixed free( null ) ABEND during failed startup if alloc() failed

TSE Alpha 3 - 0.52c - Minor Fixes Rev

	- Fixed garbage in frame pseudoheader assembler bug
	- Increased throughput of the logging facility to about 1MB / sec

TSE Alpha 3 - 0.51u - 3rd Private Alpha Release

	- Added code to manage connection oriented rates
	- Added code to handle and debug address mask and hashing
	- Modified configuration file format to accommodate address masks
	- Fixed garbage in frame pseudoheader assembler bug
	- Added counters to handle hash statistics
	- Greatly enhanced rule processing
	- Accommodate transmitted and receive frames with same code
	- Added code to dump hash information to log file

TSE Alpha 2 - 0.37e - 2nd Private Alpha Release

	- Added completed QoS algorithm and remove test harness code.
	- Removed Rule processing test harness code
	- Added logging facility
	- Added DUMP command to create cores for debugging
	- Fixed minor bug with QoS options flags being reset during CFG load

TSE Alpha 1 - 0.21r - 1st Private Alpha Release

	- Fixed bind ABEND problem where invalid ESR address is passed
	- Fixed ABEND issue where linked list is linked to hell
	- Added code to provide a programmable delay ( QoS Test Harness )
	- Added Console Command processor and support code
	- Added code to support half-bind concept
	- Added prescan TX stack support
	- Added final RTAG allocation, memory allocation, stack reg code

TSE First Code - 0.13b - First Source, First Build

	- Implemented trivial Prescan RX ESR
	- Implemented IOCTL IF to OS
	- Implemented basic RTAG allocation
	- Fixed ESP + 3 bug in call to OS, moi mal
	- Stole C NLM husk from SCREAMER NLM sources
	- Added initial LLIST and LSLCALLS sources and test harnesses

NLMSHELL 1.43t - Prototype WATC 10.6 NLM For Lifecycle 


Testing Information
~~~~~~~~~~~~~~~~~~~
Prior to the public release of each version of the TSE, it is tested on
various servers with various versions of NetWare and other common
applications.  Each test cycle includes the processing of in excess of
5 billion ( 5E+9 ) frames at a rate of 4000 frames per second or greater.
The NLM is also tested for loading and unloading issues by loading and
unloading the NLM every 5 minutes on test servers, testing at least 1000
load unload cycles.

In response to the last bug identified in rule processing code, an
additional testing phase was added for this and future releases.  This
includes a gauntlet of rules and address masks to ensure these execution
sections function as expected.

Official Beta 2 Haiku
~~~~~~~~~~~~~~~~~~~~~

Shaper Chokes Out Fluff
Good Packets Fly In And Out
Garbage Downloads Slow

Floods Of Music Files
Rivers Of Movie Trailers
Now Just A Trickle

iMesh Met Its Match
gNutella Downloads Slowly
Useful Traffic Flows

Evil Morpheus
Your Reign of Terror Ends Now
Super Nodes Are Down
