Everyone,

SEG.NLM
========

I have spent quite a bit of time on updating SEG.NLM to fix a few issues and add new features that have been requested because of the memory fragmentation problems we've experienced with NetWare.  There are a lot of good updates in SEG that need to be known about.

I've tested it on NetWare 5.1, 6.0, and 6.5 ... although by far most of the testing has been done on NetWare 6.5.  If you find problems with it, please let me know the version of the Operating System and the support pack level where you found the problem.  I'll set it up and try to duplicate it and then fix it.

You'll want to read the comments at the bottom of the document.  I talk about some of the new stats I show and why they have values that sometimes don't make sense.

Main fixes/enhancments to know about:

1.  Memory leak.  Every version of 1.60 SEG.NLM has a memory leak.  It is very slow, but it is still there.  It is fixed now.
2.  Additional stat on the main page:  Largest VM Cache Segment (available only in NetWare 6.5).
3.  New <F10> page with options.  It looks kind of ugly, so if you have suggestions, please let me know.
4.  This is version 1.71.  I'm doing this to know what version is where because I've given out a few v1.70 instances to customers.

I have had a few people come to me and ask what happens if NLMs get unloaded and then reloaded.  In one of the previous emails I sent out, I stated that SEG.NLM does not do tracking on NLMs loading and unloading and that there is a possibility of some kind of problems (not abends), but where stats for that specific NLM are not correct (i.e., displaying that it still has memory when it doesn't ... because it isn't loaded ... or something else weird like that).  Most of this "funny" stuff will happen on the NLM screens in SEG.NLM.  This will not affect the main cache pool readings or anything else like that.  To get around this problem (if you see it and want to fix it), SEG will have to unloaded and reloaded so it can rescan all of the NLMs in memory.

I have yet to spend time to actually address this issue.  I have a few theories:  Check the number of loaded NLMs currently against the previous amount (done once every 5 seconds), and if the numbers are different (up or down), then rebuild the list and look for additions or subtractions to it.  Now ... I just need to code it in.

I still get questions (and this really is an old question) as to why the Total Cache Buffers in MONITOR do not match what the cache buffers in SEG are.  The reason for this is simple:  MONITOR takes the Current number of cache buffers and adds to it the available cache buffers that NSS says it will give back.  That combination is what the number displayed in MONITOR represents.  SEG breaks those two numbers apart and shows what NSS has, what NSS will give back, and what the current number of available cache buffers are.

I have fixed lots of little stupid display bugs, and other things that just didn't look right (I have definitely not reported everything I've changed).  However, all of the major changes are noted in the document (and some stupid ones).

Fixes:

The list was getting too big to save in an email ... so I've transferred the items fixes prior to July 15 to a Word Document (yes, MS Word).  If you'd like the full fix list, check out the doc attached.

July 15:

1.  Adjusted and made more changes to the <F10> page.
2.  Fixed the way the .ms screen refreshes so it doesn't flash (which is a good thing).

July 16:

1.  I removed information from the .ms screen (<F10> and then <F6>) because it wasn't necessary to show a ? when I didn't have the data ... just took that stuff out.  I don't know when I'll get around to putting it back it ... ???

July 17:

1.  Fixed the refreshing issue with the <F10>.  I had to make a new "clear" function to make it easy, but that wasn't too bad.
2.  Turned off CONTROL character checking.  What this means is that the NLM will not response to CTRL-C or CTRL-S anymore.  I was messing around and found out that I didn't have routines to handle the CTRL-C ... so I had to disable it.  It will normally do an "abnormal" NLM termination.  Well ... since I don't have that function, it is disabled.  If enabled, the NLM would hang (the server would be fine) and not allow you to unload it ... not good.
3.  Turned on the VM Largest Segment reporting.  It has been in the NLM for a good month or so, I just wasn't ready to "let it go".  It is now turned on by default.  If you're using a program that doesn't recognize this new column, just delete the column in the SEG.CSV log file and then the program should work.  Beyond that I need to get the two programs I have updated to include this new piece of data (along with the server up time ... they don't look at that yet either).
4.  Added more information to the <F10> screen on the set parameter.  To the right side of them (except the vm garbage collection one) I have a little blurb that talks about it if certain features are enabled.  For instance, for the garbage collection set parameter, if that is changed to 1 minute (exactly), then the blurb will talk about the server being in an aggressive mode for garbage collection (which is true).  If this is set to 1 minute 1 second, that aggressive mode is disabled, so I don't display anything.  Anyway, things like that.

I'm getting really close to being done.  I can't think of anything that is NECESSARY to put in the NLM now.  Everything is turned on, all of the refreshing issues are gone, and the data is correct (from what I can tell).

July 27:

***** I didn't want to make any more changes, but I had to make this to fix the NLM so it would work correctly. *****

1.  Made some minor changes to the <F10> page.  The changes are internal and will just help me (the developer) make new additions/changes to this page easier.  For the output on the page, the only difference is now it says "Default setting" if the garbage collection parameters are at their default setting.

2.  More changes to the <F10> page.  I've been noticing that PORTAL.NLM (NetWare Remote Manager) reports some interesting statistics for setting the "file cache maximum size" to on my server.  I have 2 GB of ram on the server, and NRM says to set this to 2.8 GB.  I've had a discussion with the developer about it, and here's what he said:

"I looked at the tuning page it that looks good to me. Remember it is logical memory. The user space only needs 300 MB, which leaves 3.7 GB for the whole server. The peak NLM memory is only 375 MB, almost doubling that by giving it say 700 MB will still leave 3 GB for the file system cache. Now with the physical memory at 2 GB, that is the largest size the file system cache will be, but just in case you add more memory later we tell you to set it where it would be with more memory."

So NRM will give a "forward looking" approximation as if the customer is running with 4 GB of ram.  Be aware of that.

The formula used for calculating this set parameter is the following:

File Cache Maximum Size = 4 GB - (OS code/data memory + userHighWaterMark + nlmHighWaterMark)

Because my server doesn't have a ton of NLM usage, that number is low.  If it were higher, then the resulting number from this formula would be lower and would "look" better to me.

Right now I'm torn between reporting what NRM reports, or putting in a formula like this:

File Cache Maximum Size = OS code/data memory + userHighWaterMark + nlmHighWaterMark

I am going to leave it the way it is ... to match NRM ... but an understanding of how the number is generated is helpful if customers question why the number is what it is.  I would suggest that customers NOT change this setting unless they know exactly what they're doing and what the change will do to the server's memory pool configuration.

3.  This isn't a change to the NLM, but a change to TID 10091980.  I've changed some instructions in the TID to say that Novell technical support should be contacted before changing the garbage collection parameters or the "file cache maximum size" parameter ... which directly relates to the above change.  The reason for this is that customers can hurt themselves if they don't know what they're doing.  We're giving them the rope to hang themselves.  I hope this prevents a few problems.  By understanding the above change, this new addition to the TID should be easy to explain.

As far as the garbage collection setting, this should only be changed on HEAVILY used servers.  By default NetWare moves memory around just fine.  In a few cases we've seen problems where it can get behind in moving memory between the cache pools ... this setting will help address that.

July 28:

1.  I completely removed the display of the "file cache maximum size" setting that duplicates what NetWare Remote Manager shows.  I'm not comfortable showing that number without having other recommendations or explanations for how it operates.  So, now I refer the admin to the TID (10091980) and hopefully they'll contact support before making any changes.

I talked with Steve Eaton about this and he and I both agree that a new formula for determining the file cache size needs to be determined.  Right now the formula assumes a little too much (especially for servers under 4 GB of ram).  So, I'll need to add that on to the ToDo list for the next version.  I'm not going to put it in for this version ... I want everything to remain stable and have no code changes (like this one I just did).  I'd like for all of the code to just stay in one place and have no major issues that I have to go make changes to fix.  Once that's done, I'll release it.

Because it has been fairly good (except for the changes to this single parameter), I still want to release it at the end of this week (July 30).

So, if two more days pass without changes ... out it goes.  It's stable, reports correct numbers, and I haven't had to fix a major issue with it in more than 2 or 3 weeks I'd say  (probably more).  The last serious issue I had was with the log file stuff on June 18th.  Most of the changes made were done for visibility and readability by customers since then.

SEG Comments:

1.  I have noticed that sometimes the suggestions in NetWare Remote Manager differ from what the suggestions in SEG.NLM are.  Here's something to keep in mind:

If the L==P space is set to 1 GB, and NoRM is recommending say 2.3 GB (or whatever), see if that makes sense.  Don't trust NoRM completely and blindly do anything.  If NSS needs that amount of ram or filesys doesn't or whatever, then go ahead and do it.  However, I have seen where NoRM recommends one thing, and SEG recommends something else.  I need to talk with the developer about it, but to me the NoRM recommendation looks wrong.  Anyway, think what you want.

Let me know if you have any questions or comments or suggestions for the NLM.

FUTURE:
========

There have been a number of requests and updates that I want to do (this is a little bit of duplication on what is in the attached document.  These updates, and maybe more *MAY* be in the next released version that I get out (when I get time).  There's no guarantee on anything here because coding is only done when I have free time at work:

1.  Winsock layer.  I want the NLM to communicate with anything else that hits it on a specific socket ... client program, another NLM, anything.  This is currently partially implemented but not running in the version shipping now.
2.  Rename the log files from seg.csv to seg-<server name>.csv
3.  Put a visual representation on the <F10><F8> page for fragmentation percentage ... some kind of user entertainment.
4.  Make a trending page ... I actually have this in the code already, I just haven't spent time on it lately (8-10 months ago ... maybe more).  I want to finish this.
5.  Make a stats page for what SEG is doing.  You get this at the system console, I'll put it on a screen now.
6.  Put color on the console screen writing.
7.  Make a menuing system on the top line of the screen.  You would be able to use this or the function keys to navigate through the NLM.
8.  Email notification.  When there's a problem (threshold reached or whatever), then send out an email notification to a list of people.
9.  Fix the "file cache maximum size" formula for displaying space to use on the server.
10.  Fix NLM issue where they load/unload and the memory tracked internally in SEG isn't updated properly.
11.  Add more stats to describe the amount of physical ram used by the server in the FS Cache pool (for NSS and TFS file systems).  Right now I don't describe any filesys.nlm caching levels.  I think I should add that.

Those are all of the issues I can think of (besides the little ones that always come up as I'm working through the code).