UDMA tips and tricks under NetWare

If you found this page helpful, interesting, or you just maintain NetWare server(s), please tell me what harddisk controller and what version of NetWare do you use HERE and see what others use. You can add your comments too, after voting... Thanks!


Have you ever tried to run your NetWare with UDMA capable drivers?
Have you ever wondered if it can be done or why is it not working?
Have you ever tried to find drivers to work with your harddisks?
Have you ever imagined your NetWare mirroring a 2*120GB software RAID-1 within 90 minutes, with 0-5% processor utilization?
[screenshots: 4.11 mirroring 2*120GB]
[screenshots: 3.12 disk, partition and volume info]

Well, then this page is for you!

        

Around the end of November 2003, I finally had the time and chance to contact Novell's IDEATA developer Mr. Noji Ratzlaff, and explained my "problems" about the driver not working on 3.x, and 4.x in DMA mode.
Knowing that 3.x and even 4.x are obsolete and unsupported products, I had no wild expectations about my problem, but to my greatest suprise, I found warm welcome and helpfulness at a range which is rare even in the open source community.

As a result of his work and my feedbacks, we finally have an UDMA capable IDEATA.HAM driver which works on 3.12, 4.10 and 4.11 and of course 5.x and above too. (And yes, of course this version is also handling disks above 8GB)

For NetWare 3.x, the NBI.NLM is outdated and probably will never be updated. Because of NBI31X's limitations, some chipset ID's were not recognized by, and not passed to IDEATA.HAM.
This meant, that if it didn't recognize the chipset (controller) then it didn't do UDMA, and was extremely slow (compared to how fast it could have been).
In the "early years" NetWare's Bus Interface (NBI) did not scan device 1F of the PCI bus. NetWare 3.x discontinued, NBI development was stopped for this version, then of course the 1E limit was "frozen".
Also, recently chipsets started to position the only important device besides LAN cards, - the ATA/SATA controllers - at device 1F, which of course NBI31X was "unable to handle". Relatively successful quirks was made for ideata.ham to "step over" the NBI limitations, but now the ultimate solution is now out: a "patched" NBI31X, where the 1E limit is changed to 1F and also patched to allow scanning of all functions on a pci device.
This NBI should be considered unofficial, even though only several significant byte is changed compared to the official one. Since the "original" is unsupported too, this doesn't make too much difference.
For easy recognition, this driver has new date (2005 November 22), and version number with a subrevision for marking the unofficial state (v2.27u).
So first, you should try these drivers along with the appropriate NBIs (packed in ideata.zip) and only if that doesn't work, only then try load ideata.ham /legacy, but be prepared that it could hang during mounting the volumes.

So, here are the newest ideata.ham + other required ingredients in one zip file.
It contains the latest drivers, and the older NWPAs and the patched NBIs for NetWare 3.12/3.2 and 4.x. Of course the drivers are to be used for NetWare 5.x and 6.x too. (Full file listing)
The drivers are not written by me, these are the official Novell drivers, can be downloaded from the following link, I just "bundle" them together and wrote this "user guide" combining the collected information.




Current Issues:
At the end of 2006 I've experienced that some motherboards hardfreeze after idehd.cdm is autoloaded. I've seen this with Gigabyte GA915 and GA945 motherboards, this happens only when used with SATA drives. Strangely, PATA drives work well on the same motherboard.
I've sent a motherboard at that time to Mr. Noji Ratzlaff (the developer of ideata and related modules), and he was kind enough to look into that issue. Not much found out though, seems to be related to some strange BIOS issue.

 - 1st mail, thoughts on the freeze
 - On SATA-PATA "differences"
 - My mail on Intel motherboards seem to be working

If you have any information on amotherboard exhibiting or not exhibiting this behavior, please let me know, I will list it here.
Please send motherboard manufacturer and model (possibly link to manufacturer's page) with BIOS and chipset info. Eg.: AWARD 6.0PG / Intel xxx.



Links:
-  Novell's certified storage drivers page - usually older drivers for 3rd parties
-  Debugging Information Provided by IDEATA.HAM v4.10 - TID 10085776. Switches and technical stuff.
-  Media Manager and NWPA updates - TID2971030 (last modified 08JUN2005) Latest NWPA for NetWare 6:
This fix can ONLY be applied to a NetWare 6 server with sp5 installed. Not intended for 5.1 or 6.5 OS.
mmft2a.exe; 303450 bytes; Date/Time: 06-08-2005/02:48PM
-  Updated NWPA for NetWare 5.1 - TID 2969829, NWPA v3.09f - 2004/08/23 (post SP7 / beta SP8). Direct download: nwpa5pt4.exe
-  Support for new IDE chipsets for Netware 5.1 - TID2969018. New NBI.NLM (v2.35o - 2003 Nov. 12) for 5.1 [note, this NBI is packed in my zip, works with 4.11/4.2]
-  Cosmetic issue with displaying incorrect mirror percentage details
-  IDEATA.HAM's supported controllers
-  www.lavalys.com Professional, multilingual system information tool (formerly aida32)
-  A good tool (and PCI database) for checking PCI devices from DOS. Make sure to download the latest pcidevs.txt for pci.exe.
-  Identify your ATA controller's chipset visually...
-  Hungarian version of this article - quite outdated...
-  Russian discussion on this topic


LAN driver related links:
-  Novell's certified LAN drivers page - usually older drivers
-  odi33g.exe contains newer MSM/MSM31X (v3.95) and ETHERTSM (v3.80) than the basic 411sp9,
but there is a separate ODI331 directory/"product" which contains newer MSM (3.98a) and ETHERTSM (3.81) modules.
However, they are for 4.x only - though ETHERTSM might work on a 3.12 too.
Anyway, odi33g.exe is definitely a "must have" for 3.1x-s.
ODI33g.exe readme and download link [local mirror of odi33g.exe]
-  3Com LAN drivers (3C509, 3C90x, etc.)
-  Broadcom B57 (57xx), B44 (4401), etc. Used by many vendors, such as Dell, HP (Compaq), etc.
-  Intel LAN drivers for IEPRO100/1000
-  Marvell drivers (Yukon, 88Exxxx, etc.)
-  Realtek drivers
-  VIA driver download page
-  Attansic/Atheros ATLX [ATL1/ATL2] driver download page



Useful links and tips for NetWare 3.12 (and 3.2):
- NetWare 3.12 Product updates - Minimum (system) patches to be applied:
  - Warning! Do not use loader.exe from ldr312ft.exe, as loading ideata.ham for the secondary channel will likely abend your server.
  - 312ptd.exe and mmattrfx.nlm from nwpaup1a.exe, but it is packaged with the ideata.zip above.

- My "patch collection" for 3.12 (can be used for 3.11 too). Contains all important updates needeed to copy to SYSTEM and PUBLIC directories.

Because of memory issues, NetWare 3.12 will have problems with larger disks. It depends on the number and size of files, etc. I have successfully installed 2*120GB HDDs but when I wanted to install another pair of 120GB HDD mirrorred, I started to have serious problems, such as no memory available to create partition, unable to establish mirror, etc.
It seems that the Maximum Short Term Alloc Memory (32MB) is not enough after about 2*150GB. To raise this settings you have to patch the server.exe. It is not hard, find more details here.

To save memory when dealing with large (for 3.12) harddisks (around 80-120GB), you should:
- check for and remove any memory handler (himem.sys, emm386.exe, etc.) from config.sys, but this is very basic...
- insert the following lines into the STARTUP.NCF
  SET Maximum Alloc Short Term Memory = 33554432
  SET Cache Buffer Size = 16384
- use 64K block size for the volumes
- have separate volume at least for the system volume.
  Let's say, you have a 120GB disk with 8GB SYS, 80GB DATA1, and 22 GB INSTALL.
  This way, if you exeperience memory problems because of full volumes, you will still be able start the system and manually mount DATA1 only.
- purge frequently to avoid unnecessary growth of DET (Directory Entry Table).
  For automatic purging use cron.nlm and toolbox.nlm
  Copy cron.nlm and toolbox.nlm to sys:system, load cron.nlm at startup (end of autoexec.ncf is OK), put the following line in sys:etc\crontab

    #start purgeall.ncf at every day 23:30
    30 23 * * * purgeall.ncf

  and create a purgeall.ncf in sys:system with the following content:

    #purging deleted files older than 14 days, on volumes: sys, data1, install - or whatever you specify
    purge sys:* -a -d=14
    purge data1:* -a -d=14
    purge install:* -a -d=14



FAQ:
Q:   In monitor.nlm disk adapter information shows: DMA - not used. Is it really not used? [screenshot]
A:   "The DMA resources listed by Monitor and similar utilities refer to the ISA DMA controllers onboard the motherboard, of which there were four, and are seldom used by today's modern controllers. Today's IDE controllers use their own (much faster) DMA controllers, implemented in what are known as SouthBridge chipsets. You will not see this resource being used by any SCSI driver dated after 1999 and any IDE driver dated after 2001." - Noji Ratzlaff, ideata developer


Q:   How can I tell if (U)DMA is used or not then?
A1:   upon loading ideata.ham with the /info switch, examples:
IDE chipset ID FFFFFFFF (Unknown or New) present: = BAD, or
IDE chipset ID 24CB8086 (Intel ICH4) present: = GOOD
Some example results on GOOD (DMA capable) configurations:
  Intel PIIX4 controller
  Intel ICH controller - Compaq Deskpro running NetWare 6.5
  VIA VT8233 controller
  VIA VT82C686A controller - ASRock M266A (Pentium 4) motherboard
A2:   when driver is loaded, you can issue a "load ideata.ham /info /?", you will see something like this:
  Hardware configuration for 1 IDE chipset:
  00  FFFFFFFF  017001F0  02000000  00010000  03F6 = 50  00 00 67 84 18 E0 50
      00000000  037603F6  00000000  00000000  0376 = 50  00 00 60 00 00 F0 50
      00000000  00000000  0000000F  0F0E0080  0002 = 19
      00010000  01000000  00000000  00000000  000A = FF
or

  Hardware configuration for 1 IDE chipset:
  00  24CB8086  000001F0  01000000  00010000  03F6 = 50  00 00 67 84 18 E0 50
      1FC6D020  000003F6  01018A45  00000045  0000 = 9F  C8 0D 6D BB 47 D2 8D
      000000F9  0000000E  00000000  000E1110  F002 = 20
      0001F000  00010000  00000002  80891043  F00A = 04
First 4 byte is the Chipset ID. So FFFFFFFF means BAD...

For more details on the values, download this excel sheet and copy your values into it, or visit the following link:
Debugging Information Provided by IDEATA.HAM v4.10 - TID 10085776

TIPS:
- Try "load ideata.ham /legacy" if it reports FFFFFFFF to see if it helps.
- You can load the newest IDEATA on any older machine not supporting UDMA, just use LOAD IDEATA /NODMA or LOAD IDEATA /SAFE.
- You can get the list of swithes by issuing a LOAD IDEATA /? and LOAD IDEHD /?
- /DEBUG and /VERBOSE can be useful in some cases too.


Q:   Does IDEATA and its belongings work on NetWare 3.11?
A:   No. IDEATA requires NWPA, and 3.11 can't load NWPA because server.exe doesn't have the MM_xxx routines, etc. So no. You need at least a NetWare3.12.


Q:   Does ideata.ham support disks above 137GB?
A:   Yes. IDEATA.HAM and IDEHD.CDM drivers support 48-bit LBA addressing (accesing ATA drives above the 137GB "limit"), for drives up to 2 TB, since June 2001. (Unlike Windows 2000 or Windows XP).
More useful links on this topic:
- http://www.48bitlba.com/issues.htm
- http://maxtor.custhelp.com/cgi-bin/maxtor.cfg/php/enduser/std_adp.php?p_faqid=960
- http://www.dewassoc.com/kbase/hard_drives/hard_drive_size_barriers.htm




How to collect and export information on the system: If you're successful, don't hesitate to tell it to me, preferably with the tested configuration and NetWare version, and a related conlog.
The UDMA capabilites can be tested the following way:
    LOAD IDEATA.HAM /INFO and LOAD IDEATA.HAM /INFO /?

If you want to send me a snapshot of your working and unlisted controller (please do so), you need to load conlog, which is very painful, as it requires CLIB and it's belongings, plus NIT.
1) you can simply copy the whole SYS:SYSTEM to C:\SYSTEM, then start NetWare without loading any drivers.
    Then do a "search add c:\system" and a "load nit", "load conlog file=c:\logfile".
    From then on, you can load the IDEATA driver.
2) Or you can start the server normally, "load conlog file=c:\logfile", then unload and load IDEATA. Might be easier.
3) As a 3rd solution for 3.x, 4.x; download a "hacked" conlog.nlm v1.0. It has a hardcoded log directory, this one will log to c:\console.log
    Requires clib.nlm, streams.nlm and a3112.nlm.
Download the required files in one pack: for 3.12 and for 4.11.


If you found this page helpful, interesting, or you just maintain NetWare server(s), please tell me what harddisk controller and what version of NetWare do you use HERE and see what others use. You can add your comments too, after voting... Thanks!




Comments, suggestion, flame, praise to:

version - yyyy.mm.ddDescription
v1.00 - 2003.12.08First version
v1.01 - 2003.12.12text modifications, vt82c686a
v2.00 - 2004.01.20new version, aka: success story
v2.01 - 2004.01.24updated ideata package with version 2004.01.13 (was 2004.01.08)
v2.02 - 2004.02.22minor corrections
v2.03 - 2004.03.123.12 links and tips
v2.03 - 2004.03.13modified conlog tips
v2.04 - 2004.04.17added 411base.zip
v2.05 - 2004.04.18screenshots
v2.06 - 2004.05.03more screenshots, NetWare v3.12
v2.07 - 2004.06.01updated ideata package with ideata.ham v4.15 (2004.04.24), links, tidy-up, added FAQ and Current Issues
v2.08 - 2004.06.02updated ideata package with ideata.ham v4.15 (2004.06.01), updated suppctrs.txt
v2.09 - 2004.06.07updated ideata package with ideata.ham v4.15 (2004.06.04)
v2.10 - 2004.06.09link to new NBI.NLM (v2.35o - 2003 November 12) for 5.1 added. Repackaged ideata.zip, with latest nbi.nlm and nwpa.nlm I found for 5.x/6.x
v2.11 - 2004.07.16expanded 48bit section.
v2.12 - 2004.07.29Added survey on NetWare.
v2.13 - 2004.10.05updated ideata package with ideata.ham v4.16 and idecd.cdm v4.03 (2004.09.11)
Even more screenshots, NetWare v3.12 on a XEON 2.8GHz with 2GB RAM
v2.14 - 2004.10.20updated ideata package with ideata.ham v4.18 (2004.10.10), idecd.cdm v4.04 and idehd.cdm v4.13 (2004.09.30)
v2.15 - 2004.11.15Added (Intel) BIOS - Legacy ATA settings screenshots
v2.16 - 2004.11.18link to new NWPA.NLM (v3.09f - 2004 August 8) for NetWare 5.1 added.
updated ideata package with ideata.ham v4.19 (2004.11.13)
v2.17 - 2004.12.09updated ideata package with ideata.ham v4.19 (2004.11.30)
v2.18 - 2004.12.18updated ideata package with ideata.ham v4.20 (2004.12.15)
v2.19 - 2005.01.19updated ideata package with ideata.ham v4.21 (2005.01.15)
v2.20 - 2005.02.01updated ideata package with idehd/idecd.cdm (v4.14/4.05 - 2005.01.25)
v2.21 - 2005.03.14recommendation on not to use ldr312ft.exe
v2.22 - 2005.06.01updated with ideata.ham and idehd/idecd.cdm (v4.22/4.15/4.06 - 2005.05.05)
v2.23 - 2005.07.08updated with ideata.ham and idehd/idecd.cdm (v4.23/4.16/4.07 - 2005.05.15)
v2.24 - 2005.07.10Added my all-in-one 3.1x public/system update pack for "public demand".
v2.25 - 2005.07.14Added fresh nwpa update link for NetWare 6.0
v2.26 - 2005.08.02ideata.ham 4.24, refreshed support modules, see content.txt
v2.27 - 2005.08.08ideata.ham 4.25
v2.28 - 2005.09.20patched NBIs [1E->1F]
v2.29 - 2005.09.27found "developer at work" sticker...
v2.30 - 2005.11.20updated with ideata.ham and idehd/idecd.cdm (v4.26/4.17/4.08 - 2005.08.31)
v2.31 - 2005.11.22another patches for NBI to allow to scan all functions
v2.32 - 2005.11.24was told that NBI (v2.35o) for 5.1 is working for 4.x and 1F and function scan is fixed in that... Silly me.
v2.33 - 2006.03.10updated with ideata.ham and idecd.cdm (v4.27/4.08 - 2006.01.30)
v2.34 - 2006.09.21updated with ideata.ham and idehd.cdm (v4.28/4.18 - 2006.06.06)
v2.35 - 2006.09.28Details for 3.12 / Short term memory alloc limit patch added
v2.36 - 2006.11.06updated with ideata.ham (v4.29 - 2006.10.15)
v2.37 - 2007.01.31updated ideata.ham/idecd.cdm/idehd.cdm (v4.31/4.11/4.19 - 2007.01.01 [Released 2007.01.13])
v2.38 - 2007.04.03updated ideata.ham/idecd.cdm/idehd.cdm (v4.32/4.12/4.21 - 2007.03.03 [Released 2007.03.27])
v2.39 - 2007.05.11updated ideata.ham/idecd.cdm/idehd.cdm (v4.33/4.13/4.22 - 2007.04.04 [Released 2007.05.09])
v2.40 - 2007.06.15updated ideata.ham (v4.34 - 2007.05.05 [Released 2007.06.13])
v2.41 - 2007.11.03On the 6th of October, Mr. Noji Ratzlaff has left Novell (driver development department dissolved), so it is very likely that there will be no further ide driver updates. :-(
v2.42 - 2008.07.24Freeze issues with some motherboards
v2.43 - 2008.08.08For popular demand, LAN driver section is added with download links. Fixed broken links.
v2.44 - 2010.04.20Updated/removed some dead links to Novell's site, local odi33g.exe mirror, MSM/ETHERTSM versions


[Hungarian version] - [Russian discussion on this topic]

Backreferences:
http://www.novell.com/coolsolutions/netware/features/tips/t_ideata_ham_nw.html
http://www.novell.hu/faq/netware.shtml


[main page]