Registry Entries on Interix and Subsystem for Unix-based Applications

by Rodney Ruddock

Overview

There are several registry entries that can be set for Interix and the Subsystem for Unix-based Applications (SUA). Many of these have default settings or meanings when they do not appear in the registry. Having the registry entries set correctly provides a better experience and better expected behavior when using programs. Worse, there have been some Microsoft Hotfixes that have clobbered these registry entries.

What are Registry Entries?

Registry entries are a database of entries that programs and device drivers on the Windows OS use or can use for their configuration. Programs are not required or forced to use the registry. However, this was set by Microsoft as the norm to be followed instead of information be stored in a disperate collection of configuration files. Each of the registries is known as a "hive" and can be backed-up and restored. Many programs continue to use configuration files for historical and ease of use reasons.

How to View and Change Registry Entries

You can view registry entries in three ways:

Using the REGEDIT (win32) program
Using the regdump utility
Writing your own program

Writing your own program isn't going to be addressed in this article.

The regdump utility is part of the lsreg package that can be installed. While the lsreg utility allows you to print out the value of a known registry entry, similar to the ls utility, the regdump utility prints out the key values that affect Interix. The importance of certain registry values have changes over time. While some entries were important for Interix 3.0 and are not for Interix/SUA 6.0 they are still currently output. This can be used for quickly checking that registry entries have not be modified by Windows updates, hotfixes or other programs. You can only view the values with regdump and lsreg at this time.

The REGEDIT win32 GUI utility is probably the most well known tool for looking at and changing registry entries. REGEDIT is also known as "regedit.exe" and can be found in the "%windir%" directory. You can start it by creating a shortcut on your desktop or from Run on the Start menu. The shortcut is usually the best idea because you will need to use this tool for many other programs too.

List of Registry Entries Interix and Subsystem for Unix-based Applications

There are many registry entries. They are grouped into four groups below based on their importance.

Most Important Registry Entries

These three are the most important registry entries because they affect the behavior of programs for users, developers and administrators the most. You will need to reboot the system after changing any of the registry entries for them to take effect. Here is a table with brief descriptions:

Entry Description	Location	Recommend Value
Case sensitive filesytem	HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\ObCaseInsensitive	0x0 (DWORD)
SetUid for executable binaries	HKLM\SOFTWARE\Microsoft\SUA\EnableSetuidBinaries	0x1 (DWORD)
Allow 'su' to root (Administrator)	HKLM\SOFTWARE\Microsoft\SUA\EnableSuToRoot	0x1 (DWORD)

In the table above you can substitute "SUA" with "Services for UNIX" when you are using Interix 3.5 (component of Services for Unix, aka SFU).

ObCaseInsensitive

It is important to note that this entry is labeled in the reverse of most registry entries. By having this entry set to 0x0 (off) you enable case sensitive fileystem behavior. Thus a file name "Foo" and "foo" are treated as different files just like on Unix systems. When this registry entry is active (0x1) and both files "Foo" and "foo" exists only one of them will be accessible though both continue to exist.

This is one of the few registry entries that affects Interix, Subsystem for Unix-based Applications and NFS while located in a "different" section of the registry hive. As such, several different hotfixes over the years have clobbered this entry when they should not have.

When a filesystem is case insensitive and it remains "case preserving". This means when a file is named "Expo" the capital 'E' is preserved even when it is accessed as "expo".

SetUid

This enables executable files with the "setuid" and/or "setgid" permission bit turned ON to be run as the owner and/or group of the file. Normally a file is run with the owner and group based on the user account that made the request to execute the file. Normal Unix (and Linux) operation have this capability active. Having this ability active with Interx/SUA means the behavior of the system overall can be more Unix-like.

Having the SetUid ability active or ON means special programs such as daemons (servers in Win32-speak) can have the powers of the local administrator ("root" in Unix-speak). This makes it possible for the program to change to another user to be that user (after a login for example).

EnableSuToRoot

The setuser or "su" capability allows one user to assume the identity of another user. The switch is controlled and authorized usually by password. It is possible for the local administrator to assume any user's identity without a password but this instance is restricted to local device access only. The EnableSuToRoot allows users to 'su' to the Administrator account by password. This includes users in the Administrators group. Otherwise the Administrator account cannot be su'd to from any account.

Useful Registry Entries

The registry entries in this seection are useful for tuning the system to make it easier to deal with. These registry entries are all found in:

        HKLM\SOFTWARE\Microsoft\SUA\

You will need to reboot the system after changing any of the registry entries for them to take effect.

PrincipalDomain -- This is the value returned by the pdomain utility. When not set the subsystem determines the default value (usually the domain the machine belongs to rather than the name of the local machine). If the machine is part of more than one domain you can explicitly set it here so that the correct value is always used. It may also be set if there is some technical confusion. The Principal Domain is the domain name used when a fully qualified user name (FQUN) is not specified (e.g. DOM+user).
LoginDefaultDomain -- This is different than the registry entry PrincipalDomain. This registry entry affects the daemons telnet and rlogin. You will see this domain name displayed with the login prompt. When not set the prinicpal domain is the assumed value. This can make remote logins easier when wanting to avoid entering the FQUN.
MaxOpenFiles -- This affects the select() and sysconf() API's. It can also affect user processes' "rlimit" value for descriptors. The default value is 1024 when this entry does not exist or is not set. The maximum value this entry can be set to is 4096. Programatically use the sysconf() API call for _SC_OPEN_MAX for the current active subsystem value. With select() API calls you should always use the actual maximum descriptor value + 1 instead of the value from sysconf() though it will be valid -- never use FD_SETSIZE.
MaxProc -- This registry entry is used with older Interix releases. Current releases no longer use this entry. When this entry became active there was a fixed table size for the number of available process slot. Currently the process slots are increased when the table becomes full.
ApiThreads -- Adjusting this entry should be avoided by most sites. The Interix subsystem has a pool of threads to service user system calls. The number of subsystem threads is normally adjusted based on the call load and available system resources. Setting this registry entry will start the subsystem with a minimum number of threads in the pool. Minimum value is 1 and maximum is 3000. The initial starting minimum is usually 5.
ApiThreadsMax -- Adjusting this entry should be avoided by most sites. The Interix subsystem has a pool of threads to service user system calls. The default maximum is 30. The maximum value for this entry is 3000. The larger the value the more resources used. It is usually best to let the system balance the number of threads in the pool.
ForkWait -- This is an older entry. It was originally created as a workaround by Softway Systems (the original Interix developers) to a NT kernel bug with multiple CPU's. Current Interix/SUA versions should not need to use this registry enrty.
ConsoleTtyName -- an entry that appears not to have been implemented though the include file has it listed.

Least Used Registry Entries

These registry entries for Interix and Subsystem for Unix-based Applications date back enough years that some of them may continue to be present with the most current version/release of Interix/SUA but adjusting them serves no helpful purpose since much of this information is internal. While the PriorityAdjustment may look interesting it is best left alone because of the serious side-effects it can have.

PriorityAdjustment -- do not play with this entry.
_CS_PATH -- important for Interix 3.0 and earlier.
Current_Release -- ignored.
Current_Version -- ignored.
InstallPath -- important for Interix 3.0 and earlier.
InstallRoot -- important for Interix 3.0 and earlier.
InstallRoot_Win -- important for Interix 3.0 and earlier.
Version -- ignored.
ProductVerMajor -- ignored.
ProductVerMinor -- ignored.

System Startup Entries

There are registry entries that normally are set once and should never need to be changed when the OS is installed. For a period of time there were some dim-witted companies who kept trying to change them as a "security measure". This seems to have been cleared up over the past several years. However, if you have an older system (Windows 2000 or XP) and have some very older AV or security software they may get clobbered.

There are registry entries to direct the subsystem manager which subsystems are to be started and where the binaries are to be found. These entries affect the Win32/Win64 subsystems (AKA CSRSS.EXE), the Interix/SUA subsystems (aka PSXSS.EXE) and once upon a time the OS/2 subsystem. Here are the key entries that affect Interix, Subsystem for Unix-based Applications (SUA) and OpenNT (the old name for Interix).

If the Interix subsystem (includes Interix, Subsystem for Unix-based Applications (SUA), SFU and OpenNT) cannot be started then it is possible that one of these registry entries has been affected.

If you need to restore any of these values you will need to REBOOT the system for the new value to take effect.

Entry Description	Location	Recommend Value
Required Subsystems to Activate (2000, XP)	HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\Required	Debug Windows
Required Subsystems to Activate (2003, Vista, 2008, 2008/R2, Win7)	HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\Required	Debug Windows Posix
Optional Subsystems to Activate (2000, XP)	HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\Optional	OS2 Posix
Optional Subsystems to Activate (2003, Vista, 2008, 2008/R2, Win7)	HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\Optional	Posix
Binary for Subsystem	HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\Posix	%SystemRoot%\system32\psxss.exe

Required

You should never adjust these entries. These indicate the subsystems that are required to be started by the Subsystem Manager. With little surprise you see "Windows" listed. The "Debug" is also important if you want to get any useful information when something goes wrong. Again, let these entries alone: do not delete or modify.

Starting with Windows 2003/R2 Interix/SUA began shipping as a feature (formerly known as a "component") of the base OS installation. Thus for Windows 2003/R2, Vista and 2008 "Posix" has been added to Required. This means that "zzInterix" (described below) is no longer needed.

If "Posix" is missing from this entry then the Interix subsystem will not be able to be started on W2K3/R2, Vista, W2K8, W2K8/R2 and Win7. Before adding "Posix" to this entry you should be sure that you have already installed the Subsystem for Unix-based Applications (SUA) feature.

Optional

This entry lists the optional subsystems that the Subsystem Manager can start. In this case "optional" means that these are not required to be started as part of the boot process. Originally these subsystems where started by the Subsystem Manager once a program requested service support. That is, when an Interix 2.2 program started (the binary executed) a request was made to the Subsystem Manager to start the Interix subsystem. Since this would make the response for the first program to run slow the "zzinterix" program used to be listed to start at boot-time so the first user program would have fast response.

As you may be aware, during the orignal development of NT the OS/2 operating system was going to be used as the primary subsystem. Things changed to have Windows as the primary subsystem and OS/2 was moved to optional. Having "OS2" are part of this entry remained for many years but was finally removed for Windows 2003/R2 and after.

As mentioned above in the Required description, "Posix" (meaning Interix/SUA) was added to the Required section starting with Windows 2003/R2. However it still remains as the only Optional entry starting with 2003/R2 too. Earlier releases (W2K and XP) need to have "Posix" listed in the Optional section so the Subsystem Manager knows how to start it. "Posix" should continue to be listed here and not be missing.

Posix

This is the registry entry that specifies the path to the subsystem that the Subsystem Manager should use. This value (path) has been the same since NT 3.0 until now (Win7). It should continue to remain the same into the future. There should be no reason for you to modify the value.

If the value has been modified restore it. Then reboot the system.

Conclusion

The Interix (and Subsystem for Unix-based Applications) subsystem use settings in the registry to control some of the actions, behavior and available functionality. By knowing which registry entries have what effects you can correct problems, adjust settings to your liking and check when your settings have been incorrectly changed.

You may also want to review the FAQ entries if you are having problems.