Table Of ContentWindows 10 System Programming, Part 2
Pavel Yosifovich
Thisbookisforsaleathttp://leanpub.com/windows10systemprogrammingpart2
Thisversionwaspublishedon2020-08-09
ThisisaLeanpubbook.LeanpubempowersauthorsandpublisherswiththeLeanPublishing
process.LeanPublishingistheactofpublishinganin-progressebookusinglightweighttoolsand
manyiterationstogetreaderfeedback,pivotuntilyouhavetherightbookandbuildtractiononce
youdo.
©2020PavelYosifovich
Contents
Chapter13:WorkingWithMemory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
MemoryAPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
TheVirtualAlloc*Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Decommitting/ReleasingMemory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
ReservingandCommittingMemory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
TheMicroExcel Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
WorkingSets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
TheWorkingSets Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
PrivateHeaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
HeapTypes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
HeapDebuggingFeatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
TheC/C++Runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
TheLocal/GlobalAPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
OtherHeapFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
OtherVirtualFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
MemoryProtection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
LockingMemory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
MemoryHintFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
WritingandReadingOtherProcesses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
LargePages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
AddressWindowingExtensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
NUMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
TheVirtualAlloc2Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Sumamry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Chapter14:MemoryMappedFiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
MappingFiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Thefilehist Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
SharingMemory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
SharingMemorywithFileBacking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
TheMicroExcel2 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
OtherMemoryMappingFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
CONTENTS
DataCoherence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Chapter15:DynamicLinkLibraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
BuildingaDLL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
ImplicitandExplicitLinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
ImplicitLinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
ExplicitLinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
CallingConventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
DLLSearchandRedirection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
TheDllMainFunction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
DLLInjection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
InjectionwithRemoteThread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
WindowsHooks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
DLLInjectingandHookingwithSetWindowsHookEx . . . . . . . . . . . . . . . . . . . . . 95
APIHooking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
IATHooking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
“Detours”StyleHooking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
DLLBaseAddress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Delay-LoadDLLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
TheLoadLibraryExFunction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
MiscellaneousFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
Chapter16:Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
WinLogon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
LogonUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
LSASS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
LsaIso . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
SecurityReferenceMonitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
EventLogger. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
SIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Tokens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
TheSecondaryLogonService . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Impersonation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
ImpersonationinClient/Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
SuperPrivileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
AccessMasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
SecurityDescriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
TheDefaultSecurityDescriptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
CONTENTS
BuildingSecurityDescriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
UserAccessControl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Elevation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
RunningAsAdminRequired . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
UACVirtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
IntegrityLevels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
UIPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
SpecializedSecurityMechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
ControlFlowGuard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
ProcessMitigations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Chapter17:TheRegistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
TheHives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
HKEY_LOCAL_MACHINE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
HKEY_USERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
HKEY_CURRENT_USER(HKCU). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
HKEY_CLASSES_ROOT(HKCR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
HKEY_CURRENT_CONFIG(HKCC). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
HKEY_PERFORMANCE_DATA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
32-bitSpecificHives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
WorkingwithKeysandValues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
ReadingValues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
WritingValues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
DeletingKeysandValues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
CreatingRegistryLinks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
EnumeratingKeysandValues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
RegistryNotifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
TransactionalRegistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
RemoteRegistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
MiscellaneousRegistryFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
Chapter 13: Working With Memory
Inchapter12,welookedatthebasicsofvirtualandphysicalmemory.Inthischapter,we’lldiscussthe
various APIs available to developers for managing memory. Some APIs are better to use for large
allocations, while others are more suited to managing small allocations. After you complete this
chapter, you should have a good understanding of the various APIs and their capabilities, allowing
youtochoosetherighttoolforthejobwherememoryisinvolved.
Inthischapter:
• MemoryAPIs
• TheVirtualAlloc*Functions
• ReservingandCommittingMemory
• WorkingSets
• Heaps
• OtherVirtualFunctions
• WritingandReadingOtherProcesses
• LargePages
• AddressWindowingExtensions
• NUMA
• TheVirtualAlloc2Function
Memory APIs
Windows provides several sets of APIs to work with memory. Figure 13-1 shows the available sets
andtheirdependencyrelationship.
Chapter13: WorkingWithMemory 2
Figure13-1:Windowsuser-modeAPIs
We’ll look at the APIs from the lowest level to the highest. Each API set has its strengths and
shortcomings.
The VirtualAlloc* Functions
The lowest layer - the Virtual API is the closest to the memory manager, which has several
implications:
• It’s the most powerful API, providing practically everything that can be done with virtual
memory.
• Italwaysworksinunitsofpagesandonpageboundaries.
• It’susedbyhigher-levelAPIs,aswe’llseethroughoutthischapter.
Themostfundamentalfunctionthatallowsreservingand/orcommittingmemoryisVirtualAlloc:
LPVOID VirtualAlloc(
_In_opt_ LPVOID lpAddress,
_In_ SIZE_T dwSize,
_In_ DWORD flAllocationType,
_In_ DWORD flProtect);
Anextendedfunction,VirtualAllocEx,worksonapotentiallydifferentprocess:
Chapter13: WorkingWithMemory 3
LPVOID VirtualAllocEx(
_In_ HANDLE hProcess,
_In_opt_ LPVOID lpAddress,
_In_ SIZE_T dwSize,
_In_ DWORD flAllocationType,
_In_ DWORD flProtect);
VirtualAllocEx is identical to VirtualAlloc except for the process handle parameter, that must
havethePROCESS_VM_OPERATIONaccessmask.
VirtualAlloc(Ex)cannotbecalledfromaUWPprocess.Windows10addedavariantofVirtualAlloc
thatcanbecalledfromaUWPprocess:
PVOID VirtualAllocFromApp(
_In_opt_ PVOID BaseAddress,
_In_ SIZE_T Size,
_In_ ULONG AllocationType,
_In_ ULONG Protection);
TomakethingssimplerforUWPprocesses,VirtualAllocisdefinedinlineandcallsVirtualAllocFromApp,
sotechnicallyyoucancallVirtualAllocfromaUWPprocess.
ThereisanotherVirtualAllocvariantintroducedinWindows10version1803calledVirtualAlloc2.
It’s dealt with in its own section. There is yet anotherVirtualAlloc variant (VirtualAllocExNuma)
that is used specifically with Non-Uniform Memory Architecture (NUMA). We’ll discuss NUMA in
itsownsectionaswell.
We’ll start by describing the basic VirtualAlloc function upon which all the rest are built.
VirtualAlloc’smainpurposeistoreserveand/orcommitablockofmemory.
ThefirstparametertoVirtualAllocisanoptionalpointerwherethereservation/committingshould
takeplace.Ifit’sanewallocation,NULListypicallypassed-in,indicatingthatthememorymanager
shouldfindsomefreeaddress.Iftheregionisalreadyreserved,andacommitmentinsidetheregion
is needed, then lpAddress indicates where the committing should start. In any case, the address
is rounded down to the nearest page. For new reservations, it’s rounded down to the allocation
granularity.
Allocationgranularityiscurrently64KBonallWindowsarchitecturesandversions.Youcanalways
getthevaluedynamicallybycallingGetSystemInfo.
dwSize is the size of the block to reserve/commit. If lpAddress is NULL, the size is rounded up to
the nearest page boundary. For example, 1 KB is rounded to 4 KB, 50 KB is rounded to 52 KB. If
lpAddressisnotNULL,thenallpagesintherangeof lpAddresstolpAddress+dwSizeareincluded.
Chapter13: WorkingWithMemory 4
flAllocationType indicates the type of operation to perform. The most common flags are MEM_-
RESERVE and MEM_COMMIT. With MEM_RESERVE, the region is reserved, although the function fails if
lpAddressspecifiesanalreadyreservedregion.
MEM_COMMITcommitsaregion(orpartofaregion)previouslyreserved.ThismeanslpAddresscannot
be NULL in this case. However, it is possible to reserve and commit memory at the same time by
combiningbothflags.Forexample,thefollowingcodereservesandcommits128KBofmemory:
void* p = ::VirtualAlloc(nullptr, 128 << 10, MEM_COMMIT | MEM_RESERVE,
PAGE_READWRITE);
if(!p) {
// some error occurred
}
A VirtualAlloc Bug
Technically, you can commit and reserve memory at the same time by using MEM_COMMIT alone.
Strictlyspeaking,thisisincorrect.ThereasonthisworksgoesbacktoabugintheAPIthatallowed
it. Unfortunately, many developers (knowingly or not) abused this bug, and so Microsoft decided
nottofixitsothatexistingcodewouldnotbreak.Youshouldalwaysusebothflagsifreservingand
committingatthesametime.
Any committed pages are guaranteed to be filled with zeros. The reason has to do with a security
requirementstatingthataprocesscanneverseeanymemorybelongingtoanotherprocess,evenif
thatprocessnolongerexists.Tomakeitexplicit,thememoryisalwayszeroedout.
Thisisnotthecasewithfunctionssuchasmallocandsimilar.Thereasonwillbeclearlater
inthischapter.
Reserving a region of memory that is already reserved is an error. On the other hand, committing
memorythatisalreadycommittedsucceedsimplicitly.
ThelastparametertoVirtualAllocisthepageprotectiontosetforthereserved/committedmemory
(see chapter 12 in part 1 for more on protection flags). For committed memory, it’s the page
protectiontoset.Forreservedmemory,thissetstheinitialprotection(AllocationProtectmember
in
MEMORY_BASIC_INFORMATION),althoughitcanchangewhenmemoryislatercommitted.Theprotec-
tionflaghasnoeffectonreservedmemory,sincereservedmemoryisinaccessible.Still,avalidvalue
mustbesuppliedeveninthiscase.
The return value of VirtualAlloc is the base address for the operation if successful, or NULL
otherwise.If lpAddressisnotNULL,thereturnedvaluemayormaynotequallpAddress,depending
onitspageorallocationgranularityalignment(asdescribedearlier).
Chapter13: WorkingWithMemory 5
ThereareotherpossibleflagstoVirtualAllocexceptMEM_RESERVEandMEM_COMMIT:
• MEM_RESETisaflag,thatifused,mustbetheonlyone.Itindicatestothememorymanagerthat
thecommittedmemoryintherangeisnolongerneeded,andsothememorymanagershould
notbotherwritingittoapagefile.Thecommittedmemorycannotbebackedbyamappedfile,
only by a page file. Note that this is not the same as decommitting the memory; the memory
isstillcommittedandcanbeusedlater(seenextflag).
• MEM_RESET_UNDOistheoppositeof MEM_RESET,statingthatthecommittedmemoryregionisof
interest again. The values in the range are not necessarily zero, since the memory manager
mayormaynothavereusedthemappedphysicalpages.
• MEM_LARGE_PAGESindicatestheoperationshoulduselargepagesratherthansmallpages.We’ll
discussthisoptioninthe“LargePages”section,laterinthischapter.
• MEM_PHYSICAL is a flag that can only be specified with MEM_RESERVE, for use with Address
WindowingExtensions (AWE),describedlaterinthischapter.
• MEM_TOP_DOWNisanadvisoryflagtothememorymanagertopreferhighaddressesratherthan
lowones.
• MEM_WRITE_WATCH is a flag that must be specified with MEM_RESERVE. This flag indicates the
systemshouldtrackmemorywritestothisregion(oncecommitted,ofcourse).Thisisdescribed
furtherinthe“MemoryTracking”section.
Decommitting / Releasing Memory
VirtualAlloc must have an opposite function that can de-commit and/or release (the opposite of
reserve)ablockofmemory.Thisistheroleof VirtualFreeandVirtualFreeEx:
BOOL VirtualFree(
_in_ LPVOID lpAddress,
_In_ SIZE_T dwSize,
_In_ DWORD dwFreeType);
BOOL VirtualFreeEx(
_In_ HANDLE hProcess,
_In_ LPVOID lpAddress,
_In_ SIZE_T dwSize,
_In_ DWORD dwFreeType);
VirtualFreeExisanextendedversionof VirtualFreethatperformstherequestedoperationinthe
process specified by hProcess, which must have the PROCESS_VM_OPERATION access mask (just like
VirtualAllocEx). Only two flags are supported by the dwFreeType parameter - MEM_DECOMMIT and
MEM_RELEASE-oneofwhich(andonlyone)mustbespecified.
MEM_DECOMMIT decommits the pages that span lpAddress to lpAddress+dwSize, returning the
memory region to the reserved state. MEM_RELEASE indicates the region should be freed completely.
