Programming Linux Anti-Reversing Techniques

248 Pages·2016·0.91 MB·English

by  Jacob Baines

Programming Linux Anti-Reversing
Techniques
Jacob Baines
Thisbookisforsaleathttp://leanpub.com/anti-reverse-engineering-linux
Thisversionwaspublishedon2016-12-20
ThisisaLeanpubbook.LeanpubempowersauthorsandpublisherswiththeLean
Publishingprocess.LeanPublishingistheactofpublishinganin-progressebook
usinglightweighttoolsandmanyiterationstogetreaderfeedback,pivotuntilyou
havetherightbookandbuildtractiononceyoudo.
©2016JacobBaines
43.953333,-69.052500
“Everything’sNotLost”
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
WhyReadThisBook? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
TopicsNotCovered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
CodeandCommandOutput . . . . . . . . . . . . . . . . . . . . . . . . 2
Chapter1:Introductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Introducing“Trouble” . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
UsingCMake . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
TheCode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Compiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Executing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
AccessingtheShell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Chapter2:CompilerOptions . . . . . . . . . . . . . . . . . . . . . . . . . . 11
-g . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
RecoveringtheBindShellPasswordwithHexdump . . . . . . . . . . 17
RecoveringtheBindShellPasswordwithGDB . . . . . . . . . . . . . 19
TheDebuggingInformationinIDA . . . . . . . . . . . . . . . . . . . 20
RemovingtheDebuggingInformation . . . . . . . . . . . . . . . . . 21
CaseStudy:XORDDOS . . . . . . . . . . . . . . . . . . . . . . . . . 23
-s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
SYMTABvs.DYNSYM . . . . . . . . . . . . . . . . . . . . . . . . . . 25
FindingtheBindShellPasswordUsing.symtab . . . . . . . . . . . . 29
CaseStudy:TheFILESymbol . . . . . . . . . . . . . . . . . . . . . . 29
ExamingTrouble After-s . . . . . . . . . . . . . . . . . . . . . . . . 30
-fvisibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
CONTENTS
LookingatFUNCsymbols . . . . . . . . . . . . . . . . . . . . . . . . 32
HidingFUNCsymbols . . . . . . . . . . . . . . . . . . . . . . . . . . 33
-O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
CorrectedBlockTinyEncryptionAlgorithm(XXTEA) . . . . . . . . . 37
-Os. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
-O3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
-funroll-loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
-static . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
ResolvingFunctionsatRuntime . . . . . . . . . . . . . . . . . . . . . 52
ltrace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
LD_PRELOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Usingmusl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Chapter3:FileFormatHacks. . . . . . . . . . . . . . . . . . . . . . . . . . 64
TheStripUtility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
RemovingtheSectionHeadersTable . . . . . . . . . . . . . . . . . . . . 68
LittleEndianorBigEndian? . . . . . . . . . . . . . . . . . . . . . . . . 78
TheSectionsAreaLie . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
FlippingtheExecutableBit . . . . . . . . . . . . . . . . . . . . . . . . 84
Lyingwith.init . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
HidingtheEntryPoint . . . . . . . . . . . . . . . . . . . . . . . . . . 107
MixingtheSymbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Chapter4:FightingOffStringAnalysis . . . . . . . . . . . . . . . . . . . . 129
CodeReorganization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
StackStrings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
XORStackString . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
FunctionEncryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
ComputingtheFunction’sSizeUsingaLinkerScript . . . . . . . . . . 141
DecryptionLogic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
EncryptionLogic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
CreatingaCryptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
ImplementingtheCryptor . . . . . . . . . . . . . . . . . . . . . . . . 163
AnalyzingtheCryptor . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Chapter5:ObstructingCodeFlowAnalysis . . . . . . . . . . . . . . . . . 174
CONTENTS
IndirectFunctionCalls . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
EarlyReturn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
JumpOveranInvalidByte . . . . . . . . . . . . . . . . . . . . . . . . . 189
Jump!Jump! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
AlwaysFollowtheConditional . . . . . . . . . . . . . . . . . . . . . . . 195
OverlappingInstructions . . . . . . . . . . . . . . . . . . . . . . . . . . 198
Chapter6:EvadingtheDebugger . . . . . . . . . . . . . . . . . . . . . . . 203
TraceMe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
TrappingtheDebugger . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
BecomingAttached . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
/proc/self/status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
madvise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
prctl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
DetectionBeforemain() . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
ComputingFunctionChecksums . . . . . . . . . . . . . . . . . . . . . . 233
Conclusion:AllThatWeFallFor . . . . . . . . . . . . . . . . . . . . . . . 242
Preface
Why Read This Book?
There are many articles and books that talk about anti-reverse engineering tech-
niques and how to break them. However, the writing is often limited to small code
snippets or a screenshot of assembly in IDA. This book seeks to be different by
startingwithasimpleprogramyou’llupdatewithanti-retechniquesasyouprogress
through the book. This gives the reader the oppurtunity to compile and analyze the
binaryontheirown.Ibelievetheemphasisonthe“proofofconcept”makesthisbook
unique.
Another unique aspect of this book is the emphasis onLinux anti-reverse engineer-
ing. Hackers have been writing about these techniques since the late 90’s, but much
ofwhathasbeenwrittenisstrewnacrosstheinternet.Thisbookattemptstocoalesce
amajorityoftechniquesandpresentthemtothereaderasoneunit.
Finally, a lot of the proof of concept code that currently exists on the internet is
bad.Nottobealljudgemental,butsomeofitdoesn’tcompile.Somesimplycrashes.
Othersarewritteninconfusingandbizarremanners.Thecodeprovidedinthisbook
attemptstobeclean,useful,andmaintainable.
Topics Not Covered
There are some topics that have been intentionally left out of this book. I want to
statethesetopicsupfrontsothatthereaderknowswhattheyaregetting.
1. VirtualMachineDetection:VMdetectiontechniquesarecontinuouslyinflux
asbothsidesseektooneupeachother.Atechniquethatworksoneweekcould
easilybebrokenthenext.Assuch,I’veavoidedthissubjectaltogether.
Preface 2
2. Hiding Network Communication: Understanding a binary’s network com-
municationisahugeaidinreverseengineering.However,thistopicistoodeep
todojusticewithinthesepages.
3. RootkitTopics:Hidingandpersistenceareoutofscope.
4. AnythingRelatedtotheKernel:Thisbookfocusesonlyonuserlandbinaries.
Prerequisites
The reader should have access to a Linux host using the x86-64 architecture. The
codeforthisbookwaswrittenandtestedonUbuntu16.04.
This book does discuss the use of IDA, but I understand that the high cost of IDA is
anon-starterformany.Therefore,I’vedonemybesttoalsoincludeexamplesusing
Radare2andHopper.
Finally, the code for this book is largely written in C with a small amount of x86-
64 assembler. Some of the tooling is written in C++. However, I do not expect the
reader to be well versed in any of these languages. Part of the beauty of having
complete code examples to work from is that it gives the author a chance to point
outanyidiosyncrasiesandprovidesthereadertheoppurtunitytopullapartthecode
ontheirowntime.
Code and Command Output
All the code is available on Github: https://github.com/antire-book/antire_book. I
understand that the reader may not always be close to a computer or have access
to GitHub in order to browse the book’s code. As such, all of the code is also listed
within the book. I know some people don’t like that, but since the code is essential
tothisbookIcan’thavethereadergowithout.
Also related to formatting, any output from a command line tool included in this
bookwillgenerallyincludeall oftheoutput.Forexample,whenGDBstartsitprints
outalotofversion,license,andhelpinformation.Iwon’tcutanyofthat.Iwantthe
reader to be able to match the output from their computer with what I’ve listed in
thebook.Thislargelyaidsintroubleshootinganyissuesthereaderrunsinto.
Chapter 1: Introductions
Introducing “Trouble”
ThisbookcentersaroundtheobfuscationofabindshellcalledTrouble.
What is a bind shell?
“Bind shell is a type of shell in which the target machine opens up a
communicationportoralisteneronthevictimmachineandwaitsforan
incomingconnection.Theattackerthenconnectstothevictimmachine’s
listenerwhichthenleadstocodeorcommandexecutionontheserver.”¹
TheTroublebindshellisusedasanexample,notbecauseitisuniqueorinteresting,
but because it is small and simple. It also has a property that should be interesting
toareverseengineer:itrequiresapasswordtoaccesstheshell.Alloftheactivityin
thisbookattemptstoeitherhideorrecovertheshell’spassword.
Using CMake
You’ll be using CMake² to compile Trouble. CMake is an open source Makefile
generation tool. It’s useful for dependency checking and supports a simple syntax.
Don’tworryifyouaren’tfamiliarwithCMake.You’llpickitasthebookprogresses.
CMake relies on “CMakeList.txt” files to generate Makefiles. For chapter one, you’ll
findTrouble’s“CMakeList.txt”inthechapt_1_introduction/trouble directory.
¹http://resources.infosecinstitute.com/icmp-reverse-shell/
²https://cmake.org/
Chapter1: Introductions 4
chap_1_introduction/trouble/CMakeLists.txt
project(trouble C)
cmake_minimum_required(VERSION 3.0)
# This will create a 32 byte "password" for the bind shell. This command
# is only run when "cmake" is run, so if you want to generate a new password
# then "cmake ..; make" should be run from the command line.
exec_program("/bin/sh"
${CMAKE_CURRENT_SOURCE_DIR}
ARGS "-c 'cat /dev/urandom | tr -dc a-zA-Z0-9 | head -c 32'"
OUTPUT_VARIABLE random_password )
# Pass the random password into ${PROJECT_NAME} as a macro
add_definitions(-Dpassword="${random_password}")
set(CMAKE_C_FLAGS "-Wall -Wextra -Wshadow -g -std=gnu11")
add_executable(${PROJECT_NAME} src/trouble.c)
# After the build is successful, display the random password to the user
add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E echo
"The bind shell password is:" ${random_password})
NotonlywillthisfilegenerateTrouble’s Makefilebutitalsogeneratestheshell’s32
byte password everytime it is executed. The password create using urandom with
thefollowingcommand:
cat /dev/urandom | tr -dc a-zA-Z0-9 | head -c 32
More will be expalined about using CMake file in the upcoming section about
compilingTrouble.
The Code
The code for the Trouble bind shell is located within the “source” directory. It is
comprisedofasinglefile: