Home Page Icon
Home Page
Table of Contents for
Testing the exec axiom
Close
Testing the exec axiom
by Kaiwan N Billimoria
Hands-On System Programming with Linux
Title Page
Copyright and Credits
Hands-On System Programming with Linux
Packt Upsell
Why subscribe?
Packt.com
Contributors
About the author
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Linux System Architecture
Technical requirements
Linux and the Unix operating system
The Unix philosophy in a nutshell
Everything is a process – if it's not a process, it's a file
One tool to do one task
Three standard I/O channels
Word count 
cat
Combine tools seamlessly
Plain text preferred
CLI, not GUI
Modular, designed to be repurposed by others
Provide mechanisms, not policies
Pseudocode
Linux system architecture
Preliminaries
The ABI
Accessing a register's content via inline assembly
Accessing a control register's content via inline assembly
CPU privilege levels
Privilege levels or rings on the x86
Linux architecture
Libraries
System calls
Linux – a monolithic OS
What does that mean?
Execution contexts within the kernel
Process context
Interrupt context
Summary
Virtual Memory
Technical requirements
Virtual memory
No VM – the problem
Objective
Virtual memory
Addressing 1 – the simplistic flawed approach
Addressing 2 – paging in brief
Paging tables – simplified
Indirection
Address-translation
Benefits of using VM
Process-isolation
The programmer need not worry about physical memory
Memory-region protection
SIDEBAR :: Testing the memcpy() C program
Process memory layout
Segments or mappings
Text segment
Data segments
Library segments
Stack segment
What is stack memory?
Why a process stack?
Peeking at the stack
Advanced – the VM split
Summary
Resource Limits
Resource limits
Granularity of resource limits
Resource types
Available resource limits
Hard and soft limits
Querying and changing resource limit values
Caveats
A quick note on the prlimit utility
Using prlimit(1) – examples
API interfaces
Code examples
Permanence
Summary
Dynamic Memory Allocation
The glibc malloc(3) API family
The malloc(3) API
malloc(3) – some FAQs
malloc(3) – a quick summary
The free API
free – a quick summary
The calloc API
The realloc API
The realloc(3) – corner cases
The reallocarray API
Beyond the basics
The program break
Using the sbrk() API
How malloc(3) really behaves
Code example – malloc(3) and the program break
Scenario 1 – default options
Scenario 2 – showing malloc statistics
Scenario 3 – large allocations option
Where does freed memory go?
Advanced features
Demand-paging
Resident or not?
Locking memory
Limits and privileges
Locking all pages
Memory protection
Memory protection – a code example
An Aside – LSM logs, Ftrace
LSM logs
Ftrace
An experiment – running the memprot program on an ARM-32
Memory protection keys – a brief note
Using alloca to allocate automatic memory
Summary
Linux Memory Issues
Common memory issues
Incorrect memory accesses
Accessing and/or using uninitialized variables
Test case 1: Uninitialized memory access
Out-of-bounds memory accesses
Test case 2
Test case 3
Test case 4
Test case 5
Test case 6
Test case 7
Use-after-free/Use-after-return bugs
Test case 8
Test case 9
Test case 10
Leakage
Test case 11
Test case 12
Test case 13 
Test case 13.1
Test case 13.2
Test case 13.3
Undefined behavior
Fragmentation
Miscellaneous
Summary
Debugging Tools for Memory Issues
Tool types
Valgrind
Using Valgrind's Memcheck tool
Valgrind summary table
Valgrind pros and cons : a quick summary
Sanitizer tools
Sanitizer toolset
Building programs for use with ASan
Running the test cases with ASan
AddressSanitizer (ASan) summary table
AddressSanitizer pros and cons – a quick summary
Glibc mallopt
Malloc options via the environment 
Some key points
Code coverage while testing
What is the modern C/C++ developer to do?
A mention of the malloc API helpers
Summary
Process Credentials
The traditional Unix permissions model
Permissions at the user level
How the Unix permission model works
Determining the access category
Real and effective IDs
A puzzle – how can a regular user change their password?
The setuid and setgid special permission bits
Setting the setuid and setgid bits with chmod
Hacking attempt 1
System calls
Querying the process credentials
Code example
Sudo – how it works
What is a saved-set ID?
Setting the process credentials
Hacking attempt 2
An aside – a script to identify setuid-root and setgid  installed programs
setgid example – wall
Giving up privileges
Saved-set UID – a quick demo
The setres[u|g]id(2) system calls
Important security notes
Summary
Process Capabilities
The modern POSIX capabilities model
Motivation
POSIX capabilities
Capabilities – some gory details
OS support
Viewing process capabilities via procfs
Thread capability sets
File capability sets
Embedding capabilities into a program binary
Capability-dumb binaries
Getcap and similar utilities
Wireshark – a case in point
Setting capabilities programmatically
Miscellaneous
How ls displays different binaries
Permission models layering
Security tips
FYI – under the hood, at the level of the Kernel
Summary
Process Execution
Technical requirements
Process execution
Converting a program to a process
The exec Unix axiom
Key points during an exec operation
Testing the exec axiom
Experiment 1 – on the CLI, no frills
Experiment 2 – on the CLI, again
The point of no return
Family time – the exec family APIs
The wrong way
Error handling and the exec
Passing a zero as an argument
Specifying the name of the successor
The remaining exec family APIs
The execlp API
The execle API
The execv API
Exec at the OS level
Summary table – exec family of APIs
Code example
Summary
Process Creation
Process creation
How fork works
Using the fork system call
Fork rule #1
Fork rule #2 – the return
Fork rule #3
Atomic execution?
Fork rule #4 – data
Fork rule #5 – racing
The process and open files
Fork rule #6 – open files
Open files and security
Malloc and the fork
COW in a nutshell
Waiting and our simpsh project
The Unix fork-exec semantic
The need to wait
Performing the wait
Defeating the race after fork
Putting it together – our simpsh project
The wait API – details
The scenarios of wait
Wait scenario #1
Wait scenario #2
Fork bombs and creating more than one child
Wait scenario #3
Variations on the wait  – APIs
The waitpid(2)
The waitid (2)
The actual system call
A note on the vfork
More Unix weirdness
Orphans
Zombies
Fork rule #7
The rules of fork – a summary
Summary
Signaling - Part I
Why signals?
The signal mechanism in brief
Available signals
The standard or Unix signals
Handling signals
Using the sigaction system call to trap signals
Sidebar – the feature test macros
The sigaction structure
Masking signals
Signal masking with the sigprocmask API
Querying the signal mask
Sidebar – signal handling within the OS – polling not interrupts
Reentrant safety and signalling
Reentrant functions
Async-signal-safe functions
Alternate ways to be safe within a signal handler
Signal-safe atomic integers
Powerful sigaction flags
Zombies not invited
No zombies! – the classic way
No zombies! – the modern way
The SA_NOCLDSTOP flag
Interrupted system calls and how to fix them with the SA_RESTART
The once only SA_RESETHAND flag
To defer or not? Working with SA_NODEFER
Signal behavior when masked
Case 1 : Default : SA_NODEFER bit cleared
Case 2 : SA_NODEFER bit set
Running of case 1 – SA_NODEFER bit cleared [default]
Running of case 2 – SA_NODEFER bit set
Using an alternate signal stack
Implementation to handle high-volume signals with an alternate signal stack
Case 1 – very small (100 KB) alternate signal stack
Case 2 : A large (16 MB) alternate signal stack 
Different approaches to handling signals at high volume
Summary
Signaling - Part II
Gracefully handling process crashes
Detailing information with the SA_SIGINFO
The siginfo_t structure
Getting system-level details when a process crashes
Trapping and extracting information from a crash
Register dumping
Finding the crash location in source code
Signaling – caveats and gotchas
Handling errno gracefully
What does errno do?
The errno race
Fixing the errno race
Sleeping correctly
The nanosleep system call
Real-time signals
Differences from standard signals
Real time signals and priority
Sending signals
Just kill 'em
Killing yourself with a raise
Agent 00 – permission to kill
Are you there?
Signaling as IPC
Crude IPC
Better IPC – sending a data item
Sidebar – LTTng
Alternative signal-handling techniques
Synchronously waiting for signals
Pause, please
Waiting forever or until a signal arrives
Synchronously blocking for signals via the sigwait* APIs
The sigwait library API
The sigwaitinfo and the sigtimedwait system calls
The signalfd(2) API
Summary
Timers
Older interfaces
The good ol' alarm clock
Alarm API – the downer
Interval timers
A simple CLI digital clock
Obtaining the current time
Trial runs
A word on using the profiling timers
The newer POSIX (interval) timers mechanism
Typical application workflow
Creating and using a POSIX (interval) timer
The arms race – arming and disarming a POSIX timer
Querying the timer
Example code snippet showing the workflow
Figuring the overrun
POSIX interval timers – example programs
The reaction – time game
How fast is fast?
Our react game – how it works
React – trial runs
The react game – code view
The run:walk interval timer application
A few trial runs
The low – level design and code
Timer lookup via proc
A quick mention
Timers via file descriptors
A quick note on watchdog timers
Summary
Multithreading with Pthreads Part I - Essentials
Multithreading concepts
What exactly is a thread?
Resource sharing
Multiprocess versus multithreaded
Example 1 – creation/destruction – process/thread
The multithreading model
Example 2 – matrix multiplication – process/thread
Example 3 – kernel build
On a VM with 1 GB RAM, two CPU cores and parallelized make -j4
On a VM with 1 GB RAM, one CPU core and sequential make -j1
Motivation – why threads?
Design motivation
Taking advantage of potential parallelism
Logical separation
Overlapping CPU with I/O
Manager-worker model
IPC becoming simple(r)
Performance motivation
Creation and destruction
Automatically taking advantage of modern hardware
Resource sharing
Context switching
A brief history of threading
POSIX threads
Pthreads and Linux
Thread management – the essential pthread APIs
Thread creation
Termination
The return of the ghost
So many ways to die
How many threads is too many?
How many threads can you create?
Code example – creating any number of threads
How many threads should one create?
Thread attributes
Code example – querying the default thread attributes
Joining
The thread model join and the process model wait
Checking for life, timing out
Join or not?
Parameter passing
Passing a structure as a parameter
Thread parameters – what not to do
Thread stacks
Get and set thread stack size
Stack location
Stack guards
Summary
Multithreading with Pthreads Part II - Synchronization
The racing problem
Concurrency and atomicity
The pedagogical bank account example
Critical sections
Locking concepts
Is it atomic?
Dirty reads
Locking guidelines
Locking granularity
Deadlock and its avoidance
Common deadlock types
Self deadlock (relock)
The ABBA deadlock
Avoiding deadlock
Using the pthread APIs for synchronization
The mutex lock
Seeing the race
Mutex attributes
Mutex types
The robust mutex attribute
IPC, threads, and the process-shared mutex
Priority inversion, watchdogs, and Mars
Priority inversion
Watchdog timer in brief
The Mars Pathfinder mission in brief
Priority inheritance – avoiding priority inversion
Summary of mutex attribute usage
Mutex locking – additional variants
Timing out on a mutex lock attempt
Busy-waiting (non-blocking variant) for the lock
The reader-writer mutex lock
The spinlock variant
A few more mutex usage guidelines
Is the mutex locked?
Condition variables
No CV – the naive approach
Using the condition variable
A simple CV usage demo application
CV broadcast wakeup
Summary
Multithreading with Pthreads Part III
Thread safety
Making code thread-safe
Reentrant-safe versus thread-safe
Summary table – approaches to making functions thread-safe
Thread safety via mutex locks
Thread safety via function refactoring
The standard C library and thread safety
List of APIs not required to be thread-safe
Refactoring glibc APIs from foo to foo_r
Some glibc foo and foo_r APIs
Thread safety via TLS 
Thread safety via TSD 
Thread cancelation and cleanup
Canceling a thread
The thread cancelation framework
The cancelability state
The cancelability type
Canceling a thread – a code example
Cleaning up at thread exit
Thread cleanup – code example
Threads and signaling
The issue
The POSIX solution to handling signals on MT
Code example – handling signals in an MT app
Threads vs processes – look again
The multiprocess vs the multithreading model – pros of the MT model
The multiprocess vs the multithreading model – cons of the MT model
Pthreads – a few random tips and FAQs
Pthreads – some FAQs
Debugging multithreaded (pthreads) applications with GDB
Summary
CPU Scheduling on Linux
The Linux OS and the POSIX scheduling model
The Linux process state machine
The sleep states
What is real time?
Types of real time
Scheduling policies
Peeking at the scheduling policy and priority
The nice value
CPU affinity
Exploiting Linux's soft real-time capabilities
Scheduling policy and priority APIs
Code example – setting a thread scheduling policy and priority
Soft real-time – additional considerations
RTL – Linux as an RTOS
Summary
Advanced File I/O
I/O performance recommendations
The kernel page cache
Giving hints to the kernel on file I/O patterns
Via the posix_fadvise(2) API
Via the readahead(2) API
MT app file I/O with the pread, pwrite APIs
Scatter – gather I/O
Discontiguous data file – traditional approach
Discontiguous data file – the SG – I/O approach
SG – I/O variations
File I/O via memory mapping
The Linux I/O code path in brief
Memory mapping a file for I/O
File and anonymous mappings
The mmap advantage
Code example
Memory mapping – additional points
DIO and AIO
Direct I/O (DIO)
Asynchronous I/O (AIO)
I/O technologies – a quick comparison
Multiplexing or async blocking I/O – a quick note
I/O – miscellaneous
Linux's inotify framework
I/O schedulers
Ensuring sufficient disk space
Utilities for I/O monitoring, analysis, and bandwidth control
Summary
Troubleshooting and Best Practices
Troubleshooting tools
perf
Tracing tools
The Linux proc filesystem
Best practices
The empirical approach
Software engineering wisdom in a nutshell
Programming
A programmer’s checklist – seven rules
Better testing
Using the Linux kernel's control groups
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Search in book...
Toggle Font Controls
Playlists
Add To
Create new playlist
Name your new playlist
Playlist description (optional)
Cancel
Create playlist
Sign In
Email address
Password
Forgot Password?
Create account
Login
or
Continue with Facebook
Continue with Google
Sign Up
Full Name
Email address
Confirm Email Address
Password
Login
Create account
or
Continue with Facebook
Continue with Google
Prev
Previous Chapter
Key points during an exec operation
Next
Next Chapter
Experiment 1 – on the CLI, no frills
Testing the exec axiom
Can you test this
exec
axiom described above? Sure. Let's try this in three different ways.
Add Highlight
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click
here
login for view all page.
Day Mode
Cloud Mode
Night Mode
Reset