Process Address Space: The way to create virtual address (page table) of userspace application
6 likes2,310 views
Process Address Space: The way to create virtual address (page table) of userspace application. Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
![64-bit Process Virtual Address
Kernel Space
0x0000_7FFF_FFFF_FFFF
0xFFFF_8000_0000_0000
128TB
64-bit Virtual Address
0
0xFFFF_FFFF_FFFF_FFFF
• [Option 1] Disable ASLR (Address Space Layout Randomization)
# echo 0 > /proc/sys/kernel/randomize_va_space
• [Option 2] kernel parameter: norandmaps
Note: Disable User Space Address Randomization
Empty Space
User Space
128TB
128MB gap](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-3-320.jpg)
![wait for user input
while(1)
fork
child
parent
shell_execve
execve(command, args, env)
shell (ex: bash) - `cat hello.c` File name (Ex: /bin/mount)
4KB - Hole
Environment strings
Command-line arguments
Dynamic linker’s table
(Auxiliary Vector)
envp[]
argv[]
argc
Return address
User space stack created by kernel
Kernel
copy from user
copy from user
copy from user
Stack layout when executing a command](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-8-320.jpg)
![vma->vm_end = STACK_TOP_MAX = 0x7FFF_FFFF_F000
4KB (PAGE_SIZE) vma->vm_start = 0x7FFF_FFFF_E000
4KB - hole
0x7FFF_FFFF_FFFF
User Space Stack
bprm->p = vma->vm_end - sizeof(void *)
= 0x7FFF_FFFF_EFF8 (Current top of memory)
kernel_init -> run_init_process: init process (pid = 1) - Stack
1
2
1
2
linux_binprm
vma
vma_pages
mm
struct file *executable
struct file *interpreter
struct file *file
const char *filename = “/init”
const char *interp = “/init”
argc = 2
envc = 3
rlmit_stack
char buf[]](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-9-320.jpg)
![[Argument/environment space] Case #1
[Argument/environment space] Case #2
kernel_init -> run_init_process: init process (pid = 1) - Stack](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-10-320.jpg)
![struct file *executable
struct file *interpreter
struct file *file
const char *filename = “/init”
const char *interp = “/init”
argc = 2
envc = 3
kernel_read
rlmit_stack
char buf[]
linux_binprm
vma
vma_pages
mm
kernel_init -> run_init_process: init process (pid = 1) - Stack](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-16-320.jpg)
![struct file *executable
struct file *interpreter
struct file *file
const char *filename = “/init”
const char *interp = “/init” “/bin/sh”
argc = 2 3
envc = 3
kernel_read
rlmit_stack
char buf[]
linux_binprm
vma
vma_pages
mm
file: /bin/sh
file: /init
kernel_init -> run_init_process: init process (pid = 1) - Stack](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-17-320.jpg)
![struct file *executable
struct file *interpreter
struct file *file
const char *filename = “/init”
const char *interp = “/init” “/bin/sh”
argc = 2 3
envc = 3
kernel_read
rlmit_stack
char buf[]
linux_binprm
vma
vma_pages
mm
file: /bin/sh
file: /init
struct file *executable
struct file *interpreter = NULL
struct file *file
const char *filename = “/init”
const char *interp = “/init” “/bin/sh”
argc = 2 3
envc = 3
kernel_read
rlmit_stack
char buf[]
linux_binprm
vma
vma_pages
mm
file: /bin/sh
kernel_init -> run_init_process: init process (pid = 1)](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-18-320.jpg)
![struct file *executable
struct file *interpreter = NULL
struct file *file
const char *filename = “/init”
const char *interp = “/init” “/bin/sh”
argc = 2 3
envc = 3
kernel_read
rlmit_stack
char buf[]
linux_binprm
vma
vma_pages
mm
file: /bin/sh
depth = 1
kernel_init -> run_init_process: init process (pid = 1)](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-19-320.jpg)
![kernel_init -> run_init_process: init process (pid = 1)
[Dynamic linking] ld-Linux.so: dynamic linker/loader
• Find and load the shared objects (shared libraries) needed
by a program, prepare the program to run, and then run it
• `man ld-linux`
depth = 1](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-20-320.jpg)
![mm_struct
unsigned long saved_auxv[]
create_elf_tables() - Auxiliary Vector
Auxiliary Table Entry Value
Auxiliary Table Entry ID
Auxiliary Table Entry Value
Auxiliary Table Entry ID
.
.
saved_auxv[0]
saved_auxv[1]
saved_auxv[n]
filename: “/init”
4KB - hole
8-byte hole
dyndbg=file arch/x86/mm/init.c +p
TERM=linux
HOME=/
nokaslr
/init
Stack Guard Gap
/bin/sh
mmap
ELF_PLATFORM: “x86_64”
16-byte random characters
Auxiliary Vector
envp[2]
envp[1]
envp[0]
argv[2]
argv[1]
argv[0]
argc = bprm->argc (=3)
0
0
User Space Virtual Memory Layout - Stack
More Info
• `man getauxval`
• https://lwn.net/Articles
/519085/](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-34-320.jpg)
![STACK_END_MAGIC
ip = 0x401c10
kernel stack
usage space
struct inactive_task_frame
Kernel Stack
…
pt_regs
File name (Ex: /tmp/hello)
4KB - Hole
Environment strings
Command-line arguments
Dynamic linker’s table
(Auxiliary Vector)
envp[]
argv[]
argc
Return address
User Space Stack
/tmp/hello
_start
__libc_start_main - LIBC_START_MAIN in csu/libc-start.c
(/lib/x86_64-linux-gnu/libc.so.6 -> libc-2.31.so)
Pass address of main()
main
Statically-linked program (OS: Ubuntu 20.04.3)](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-38-320.jpg)
![STACK_END_MAGIC
ip = 0x7ffff7fd0100
kernel stack
usage space
struct inactive_task_frame
Kernel Stack
Dynamically-linked program (OS: Ubuntu 20.04.3)
…
pt_regs
_start
_dl_start
_dl_start_final
_dl_sysdep_start
Parse auxiliary vector:
For example: Get ‘AT_ENTRY’ value = 0x555555555060
dl_main
File name (Ex: /tmp/hello)
4KB - Hole
Environment strings
Command-line arguments
Dynamic linker’s table
(Auxiliary Vector)
envp[]
argv[]
argc
Return address
User Space Stack
load binary/libraries and
perform relocation
_dl_start_user
[Function Return]
Next IP of _dl_start()
Return value of _dl_start() = Address
of _start() of the executable file
/lib64/ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.31.so
Jump to ‘AT_ENTRY’ value = 0x555555555060
/tmp/hello
_start
__libc_start_main - LIBC_START_MAIN in csu/libc-start.c
(/lib/x86_64-linux-gnu/libc.so.6 -> libc-2.31.so)
Pass address of main()
main](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-39-320.jpg)
![/lib64/ld-linux-x86-64.so.2 _start
_dl_start
_dl_start_user
[Function Return]
Next IP of _dl_start()
Return value of _dl_start() = Address
of _start() of the executable file
/lib64/ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.31.so
Jump to ‘AT_ENTRY’ value = 0x555555555060
Ubuntu 20.04.3 RHEL8.2](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-41-320.jpg)
![[Dynamically-linked program] Executable file: where is “_start()” from?
*crt*.o (C Runtime): A set of execution startup routines linked into a C program that
performs initialization work before calling the program’s main function.
Reference
https://dev.gentoo.org/~vapier/crt.txt
https://en.wikipedia.org/wiki/Crt0](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-42-320.jpg)
![[Dynamically-linked program] Executable file: where is “_start()” from?](https://image.slidesharecdn.com/processaddressspace-220325100710/85/Process-Address-Space-The-way-to-create-virtual-address-page-table-of-userspace-application-43-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to Process Address Space: The way to create virtual address (page table) of userspace application (20)
Ad
Recently uploaded (20)
![Download Wondershare Filmora Crack [2025] With Latest](https://cdn.slidesharecdn.com/ss_thumbnails/neo4j-howkgsareshapingthefutureofgenerativeaiatawssummitlondonapril2024-240426125209-2d9db05d-250419-250428115407-a04afffa-thumbnail.jpg?width=560&fit=bounds)
Ad
Process Address Space: The way to create virtual address (page table) of userspace application
- 1. Process Address Space: The way to create virtual address (page table) of userspace application Adrian Huang | Oct, 2021 * Based on kernel 5.11 (x86_64) – QEMU * SMP (4 CPUs) and 8GB memory * Kernel parameter: nokaslr norandmaps * Userspace: ASLR is disabled * Legacy BIOS
- 2. Agenda • 64-bit Virtual Address • mm_struct & VMA • Detail about stack • Stack configuration via bprm_execve() • Important function: load_elf_binary() • Auxiliary Vector • How does Linux call your main() function? The call path? • Statically-linked program • Base address: 0x400000 • Dynamically-linked program • Base address: 0x555555554000 • Do you know why the base address of a dynamically-linked program is the base address ‘0x555555554000’?
- 3. 64-bit Process Virtual Address Kernel Space 0x0000_7FFF_FFFF_FFFF 0xFFFF_8000_0000_0000 128TB 64-bit Virtual Address 0 0xFFFF_FFFF_FFFF_FFFF • [Option 1] Disable ASLR (Address Space Layout Randomization) # echo 0 > /proc/sys/kernel/randomize_va_space • [Option 2] kernel parameter: norandmaps Note: Disable User Space Address Randomization Empty Space User Space 128TB 128MB gap
- 4. Process Address Space – mm_struct & VMA
- 5. Process Address Space – mm_struct & VMA
- 6. Process Address Space – mm_struct & VMA
- 7. 64-bit Process Virtual Address – Stack Layout
- 8. wait for user input while(1) fork child parent shell_execve execve(command, args, env) shell (ex: bash) - `cat hello.c` File name (Ex: /bin/mount) 4KB - Hole Environment strings Command-line arguments Dynamic linker’s table (Auxiliary Vector) envp[] argv[] argc Return address User space stack created by kernel Kernel copy from user copy from user copy from user Stack layout when executing a command
- 9. vma->vm_end = STACK_TOP_MAX = 0x7FFF_FFFF_F000 4KB (PAGE_SIZE) vma->vm_start = 0x7FFF_FFFF_E000 4KB - hole 0x7FFF_FFFF_FFFF User Space Stack bprm->p = vma->vm_end - sizeof(void *) = 0x7FFF_FFFF_EFF8 (Current top of memory) kernel_init -> run_init_process: init process (pid = 1) - Stack 1 2 1 2 linux_binprm vma vma_pages mm struct file *executable struct file *interpreter struct file *file const char *filename = “/init” const char *interp = “/init” argc = 2 envc = 3 rlmit_stack char buf[]
- 10. [Argument/environment space] Case #1 [Argument/environment space] Case #2 kernel_init -> run_init_process: init process (pid = 1) - Stack
- 11. kernel_init -> run_init_process: init process (pid = 1) - Stack
- 12. 1 2 3 4 kernel_init -> run_init_process: init process (pid = 1) - Stack
- 13. kernel_init -> run_init_process: init process (pid = 1) - Stack
- 14. kernel_init -> run_init_process: init process (pid = 1) - Stack
- 15. kernel_init -> run_init_process: init process (pid = 1) - Stack
- 16. struct file *executable struct file *interpreter struct file *file const char *filename = “/init” const char *interp = “/init” argc = 2 envc = 3 kernel_read rlmit_stack char buf[] linux_binprm vma vma_pages mm kernel_init -> run_init_process: init process (pid = 1) - Stack
- 17. struct file *executable struct file *interpreter struct file *file const char *filename = “/init” const char *interp = “/init” “/bin/sh” argc = 2 3 envc = 3 kernel_read rlmit_stack char buf[] linux_binprm vma vma_pages mm file: /bin/sh file: /init kernel_init -> run_init_process: init process (pid = 1) - Stack
- 18. struct file *executable struct file *interpreter struct file *file const char *filename = “/init” const char *interp = “/init” “/bin/sh” argc = 2 3 envc = 3 kernel_read rlmit_stack char buf[] linux_binprm vma vma_pages mm file: /bin/sh file: /init struct file *executable struct file *interpreter = NULL struct file *file const char *filename = “/init” const char *interp = “/init” “/bin/sh” argc = 2 3 envc = 3 kernel_read rlmit_stack char buf[] linux_binprm vma vma_pages mm file: /bin/sh kernel_init -> run_init_process: init process (pid = 1)
- 19. struct file *executable struct file *interpreter = NULL struct file *file const char *filename = “/init” const char *interp = “/init” “/bin/sh” argc = 2 3 envc = 3 kernel_read rlmit_stack char buf[] linux_binprm vma vma_pages mm file: /bin/sh depth = 1 kernel_init -> run_init_process: init process (pid = 1)
- 20. kernel_init -> run_init_process: init process (pid = 1) [Dynamic linking] ld-Linux.so: dynamic linker/loader • Find and load the shared objects (shared libraries) needed by a program, prepare the program to run, and then run it • `man ld-linux` depth = 1
- 21. kernel_init -> run_init_process: init process (pid = 1) de_thread(): All threads other than the calling thread are destroyed during an execve(). Mutexes, condition variables, and other pthreads objects are not preserved. unshare_files(): The file descriptor table is unshared, undoing the effect of the CLONE_FILES flag of clone(2). exec_mmap(): 1. The program that is currently being run by the calling process to be replaced with a new program, with newly initialized stack, heap, and (initialized and uninitialized) data segments. 2. Memory mappings are not preserved. unshare_sighand(): The signal dispositions is unshared, undoing the effect of the CLONE_SIGHAND flag of clone(2) – Not from `man execve` do_close_on_exec(): By default, file descriptors remain open across an execve(). File descriptors that are marked close-on-exec are closed. Description from `man execve`
- 22. kernel_init -> run_init_process: init process (pid = 1) CLONE_FILES flag is set
- 23. kernel_init -> run_init_process: init process (pid = 1)
- 25. kernel_init -> run_init_process: init process (pid = 1)
- 28. load_elf_binary(): load PT_LOAD program headers
- 29. load_elf_binary(): load PT_LOAD program headers Note: statically linked program
- 31. load_elf_binary(): set_brk() & padzero() Note: statically linked program
- 33. create_elf_tables() – User space stack
- 34. mm_struct unsigned long saved_auxv[] create_elf_tables() - Auxiliary Vector Auxiliary Table Entry Value Auxiliary Table Entry ID Auxiliary Table Entry Value Auxiliary Table Entry ID . . saved_auxv[0] saved_auxv[1] saved_auxv[n] filename: “/init” 4KB - hole 8-byte hole dyndbg=file arch/x86/mm/init.c +p TERM=linux HOME=/ nokaslr /init Stack Guard Gap /bin/sh mmap ELF_PLATFORM: “x86_64” 16-byte random characters Auxiliary Vector envp[2] envp[1] envp[0] argv[2] argv[1] argv[0] argc = bprm->argc (=3) 0 0 User Space Virtual Memory Layout - Stack More Info • `man getauxval` • https://lwn.net/Articles /519085/
- 37. load_elf_binary() -> START_THREAD() – Statically-linked program STACK_END_MAGIC = 0x57AC6E9D struct pt_regs (save CPU registers for userspace application) task.stack THREAD_SIZE = 16KB kernel stack usage space task.stack + THREAD_SIZE struct inactive_task_frame task.thread_struct.sp struct fork_frame Kernel Stack _start() in executable file ‘busybox’
- 38. STACK_END_MAGIC ip = 0x401c10 kernel stack usage space struct inactive_task_frame Kernel Stack … pt_regs File name (Ex: /tmp/hello) 4KB - Hole Environment strings Command-line arguments Dynamic linker’s table (Auxiliary Vector) envp[] argv[] argc Return address User Space Stack /tmp/hello _start __libc_start_main - LIBC_START_MAIN in csu/libc-start.c (/lib/x86_64-linux-gnu/libc.so.6 -> libc-2.31.so) Pass address of main() main Statically-linked program (OS: Ubuntu 20.04.3)
- 39. STACK_END_MAGIC ip = 0x7ffff7fd0100 kernel stack usage space struct inactive_task_frame Kernel Stack Dynamically-linked program (OS: Ubuntu 20.04.3) … pt_regs _start _dl_start _dl_start_final _dl_sysdep_start Parse auxiliary vector: For example: Get ‘AT_ENTRY’ value = 0x555555555060 dl_main File name (Ex: /tmp/hello) 4KB - Hole Environment strings Command-line arguments Dynamic linker’s table (Auxiliary Vector) envp[] argv[] argc Return address User Space Stack load binary/libraries and perform relocation _dl_start_user [Function Return] Next IP of _dl_start() Return value of _dl_start() = Address of _start() of the executable file /lib64/ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.31.so Jump to ‘AT_ENTRY’ value = 0x555555555060 /tmp/hello _start __libc_start_main - LIBC_START_MAIN in csu/libc-start.c (/lib/x86_64-linux-gnu/libc.so.6 -> libc-2.31.so) Pass address of main() main
- 40. STACK_END_MAGIC ip = 0x7ffff7fd0100 kernel stack usage space struct inactive_task_frame Kernel Stack Dynamically-linked program (OS: Ubuntu 20.04.3) … pt_regs +
- 41. /lib64/ld-linux-x86-64.so.2 _start _dl_start _dl_start_user [Function Return] Next IP of _dl_start() Return value of _dl_start() = Address of _start() of the executable file /lib64/ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.31.so Jump to ‘AT_ENTRY’ value = 0x555555555060 Ubuntu 20.04.3 RHEL8.2
- 42. [Dynamically-linked program] Executable file: where is “_start()” from? *crt*.o (C Runtime): A set of execution startup routines linked into a C program that performs initialization work before calling the program’s main function. Reference https://dev.gentoo.org/~vapier/crt.txt https://en.wikipedia.org/wiki/Crt0
- 43. [Dynamically-linked program] Executable file: where is “_start()” from?
- 44. Auxiliary vector & base address of a program Dynamically-linked program Statically-linked program + Why is the base address ‘0x555555554000’ for a dynamically-linked program?
- 45. Why is the base address ‘0x555555554000’ for a dynamically-linked program?
- 46. Demand paging Demand paging: copy a disk page into physical memory if a page fault occurs
- 47. Demand paging Demand paging: copy a disk page into physical memory if a page fault occurs