Loading document…
Opening in Pages for Mac...
Choose Media…
Your browser isn’t fully supported.
For the best Pages for iCloud experience, use a supported browser.
Learn More
Cancel
Continue
PROPOSAL
Add support for chdir(2) support in posix_spawn(3)
Table of Contents
About the Project 3
What is the goal of the project? 3
What will be the deliverables of the project? 3
Give an overview of how you intend to reach the project's goal in the form of milestones and a
schedule. 3
Is similar software already available elsewhere 4
Is the project a port of software, or a rewrite? 4
About the Project and NetBSD 5
If your working area is the core NetBSD operating system: have you installed NetBSD and made
fi rst experiences with hands-on confi guration? Have you rebuilt the kernel and the userland, either
in full or in parts? If you plan to work on pkgsrc, have you installed packages from source and
binary? Have you created a package on your own? 5
Have you found the relevant places that your project is based on in the source code, and read
through it? 5
How will your project integrate into NetBSD? 6
What interfaces in NetBSD will your project use? 6
To what degree are you familiar with those interfaces? 7
Is knowledge on other topics required for this project, e.g., on hardware, software other than
NetBSD, APIs, protocols, etc.? If so, give details and references. 7
To what degree are you familiar with those? (not/some/very, details?) 7
If the project involves hardware (e.g., writing drivers, doing a port to new hardware, ...): do you own
the hardware or have access to? 7
About Me 8
Can you list some prior projects that you have worked on so far? Include details like programming
language, duration, number of people involved, project goal, if you used CVS, SVN or similar, and
whatever else we may fi nd thrilling! If you have a CV/resume online, feel free to include a link. 8
Do you have any prior experience with programming NetBSD? In what area? If you did send some
problem reports (PRs) or patches, please include references. 8
Have you previously discussed your project within NetBSD, either on a mailing list or with some
specifi c developers?. 8
How do we contact you for questions, comments, suggestions, etc.? 8
Is there anything else you'd like us to know? Did we forget any important details or questions? 8
About the Project
What is the goal of the project?
The primary goal of the project is to extend the posix_spawn(3) to include chdir(2)
for the newly created process.
posix_spawn() is a POSIX standard method to create a child process. It takes the
necessary arguments, i .e process identifi er, name of the fi le, fi le actions, spanning
attributes, arguments, and environment variables, to spawn a new process.
The requirement here is to change the current working directory using chdir before
executing the child process.
As two possible functions for chdir and fchdir will be implemented, it is essential that
necessary documentation be provided. This part will happen according to the
sequence of actions mentioned in the Austin Groups bug’s tracker.
What will be the deliverables of the project?
Successful implementation of the function to change the working directory will be
the key deliverable of this project, along with the necessary documentation at all the
places where changes are made.
I will also do my best to resolve the issues mentioned relating to
posix_spawn_addclose() and posix_spawn_adddup2().
Give an overview of how you intend to reach the project's goal
in the
form of milestones and a schedule.
Pre-Project
:
April 15-May 17, 2021
•
In-depth understanding of the functionality and implementation structures used
in the kern_exec.c fi le and in the libc fi les.
•
Getting a handle on the in-kernel functions for performing numerous activities
such as memory allocations, getting fi le descriptors, etc. Eg- fd_getline(),
kmem_alloc(), etc.
May 17-June 6, 2021
•
Setting up KTRACE for the project.
•
Designing a proper, detailed Plan of Action for the project. This will include
specifi c places in the kernel fi les and libc fi les where changes/additions will be
made.
Coding Timeline
:
The complete coding duration is of 10 weeks from June 7 to August 16, 2021 which
will be divided as follows:
Week 1-3 -
The bulk of this phase will happen inside the kernel, to be specifi c, in
kern_exec.c fi le.
Week 4, 5
- During this phase a major chunk of the work will happen on the libc
functions/fi les.
Week 6, 7
- In this phase, the work done in the fi rst fi ve weeks will be checked for
redundancies. To say, basic test cases will be verifi ed.
Week 8, 9
- The code will be checked for style guidelines and the required
documentation will be added.
Week 10
- A fi nal review of the code.
Is similar software already available elsewhere
Yes, both linux and Solaris kernels, have the ability to change directory when
spawning a process using posix_spawn(3).
Is the project a port of software, or a rewrite?
Both posix_spawn and posix_spawnp already exist. But the flexibility of changing the
current working directory is absent. Users often exec a shim process, but even that
is not in the standard, so users have to come up with that as well. The parent
process has the ability to change the directory but the child is not provided one.
Also, because it is expensive to convert relative names into absolute ones, it will be
benefi cial to extend such a functionality. But as the main posix_spawn functionality
already exists; I believe this project could be characterized as a port.
About the Project and NetBSD
If your working area is the core NetBSD operating system: have you
installed NetBSD and made fi rst experiences with hands-on
confi guration? Have you rebuilt the kernel and the userland, either
in full or in parts? If you plan to work on pkgsrc, have you installed
packages from source and binary? Have you created a package on
your own?
The changes required to the fi les, will happen in the libc functions and the most
important work will be in the kern_exec.c fi le. Once the required functionality is
added, there will be a need to rebuild from source. For that I have already
downloaded Virtual Box. In Virtual Box an instance of the NetBSD OS is up and
running and I have explored it. Provided that I am a Mac user, NetBSD is already
familiar to me. Also, In the past week, I have gone through the NetBSD Guide’s fi fth
unit which corresponds to building the kernel. So to sum up: yes: I have made fi rst
experiences with hands-on confi gurations.
Have you found the relevant places that your project is based on in
the source code, and read through it?
After going through the mail, I received from Martin, I examined the following fi les:
•
/usr/src/lib/libc/gen/posix_spawnp.c
•
posix_spawn_fi leactions.c
•
posix_spawn_shed.c
To understand the data structures used in these three fi les, I have also referred to:
•
/usr/src/lib/libc/include/namespace.h
•
/usr/src/sys/sys/spawn.h
Then, to see how the syscall is actually made and how posix_spawn is actually
implemented, inside the kernel, I have gone through the major functions in the
following fi les:
•
/usr/src/sys/kern/kern_exec.c
•
/usr/src/sys/sys/proc.h
•
/usr/src/sys/sys/lwp.h.
The functions and data structures include-
-
line
256 - struct execve_data
-
line
274 - struct spawn_exec_data
-
line
1180 - execve_runproc()
-> error/return code : performs the exec
-
line
2052 - spawn_exec_data_release()
-> void : free the spawn_exec_data
struct
-
line
2070 - spawn_handle_posix_fi le_actions() -> error/return code
: deals with
the possible fi le actions.
-
line
2118 - spawn_handle_posix_fi le_attrs() -> error/return code : handles
group ids, signals and scheduling policy
, etc.
-
line
2220 - note about child process
-
line
2232 - spawn_return() -> void :
acquires
the locks, calls both fi le and attr
s
functions and fi nally calls execve_runproc()
to replace the child process.
-
line
2340 - posix_spawn_fa_free -> void
: free’s the memory allocated for
posix_spawn_fi le_actions struct.
-
line
2355 - posix_spawn_fa_alloc -> error/return codes
: allocates the memory
allocated for posix_spawn_fi le_actions struct.
-
line
2453 - do_posix_spawn() 2631 -
> error/return codes: does all the heavy
lifting, allocates the vm space, creates the child process, then creates the exec
process, handles locks, id’s, authenticator values, etc and takes us back to user
land with the help of spawn_return().
-
line
2753 - sys_posix_spawn() :
-> error/return codes : call functions to allocate
space, then calls do_spawn_exec and then frees the memory.
These line numbers correspond to the fi le /usr/src/sys/kern/kern_exec.c.
How will your project integrate into NetBSD?
This project is a part of the user-land, and an integral part of the kernel space. Once
the kernel changes are made (in the kern_exec.c fi le), and user-land
(posix_spawn_fi le_actions.c, posix_spawnp.c) complies with those changes, a major
chunk of the project is completed. that is when testing will be necessary to check for
things like memory leaks and properties of the child process. The work done will be
submitted in the form of patches.
What interfaces in NetBSD will your project use?
To extend posix_spawn(3) to also include the ability to chdir(2) and fchdir(2), the
following probable tasks will be required.
•
In fi le “/usr/src/sys/sys/spawn.h” the enum fae_actions needs two more entries for
chdir and fchdir.
•
In fi le “usr/src/lib/libc/gen/posix_spawn_fi leactions.c” two more functions
corresponding to chdir and fchdir will be added.
•
Once these two are done, we can go ahead and implement the two functions in the
“usr/src/lib/libc/gen/
posix_spawnp.c” fi le.
But to validate the success of the above-mentioned steps, we need to make sure
that the kernel has the desired implementations. For that, in the /src/sys/kern/
kern_exec.c fi les, the following changes are required:
•
First we need to add two more cases for each chdir and fchdir in the switch block
of the handle_posix_spawn_fi le_actions() function. Line - 2108
•
Further we make sure that the result of chdir or fchdir has been implemented
before the call to execve_runproc() is made. execve_runproc() method is called by
the spawn_return() method, which in turn is called by do_posix_spawn().
We can describe posix_spawn happening in three simple stages -
-
The fork step, that creates the child process. This step mainly happens in the
do_posix_spawn() function, where virtual address space is allocated, process
table entry is made, process space for exec is borrowed, flags are setup, locks
are acquired, etc.
-
Next is the pre-exec step. Here we make sure that the path for the about-to-be-
created child process is what the user wanted. Essentially the chdir and fchdir
’s success needs to be checked.
-
Finally the exec step, where the fork created child process is replaced by the
new exec process through the call to spawn_return(). Once the light weight
process is created, it is exposed to the scheduler and voila!
•
All the above mentioned steps are done by the functions mentioned above, by
calling each other. However the main function, which makes the calls to start this
domino effect is the sys_posix_spawn() function.
•
The sys_posix_spawn() method checks the data structures using
check_posix_spawn(), allocates space for fi le_actions struct by calling
posix_spawn_fa_alloc(), copies posix_spawnattr struct. Then the main work of
calling do_posix_spawn is done, and fi nally all the allocated memory is freed.
These are the most important tasks that need to be performed.
To what degree are you familiar with those interfaces?
As the data structures that are used in this are trivial, like struct, enum and union, I
am well acquainted with them. Also, because I have gone through the list of fi les
mentioned above, I fully grasp the working and manipulation of the data types and
data structures used. However the kernel space fi les use custom implemented
functions like, kmem_free(), fd_getfi le() and structs like proc and lwp. I have gone
through them, and I’d say, that I have a decent bird’s eye view of their
implementation. I do completely comprehend what these functions and structures
are there for. I also have a good idea about the Read-Write lock from when I read
Operating Systems.
Is knowledge on other topics required for this project, e.g.
,
on
hardware, software other than NetBSD, APIs, protocols, etc.? If so,
give details and references.
This project focuses on the creation of child processes and the address space of the
created child process, therefor one should know about the mechanics of process
creation and address space management. Moreover, a basic idea of how locks work,
fi le permissions, signal response etc., will go a long way for the purpose of this
project, In addition to this, a sound background in C programming is necessary.
Knowledge of other protocols or hardware is not a dire need for this project.
To what degree are you familiar with those? (not/some/very,
details?)
I am conversant with all the requirements.
If the project involves hardware (e.g.
,
writing drivers, doing a port
to new hardware, ...): do you own the hardware or have access to?
No hardware is required.
About Me
Can you list some prior projects that you have worked on so far?
Include details like programming language, duration, number of
people involved, project goal, if you used CVS, SVN or similar, and
whatever else we may fi nd thrilling! If you have a CV/resume online,
feel free to include a link.
I haven’t worked on any big projects per-se, and I want to change that with this
project.
However, I have implemented a few UNIX utilities such as, cat, grep, reverse(not
standard). Further more I have implemented the working of a basic shell as well.
Other than this, I have a scrupulous background in C and a thorough knowledge of all
the skills required for this project.
This is the link to my resume:
Resume
Do you have any prior experience with programming NetBSD? In
what area? If you did send some problem reports (PRs) or patches,
please include references.
I don’t have any prior experience with programming NetBSD.
Have you previously discussed your project within NetBSD, either
on a mailing list or with some specifi c developers?.
I did send a mail a few days back regarding the project.
The mail was sent at:
tech-kern
and I got a reply from Martin Husemann.
The link from the mailing list is:
fi rst-mail
This document is the revision of the fi rst draft of the proposal sent on 1st April, 2021,
the link to which is:
fi rst-draft
How do we contact you for question
s
, comments, suggestions, etc.?
My contact details are as follows:
Phone:
+91 7895201011 (available on WhatsApp and iMessage)
Email:
cosmologist_piyush@icloud.com
(available on iMessage as well)
Is there anything else you'd like us to know? Did we forget any
important details or questions?
It would be an absolute honor of mine to be able to stick with this project until the
code gets accepted in the kernel, and therefore I would love to keep contributing to
this and NetBSD even after GSOC.
