This is a classic article written by Jack Wallen from the Linux.com archives. For more great SysAdmin tips and techniques check out our free intro to Linux course.

Quick question: How much space do you have left on your drives? A little or a lot? Follow up question: Do you know how to find out? If you happen to use a GUI desktop (e.g., GNOME, KDE, Mate, Pantheon, etc.), the task is probably pretty simple. But what if you’re looking at a headless server, with no GUI? Do you need to install tools for the task? The answer is a resounding no. All the necessary bits are already in place to help you find out exactly how much space remains on your drives. In fact, you have two very easy-to-use options at the ready.

In this article, I’ll demonstrate these tools. I’ll be using Elementary OS, which also includes a GUI option, but we’re going to limit ourselves to the command line. The good news is these command-line tools are readily available for every Linux distribution. On my testing system, there are a number of attached drives (both internal and external). The commands used are agnostic to where a drive is plugged in; they only care that the drive is mounted and visible to the operating system.

With that said, let’s take a look at the tools.

df

The df command is the tool I first used to discover drive space on Linux, way back in the 1990s. It’s very simple in both usage and reporting. To this day, df is my go-to command for this task. This command has a few switches but, for basic reporting, you really only need one. That command is df -H. The -H switch is for human-readable format. The output of df -H will report how much space is used, available, percentage used, and the mount point of every disk attached to your system (Figure 1).

Figure 1: The output of df -H on my Elementary OS system.

What if your list of drives is exceedingly long and you just want to view the space used on a single drive? With df, that is possible. Let’s take a look at how much space has been used up on our primary drive, located at /dev/sda1. To do that, issue the command:

df -H /dev/sda1

The output will be limited to that one drive (Figure 2).

Figure 2: How much space is on one particular drive?

You can also limit the reported fields shown in the df output. Available fields are:

  • source — the file system source

  • size — total number of blocks

  • used — spaced used on a drive

  • avail — space available on a drive

  • pcent — percent of used space, divided by total size

  • target — mount point of a drive

Let’s display the output of all our drives, showing only the size, used, and avail (or availability) fields. The command for this would be:

df -H --output=size,used,avail

The output of this command is quite easy to read (Figure 3).

Figure 3: Specifying what output to display for our drives.

The only caveat here is that we don’t know the source of the output, so we’d want to include source like so:

df -H --output=source,size,used,avail

Now the output makes more sense (Figure 4).

Figure 4: We now know the source of our disk usage.

du

Our next command is du. As you might expect, that stands for disk usage. The du command is quite different to the df command, in that it reports on directories and not drives. Because of this, you’ll want to know the names of directories to be checked. Let’s say I have a directory containing virtual machine files on my machine. That directory is /media/jack/HALEY/VIRTUALBOX. If I want to find out how much space is used by that particular directory, I’d issue the command:

du -h /media/jack/HALEY/VIRTUALBOX

The output of the above command will display the size of every file in the directory (Figure 5).

Figure 5: The output of the du command on a specific directory.

So far, this command isn’t all that helpful. What if we want to know the total usage of a particular directory? Fortunately, du can handle that task. On the same directory, the command would be:

du -sh /media/jack/HALEY/VIRTUALBOX/

Now we know how much total space the files are using up in that directory (Figure 6).

Figure 6: My virtual machine files are using 559GB of space.

You can also use this command to see how much space is being used on all child directories of a parent, like so:

du -h /media/jack/HALEY

The output of this command (Figure 7) is a good way to find out what subdirectories are hogging up space on a drive.

Figure 7: How much space are my subdirectories using?

The du command is also a great tool to use in order to see a list of directories that are using the most disk space on your system. The way to do this is by piping the output of du to two other commands: sort and head. The command to find out the top 10 directories eating space on a drive would look something like this:

du -a /media/jack | sort -n -r | head -n 10

The output would list out those directories, from largest to least offender (Figure 8).

Figure 8: Our top ten directories using up space on a drive.

Not as hard as you thought

Finding out how much space is being used on your Linux-attached drives is quite simple. As long as your drives are mounted to the Linux system, both df and du will do an outstanding job of reporting the necessary information. With df you can quickly see an overview of how much space is used on a disk and with du you can discover how much space is being used by specific directories. These two tools in combination should be considered must-know for every Linux administrator.

And, in case you missed it, I recently showed how to determine your memory usage on Linux. Together, these tips will go a long way toward helping you successfully manage your Linux servers.

This is a classic article written by Jack Wallen from the Linux.com archives. For more great SysAdmin tips and techniques check out our free intro to Linux course.

Although there are already a lot of good security features built into Linux-based systems, one very important potential vulnerability can exist when local access is granted – – that is file permission-based issues resulting from a user not assigning the correct permissions to files and directories. So based upon the need for proper permissions, I will go over the ways to assign permissions and show you some examples where modification may be necessary.

Permission Groups

Each file and directory has three user based permission groups:

  • owner – The Owner permissions apply only to the owner of the file or directory, they will not impact the actions of other users.
  • group – The Group permissions apply only to the group that has been assigned to the file or directory, they will not affect the actions of other users.
  • all users – The All Users permissions apply to all other users on the system, this is the permission group that you want to watch the most.

Permission Types

Each file or directory has three basic permission types:

  • read – The Read permission refers to a user’s capability to read the contents of the file.
  • write – The Write permissions refer to a user’s capability to write or modify a file or directory.
  • execute – The Execute permission affects a user’s capability to execute a file or view the contents of a directory.

Viewing the Permissions

You can view the permissions by checking the file or directory permissions in your favorite GUI File Manager (which I will not cover here) or by reviewing the output of the “ls -l” command while in the terminal and while working in the directory which contains the file or folder.

The permission in the command line is displayed as: _rwxrwxrwx 1 owner:group

  1. User rights/Permissions
    1. The first character that I marked with an underscore is the special permission flag that can vary.
    2. The following set of three characters (rwx) is for the owner permissions.
    3. The second set of three characters (rwx) is for the Group permissions.
    4. The third set of three characters (rwx) is for the All Users permissions.
  2. Following that grouping since the integer/number displays the number of hardlinks to the file.
  3. The last piece is the Owner and Group assignment formatted as Owner:Group.

Modifying the Permissions

When in the command line, the permissions are edited by using the command chmod. You can assign the permissions explicitly or by using a binary reference as described below.

Explicitly Defining Permissions

To explicitly define permissions you will need to reference the Permission Group and Permission Types.

The Permission Groups used are:

  • u – Owner
  • g – Group
  • o – Others
  • a – All users

The potential Assignment Operators are + (plus) and – (minus); these are used to tell the system whether to add or remove the specific permissions.

The Permission Types that are used are:

  • r – Read
  • w – Write
  • x – Execute

So for example, let’s say I have a file named file1 that currently has the permissions set to _rw_rw_rw, which means that the owner, group, and all users have read and write permission. Now we want to remove the read and write permissions from the all users group.

To make this modification you would invoke the command: chmod a-rw file1
To add the permissions above you would invoke the command: chmod a+rw file1

As you can see, if you want to grant those permissions you would change the minus character to a plus to add those permissions.

Using Binary References to Set permissions

Now that you understand the permissions groups and types this one should feel natural. To set the permission using binary references you must first understand that the input is done by entering three integers/numbers.

A sample permission string would be chmod 640 file1, which means that the owner has read and write permissions, the group has read permissions, and all other user have no rights to the file.

The first number represents the Owner permission; the second represents the Group permissions; and the last number represents the permissions for all other users. The numbers are a binary representation of the rwx string.

  • r = 4
  • w = 2
  • x = 1

You add the numbers to get the integer/number representing the permissions you wish to set. You will need to include the binary permissions for each of the three permission groups.

So to set a file to permissions on file1 to read _rwxr_____, you would enter chmod 740 file1.

Owners and Groups

I have made several references to Owners and Groups above, but have not yet told you how to assign or change the Owner and Group assigned to a file or directory.

You use the chown command to change owner and group assignments, the syntax is simple

chown owner:group filename,

so to change the owner of file1 to user1 and the group to family you would enter chown user1:family file1.

Advanced Permissions

The special permissions flag can be marked with any of the following:

  • _ – no special permissions
  • d – directory
  • l– The file or directory is a symbolic link
  • s – This indicated the setuid/setgid permissions. This is not set displayed in the special permission part of the permissions display, but is represented as a s in the read portion of the owner or group permissions.
  • t – This indicates the sticky bit permissions. This is not set displayed in the special permission part of the permissions display, but is represented as a t in the executable portion of the all users permissions

Setuid/Setgid Special Permissions

The setuid/setguid permissions are used to tell the system to run an executable as the owner with the owner’s permissions.

Be careful using setuid/setgid bits in permissions. If you incorrectly assign permissions to a file owned by root with the setuid/setgid bit set, then you can open your system to intrusion.

You can only assign the setuid/setgid bit by explicitly defining permissions. The character for the setuid/setguid bit is s.

So do set the setuid/setguid bit on file2.sh you would issue the command chmod g+s file2.sh.

Sticky Bit Special Permissions

The sticky bit can be very useful in shared environment because when it has been assigned to the permissions on a directory it sets it so only file owner can rename or delete the said file.

You can only assign the sticky bit by explicitly defining permissions. The character for the sticky bit is t.

To set the sticky bit on a directory named dir1 you would issue the command chmod +t dir1.

When Permissions Are Important

To some users of Mac- or Windows-based computers, you don’t think about permissions, but those environments don’t focus so aggressively on user-based rights on files unless you are in a corporate environment. But now you are running a Linux-based system and permission-based security is simplified and can be easily used to restrict access as you please.

So I will show you some documents and folders that you want to focus on and show you how the optimal permissions should be set.

  • home directories– The users’ home directories are important because you do not want other users to be able to view and modify the files in another user’s documents of desktop. To remedy this you will want the directory to have the drwx______ (700) permissions, so lets say we want to enforce the correct permissions on the user user1’s home directory that can be done by issuing the command chmod 700 /home/user1.
  • bootloader configuration files– If you decide to implement password to boot specific operating systems then you will want to remove read and write permissions from the configuration file from all users but root. To do you can change the permissions of the file to 700.
  • system and daemon configuration files– It is very important to restrict rights to system and daemon configuration files to restrict users from editing the contents, it may not be advisable to restrict read permissions, but restricting write permissions is a must. In these cases it may be best to modify the rights to 644.
  • firewall scripts – It may not always be necessary to block all users from reading the firewall file, but it is advisable to restrict the users from writing to the file. In this case the firewall script is run by the root user automatically on boot, so all other users need no rights, so you can assign the 700 permissions.

Other examples can be given, but this article is already very lengthy, so if you want to share other examples of needed restrictions please do so in the comments.

This is a classic article written by Jack Wallen from the Linux.com archives. For more great SysAdmin tips and techniques check out our free intro to Linux course.

It goes without saying that every good Linux desktop environment offers the ability to search your file system for files and folders. If your default desktop doesn’t — because this is Linux — you can always install an app to make searching your directory hierarchy a breeze.

But what about the command line? If you happen to frequently work in the command line or you administer GUI-less Linux servers, where do you turn when you need to locate a file? Fortunately, Linux has exactly what you need to locate the files in question, built right into the system.

The command in question is find. To make the understanding of this command even more enticing, once you know it, you can start working it into your Bash scripts. That’s not only convenience, that’s power.

Let’s get up to speed with the find command so you can take control of locating files on your Linux servers and desktops, without the need of a GUI.

How to use the find command

When I first glimpsed Linux, back in 1997, I didn’t quite understand how the find command worked; therefore, it never seemed to function as I expected. It seemed simple; issue the command find FILENAME (where FILENAME is the name of the file) and the command was supposed to locate the file and report back. Little did I know there was more to the command than that. Much more.

If you issue the command man find, you’ll see the syntax of the find command is:

find [-H] [-L] [-P] [-D debugopts] [-Olevel] [starting-point...] [expression]

Naturally, if you’re unfamiliar with how man works, you might be confused about or overwhelmed by that syntax. For ease of understanding, let’s simplify that. The most basic syntax of a basic find command would look like this:

find /path option filename

Now we’ll see it at work.

Find by name

Let’s break down that basic command to make it as clear as possible. The most simplistic  structure of the find command should include a path for the file, an option, and the filename itself. You may be thinking, “If I know the path to the file, I’d already know where to find it!”. Well, the path for the file could be the root of your drive; so / would be a legitimate path. Entering that as your path would take find longer to process — because it has to start from scratch — but if you have no idea where the file is, you can start from there. In the name of efficiency, it is always best to have at least an idea where to start searching.

The next bit of the command is the option. As with most Linux commands, you have a number of available options. However, we are starting from the beginning, so let’s make it easy. Because we are attempting to find a file by name, we’ll use one of two options:

  • name – case sensitive

  • iname – case insensitive

Remember, Linux is very particular about case, so if you’re looking for a file named Linux.odt, the following command will return no results.

find / -name linux.odt

If, however, you were to alter the command by using the -iname option, the find command would locate your file, regardless of case. So the new command looks like:

find / -iname linux.odt

Find by type

What if you’re not so concerned with locating a file by name but would rather locate all files of a certain type? Some of the more common file descriptors are:

  • f – regular file

  • d – directory

  • l – symbolic link

  • c – character devices

  • b – block devices

Now, suppose you want to locate all block devices (a file that refers to a device) on your system. With the help of the -type option, we can do that like so:

find / -type c

The above command would result in quite a lot of output (much of it indicating permission denied), but would include output similar to:

/dev/hidraw6
/dev/hidraw5
/dev/vboxnetctl
/dev/vboxdrvu
/dev/vboxdrv
/dev/dmmidi2
/dev/midi2
/dev/kvm

Voilà! Block devices.

We can use the same option to help us look for configuration files. Say, for instance, you want to locate all regular files that end in the .conf extension. This command would look something like:

find / -type f -name "*.conf"

The above command would traverse the entire directory structure to locate all regular files ending in .conf. If you know most of your configuration files are housed in /etc, you could specify that like so:

find /etc -type f -name “*.conf”

The above command would list all of your .conf files from /etc (Figure 1).

Figure 1: Locating all of your configuration files in /etc.

Outputting results to a file

One really handy trick is to output the results of the search into a file. When you know the output might be extensive, or if you want to comb through the results later, this can be incredibly helpful. For this, we’ll use the same example as above and pipe the results into a file called conf_search. This new command would look like: ​

find /etc -type f -name “*.conf” > conf_search

You will now have a file (conf_search) that contains all of the results from the find command issued.

Finding files by size

Now we get to a moment where the find command becomes incredibly helpful. I’ve had instances where desktops or servers have found their drives mysteriously filled. To quickly make space (or help locate the problem), you can use the find command to locate files of a certain size. Say, for instance, you want to go large and locate files that are over 1000MB. The find command can be issued, with the help of the -size option, like so:

find / -size +1000MB

You might be surprised at how many files turn up. With the output from the command, you can comb through the directory structure and free up space or troubleshoot to find out what is mysteriously filling up your drive.

You can search with the following size descriptions:

  • c – bytes

  • k – Kilobytes

  • M – Megabytes

  • G – Gigabytes

  • b – 512-byte blocks

Keep learning

We’ve only scratched the surface of the find command, but you now have a fundamental understanding of how to locate files on your Linux systems. Make sure to issue the command man find to get a deeper, more complete, knowledge of how to make this powerful tool work for you.

This is a classic article written by Jack Wallen from the Linux.com archives. For more great SysAdmin tips and techniques check out our free intro to Linux course.

Picture this: You’ve launched an application (be it from your favorite desktop menu or from the command line) and you start using that launched app, only to have it lock up on you, stop performing, or unexpectedly die. You try to run the app again, but it turns out the original never truly shut down completely.

What do you do? You kill the process. But how? Believe it or not, your best bet most often lies within the command line. Thankfully, Linux has every tool necessary to empower you, the user, to kill an errant process. However, before you immediately launch that command to kill the process, you first have to know what the process is. How do you take care of this layered task? It’s actually quite simple…once you know the tools at your disposal.

Let me introduce you to said tools.

The steps I’m going to outline will work on almost every Linux distribution, whether it is a desktop or a server. I will be dealing strictly with the command line, so open up your terminal and prepare to type.

Locating the process

The first step in killing the unresponsive process is locating it. There are two commands I use to locate a process: top and ps. Top is a tool every administrator should get to know. With top, you get a full listing of currently running process. From the command line, issue top to see a list of your running processes (Figure 1).

Figure 1: The top command gives you plenty of information.

From this list you will see some rather important information. Say, for example, Chrome has become unresponsive. According to our top display, we can discern there are four instances of chrome running with Process IDs (PID) 3827, 3919, 10764, and 11679. This information will be important to have with one particular method of killing the process.

Although top is incredibly handy, it’s not always the most efficient means of getting the information you need. Let’s say you know the Chrome process is what you need to kill, and you don’t want to have to glance through the real-time information offered by top. For that, you can make use of the ps command and filter the output through grep. The ps command reports a snapshot of a current process and grep prints lines matching a pattern. The reason why we filter ps through grep is simple: If you issue the ps command by itself, you will get a snapshot listing of all current processes. We only want the listing associated with Chrome. So this command would look like:

ps aux | grep chrome

The aux options are as follows:

  • a = show processes for all users

  • u = display the process’s user/owner

  • x = also show processes not attached to a terminal

The x option is important when you’re hunting for information regarding a graphical application.

When you issue the command above, you’ll be given more information than you need (Figure 2) for the killing of a process, but it is sometimes more efficient than using top.

Figure 2: Locating the necessary information with the ps command.

Killing the process

Now we come to the task of killing the process. We have two pieces of information that will help us kill the errant process:

  • Process name
  • Process ID

Which you use will determine the command used for termination. There are two commands used to kill a process:

  • kill – Kill a process by ID
  • killall – Kill a process by name

There are also different signals that can be sent to both kill commands. What signal you send will be determined by what results you want from the kill command. For instance, you can send the HUP (hang up) signal to the kill command, which will effectively restart the process. This is always a wise choice when you need the process to immediately restart (such as in the case of a daemon). You can get a list of all the signals that can be sent to the kill command by issuing kill -l. You’ll find quite a large number of signals (Figure 3).

Figure 3: The available kill signals.

The most common kill signals are:

Signal Name

Single Value

Effect

SIGHUP

1

Hangup

SIGINT

2

Interrupt from keyboard

SIGKILL

9

Kill signal

SIGTERM

15

Termination signal

SIGSTOP

17, 19, 23

Stop the process

What’s nice about this is that you can use the Signal Value in place of the Signal Name. So you don’t have to memorize all of the names of the various signals.
So, let’s now use the kill command to kill our instance of chrome. The structure for this command would be:

kill SIGNAL PID

Where SIGNAL is the signal to be sent and PID is the Process ID to be killed. We already know, from our ps command that the IDs we want to kill are 3827, 3919, 10764, and 11679. So to send the kill signal, we’d issue the commands:

kill -9 3827

kill -9 3919

kill -9 10764

kill -9 11679

Once we’ve issued the above commands, all of the chrome processes will have been successfully killed.

Let’s take the easy route! If we already know the process we want to kill is named chrome, we can make use of the killall command and send the same signal the process like so:

killall -9 chrome

The only caveat to the above command is that it may not catch all of the running chrome processes. If, after running the above command, you issue the ps aux|grep chrome command and see remaining processes running, your best bet is to go back to the kill command and send signal 9 to terminate the process by PID.

Ending processes made easy

As you can see, killing errant processes isn’t nearly as challenging as you might have thought. When I wind up with a stubborn process, I tend to start off with the killall command as it is the most efficient route to termination. However, when you wind up with a really feisty process, the kill command is the way to go.

Our recently published Open Source Jobs Report examined the demand for open source talent and trends among open source professionals. What did we find?

Open Source Career Opportunities are Strong

The good news is that hiring is rebounding in the wake of the pandemic, as organizations look to continue their investments in digital transformation. This is evidenced by 50% of employers surveyed who stated they are increasing hires this year. There are significant challenges though, with 92% of managers having difficulty finding enough talent and struggling to hold onto existing talent in the face of fierce competition. Other key findings from this year’s report included:

  • Cloud is on the rise. Cloud and container technology skills are most in-demand by hiring managers, surpassing Linux for the first time, with 46% of hiring managers seeking cloud talent.
  • DevOps has become the standard method for developing software. Virtually all open source professionals (88%) report using DevOps practices in their work, a 50% increase from three years ago.
  • Demand for certified talent is spiking. Managers are prioritizing hires of certified talent (88%).
  • Training is increasingly helping close skills gaps. 92% of managers report increasing requests for training. Employers also report that they prioritize training investments to close skills gaps, with 58% using this tactic.
  • Discrimination is a growing concern in the community. Open source professionals having been discriminated against or made to feel unwelcome in the community increased to 18% in 2021 — a 125% increase over the past three years.

Enabling Training and Certification

This year, ‬vendor-neutral training and certification grew in importance as demand for professionals with critical skills in open cloud technologies and DevOps increased‭.‬ Over 2 million individuals have enrolled in free Linux Foundation training courses, providing them a great way to explore different open source technologies and decide which is the best fit for them; this includes over a million students who have enrolled in our Introduction to Linux course on the edX platform. To date, over 50,000 individuals have been certified for their technical competence through Linux Foundation programs.

This year, our Training & Certification team launched over 20 new offerings. We now host over 70 eLearning courses, deliver over 20 instructor-led courses, and offer more than a dozen certification exams that enable certified professionals to demonstrate their skills, with more being released regularly. 

This year saw the addition of exam simulators to our Kubernetes certification exams, enabling exam registrants to familiarize themselves with the exam environment before sitting for their exam. In late 2021, we will launch a new Kubernetes and Cloud Native Associate certification exam, which will serve as an entry-level certification for new cloud professionals.

In 2021, The Linux Foundation directly awarded 500 scholarships for free training and certification to individuals worldwide. Hundreds more were awarded via partnerships with nonprofits, including Blacks in Technology, TransTech Social Enterprises, and Women Who Code.

New training and certification offerings launched in 2021 include:

  • Building a RISC-V CPU Core
  • Certified Kubernetes and Cloud Native Associate (KCNA)
  • Certified TARS Application Developer (CTAD)
  • FinOps for Engineering
  • Generating a Software Bill of Materials
  • GitOps: Continuous Delivery on Kubernetes with Flux
  • Hyperledger Besu Essentials:
  • Creating a Private Blockchain Network
  • Kubernetes and Cloud Native Essentials
  • Kubernetes Security Essentials
  • Kubernetes Security Fundamentals
  • Implementing DevSecOps
  • Introduction to Cloud Foundry
  • Introduction to FDC3 Standard
  • Introduction to GitOps
  • Introduction to Kubernetes on Edge with K3s
  • Introduction to Magma:
  • Cloud Native Wireless Networking
  • Introduction to Node.js
  • Introduction to RISC-V
  • Introduction to WebAssembly
  • Open Source Management and Strategy
  • RISC-V Toolchain and Compiler Optimization
  • Techniques
  • WebAssembly Actors: From Cloud to Edge

Explore the full catalog of courses at training.linuxfoundation.org/full-catalog.

Linux Foundation Blog Post Abstract Graphic

Every month there seems to be a new software vulnerability showing up on social media, which causes open source program offices and security teams to start querying their inventories to see how FOSS components they use may impact their organizations. 

Frequently this information is not available in a consistent format within an organization for automatic querying and may result in a significant amount of email and manual effort. By exchanging software metadata in a standardized software bill of materials (SBOM) format between organizations, automation within an organization becomes simpler, accelerating the discovery process and uncovering risk so that mitigations can be considered quickly. 

In the last year, we’ve also seen standards like OpenChain (ISO/IEC 5320:2020) gain adoption in the supply chain. Customers have started asking for a bill of materials from their suppliers as part of negotiation and contract discussions to conform to the standard. OpenChain has a focus on ensuring that there is sufficient information for license compliance, and as a result, expects metadata for the distributed components as well. A software bill of materials can be used to support the systematic review and approval of each component’s license terms to clarify the obligations and restrictions as it applies to the distribution of the supplied software and reduces risk. 

Kate Stewart, VP, Dependable Embedded Systems, The Linux Foundation, will host a complimentary mentorship webinar entitled Generating Software Bill Of Materials on Thursday, March 25 at 7:30 am PST. This session will work through the minimum elements included in a software bill of materials and detail the reasoning behind why those elements are included. To register, please click here

There are many ways this software metadata can be shared. The common SBOM document format options (SPDX, SWID, and CycloneDX) will be reviewed so that the participants can better understand what is available for those just starting. 

This mentorship session will work through some simple examples and then guide where to find the next level of details and further references. 

At the end of this session, participants will be on a secure footing and a path towards the automated generation of SBOMs as part of their build and release processes in the future. 

Click here to read the February 2021 Linux Foundation Newsletter

Jason Perlow, Director of Project Insights and Editorial Content at the Linux Foundation, had an opportunity to speak with Shuah Khan about her experiences as a woman in the technology industry. She discusses how mentorship can improve the overall diversity and makeup of open source projects, why software maintainers are important for the health of open source projects such as the Linux kernel, and how language inclusivity and codes of conduct can improve relationships and communication between software maintainers and individual contributors.

JP: So, Shuah, I know you wear many different hats at the Linux Foundation. What do you call yourself around here these days?

SK: <laughs> Well, I primarily call myself a Kernel Maintainer & Linux Fellow. In addition to that, I focus on two areas that are important to the continued health and sustainability of the open source projects in the Linux ecosystem. The first one is bringing more women into the Kernel community, and additionally, I am leading the mentorship program efforts overall at the Linux Foundation. And in that role, in addition to the Linux Kernel Mentorship, we are looking at how the Linux Foundation mentorship program is working overall, how it is scaling. I make sure the LFX Mentorship platform scales and serves diverse mentees and mentors’ needs in this role. 

The LF mentorships program includes several projects in the Linux kernel, LFN, HyperLedger, Open MainFrame, OpenHPC, and other technologies. The Linux Foundation’s Mentorship Programs are designed to help developers with the necessary skills–many of whom are first-time open source contributors–experiment, learn, and contribute effectively to open source communities. 

The mentorship program has been successful in its mission to train new developers and make these talented pools of prospective employees trained by experts to employers. Several graduated mentees have found jobs. New developers have improved the quality and security of various open source projects, including the Linux kernel. Several Linux kernel bugs were fixed, a new subsystem mentor was added, and a new driver maintainer is now part of the Linux kernel community. My sincere thanks to all our mentors for volunteering to share their expertise.

JP: How long have you been working on the Kernel?

SK: Since 2010, or 2011, I got involved in the Android Mainlining project. My first patch removed the Android pmem driver.

JP: Wow! Is there any particular subsystem that you specialize in?

SK: I am a self described generalist. I maintain the kernel self-test subsystem, the USB over IP driver, usbip tool, and the cpupower tool. I contributed to the media subsystem working on Media Controller Device Allocator API to resolve shared device resource management problems across device drivers from different subsystems.

JP: Hey, I’ve actually used the USB over IP driver when I worked at Microsoft on Azure. And also, when I’ve used AWS and Google Compute. 

SK: It’s a small niche driver used in cloud computing. Docker and other containers use that driver heavily. That’s how they provide remote access to USB devices on the server to export devices to be imported by other systems for use.

JP: I initially used it for IoT kinds of stuff in the embedded systems space. Were you the original lead developer on it, or was it one of those things you fell into because nobody else was maintaining it?

SK: Well, twofold. I was looking at USB over IP because I like that technology. it just so happened the driver was brought from the staging tree into the Mainline kernel, I volunteered at the time to maintain it. Over the last few years, we discovered some security issues with it, because it handles a lot of userspace data, so I had a lot of fun fixing all of those. <laugh>.

JP: What drew you into the Linux operating system, and what drew you into the kernel development community in the first place?

SK: Well, I have been doing kernel development for a very long time. I worked on the LynxOS RTOS, a while back, and then HP/UX, when I was working at HP, after which I transitioned into  doing open source development — the OpenHPI project, to support HP’s rack server hardware, and that allowed me to work much more closely with Linux on the back end. And at some point, I decided I wanted to work with the kernel and become part of the Linux kernel community. I started as an independent contributor.

JP: Maybe it just displays my own ignorance, but you are the first female, hardcore Linux kernel developer I have ever met. I mean, I had met female core OS developers before — such as when I was at Microsoft and IBM — but not for Linux. Why do you suppose we lack women and diversity in general when participating in open source and the technology industry overall?

SK: So I’ll answer this question from my perspective, from what I have seen and experienced, over the years. You are right; you probably don’t come across that many hardcore women Kernel developers. I’ve been working professionally in this industry since the early 1990s, and on every project I have been involved with, I am usually the only woman sitting at the table. Some of it, I think, is culture and society. There are some roles that we are told are acceptable to women — even me, when I was thinking about going into engineering as a profession. Some of it has to do with where we are guided, as a natural path. 

There’s a natural resistance to choosing certain professions that you have to overcome first within yourself and externally. This process is different for everybody based on their personality and their origin story. And once you go through the hurdle of getting your engineering degree and figuring out which industry you want to work in, there is a level of establishing credibility in those work environments you have to endure and persevere. Sometimes when I would walk into a room, I felt like people were looking at me and thinking, “why is she here?” You aren’t accepted right away, and you have to overcome that as well. You have to go in there and say, “I am here because I want to be here, and therefore, I belong here.” You have to have that mindset. Society sends you signals that “this profession is not for me” — and you have to be aware of that and resist it. I consider myself an engineer that happens to be a woman as opposed to a woman engineer.

JP: Are you from India, originally?

SK: Yes.

JP: It’s funny; my wife really likes this Netflix show about matchmaking in India. Are you familiar with it?

SK: <laughs> Yes I enjoyed the series, and A Suitable Girl documentary film that follows three women as they navigate making decisions about their careers and family obligations.

JP: For many Americans, this is our first introduction to what home life is like for Indian people. But many of the women featured on this show are professionals, such as doctors, lawyers, and engineers. And they are very ambitious, but of course, the family tries to set them up in a marriage to find a husband for them that is compatible. As a result, you get to learn about the traditional values and roles they still want women to play there — while at the same time, many women are coming out of higher learning institutions in that country that are seeking technical careers. 

SK: India is a very fascinatingly complex place. But generally speaking, in a global sense, having an environment at home where your parents tell you that you may choose any profession you want to choose is very encouraging. I was extremely fortunate to have parents like that. They never said to me that there was a role or a mold that I needed to fit into. They have always told me, “do what you want to do.” Which is different; I don’t find that even here, in the US. Having that support system, beginning in the home to tell you, “you are open to whatever profession you want to choose,” is essential. That’s where a lot of the change has to come from. 

JP: Women in technical and STEM professions are becoming much more prominent in other countries, such as China, Japan, and Korea. For some reason, in the US, I tend to see more women enter the medical profession than hard technology — and it might be a level of effort and perceived reward thing. You can spend eight years becoming a medical doctor or eight years becoming a scientist or an engineer, and it can be equally difficult, but the compensation at the end may not be the same. It’s expensive to get an education, and it takes a long time and hard work, regardless of the professional discipline.

SK: I have also heard that women also like to enter professions where they can make a difference in the world — a human touch, if you will. So that may translate to them choosing careers where they can make a larger impact on people — and they may view careers in technology as not having those same attributes. Maybe when we think about attracting women to technology fields, we might have to promote technology aspects that make a difference. That may be changing now, such as the LF Public Health (LFPH) project we kicked off last year. And with LF AI & Data Foundation, we are also making a difference in people’s lives, such as detecting earthquakes or analyzing climate change. If we were to promote projects such as these, we might draw more women in.

JP: So clearly, one of the areas of technology where you can make a difference is in open source, as the LF is hosting some very high-concept and existential types of projects such as LF Energy, for example — I had no idea what was involved in it and what its goals were until I spoke to Shuli Goodman in-depth about it. With the mentorship program, I assume we need this to attract fresh talent — because as folks like us get older and retire, and they exit the field, we need new people to replace them. So I assume mentorship, for the Linux Foundation, is an investment in our own technologies, correct?

SK: Correct. Bringing in new developers into the fold is the primary purpose, of course — and at the same time, I view the LF as taking on mentorship provides that neutral, level playing field across the industry for all open source projects. Secondly, we offer a self-service platform, LFX Mentorship, where anyone can come in and start their project. So when the COVID-19 pandemic began, we expanded this program to help displaced people — students, et cetera, and less visible projects. Not all projects typically get as much funding or attention as others do — such as a Kubernetes or  Linux kernel — among the COVID mentorship program projects we are funding. I am particularly proud of supporting a climate change-related project, Using Machine Learning to Predict Deforestation.

The self-service approach allows us to fund and add new developers to projects where they are needed. The LF mentorships are remote work opportunities that are accessible to developers around the globe. We see people sign up for mentorship projects from places we haven’t seen before, such as Africa, and so on, thus creating a level playing field. 

The other thing that we are trying to increase focus on is how do you get maintainers? Getting new developers is a starting point, but how do we get them to continue working on the projects they are mentored on? As you said, someday, you and I and others working on these things are going to retire, maybe five or ten years from now. This is a harder problem to solve than training and adding new developers to the project itself.

JP: And that is core to our software supply chain security mission. It’s one thing to have this new, flashy project, and then all these developers say, “oh wow, this is cool, I want to join that,” but then, you have to have a certain number of people maintaining it for it to have long-term viability. As we learned in our FOSS study with Harvard, there are components in the Linux operating system that are like this. Perhaps even modules within the kernel itself, I assume that maybe you might have only one or two people actively maintaining it for many years. And what happens if that person dies or can no longer work? What happens to that code? And if someone isn’t familiar with that code, it might become abandoned. That’s a serious problem in open source right now, isn’t it?

SK: Right. We have seen that with SSH and other security-critical areas. What if you don’t have the bandwidth to fix it? Or the money to fix it? I ended up volunteering to maintain a tool for a similar reason when the maintainer could no longer contribute regularly. It is true; we have many drivers where maintainer bandwidth is an issue in the kernel. So the question is, how do we grow that talent pool?

JP: Do we need a job board or something? We need X number of maintainers. So should we say, “Hey, we know you want to join the kernel project as a contributor, and we have other people working on this thing, but we really need your help working on something else, and if you do a good job, we know tons of companies willing to hire developers just like you?” 

SK: With the kernel, we are talking about organic growth; it is just like any other open source project. It’s not a traditional hire and talent placement scenario. Organically they have to have credibility, and they have to acquire it through experience and relationships with people on those projects. We just talked about it at the previous Linux Plumbers Conference, we do have areas where we really need maintainers, and the MAINTAINERS file does show areas where they need help. 

To answer your question, it’s not one of those things where we can seek people to fill that role, like LinkedIn or one of the other job sites. It has to be an organic fulfillment of that role, so the mentorship program is essential in creating those relationships. It is the double-edged sword of open source; it is both the strength and weakness. People need to have an interest in becoming a maintainer and also a commitment to being one, long term.

JP: So, what do you see as the future of your mentorship and diversity efforts at the Linux Foundation? What are you particularly excited about that is forthcoming that you are working on?

SK: I view the Linux Foundation mentoring as a three-pronged approach to provide unstructured webinars, training courses, and structured mentoring programs. All of these efforts combine to advance a diverse, healthy, and vibrant open source community. So over the past several months, we have been morphing our speed mentorship style format into an expanded webinar format — the LF Live Mentorship series. This will have the function of growing our next level of expertise. As a complement to our traditional mentorship programs, these are webinars and courses that are an hour and a half long that we hold a few times a month that tackle specific technical areas in software development. So it might cover how to write great commit logs, for example, for your patches to be accepted, or how to find bugs in C code. Commit logs are one of those things that are important to code maintenance, so promoting good documentation is a beneficial thing. Webinars provide a way for experts short on time to share their knowledge with a few hours of time commitment and offer a self-paced learning opportunity to new developers.

Additionally, I have started the Linux Kernel Mentorship forum for developers and their mentors to connect and interact with others participating in the Linux Kernel Mentorship program and graduated mentees to mentor new developers. We kicked off Linux Kernel mentorship Spring 2021 and are planning for Summer and Fall.

A big challenge is we are short on mentors to be able to scale the structured program. Solving the problem requires help from LF member companies and others to encourage their employees to mentor, “it takes a village,” they say.

JP: So this webinar series and the expanded mentorship program will help developers cultivate both hard and soft skills, then.

SK: Correct. The thing about doing webinars is that if we are talking about this from a diversity perspective, they might not have time for a full-length mentorship, typically like a three-month or six-month commitment. This might help them expand their resources for self-study. When we ask for developers’ feedback about what else they need to learn new skill sets, we hear that they don’t have resources, don’t have time to do self-study, and learn to become open source developers and software maintainers. This webinar series covers general open source software topics such as the Linux kernel and legal issues. It could also cover topics specific to other LF projects such as CNCF, Hyperledger, LF Networking, etc.

JP: Anything else we should know about the mentorship program in 2021?

SK: In my view,  attracting diversity and new people is two-fold. One of the things we are working on is inclusive language. Now, we’re not talking about curbing harsh words, although that is a component of what we are looking at. The English you and I use in North America isn’t the same English used elsewhere. As an example, when we use North American-centric terms in our email communications, such as when a maintainer is communicating on a list with people from South Korea, something like “where the rubber meets the road” may not make sense to them at all. So we have to be aware of that.

JP: I know that you are serving on the Linux kernel Code of Conduct Committee and actively developing the handbook. When I first joined the Linux Foundation, I learned what the Community Managers do and our governance model. I didn’t realize that we even needed to have codes of conduct for open source projects. I have been covering open source for 25 years, but I come out of the corporate world, such as IBM and Microsoft. Codes of Conduct are typically things that the Human Resources officer shows you during your initial onboarding, as part of reviewing your employee manual. You are expected to follow those rules as a condition of employment. 

So why do we need Codes of Conduct in an open source project? Is it because these are people who are coming from all sorts of different backgrounds, companies, and ways of life, and may not have interacted in this form of organized and distributed project before? Or is it about personalities, people interacting with each other over long distance, and email, which creates situations that may arise due to that separation?

SK: Yes, I come out of the corporate world as well, and of course, we had to practice those codes of conduct in that setting. But conduct situations arise that you have to deal with in the corporate world. There are always interpersonal scenarios that can be difficult or challenging to work with — the corporate world isn’t better than the open source world in that respect. It is just that all of that happens behind a closed setting.

But there is no accountability in the open source world because everyone participates out of their own free will. So on a small, traditional closed project, inside the corporate world, where you might have 20 people involved, you might get one or two people that could be difficult to work with. The same thing happens and is multiplied many times in the open source community, where you have hundreds of thousands of developers working across many different open source projects. 

The biggest problem with these types of projects when you encounter situations such as this is dealing with participation in public forums. In the corporate world, this can be addressed in private. But on a public mailing list, if you are being put down or talked down to, it can be extremely humiliating. 

These interactions are not always extreme cases; they could be simple as a maintainer or a lead developer providing negative feedback — so how do you give it? It has to be done constructively. And that is true for all of us.

JP: Anything else?

SK: In addition to bringing our learnings and applying this to the kernel project, I am also doing this on the ELISA project, where I chair the Technical Steering Committee, where I am bridging communication between experts from the kernel and the safety communities. To make sure we can use the kernel the best ways in safety-critical applications, in the automotive and medical industry, and so on. Many lessons can be learned in terms of connecting the dots, defining clearly what is essential to make Linux run effectively in these environments, in terms of dependability. How can we think more proactively instead of being engaged in fire-fighting in terms of security or kernel bugs? As a result of this, I am also working on any necessary kernel changes needed to support these safety-critical usage scenarios.

JP: Before we go, what are you passionate about besides all this software stuff? If you have any free time left, what else do you enjoy doing?

SK: I read a lot. COVID quarantine has given me plenty of opportunities to read. I like to go hiking, snowshoeing, and other outdoor activities. Living in Colorado gives me ample opportunities to be in nature. I also like backpacking — while I wasn’t able to do it last year because of COVID — I like to take backpacking trips with my son. I also love to go to conferences and travel, so I am looking forward to doing that again as soon as we are able.

Talking about backpacking reminded me of the two-day, 22-mile backpacking trip during the summer of 2019 with my son. You can see me in the picture above at the end of the road, carrying a bearbox, sleeping bag, and hammock. It was worth injuring my foot and hurting in places I didn’t even know I had.

JP: Awesome. I enjoyed talking to you today. So happy I finally got to meet you virtually.

Building a sustainable open source community: training and certifications

Training and professional certifications are an important part of how open source technologies establish themselves as industry-leading solutions and adopted in commercial ecosystems

Introduction

In an earlier piece, we discussed how, over the last 20 years, the Linux Foundation has grown from a single project, the Linux kernel, to an organization that has helped to convene and host hundreds of the world’s most important open source communities. 

The Linux Foundation’s support programs add value for our communities as they enable our projects to engage and grow a technology ecosystem worldwide.  

The Linux Foundation has over 1,600 member companies, representing 100% of the Fortune 100 tech and telecommunication firms, small businesses and startups, hundreds of end-user companies, and everything in between. It also has over 25,000 software developers contributing code, a shared investment that we estimate to be valued at $15.7B – and growing. Our hosted projects enable advancements in many technology areas and across many vertical industries, from security to networking, edge computing, cloud, automotive, blockchain, embedded systems, and web applications.

With the increased demand and adoption of open source technologies comes the desire for professionals with the skill sets to deploy, manage, and operate systems and support end-users. According to the Linux Foundation’s most recent Jobs Report, some key findings were revealed about open source employment opportunities:

Building a sustainable open source community: training and certifications

  • Hiring open source talent is a priority for 83% of hiring managers, a 7% increase from 76% in 2017. 
  • Hiring managers cited cloud (66%) as the technology most affecting their hiring decisions. Containers placed second at 57%, followed by security (49%) and networking (47%).
  • Finding the right mix of experience and skills is difficult for 87% of hiring managers. That included the 44% who rated it very difficult, a percentage that leaped from 34% in 2017.
  • Thirty percent of respondents working in open source technologies improved their ability to work on exciting projects, collaborate with a global community (19%), and work on the most cutting-edge technology challenges (16%). 

This report will be updated this autumn, and early indications show that these trends are accelerating given current market conditions.

The Linux Foundation provides a complete portfolio of support programs for training and certification, which align with the technologies that its communities develop. The support programs currently focus on eight primary domain areas:

  • Linux Internals
  • Open Source Developer Compliance
  • Systems Administration
  • Security 
  • Networking/Edge Computing
  • Cloud
  • Web Development
  • Blockchain

These programs are co-developed with the communities, and we add programs all the time as communities request support. 

Why training and certification are critical for open source communities

The Linux Foundation’s communities request support for training and certification because it creates a cadre of professionals that can implement solutions using their collaboratively developed technologies, with demonstrated expertise. Additionally, without trained and certified professionals, these technologies will face challenges achieving or scaling both industry adoption and commercial ecosystems supporting them. Having end-users adopt the technology, and commercial solution and support providers also provide a pipeline of future contributors back to the project’s codebase. As the open source technology is deployed, it gets tested, bugs are found, new features are requested, and all that feedback cycles its way into the upstream project, sustaining and making the project better for everyone dependent on its continued success.

For many open source projects, to gain adoption and generate a commercial support ecosystem, they will ultimately need to have training and certification programs. While this may sound similar to how other professional communities have matured and have become validated for developer and engineering certifications for commercial clouds and proprietary software systems, there are some important distinctions as to why a commitment to developing training and certification for open source technologies is critical to their long-term success.

The open source community works more organically and cyclically, which necessitates that a cadre of expertise is built for it not just to be deployed (as the commercial training and ecosystem have worked historically over the past 40 years) but also as part of its continuing development and for it and all of its participants to thrive. 

An open source software community develops software, and it gets deployed by professionals. Those professionals often eventually move on to different organizations and implement the same software. Those organizations will ultimately need more people to support deployments and write applications to extend and customize the software. These organizations also need system administration professionals and cloud providers to support solutions based on these open source software systems.

Why should communities create training and certification programs with the Linux Foundation? 

Straight from the source, and integrated into how communities are built and run. As the home of Linux and other major open source technologies, nobody is closer to these projects than The Linux Foundation itself — its training programs are uniquely integrated with our communities and projects. We understand how to align instruction with a community development model. Training is one of the support pillars that also enable the developers and engineers to focus on the open source project’s development and leave educating users and implementers of the code to the Linux Foundation’s training team. 

Accelerating community growth through free training. Thanks to our members’ support of the Linux Foundation and its projects, we are often able to provide free training courses from our communities. Free training is one of the fastest ways to bring more people into our open source communities as they learn, test, deploy and support solutions based on the open source technology, as they usually come back to offer suggestions, feedback, and fixes.

Vendor-neutral courseware. The Linux Foundation is a nonprofit organization and does not promote any particular commercial product, solution, or service.

Excess funds received go back to the project community. Although the Linux Foundation keeps pricing affordable and frequently offers further discounts, the overall program does generate a surplus. Since we are a nonprofit, the surplus is invested back into the open source community in a variety of ways: we provide scholarships to deserving individuals to become trained and certified at no cost, and the Foundation supports projects that are important to the world but do not receive individual or corporate financial support. Surplus funding is also used for linux.com as well as other digital assets and key initiatives such as CommunityBridge. 

Up-to-date Curriculum. Linux Foundation courses are current with the most recent version of the software or technology. As the host of many of the most critical open source projects that are continually changing, the Linux Foundation is in an excellent position to find experts and ensure the materials are maintained and updated alongside the project’s evolution. Additionally, enrolled students receive access to the latest course versions at no additional cost.

Current and cutting-edge technologies. The Linux Foundation hosts the fastest-growing and most influential open source projects and is the first to release courses about them. 

Expert instruction. The Linux Foundation’s courses are created and taught by some of the top developers and practitioners in open source, with decades of collective open source experience behind their belts and a deep familiarity with our open source communities.

Relevant material. The Linux Foundation’s courses are created using feedback from its massive community of open source practitioners and companies. Students can be confident that the topics they are learning are applicable in today’s business environment. Companies and organizations can integrate certifications in their hiring search and evaluations to find professionals with qualified skills.

Conclusion

With the most popular open source projects receiving upwards of 90% of their code from commercial companies, they are continually seeking trained people with the skills to deploy, support, and operate the open source technology. With Linux Foundation training, in most cases being free to access, our communities can efficiently train a vast ecosystem of people with skills companies are seeking to employ. The online delivery of our courses also makes our training accessible to people from low-income regions around the world, where access to training can provide a considerable boost to their career prospects.

Enterprises especially value certifications as evidence that employees are qualified and have demonstrated their expertise in a particular technology. Enterprises also want to train their existing employees on new technologies in an organized, efficient manner, which professional training courses can provide.

Offering training and certification is one of the best ways to scale any growing open source project community. For a project to continue growing and get more contributors involved, the community will need individuals to be able to gain an understanding of the project in a relatively quick and straightforward way. Our organized training curriculum was designed to fill this expertise gap.

The Linux Foundation’s training and certification offerings, combined with its community-organized events, provides a well rounded and neutral path to build skills and enable people to contribute back to its projects, sustaining their efforts into the future.

2018 OS Jobs Report

The latest Open Source Jobs Report shows a strong market for open source talent, driven in part by the rapid growth of cloud technologies.

Linux expertise is again in the top spot as the most sought after open source skill, says the latest Open Source Jobs Report from Dice and The Linux Foundation. The seventh annual report shows rapidly growing demand for open source skills, particularly in areas of cloud technology.

Key findings of the report include:

  • Linux tops the list as the most in-demand open source skill, making it mandatory for most entry-level open source careers. This is due in part to the growth of cloud and container technologies, as well as DevOps practices, all of which are typically built on Linux.
  • Container technology is rapidly growing in popularity and importance, with 57% of hiring managers seeking those skills, up from 27% last year.
  • Hiring open source talent is a priority for 83% of hiring managers, up from 76% in 2017.
  • Hiring managers are increasingly opting to train existing employees on new open source technologies and help them gain certifications.
  • Many organizations are getting involved in open source with the express purpose of attracting developers.

Career Building

In terms of job seeking and job hiring, the report shows high demand for open source skills and a strong career benefit from open source experience.

  • 87% of open source professionals say knowing open source has advanced their career.
  • 87% of hiring managers experience difficulties in recruiting open source talent.

Hiring managers say they are specifically looking to recruit in the following areas:

OS Jobs skillsDiversity

This year’s survey included optional questions about companies’ initiatives to increase diversity in open source hiring, which has become a hot topic throughout the tech industry. The responses showed a significant difference between the views of hiring managers and those of open source pros — with only 52% of employees seeing those diversity efforts as effective compared with 70% of employers.

Overall, the 2018 Open Source Jobs Report indicates a strong market for open source talent, driven in part by the growth of cloud-based technologies. This market provides a wealth of opportunities for professionals with open source skills, as companies increasingly recognize the value of open source.

The 2018 Open Source Jobs Survey and Report, sponsored by Dice and The Linux Foundation, provides an overview of the latest trends for open source careers. Download the complete Open Source Jobs Report now.